TVCG2025

Abstract:
One common task in time series analysis is the visual investigation of spectra such as Fourier spectra or wavelet spectra to identify dominating frequencies. In this article, we present the propagation of data uncertainty to the spectra and its visualization. We consider the Fourier and continuous wavelet transformations, which are two common spectral analysis methods. Deriving the propagation for time series that can be modeled as a Gaussian process leads to a combination of weighted non-central chi-squared distributions in the spectrum. Percentile-based visualizations explicitly encode the non-normal uncertainty in the 1D Fourier and 2D wavelet spectrum. We enrich the visualization by including correlations, sensitivity, and signal-to-noise analysis. For visual exploration, we combine the different visualizations into an interactive approach that allows for investigating the uncertain time series in the temporal and spectral domains. Finally, we show the usefulness of our approach by applying it to several real-world data sets and by a qualitative interview study with visualization experts.

Abstract:
We present a palette-based framework for color composition for visual applications and three large-scale, wide-ranging perceptual studies on the perception of color harmonization. We abstract relationships between palette colors as a compact set of axes describing harmonic templates over perceptually uniform color wheels. Our framework provides a basis for interactive color-aware operations such as color harmonization of images and videos. Because our approach to harmonization is palette-based, we are able to conduct the first controlled perceptual experiments evaluating preferences for harmonized images and color palettes. In a third study, we compare preference for archetypical harmonic palettes. In total, our studies involved over 1000 participants. We found that participants do not prefer harmonized images and that some archetypal palettes are reliably viewed as less harmonious than random palettes. These studies raise important questions for research and artistic practice.

Abstract:
The increasing demand for larger and higher fidelity simulations has made Adaptive Mesh Refinement (AMR) and unstructured mesh techniques essential to focus compute effort and memory cost on just the areas of interest in the simulation domain. The distribution of these meshes over the compute nodes is often determined by balancing compute, memory, and network costs, leading to distributions with jagged nonconvex boundaries that fit together much like puzzle pieces. It is expensive, and sometimes impossible, to re-partition the data posing a challenge for in situ and post hoc visualization as the data cannot be rendered using standard sort-last compositing techniques that require a convex and disjoint data partitioning. We present a new distributed volume rendering and compositing algorithm, Approximate Puzzlepiece Compositing, that enables fast and high-accuracy in-place rendering of AMR and unstructured meshes. Our approach builds on Moment-Based Ordered-Independent Transparency to achieve a scalable, order-independent compositing algorithm that requires little communication and does not impose requirements on the data partitioning. We evaluate the image quality and scalability of our approach on synthetic data and two large-scale unstructured meshes on HPC systems by comparing to state-of-the-art sort-last compositing techniques, highlighting our approach’s minimal overhead at higher core counts. We demonstrate that Approximate Puzzlepiece Compositing provides a scalable, high-performance, and high-quality distributed rendering approach applicable to the complex data distributions encountered in large-scale CFD simulations.

Abstract:
Color smudge operations from digital painting software enable users to create natural shading effects in high-fidelity paintings by interactively mixing colors. To precisely control results in traditional painting software, users tend to organize flat-filled color regions in multiple layers and smudge them to generate different color gradients. However, the requirement to carefully deal with regions makes the smudging process time-consuming and laborious, especially for non-professional users. This motivates us to investigate how to infer user-desired smudging effects when users smudge over regions in a single layer. To investigate improving color smudge performance, we first conduct a formative study. Following the findings of this study, we design SmartSmudge, a novel smudge tool that offers users dynamical smudge brushes and real-time region selection for easily generating natural and efficient shading effects. We demonstrate the efficiency and effectiveness of the proposed tool via a user study and quantitative analysis.

Abstract:
We introduce PICA, a novel representation for high-fidelity animatable clothed human avatars with physics-plausible dynamics, even for loose clothing. Previous neural rendering-based representations of animatable clothed humans typically employ a single model to represent both the clothing and the underlying body. While efficient, these approaches often fail to represent complex garment dynamics, leading to incorrect deformations and noticeable rendering artifacts, especially for sliding or loose garments. Furthermore, most previous works represent garment dynamics as pose-dependent deformations and facilitate novel pose animations in a data-driven manner. This often results in outcomes that do not faithfully represent the mechanics of motion and are prone to generating artifacts in out-of-distribution poses. To address these issues, we employ two individual 2D Gaussian Splatting (2DGS) models with different deformation characteristics, modeling the human body and clothing separately. This distinction allows for better handling of their respective motion characteristics. With this representation, we integrate a graph neural network (GNN)-based clothing physics simulation module to ensure a better representation of clothing dynamics. Our method, through its carefully designed features, achieves high-fidelity rendering of clothed human bodies in complex and novel driving poses, outperforming previous methods under the same settings.

Abstract:
Edge bundling algorithms can significantly improve the visualization of dense graphs by identifying and bundling together suitable groups of edges and thus reducing visual clutter. As such, bundling is often viewed as a post-processing step applied to a drawing, and the vast majority of edge bundling algorithms consider a graph and its drawing as input. A different way of thinking about edge bundling is to simultaneously optimize both the drawing and the bundling, which we investigate in this paper. We build on an earlier work where we introduced a novel algorithmic framework for bundling-aware graph drawing consisting of three main steps, namely Filter for a skeleton subgraph, Draw the skeleton, and Bundle the remaining edges against the drawing of the skeleton. We propose several alternative implementations and experimentally compare them against each other and the simple idea of first drawing the full graph and subsequently applying edge bundling to it. The experiments confirm that bundled drawings created by our Filter-Draw-Bundle framework outperform previous approaches according to metrics for edge bundling and graph drawing.

Abstract:
With the increasing complexity of geometry and rendering effects in virtual reality (VR) scenes, existing foveated rendering methods for VR head-mounted displays (HMDs) struggle to meet users’ demands for VR scene rendering with high frame rates (\geq 60fps≥60fps for rendering binocular foveated images in VR scenes containing over 50 m triangles). Current research validates that auditory content affects the perception of the human visual system (HVS). However, existing foveated rendering methods primarily model the HVS's eccentricity-dependent visual perception ability on the visual content in VR while ignoring the impact of auditory content on the HVS's visual perception. In this article, we introduce an auditory-content-based perceived rendering quality analysis to quantify the impact of visual perception under different auditory conditions in foveated rendering. Based on the analysis results, we propose an audio-visual aware foveated rendering method (AvFR). AvFR first constructs an audio-visual feature-driven perception model that predicts the eccentricity-based visual perception in real time by combining the scene's audio-visual content, and then proposes a foveated rendering cost optimization algorithm to adaptively control the shading rate of different regions with the guidance of the perception model. In complex scenes with visual and auditory content containing over 1.17 m triangles, AvFR renders high-quality binocular foveated images at an average frame rate of 116fpsfps. The results of the main user study and performance evaluation validate that AvFR achieves significant performance improvement (up to 1.4× speedup) without lowering the perceived visual quality compared with the state-of-the-art VR-HMD foveated rendering method.

Abstract:
Remote sensing technologies, such as LiDAR, produce billions of points that commonly exceed the storage capacity of the GPU, restricting their processing and rendering. Level of detail (LoD) techniques have been widely investigated, but building the LoD structures is also time-consuming. This study proposes a GPU-driven culling system focused on determining the number of points visible in every frame. It can manipulate point clouds of any arbitrary size while maintaining a low memory footprint in both the CPU and GPU. Instead of organizing point clouds into hierarchical data structures, these are split into groups of points sorted using the Hilbert encoding. This alternative alleviates the occurrence of anomalous groups found in Morton curves. Instead of keeping the entire point cloud in the GPU, points are transferred on demand to ensure real-time capability. Accordingly, our solution can manipulate huge point clouds even in commodity hardware with low memory capacities. Moreover, hole filling is implemented to cover the gaps derived from insufficient density and our LoD system. Our proposal was evaluated with point clouds of up to 18 billion points, achieving an average of 80 frames per second (FPS) without perceptible quality loss. Relaxing memory constraints further enhances visual quality while maintaining an interactive frame rate. We assessed our method on real-world data, comparing it against three state-of-the-art methods, demonstrating its ability to handle significantly larger point clouds.

Abstract:
Rendering on conventional computers is capable of generating realistic imagery, but the computational complexity of these light transport algorithms is a limiting factor of image synthesis. Quantum computers have the potential to significantly improve rendering performance through reducing the underlying complexity of the algorithms behind light transport. This article investigates hybrid quantum-classical algorithms for ray tracing, a core component of most rendering techniques. Through a practical implementation of quantum ray tracing in a 3D environment, we show quantum approaches provide a quadratic improvement in query complexity compared to the equivalent classical approach. Based on domain specific knowledge, we then propose algorithms to significantly reduce the computation required for quantum ray tracing through exploiting image space coherence and a principled termination criteria for quantum searching. We show results obtained using a simulator for both Whitted style ray tracing, and for accelerating ray tracing operations when performing classical Monte Carlo integration for area lights and indirect illumination.

Abstract:
Welcome to the January 2025 issue of the IEEE Transactions on Visualization and Computer Graphics (TVCG). This is my second IEEE VIS special issue as Editor-in-Chief, and I am very excited to introduce it to you all. As in the previous three years, the papers submitted to IEEE VIS were categorized into six major research subareas: Theoretical & Empirical (112 papers), Applications (154), Systems & Rendering (51), Representations & Interaction (110), Data Transformations (53) and Analytics & Decisions (77). The conference took place in St. Pete Beach, Florida, USA, during October 13-18, 2024. Included in this special issue are the top 124 papers selected by the Program Committee from a total of 557 submissions.

Abstract:
Thunderstorms are complex multiphysics phenomena driven by charge transfer processes arising from interactions between ice and water particles in the atmosphere. We present a physically grounded model for simulating cloud electrification and lightning discharge, capable of generating diverse lightning types as emergent responses to evolving atmospheric conditions. Our approach requires only a minimal set of atmospheric parameters and no user-defined triggers. Charge separation is modeled at the microphysical level using a statistical mechanics framework, while discharges are captured through a novel gauge-invariant dielectric breakdown model that accounts for bipolar channels, dynamic electric fields, and air resistance. We validate our method through comparisons with observational data and prior models, demonstrating its ability to simulate distinct discharge types and the full life cycle of thunderstorms. Beyond scientific accuracy, our framework supports real-time nowcasting, civil engineering assessments, virtual environment generation, and the simulation of complex dielectric breakdown in varied contexts.

Abstract:
3D scene graph generation is essential for spatial computing in Extended Reality (XR), providing structured semantics for task planning and intelligent perception. However, unlike instance-segmentation-driven setups, generating semantic scene graphs still suffer from limited accuracy due to coarse and noisy point cloud data typically acquired in practice, and from the lack of interactive strategies to incorporate users' spatialized and intuitive guidance. We identify three key challenges: designing controllable interaction forms, involving guidance in inference, and generalizing from local corrections. To address these, we propose SGSG, a Stroke-Guided Scene Graph generation method that enables users to interactively refine 3D semantic relationships and improve predictions in real time. We propose three types of strokes and a lightweight SGstrokes dataset tailored for this modality. Our model integrates stroke guidance representation and injection for spatio-temporal feature learning and reasoning correction, along with intervention losses that combine consistency-repulsive and geometry-sensitive constraints to enhance accuracy and generalization. Experiments and the user study show that SGSG outperforms state-of-the-art methods 3DSSG and SGFN in overall accuracy and precision, surpasses JointSSG in predicate-level metrics, and reduces task load across all control conditions, establishing SGSG as a new benchmark for interactive 3D scene graph generation and semantic understanding in XR. Implementation resources are available at: https://github.com/Sycamore-Ma/SGSG-runtime.

Abstract:
We present GS-ProCams, the first Gaussian Splatting-based framework for projector-camera systems (ProCams). GS-ProCams is not only view-agnostic but also significantly enhances the efficiency of projection mapping (PM) that requires establishing geometric and radiometric mappings between the projector and the camera. Previous CNN-based ProCams are constrained to a specific viewpoint, limiting their applicability to novel perspectives. In contrast, NeRF-based ProCams support view-agnostic projection mapping, however, they require an additional co-located light source and demand significant computational and memory resources. To address this issue, we propose GS-ProCams that employs 2D Gaussian for scene representations, and enables efficient view-agnostic ProCams applications. In particular, we explicitly model the complex geometric and photometric mappings of ProCams using projector responses, the projection surface's geometry and materials represented by Gaussians, and the global illumination component. Then, we employ differentiable physically-based rendering to jointly estimate them from captured multi-view projections. Compared to state-of-the-art NeRF-based methods, our GS-ProCams eliminates the need for additional devices, achieving superior ProCams simulation quality. It also uses only 1/10 of the GPU memory for training and is 900 times faster in inference speed. Please refer to our project page for the code and dataset: https://realqingyue.github.io/GS-ProCams/.

Abstract:
Foveated rendering provides an idea for improving the image synthesis performance of neural radiance fields (NeRF) methods. In this article, we propose a scene-aware foveated neural radiance fields method to synthesize high-quality foveated images in complex VR scenes at high frame rates. First, we construct a multi-ellipsoidal neural representation to enhance the neural radiance field's representation capability in salient regions of complex VR scenes based on the scene content. Then, we introduce a uniform sampling based foveated neural radiance field framework to improve the foveated image synthesis performance with one-pass color inference, and improve the synthesis quality by leveraging the foveated scene-aware objective function. Our method synthesizes high-quality binocular foveated images at the average frame rate of 66 frames per second (FPSFPS) in complex scenes with high occlusion, intricate textures, and sophisticated geometries. Compared with the state-of-the-art foveated NeRF method, our method achieves significantly higher synthesis quality in both the foveal and peripheral regions with 1.41–1.46× speedup. We also conduct a user study to prove that the perceived quality of our method has a high visual similarity with the ground truth.

Abstract:
With the remarkable progress of 3D scanning technique, the captured indoor scenes appear increasingly in last decade. Generating orientation-consistent normals for indoor point clouds is a fundamental and important task. The existing orientation rectification methods pay more attention to object-level targets with connected surface. However, it is challenging to compute consistent surface orientation for real scanned indoor point clouds. In this paper, we analyze the causes of this difficulty and propose a new normal reorienting framework for indoor scene consistency, namely NRSC. It first estimates normals for an indoor point cloud and extracts all the connected regions. We then design and construct an abstract orientation bridging tree (OBT) to organize the extracted regions in a hierarchical way. For all node regions, NRSC iteratively implements a set of orientation propagations to generate locally orientation-consistent regions. Moreover, we define an auxiliary viewpoint set for each pairwise parent-child node regions and introduce a voting mechanism to rectify the region orientation of child node according to its parent. After processing all the child node regions along OBT, we finally eliminate the orientation inconsistencies between related regions. Multi-groups of experimental results on both fused indoor scenes and single-view-scenes show that our method generates globally consistent orientation for indoor point clouds.

Abstract:
Software developers often have to gain an understanding of a codebase. Be it programmers getting onboarded onto a team project or, for example, developers striving to grasp an external open-source library. In either case, they frequently turn to the project’s documentation. However, documentation in its traditional textual form is ill-suited for this kind of high-level exploratory analysis, since it is immutable from the readers’ perspective and thus forces them to follow a predefined path. We have designed an approach bringing aspects of software architecture visualization to API reference documentation. It utilizes a highly interactive node-link diagram with expressive node glyphs and flexible filtering capabilities, providing a high-level overview of the codebase as well as details on demand. To test our design, we have implemented a prototype named Helveg, capable of automatically generating diagrams of C# codebases. User testing of Helveg confirmed its potential, but it also revealed problems with the readability, intuitiveness, and user experience of our tool. Therefore, in this paper, which is an extended version of our VISSOFT paper with DOI 10.1109/VISSOFT64034.2024.00012, we address many of these problems through major changes to the glyph design, means of interaction, and user interface of the tool. To assess the improvements, this new version of Helveg was evaluated again with the same group of participants as the previous version.

Abstract:
This article reviews published research in the field of computer-aided colorization technology. We argue that within this context, the colorization task can be considered to originate from computer graphics, advance by introducing computer vision, and progress towards the fusion of vision and graphics. Hence, we propose a specific taxonomy and organize the research work chronologically. We extend the existing reconstruction-based colorization evaluation techniques on the basis that aesthetic assessment should be introduced to ensure the computer-coloredimages closely satisfy human visual-related requirements. We then perform an aesthetic assessment using the proposed metric and existing evaluations, comparing the colorization performance of seven representative unconditional colorization models. Finally, we identify unresolved issues and propose fruitful areas for future research and development.

Abstract:
We introduce Squishicalization, a pipeline for generating physicalizations of volumetric data that encode scalar information through their physical characteristics—specifically, by varying their “squishiness” or local elasticity. Data physicalization research is increasingly exploring multisensory information encoding, with a particular focus on enhancing direct interactivity. With Squishicalization, we leverage the tactile dimension of physicalization as a means of direct interactivity. Inspired by conventional volume rendering, we adapt the concept of transfer functions to encode scalar values from volumetric data into local elasticity levels. In this way, volumetric scalar data are transformed into sculptures, where the elasticity represents physical properties such as the material’s density distribution within the volume. In our pipeline, scalar values guide the weighted sampling of the scalar field. The sampled data is then processed through Voronoi tessellation to create a sponge-like structure, which can be printed with consumer-grade 3D printers and readily available filament. To validate our pipeline, we conduct a computational and mechanical evaluation, as well as a two-stage perceptual study of the capabilities of our generated squishicalizations. To further investigate potential application scenarios, we interview experts across several domains. Finally, we summarize actionable insights and future avenues for the application of our Squishicalization.

Abstract:
Scatterplots provide a visual representation of bivariate data (or 2D embeddings of multivariate data) that allows for effective analyses of data dependencies, clusters, trends, and outliers. Unfortunately, classical scatterplots suffer from scalability issues, since growing data sizes eventually lead to overplotting and visual clutter on a screen with a fixed resolution, which hinders the data analysis process. We propose an algorithm that compensates for irregular sample distributions by a smooth transformation of the scatterplot's visual domain. Our algorithm evaluates the scatterplot's density distribution to compute a regularization mapping based on integral images of the rasterized density function. The mapping preserves the samples’ neighborhood relations. Few regularization iterations suffice to achieve a nearly uniform sample distribution that efficiently uses the available screen space. We further propose approaches to visually convey the transformation that was applied to the scatterplot and compare them in a user study. We present a novel parallel algorithm for fast GPU-based integral-image computation, which allows for integrating our de-cluttering approach into interactive visual data analysis systems.

Abstract:
An ‘invariant descriptor’ captures meaningful structural features of networks, useful where traditional visualizations, like node-link views, face challenges like the ’hairball phenomenon’ (inscrutable overlap of points and lines). Designing invariant descriptors involves balancing abstraction and information retention, as richer data summaries demand more storage and computational resources. Building on prior work, chiefly the BMatrix—a matrix descriptor visualized as the invariant ’network portrait’ heatmap—we introduce BFS-Census, a new algorithm computing our Census data structures: Census-Node, Census-Edge, and Census-Stub. Our experiments show Census-Stub, which focuses on ’stubs’ (half-edges), has orders of magnitude greater discerning power (ability to tell non-isomorphic graphs apart) than any other descriptor in this study, without a difficult trade-off: the substantial increase in resolution doesn't come at a commensurate cost in storage space or computation power. We also present new visualizations—our Hop-Census polylines and Census-Census trajectories—and evaluate them using real-world graphs, including a sensitivity analysis that shows graph topology change maps to visual Census change.

Abstract:
Reflectance models capture many types of visual appearances. The most plausible reflectance models follow the Microfacet theory, which is specifically based on statistical representations, with an analytic visibility term. This visibility term has a significant impact on appearance. Visibility computed with the masking term proposed by Smith (1967), and revisited by Ashikhmin et al. (2000), is nowadays considered as the most plausible in the literature. It is simple and efficient to evaluate for statistical distributions, but it relies on assumptions that are not necessarily respected by real surfaces. This article proposes an in-depth study of masking for meshed height-field surfaces, generated either from measured real-world materials or from functions derived from distributions of surface normals. We experimentally estimate the masking (and shadowing) of surfaces using a ray-casting technique, and compare their measurements with the theoretical model from Smith and Ashikhmin et al. We show that their assumptions are too restrictive for a majority of real-world surfaces. We propose a model capable of predicting how close the theoretical masking term can be from the masking term estimated by a ray-casting approach. Although most surfaces break their assumptions, our results show that the term from Smith and Ashikhmin et al. can still be reasonably employed for a fraction in a set of more than 400 measured surfaces, with low errors compared to a ray-casting masking estimation, much lower computation times, and very similar visual appearances. Our model can be used to predict the incurred error on a physically-based rendering simulation with a microfacet-based BRDF created from real-world surfaces, instead of explicitly calculating the masking term from its height field.

Abstract:
We present a novel visualization-driven illumination model for density plots, a new technique to enhance density plots by effectively revealing the detailed structures in high- and medium-density regions and outliers in low-density regions, while avoiding artifacts in the density field's colors. When visualizing large and dense discrete point samples, scatterplots and dot density maps often suffer from overplotting, and density plots are commonly employed to provide aggregated views while revealing underlying structures. Yet, in such density plots, existing illumination models may produce color distortion and hide details in low-density regions, making it challenging to look up density values, compare them, and find outliers. The key novelty in this work includes (i) a visualization-driven illumination model that inherently supports density-plot-specific analysis tasks and (ii) a new image composition technique to reduce the interference between the image shading and the color-encoded density values. To demonstrate the effectiveness of our technique, we conducted a quantitative study, an empirical evaluation of our technique in a controlled study, and two case studies, exploring twelve datasets with up to two million data point samples.

Abstract:
Partitionings (or segmentations) divide a given domain into disjoint connected regions whose union forms again the entire domain. Multi-dimensional partitionings occur, for example, when analyzing parameter spaces of simulation models, where each segment of the partitioning represents a region of similar model behavior. Having computed a partitioning, one is commonly interested in understanding how large the segments are and which segments lie next to each other. While visual representations of 2D domain partitionings that reveal sizes and neighborhoods are straightforward, this is no longer the case when considering multi-dimensional domains of three or more dimensions. We propose an algorithm for computing 2D embeddings of multi-dimensional partitionings. The embedding shall have the following properties: It shall maintain the topology of the partitioning and optimize the area sizes and joint boundary lengths of the embedded segments to match the respective sizes and lengths in the multi-dimensional domain. We demonstrate the effectiveness of our approach by applying it to different use cases, including the visual exploration of 3D spatial domain segmentations and multi-dimensional parameter space partitionings of simulation ensembles. We numerically evaluate our algorithm with respect to how well sizes and lengths are preserved depending on the dimensionality of the domain and the number of segments.

Abstract:
Hypergraphs provide a natural way to represent polyadic relationships in network data. For large hypergraphs, it is often difficult to visually detect structures within the data. Recently, a scalable polygon-based visualization approach was developed allowing hypergraphs with thousands of hyperedges to be simplified and examined at different levels of detail. However, this approach is not guaranteed to eliminate all of the visual clutter caused by unavoidable overlaps. Furthermore, meaningful structures can be lost at simplified scales, making their interpretation unreliable. In this paper, we define hypergraph structures using the bipartite graph representation, allowing us to decompose the hypergraph into a union of structures including topological blocks, bridges, and branches, and to identify exactly where unavoidable overlaps must occur. We also introduce a set of topology preserving and topology altering atomic operations, enabling the preservation of important structures while reducing unavoidable overlaps to improve visual clarity and interpretability in simplified scales. We demonstrate our approach in several real-world applications.

Abstract:
Egocentric networks, often visualized as node-link diagrams, portray the complex relationship (link) dynamics between an entity (node) and others. However, common analytics tasks are multifaceted, encompassing interactions among four key aspects: strength, function, structure, and content. Current node-link visualization designs may fall short, focusing narrowly on certain aspects and neglecting the holistic, dynamic nature of egocentric networks. To bridge this gap, we introduce SpreadLine, a novel visualization framework designed to enable the visual exploration of egocentric networks from these four aspects at the microscopic level. Leveraging the intuitive appeal of storyline visualizations, SpreadLine adopts a storyline-based design to represent entities and their evolving relationships. We further encode essential topological information in the layout and condense the contextual information in a metro map metaphor, allowing for a more engaging and effective way to explore temporal and attribute-based information. To guide our work, with a thorough review of pertinent literature, we have distilled a task taxonomy that addresses the analytical needs specific to egocentric network exploration. Acknowledging the diverse analytical requirements of users, SpreadLine offers customizable encodings to enable users to tailor the framework for their tasks. We demonstrate the efficacy and general applicability of SpreadLine through three diverse real-world case studies (disease surveillance, social media trends, and academic career evolution) and a usability study.

Abstract:
Placing text labels is a common way to explain key elements in a given scene. Given a graphic input and original label information, how to place labels to meet both geometric and aesthetic requirements is an open challenging problem. Geometry-wise, traditional rule-driven solutions struggle to capture the complex interactions between labels, let alone consider graphical/appearance content. In terms of aesthetics, training/evaluation data ideally require nontrivial effort and expertise in design, thus resulting in a lack of decent datasets for learning-based methods. To address the above challenges, we formulate the task with a graph representation, where nodes correspond to labels and edges to interactions between labels, and treat label placement as a node position prediction problem. With this novel representation, we design a Label Placement Graph Transformer (LPGT) to predict label positions. Specifically, edge-level attention, conditioned on node representations, is introduced to reveal potential relationships between labels. To integrate graphic/image information, we design a feature aligning strategy that extracts deep features for nodes and edges efficiently. Next, to address the dataset issue, we collect commercial illustrations with professionally designed label layouts from household appliance manuals, and annotate them with useful information to create a novel dataset named the Appliance Manual Illustration Labels (AMIL) dataset. In the thorough evaluation on AMIL, our LPGT solution achieves promising label placement performance compared with popular baselines. Our algorithm and dataset are available at https://github.com/JingweiQu/LPGT.

Abstract:
Mediated reality, where augmented reality (AR) and diminished reality (DR) meet, enables visual modifications to real-world objects. A physical object with a mediated reality visual change retains its original physical properties. However, it is perceived differently from the original when interacted with. We present such a mediated reality object, a stick with different lengths or a stick with a missing portion in the middle, to investigate how users perceive its weight and center of gravity. We conducted two user studies (N=10N=10), each of which consisted of two substudies. We found that the length of mediated reality sticks influences the perceived weight. A longer stick is perceived as lighter, and vice versa. The stick with a missing portion tends to be recognized as one continuous stick. Thus, its weight and center of gravity (COG) remain the same. We formulated the relationship between inertia based on the reported COG and perceived weight in the context of dynamic touch.

Abstract:
The locomotion and interaction of multi-user groups are critical components of social virtual reality (VR), where users collaboratively navigate shared spaces defined by their relationships. As metaverse and social VR platforms evolve, safeguarding group spatial integrity becomes paramount. While teleportation-widely adopted for efficient navigation-enhances communication, it risks unintended intrusion into group spaces, compromising privacy and security. This paper presents two novel negotiated user-to-group teleportation techniques, paired with dynamic zone computation methods, to address these challenges. Our approach enables guest users to join groups through spatially aware teleportation, mediated by real-time negotiation. The negotiation interaction process is designed to facilitate users to negotiate teleportation locations efficiently and smoothly. To validate our techniques, we conducted a user study with 36 participants in a VR art museum environment, where they performed collaborative social-tour tasks. The findings demonstrate that our techniques significantly enhance group privacy protection, effectively support user-to-group negotiation of teleportation joining requirements, and alleviate anxiety associated with unwanted proximity within social VR groups.

Abstract:
Holography has immense potential for near-eye displays in virtual and augmented reality (VR/AR), providing natural 3D depth cues through wavefront reconstruction. However, balancing the field of view (FOV) with the eyebox remains challenging, constrained by the étendue limitation. Additionally, holographic image quality is often compromised due to differences between actual wave propagation and simulation models. This study addresses these by expanding the eyebox via multi-angle illumination, and enhancing image quality with end-to-end pupil-aware hologram optimization. Further, energy efficiency is improved by incorporating higher-order diffractions and pupil constraints. We explore a Pupil-HOGD algorithm for multi-angle illumination and validate it with a dual-angle holographic display prototype. Integrated with camera calibration and tracked eye position, the developed Pupil-HOGD algorithm improves image quality and expands the eyebox by 50% horizontally. We envision this approach extends the space-bandwidth product (SBP) of holographic displays, enabling broader applications in immersive, high-quality visual computing.

Abstract:
Extraction of a high-fidelity 3D medial axis is a crucial operation in CAD. When dealing with a polygonal model as input, ensuring accuracy and tidiness becomes challenging due to discretization errors inherent in the mesh surface. Commonly, existing approaches yield medial-axis surfaces with various artifacts, including zigzag boundaries, bumpy surfaces, unwanted spikes, and non-smooth stitching curves. Considering that the surface of a CAD model can be easily decomposed into a collection of surface patches, its 3D medial axis can be extracted by computing the Voronoi diagram of these surface patches, where each surface patch serves as a generator. However, no solver currently exists for accurately computing such an extended Voronoi diagram. Under the assumption that each generator defines a linear distance field over a sufficiently small range, our approach operates by tetrahedralizing the region of interest and computing the medial axis within each tetrahedral element. Just as SurfaceVoronoi computes surface-based Voronoi diagrams by cutting a 3D prism with 3D planes (each plane encodes a linear field in a triangle), the key operation in this paper is to conduct the hyperplane cutting process in 4D, where each hyperplane encodes a linear field in a tetrahedron. In comparison with the state-of-the-art, our algorithm produces better outcomes. Furthermore, it can also be used to compute the offset surface.

Abstract:
Newcomers to immersive analytics systems would benefit from informally learning data analysis skills (e.g., reading a vis, using a system) when collaborating with more expert participants. We want to design tools that facilitate this informal learning by encouraging the informal sharing of data literacy skills during collaborative immersive analysis. A first step toward this goal is to understand how informal skill sharing takes place in order to identify how it could be improved. We experimentally studied informal skill-sharing in pairs of participants analyzing scatterplots in a shared virtual reality environment. We used an original mixed-method to analyze video and log recordings as well as subjective experiential data, based on common ground theory and the grounding process. We uncovered 101 episodes of skill-sharing, organized in 14 recurring types, and identified associated problems from which we could propose six implications for designing systems that favor informal skill-sharing, hence skill learning. The method can be used to study informal skill sharing in other systems enabling embodied face-to-face collaboration, but would need to be simplified for large-scale use.

Abstract:
3D Gaussian Splatting (3DGS) models radiance fields as sparsely distributed 3D Gaussians, providing a compelling solution to novel view synthesis at high resolutions and real-time frame rates. However, deforming objects represented by 3D Gaussians remains a challenging task. Existing methods deform a 3DGS object by editing Gaussians geometrically. These approaches ignore the fact that it is the radiance field that rasterizes and renders the final image. The inconsistency between the deformed 3D Gaussians and the desired radiance field inevitably leads to artifacts in the final results. In this paper, we propose an interactive method for as-rigid-as-possible (ARAP) deformation of the Gaussian radiance fields. Specifically, after performing geometric edits on the Gaussians, we further optimize Gaussians to ensure its rasterization yields a similar result as the deformed radiance field. To facilitate this objective, we design radial features to mathematically describe the radial difference before and after the deformation, which are densely sampled across the radiance field. Additionally, we propose an adaptive anisotropic spatial low-pass filter to prevent aliasing issues during sampling and to preserve the field with the varying non-uniform sampling intervals. Users can interactively employ this tool to achieve large-scale ARAP deformations of the radiance field. Since our method maintains the consistency of the Gaussian radiance field before and after deformation, it avoids artifacts that are common in existing 3DGS deformation frameworks. Meanwhile, our method keeps the high quality and efficiency of 3DGS in rendering.

Abstract:
The emergence of distinct machine learning explanation methods has leveraged a number of new issues to be investigated. The disagreement problem is one such issue, as there may be scenarios where the output of different explanation methods disagree with each other. Although understanding how often, when, and where explanation methods agree or disagree is important to increase confidence in the explanations, few works have been dedicated to investigating such a problem. In this work, we proposed Visagreement, a visualization tool designed to assist practitioners in investigating the disagreement problem. Visagreement builds upon metrics to quantitatively compare and evaluate explanations, enabling visual resources to uncover where and why methods mostly agree or disagree. The tool is tailored for tabular data with binary classification and focuses on local feature importance methods. In the provided use cases, Visagreement turned out to be effective in revealing, among other phenomena, how disagreements relate to the quality of the explanations and machine learning model accuracy, thus assisting users in deciding where and when to trust explanations. To assess the effectiveness and practical utility of Visagreement, we conducted an evaluation involving four experts. These experts assessed the tool's Effectiveness, Usability, and Impact on Decision-Making. The experts confirm the Visagreement tool's effectiveness and user-friendliness, making it a valuable asset for analyzing and exploring (dis)agreements.

Abstract:
Sensitivity analyses of simulation ensembles determine how simulation parameters influence the simulation's outcome. Commonly, one global numerical sensitivity value is computed per simulation parameter. However, when considering 3D spatial simulations, the analysis of localized sensitivities in different spatial regions is of importance in many applications. For analyzing the spatial variation of parameter sensitivity, one needs to compute a spatial sensitivity scalar field per simulation parameter. Given nn simulation parameters, we obtain multi-field data consisting of nn scalar fields when considering all simulation parameters. We propose an interactive visual analytics solution to analyze the multi-field sensitivity data. It supports the investigation of how strongly and in what way individual parameters influence the simulation outcome, in which spatial regions this is happening, and what the interplay of the simulation parameters is. Its central component is an overview visualization of all sensitivity fields that avoids 3D occlusions by linearizing the data using an adapted scheme of data-driven space-filling curves. The spatial sensitivity values are visualized in a combination of a Horizon Graph and a line chart. We validate our approach by applying it to synthetic and real-world ensemble data.

Abstract:
Data visualization designers and clients need to communicate effectively with each other to achieve a successful project. Unlike a personal or solo project, working with a client introduces a layer of complexity to the process. Client and designer might have different ideas about what is an acceptable solution that would satisfy the goals and constraints of the project. Thus, the client-designer relationship is an important part of the design process. To better understand the relationship, we conducted an interview study with 12 data visualization designers. We develop a model of a client-designer project space consisting of three aspects: surfacing project goals, agreeing on resource allocation, and creating a successful design. For each aspect, designer and client have their own mental model of how they envision the project. Disagreements between these models can be resolved by negotiation that brings them closer to alignment. We identified three main negotiation strategies to navigate the project space: 1) expanding the project space to consider more potential options, 2) constraining the project space to narrow in on the boundaries, and 3) shifting the project space to different options. We discuss client-designer collaboration as a negotiated relationship, with opportunities and challenges for each side. We suggest ways to mitigate challenges to avoid friction from developing into conflict.

Abstract:
The task of 3D shape captioning occupies a significant place within the domain of computer graphics and has garnered considerable interest in recent years. Traditional approaches to this challenge frequently depend on the utilization of costly voxel representations or object detection techniques, yet often fail to deliver satisfactory outcomes. To address the above challenges, in this paper, we introduce Diff-3DCap, which employs a sequence of projected views to represent a 3D object and a continuous diffusion model to facilitate the captioning process. More precisely, our approach utilizes the continuous diffusion model to perturb the embedded captions during the forward phase by introducing Gaussian noise and then predicts the reconstructed annotation during the reverse phase. Embedded within the diffusion framework is a commitment to leveraging a visual embedding obtained from a pre-trained visual-language model, which naturally allows the embedding to serve as a guiding signal, eliminating the need for an additional classifier. Extensive results of our experiments indicate that Diff-3DCap can achieve performance comparable to that of the current state-of-the-art methods.

Abstract:
This paper addresses a challenging and novel problem in 2D shape cloud visualization: arranging irregular 2D shapes on an irregular canvas to minimize gaps and overlaps while emphasizing critical shapes by displaying them in larger sizes. The concept of a shape cloud is inspired by word clouds, which are widely used in visualization research to aesthetically summarize textual datasets by highlighting significant words with larger font sizes. We extend this concept to images, introducing shape clouds as a powerful and expressive visualization tool, guided by the principle that “a picture is worth a thousand words. Despite the potential of this approach, solutions in this domain remain largely unexplored.” To bridge this gap, we develop a 2D shape cloud collage framework that compactly arranges 2D shapes, emphasizing important objects with larger sizes, analogous to the principles of word clouds. This task presents unique challenges, as existing 2D shape layout methods are not designed for scalable irregular packing. Applying these methods often results in suboptimal layouts, such as excessive empty spaces or inaccurate representations of the underlying data. To overcome these limitations, we propose a novel layout framework that leverages recent advances in differentiable optimization. Specifically, we formulate the irregular packing problem as an optimization task, modeling the object arrangement process as a differentiable pipeline. This approach enables fast and accurate end-to-end optimization, producing high-quality layouts. Experimental results show that our system efficiently creates visually appealing and high-quality shape clouds on arbitrary canvas shapes, outperforming existing methods.

Abstract:
We propose a fast intersection-free remeshing of triangular meshes that robustly and efficiently generates high-quality non-intersecting meshes. Conducting intersection checks on all local operations during remeshing to prevent intersections represents the principal efficiency bottleneck. Our method is based on a key observation: intersections primarily occur in structurally complex regions. Accordingly, we develop an adaptive method to identify these key regions and perform intersection checks only for local operations within these regions during remeshing, significantly improving the algorithmic efficiency. Our method is an order of magnitude faster than traditional approaches that perform intersection checks on all local operations. Furthermore, we introduce a flip-aware extension mechanism that effectively avoids triangle flipping by constraining the optimization space of local operations, thereby avoiding the formation of irregular sharp edges. We also employ an adaptive iterative size field to eliminate banding phenomenon and propose a quasi-geometric size field adjustment method to quickly achieve smooth size transitions, thereby improving mesh quality. Compared to state-of-the-art methods, our method consistently and quickly generates higher quality non-intersecting meshes. In addition, we have validated the robustness and efficiency of our method, using all 5,469 non-intersecting valid models from the Thingi10K dataset.

Affiliations: School of Computation, Information and Technology, Technical University of Munich, Heilbronn, Germany; Institute of Computer Science, University of Würzburg, Würzburg, Germany; Faculty of Information Technology, Monash University, Clayton, VIC, Australia; School of Computer Science, University of Sydney, Camperdown, NSW, Australia; Department of Engineering, University of Perugia, Perugia, Italy; Department of Computer Science, University of Tsukuba, Ibaraki, Japan; Department of Psychology, University of Bamberg, Bamberg, Germany; Department of Computer Science, University of Konstanz, Konstanz, Germany

Abstract:
Graph and network visualization supports exploration, analysis and communication of relational data arising in many domains: from biological and social networks, to transportation and powergrid systems. With the arrival of AI-based question-answering tools, issues of trustworthiness and explainability of generated answers motivate a significant new role for visualization. In the context of graphs, we see the need for visualizations that can convince a critical audience that an assertion (e. g., from an AI) about the graph under analysis is valid. The requirements for such representations that convey precisely one specific graph property are quite different from standard network visualization criteria which optimize general aesthetics and readability. In this paper, we aim to provide a comprehensive introduction to visual proofs of graph properties and a foundation for further research in the area. We present a framework that defines what it means to visually prove a graph property. In the process, we introduce the notion of a visual certificate, that is, a specialized faithful graph visualization that leverages the viewer’s perception, in particular, pre-attentive processing (e. g., via pop-out effects), to verify a given assertion about the represented graph. We also discuss the relationships between visual complexity, cognitive load and complexity theory, and propose a classification based on visual proof complexity. Then, we provide further examples of visual certificates for problems in different visual proof complexity classes. Finally, we conclude the paper with a discussion of the limitations of our model and some open problems.

Abstract:
Augmented Reality (AR) has grown from specialised uses to applications for the common public. One of these developments led to Augmented Vision (AV), which enhances vision beyond traditional methods like glasses or contact lenses. This review aims to compare and categorise AV systems according to the paradigms they implement to enhance the users’ vision. Additionally, the review examines whether researchers conduct measurements and analysis on the human visual system (HVS) when evaluating their system. Such an overall view will help future researchers position their work on AV. By understanding AV systems’ paradigms and approaches, researchers will be well-equipped to identify gaps, explore novel directions, and leverage existing advancements. We searched Scopus, Web of Science, and PubMed databases for publications until February 26, 2025, exploring citations and references for the selected articles to avoid missing out on relevant articles. We then conducted a two-step screening process that involved LLM-assisted screening of the article’s abstracts and an in-depth assessment of the article. This review follows the PRISMA statement, reducing bias risk. We selected 113 of 469 articles, as they improved users’ visual performance. We defined three main categories: (1) adding light to the incoming light field, (2) modifying the incoming light field, and (3) intersecting approaches. We found three main application areas: (1) task-specific, (2) vision correction, and (3) visual perception enhancement. The most typical application is task-specific. We identified a gap in the literature since just four of the papers we reviewed measured and analysed the accommodation while utilising the device.

Abstract:
It is demanding to author an existing hairstyle for novel characters in games and VR applications. However, it is a non-trivial task for artists due to the complicated hair geometries and spatial interactions to preserve. In this paper, we present an automatic shape adaptation method to retarget 3D hairstyles. We formulate the adaptation process as a constrained optimization problem, where all the shape properties and spatial relationships are converted into individual objectives and constraints. To make such an optimization on high-resolution hairstyles tractable, we adopt a multi-scale strategy to compute the target positions of the hair strands in a coarse-to-fine manner. The global solving for the inter-strands coupling is restricted to the coarse level, and the solving for fine details is made local and parallel. In addition, we present a novel hairline edit tool to allow for user customization during retargeting. We achieve it by solving physics-based deformations of an embedded membrane to redistribute the hair roots with minimal distortion. We demonstrate the efficacy of our method through quantitative and qualitative experiments on various hairstyles and characters.

Abstract:
Querying time series based on their relations is a crucial part of multiple time series analysis. By retrieving and understanding time series relations, analysts can easily detect anomalies and validate hypotheses in complex time series datasets. However, current relation extraction approaches, including knowledge- and data-driven ones, tend to be laborious and do not support heterogeneous relations. By conducting a formative study with 11 experts, we concluded six time series relations, including correlation, causality, similarity, lag, arithmetic, and meta, and summarized three pain points in querying time series involving these relations. We proposed RelaQ, an interactive system that supports the time series query via relation specifications. RelaQ allows users to intuitively specify heterogeneous relations when querying multiple time series, understand the query results based on a scalable, multi-level visualization, and explore possible relations beyond the existing queries. RelaQ is evaluated with two cases and a user study with 12 participants, showing promising effectiveness and usability.

Abstract:
Data charts are prevalent across various fields due to their efficacy in conveying complex data relationships. However, static charts may sometimes struggle to engage readers and efficiently present intricate information, potentially resulting in limited understanding. We introduce “Live Charts,” a new format of presentation that decomposes complex information within a chart and explains the information pieces sequentially through rich animations and accompanying audio narration. We propose an automated approach to revive static charts into Live Charts. Our method integrates GNN-based techniques to analyze the chart components and extract data from charts. Then we adopt large natural language models to generate appropriate animated visuals along with a voice-over to produce Live Charts from static ones. We conducted a thorough evaluation of our approach, which involved the model performance, use cases, a crowd-sourced user study, and expert interviews. The results demonstrate Live Charts offer a multi-sensory experience where readers can follow the information and understand the data insights better. We analyze the benefits and drawbacks of Live Charts over static charts as a new information consumption experience.

Abstract:
Matrix reordering permutes the rows and columns of a matrix to reveal meaningful visual patterns, such as blocks that represent clusters. A comprehensive collection of matrices, along with a scoring method for measuring the quality of visual patterns in these matrices, contributes to building a benchmark. This benchmark is essential for selecting or designing suitable reordering algorithms for revealing specific patterns. In this paper, we build a matrix-reordering benchmark, ReorderBench, with the goal of evaluating and improving matrix-reordering techniques. This is achieved by generating a large set of representative and diverse matrices and scoring these matrices with a convolution- and entropy-based method. Our benchmark contains 2,835,000 binary matrices and 5,670,000 continuous matrices, each generated to exhibit one of four visual patterns: block, off-diagonal block, star, or band, along with 450 real-world matrices featuring hybrid visual patterns. We demonstrate the usefulness of ReorderBench through three main applications in matrix reordering: 1) evaluating different reordering algorithms, 2) creating a unified scoring model to measure the visual patterns in any matrix, and 3) developing a deep learning model for matrix reordering.

Abstract:
Projector-camera systems (ProCams) simulation aims to model the physical project-and-capture process and associated scene parameters of a ProCams, and is crucial for spatial augmented reality (SAR) applications such as ProCams relighting and projector compensation. Recent advances use an end-to-end neural network to learn the project-and-capture process. However, these neural network-based methods often implicitly encapsulate scene parameters, such as surface material, gamma, and white balance in the network parameters, and are less interpretable and hard for novel scene simulation. Moreover, neural networks usually learn the indirect illumination implicitly in an image-to-image translation way which leads to poor performance in simulating complex projection effects such as soft-shadow and interreflection. In this paper, we introduce a novel path tracing-based differentiable projector-camera systems (DPCS), offering a differentiable ProCams simulation method that explicitly integrates multi-bounce path tracing. Our DPCS models the physical project-and-capture process using differentiable physically-based rendering (PBR), enabling the scene parameters to be explicitly decoupled and learned using much fewer samples. Moreover, our physically-based method not only enables high-quality downstream ProCams tasks, such as ProCams relighting and projector compensation, but also allows novel scene simulation using the learned scene parameters. In experiments, DPCS demonstrates clear advantages over previous approaches in ProCams simulation, offering better interpretability, more efficient handling of complex interreflection and shadow, and requiring fewer training samples. The code and dataset are available on the project page: https://jijiangli.github.io/DPCS/.

Abstract:
Topology optimization can maximally leverage the high DOFs and mechanical potentiality of porous foams but faces challenges in adapting to free-form outer shapes, maintaining full connectivity between adjacent foam cells, and achieving high simulation accuracy. Utilizing the concept of Voronoi tessellation may help overcome the challenges owing to its distinguished properties on highly flexible topology, natural edge connectivity, and easy shape conforming. However, a variational optimization of the so-called Voronoi foams has not yet been fully explored. In addressing the issue, a concept of explicit topology optimization of open-cell Voronoi foams is proposed that can efficiently and reliably guide the foam's topology and geometry variations under critical physical and geometric requirements. Taking the site (or seed) positions and beam radii as the DOFs, we explore the differentiability of the open-cell Voronoi foams w.r.t. its seed locations, and propose a highly efficient local finite difference method to estimate the derivatives. During the gradient-based optimization, the foam topology can change freely, and some seeds may even be pushed out of shape, which greatly alleviates the challenges of prescribing a fixed underlying grid. The foam's mechanical property is also computed with a much-improved efficiency by an order of magnitude, in comparison with benchmark FEM, via a new material-aware numerical coarsening method on its highly heterogeneous density field counterpart. We show the improved performance of our Voronoi foam in comparison with classical topology optimization approaches and demonstrate its advantages in various settings.

Abstract:
As eusocial creatures, bees display unique macro collective behavior and local body dynamics that hold potential applications in various fields, such as computer animation, robotics, and social behavior. Unlike birds and fish, bees fly in a low-aligned zigzag pattern. Additionally, bees rely on visual signals for foraging and predator avoidance, exhibiting distinctive local body oscillations, such as body lifting, thrusting, and swaying. These inherent features pose significant challenges to realistic bee simulations in practical animation applications. In this article, we present a bio-inspired model for bee simulations capable of replicating both macro collective behavior and local body dynamics of bees. Our approach utilizes a visually-driven system to simulate a bee's local body dynamics, incorporating obstacle perception and body rolling control for effective collision avoidance. Moreover, we develop an oscillation rule that captures the dynamics of the bee's local bodies, drawing on insights from biological research. Our model extends beyond simulating individual bees’ dynamics; it can also represent bee swarms by integrating a fluid-based field with the bees’ innate noise and zigzag motions. To fine-tune our model, we utilize pre-collected honeybee flight data. Through extensive simulations and comparative experiments, we demonstrate that our model can efficiently generate realistic low-aligned and inherently noisy bee swarms.

Abstract:
Seasonal-trend decomposition based on loess (STL) is a powerful tool to explore time series data visually. In this article, we present an extension of STL to uncertain data, named uncertainty-aware STL (UASTL). Our method propagates multivariate Gaussian distributions mathematically exactly through the entire analysis and visualization pipeline. Thereby, stochastic quantities shared between the components of the decomposition are preserved. Moreover, we present application scenarios with uncertainty modeling based on Gaussian processes, e.g., data with uncertain areas or missing values. Besides these mathematical results and modeling aspects, we introduce visualization techniques that address the challenges of uncertainty visualization and the problem of visualizing highly correlated components of a decomposition. The global uncertainty propagation enables the time series visualization with STL-consistent samples, the exploration of correlation between and within decomposition's components, and the analysis of the impact of varying uncertainty. Finally, we show the usefulness of UASTL and the importance of uncertainty visualization with several examples. Thereby, a comparison with conventional STL is performed.

Abstract:
The 2024 VGTC Visualization Lifetime Achievement Award goes to Hans-Christian Hege for his fundamental technical contributions to visualization and visualization software with a focus on applications in the natural sciences, medicine and engineering. The 2024 VGTC Visualization Lifetime Achievement Award goes to Min Chen for his contributions to the theoretical foundations of visualization as well as outstanding work on novel and impactful applications. Min Chen started his journey of research in 1984 at Swansea University where he developed his academic career. Since 2011, he has been professor of scientific visualization in the Department of Engineering Science and a fellow of Pembroke College, University of Oxford.

Abstract:
The 2024 VGTC Visualization Technical Achievement Award goes to Han-Wei Shen for his research on extreme-scale and multivariate time-varying data visualization, uncertainty visualization, and novel approaches to inclusion of AI in scientific workflows. The 2024 VGTC Visualization Technical Achievement Award goes to Bongshin Lee for her groundbreaking contributions in advancing visualization technologies through intuitive interaction designs, data-driven storytelling, and integration of machine learning.

Abstract:
The optimization of cooling systems is important in many cases, for example for cabin and battery cooling in electric cars. Such an optimization is governed by multiple, conflicting objectives and it is performed across a multi-dimensional parameter space. The extent of the parameter space, the complexity of the non-linear model of the system, as well as the time needed per simulation run and factors that are not modeled in the simulation necessitate an iterative, semi-automatic approach. We present an interactive visual optimization approach, where the user works with a p-h diagram to steer an iterative, guided optimization process. A deep learning (DL) model provides estimates for parameters, given a target characterization of the system, while numerical simulation is used to compute system characteristics for an ensemble of parameter sets. Since the DL model only serves as an approximation of the inverse of the cooling system and since target characteristics can be chosen according to different, competing objectives, an iterative optimization process is realized, developing multiple sets of intermediate solutions, which are visually related to each other. The standard p-h diagram, integrated interactively in this approach, is complemented by a dual, also interactive visual representation of additional expressive measures representing the system characteristics. We show how the known four-points semantic of the p-h diagram meaningfully transfers to the dual data representation. When evaluating this approach in the automotive domain, we found that our solution helped with the overall comprehension of the cooling system and that it lead to a faster convergence during optimization.

Abstract:
Assigning discriminable and harmonic colors to samples according to their class labels and spatial distribution can generate attractive visualizations and facilitate data exploration. However, as the number of classes increases, it is challenging to generate a high-quality color assignment result that accommodates all classes simultaneously. A practical solution is to organize classes into a hierarchy and then dynamically assign colors during exploration. However, existing color assignment methods fall short in generating high-quality color assignment results and dynamically aligning them with hierarchical structures. To address this issue, we develop a dynamic color assignment method for hierarchical data, which is formulated as a multi-objective optimization problem. This method simultaneously considers color discriminability, color harmony, and spatial distribution at each hierarchical level. By using the colors of parent classes to guide the color assignment of their child classes, our method further promotes both consistency and clarity across hierarchical levels. We demonstrate the effectiveness of our method in generating dynamic color assignment results with quantitative experiments and a user study.

Abstract:
We introduce a visual analysis method for multiple causal graphs with different outcome variables, namely, multi-outcome causal graphs. Multi-outcome causal graphs are important in healthcare for understanding multimorbidity and comorbidity. To support the visual analysis, we collaborated with medical experts to devise two comparative visualization techniques at different stages of the analysis process. First, a progressive visualization method is proposed for comparing multiple state-of-the-art causal discovery algorithms. The method can handle mixed-type datasets comprising both continuous and categorical variables and assist in the creation of a fine-tuned causal graph of a single o utcome. Second, a comparative graph layout technique and specialized visual encodings are devised for the quick comparison of multiple causal graphs. In our visual analysis approach, analysts start by building individual causal graphs for each outcome variable, and then, multi-outcome causal graphs are generated and visualized with our comparative technique for analyzing differences and commonalities of these causal graphs. Evaluation includes quantitative measurements on benchmark datasets, a case study with a medical expert, and expert user studies with real-world health research data.

Abstract:
We developed and validated an instrument to measure the perceived readability in data visualization: PREVis. Researchers and practitioners can easily use this instrument as part of their evaluations to compare the perceived readability of different visual data representations. Our instrument can complement results from controlled experiments on user task performance or provide additional data during in-depth qualitative work such as design iterations when developing a new technique. Although readability is recognized as an essential quality of data visualizations, so far there has not been a unified definition of the construct in the context of visual representations. As a result, researchers often lack guidance for determining how to ask people to rate their perceived readability of a visualization. To address this issue, we engaged in a rigorous process to develop the first validated instrument targeted at the subjective readability of visual data representations. Our final instrument consists of 11 items across 4 dimensions: understandability, layout clarity, readability of data values, and readability of data patterns. We provide the questionnaire as a document with implementation guidelines on osf.io/9cg8j. Beyond this instrument, we contribute a discussion of how researchers have previously assessed visualization readability, and an analysis of the factors underlying perceived readability in visual data representations.

Abstract:
Physically-based fluid animation in Virtual Reality (VR) significantly enhances the user experience through visually engaging flow motions. Nonetheless, such simulations are often limited by their substantial computational demands. A tailored adaptive simulation algorithm is important for high-performance VR fluid simulations, which dynamically allocate degrees of freedom (DoF) while accounting for user perception in VR. This paper proposes a novel scene-based gaze-contingent fluid simulation system for VR, featuring a highly adaptive fluid simulator integrated with a VR perceptual model that accounts for the foveation and geometry of fluid. Our method leverages an eccentricity and curvature-dependent perceptual model to dynamically allocate computational resources, improving the efficiency and maintaining spatio-temporal stability of fluid animation in VR. A user study was conducted to measure the simulation resolution thresholds for fluid animations in VR, considering various levels of eccentricity and curvature. Our findings indicate notable differences in perceptual thresholds based on these metrics. By incorporating these insights into our adaptive fluid simulator as a unified sizing function, we maintain perceptually optimal particle resolution, achieving up to a 3.62× performance improvement while delivering superior perceptual realism and user experience, as validated by a subjective evaluation study.

Abstract:
Glyph-based visualization is one of the main techniques for visualizing complex multivariate data. With small glyphs, data variables are typically encoded with relatively low visual and perceptual precision. Glyph designers have to contemplate the trade-offs in allocating visual channels when there is a large number of data variables. While there are many successful glyph designs in the literature, there is not yet a systematic method for assisting visualization designers to evaluate different design options that feature different types of trade-offs. In this paper, we present an evaluation scheme based on the multi-criteria decision analysis (MCDA) methodology. The scheme provides designers with a structured way to consider their glyph designs from a range of perspectives, while rendering a semi-quantitative template for evaluating different design options. In addition, this work provides guideposts for future empirical research to obtain more quantitative measurements that can be used in MCDA-aided glyph design processes.

Abstract:
Whole-body motion generation from speech audio is crucial for computer graphics and immersive VR/AR. Prior methods struggle to produce natural and diverse whole-body motions from speech. In this paper, we introduce a novel method, named SpeechAct, based on a hybrid point representation and contrastive motion learning to boost realism and diversity in motion generation. Our hybrid point representation leverages the advantages of keypoint representation and surface points of 3D body model, which is easy to learn and helps to achieve smooth and natural motion generation from speech audio. We design a VQ-VAE to learn a motion codebook using our hybrid presentation, and then regress the motion from the input audio using a translation model. To boost diversity in motion generation, we propose a contrastive motion learning method according to the intuitive idea that the generated motion should be different from the motions of other audios and other speakers. We collect negative samples from other audio inputs and other speakers using our translation model. With these negative samples, we pull the current motion away from them using a contrastive loss to produce more distinctive representations. In addition, we compose a face generator to generate deterministic face motion due to the strong connection between the face movements and the speech audio. Experimental results validate the superior performance of our model. The code is available at http://cic.tju.edu.cn/faculty/likun/projects/SpeechAct/index.html.

Abstract:
Data-rich documents are commonly found across various fields such as business, finance, and science. However, a general limitation of these documents for reading is their reliance on text to convey data and facts. Visual representation of text aids in providing a satisfactory reading experience in comprehension and engagement. However, existing work emphasizes presenting the insights within phrases or sentences, rather than fully conveying data stories within the whole paragraphs and engaging readers. To provide readers with satisfactory data stories, this paper presents Narrative Player, a novel method that automatically revives data narratives with consistent and contextualized visuals. Specifically, it accepts a paragraph and corresponding data table as input and leverages LLMs to characterize the clauses and extract contextualized data facts. Subsequently, the facts are transformed into a coherent visualization sequence with a carefully designed optimization-based approach. Animations are also assigned between adjacent visualizations to enable seamless transitions. Finally, the visualization sequence, transition animations, and audio narration generated by text-to-speech technologies are rendered into a data video. The evaluation results showed that the automatic-generated data videos were well-received by participants and experts for enhancing reading.

Abstract:
Keypoint detection plays a fundamental role in many applications, such as 3D reconstruction, object registration, and shape retrieval, and has attracted significant interest from researchers in computer vision and graphics. However, due to the ambiguity of the keypoint and the complexity of 3D objects, it is still tricky for existing 3D keypoint detection methods to generate stable keypoints with good coverage, especially for unsupervised detection methods. This paper proposes a 3D keypoint detection method based on hierarchical point saliency. This method can effectively and accurately locate the keypoints of a 3D point cloud, and it does not require complex training processes. First, we propose a simple and effective point descriptor called the local geometric structure feature, which can effectively characterize the geometric structure changes of 3D point clouds and has a strong feature identification ability. Second, we define two saliency measures used to characterize the saliency of points in the point cloud, which are low-level and high-level saliency. Third, we hierarchically characterize the saliency of points by combining the low-level and high-level saliency, thus measuring the probability that a point belongs to a keypoint. Finally, we extensively test our method on three benchmark 3D point cloud datasets, and the experimental results demonstrate that our method achieves state-of-the-art performance in keypoint detection tasks, significantly superior to the prior hand-crafted and deep-learning-based 3D keypoint detection methods.

Abstract:
Recent advancements in human fashion video generation have transformed the field, producing various promising effects. Existing methods mainly focus on pose control but lack the ability to achieve sketch-based control, largely due to the absence of appearance-consistent and shape-varying knowledge in existing datasets. Moreover, the necessity of sequential structure inputs to control video generation hinders real-world applications. To address these limitations, we introduce Sketch2HumanVideo, an approach that, for the first time, achieves sketch-controllable human video generation with three conditions: temporally sparse sketches, a spatially sparse pose sequence, and a reference appearance image. Our key contribution is a sparse sketch encoder, which takes the first two conditions as input, enabling precise and multi-view control of shape motion. To provide the above knowledge, we leverage the expertise of two pretrained models to synthesize a dataset comprising shape-varying yet appearance-consistent examples for model training. Furthermore, we introduce an enlarging-and-resampling scheme to enhance high-frequency details of local regions in resource-constrained scenarios, thereby promoting the generation of realistic videos. Through qualitative and quantitative experiments, our method showcases superior performance to state-of-the-art approaches and flexible control.

Abstract:
Flow fields are often partitioned into data blocks for massively parallel computation and analysis based on blockwise relationships. However, most of the previous techniques only consider the first-order dependencies among blocks, which is insufficient in describing complex flow patterns. In this work, we present FlowHON, an approach to construct higher-order networks (HONs) from flow fields. FlowHON captures the inherent higher-order dependencies in flow fields as nodes and estimates the transitions among them as edges. We formulate the HON construction as an optimization problem with three linear transformations. The first two layers correspond to the node generation and the third one corresponds to edge estimation. Our formulation allows the node generation and edge estimation to be solved in a unified framework. With FlowHON, the rich set of traditional graph algorithms can be applied without any modification to analyze flow fields, while leveraging the higher-order information to understand the inherent structure and manage flow data for efficiency. We demonstrate the effectiveness of FlowHON using a series of downstream tasks, including estimating the density of particles during tracing, partitioning flow fields for data management, and understanding flow fields using the node-link diagram representation of networks.

Abstract:
Deep learning advancements have enabled the generation of visually plausible hair geometry from a single image, but the results still do not meet the realism required for further applications (e.g., high quality hair rendering and simulation). One of the essential element that is missing in previous single-view hair reconstruction methods is the clumping effect of hair, which is influenced by scalp secretions and oils, and is a key ingredient for high-quality hair rendering and simulation. Inspired by common practices in industrial production which simulates realistic hair clumping by allowing artists to adjust clumping parameters, we aim to integrate these clumping effects into single-view hair reconstruction. We introduce a hierarchical hair representation that incorporates a clumping modifier into the guide hair and skinning-based hair expressions. This representation utilizes guide strands and skinning weights to express the basic geometric structure of the hair. The clumping modifier allows for the expression of more detailed and realistic clumping effects. Based on this representation, We design a fully differentiable framework integrating a neural measurement of clumping and a line-based rasterization renderer to iteratively solve guide strands positions and clumping parameters. Our method demonstrates superior performance both qualitatively and quantitatively compared to state-of-the-art techniques.

Abstract:
Our work aims to reconstruct the appearance and geometry of the two-hand-held object from a sequence of color images. In contrast to traditional single-hand-held manipulation, two-hand-holding allows more flexible interaction, thereby providing back views of the object, which is particularly convenient for reconstruction but generates complex view-dependent occlusions. The recent development of neural rendering provides new potential for hand-held object reconstruction. In this paper, we propose a novel neural representation-based framework to recover radiance fields of the two-hand-held object, named \rmH_2\rmOH2O-NeRF. We first design an object-centric semantic module based on the geometric signed distance function cues to predict 3D object-centric regions and develop the view-dependent visible module based on the image-related cues to label 2D occluded regions. We then combine them to obtain a 2D visible mask that adaptively guides ray sampling on the object for optimization. We also provide a newly collected \rmH_2\rmOH2O dataset to validate the proposed method. Experiments show that our method achieves superior performance on reconstruction completeness and view-consistency synthesis compared to the state-of-the-art methods.

Abstract:
The image-based Person-to-Person (P2P) virtual try-on, involving the direct transfer of garments from one person to another, is one of the most promising applications of human-centric image generation. However, existing approaches struggle to accurately learn the clothing deformation when directly warping the garment from the source pose onto the target pose. To address this, we propose Person-to-Person virtual try-on via Garment UNwrapping, a novel framework dubbed as ViTon-GUN. Specifically, we divide the P2P task into two subtasks: Person-to-Garment (P2G) and Garment-to-Person (G2P). The P2G aims to unwrap the target garment from a source pose to a canonical representation based on A-Pose. In the P2G stage, we enable the implementation of a flow-based P2G scheme by introducing an A-Pose estimator and establishing comprehensive training conditions. Building upon this step-wise strategy, we introduce a novel pipeline for P2P try-on. Once trained, the P2G strategy can serve as a “plug-and-play” module, which efficiently adapts existing diffusion-based pre-trained G2P models to P2P try-on without further training. Quantitative and qualitative experiments demonstrate that our ViTon-GUN performs remarkably well on P2P try-on, even for dresses with intricate design details.

Abstract:
We study a problem of modeling wireframe meshes where the vertices and edges fall into a set of discrete equivalence classes, respectively. This problem is motivated by the need of fabricating large wireframe structures at lower cost and faster speed since both nodes (thickened vertices) and rods (thickened edges) can be mass-produced. Given a 3D shape represented as a wireframe mesh, our goal is to compute a set of template vertices and a set of template edges, whose instances can be used to produce a fabricable wireframe mesh that approximates the input shape. To achieve this goal, we propose a computational approach that generates the template vertices and template edges by iteratively clustering and optimizing the mesh vertices and edges. At the clustering stage, we cluster mesh vertices and edges according to their shape and length, respectively. At the optimization stage, we first locally optimize the mesh to reduce the number of clusters of vertices and/or edges, and then globally optimize the mesh to reduce the intra-cluster variance for vertices and edges, while facilitating fabricability of the wireframe mesh. We demonstrate that our approach is able to model wireframe meshes with various shapes and topologies, compare it with three state-of-the-art approaches to show its superiority, and validate fabricability of our results by making three physical prototypes.

Abstract:
We present a novel method for the interactive construction and rendering of extremely large molecular scenes, capable of representing multiple biological cells in atomistic detail. Our method is designed for scenes that are procedurally constructed based on a given set of building rules. Rendering large scenes typically requires the entire scene to be available in-core, or alternatively, it requires out-of-core management to load data into the memory hierarchy as a part of the rendering loop. Instead of out-of-core memory management, we propose procedurally generating the scene on-demand on the fly. The key concept is a positional- and view-dependent procedural scene-construction strategy, where only a fraction of the atomistic scene around the camera is available in the GPU memory at any given time. The atomistic detail is populated into a uniform-space partitioning using a grid covering the entire scene. Most grid cells are not filled with geometry, only those that are potentially seen by the camera are populated. The atomistic detail is populated in a compute shader and its representation is connected with acceleration data structures for hardware ray-tracing of modern GPUs. Distant objects, where atomistic detail is not perceivable from a given viewpoint, are represented by a triangle mesh mapped with a seamless texture generated from the rendering of geometry with atomistic detail. The algorithm consists of two pipelines, the construction-compute pipeline and rendering pipeline, which work together to render molecular scenes at an atomistic resolution beyond the limit of the GPU memory containing trillions of atoms. The proposed technique is demonstrated on multiple models of SARS-CoV-2 and the red blood cell.

Abstract:
Creating an animated data video with audio narration is a time-consuming and complex task that requires expertise. It involves designing complex animations, turning written scripts into audio narrations, and synchronizing visual changes with the narrations. This paper presents WonderFlow, an interactive authoring tool, that facilitates narration-centric design of animated data videos. WonderFlow allows authors to easily specify semantic links between text and the corresponding chart elements. Then it automatically generates audio narration by leveraging text-to-speech techniques and aligns the narration with an animation. WonderFlow provides a structure-aware animation library designed to ease chart animation creation, enabling authors to apply pre-designed animation effects to common visualization components. Additionally, authors can preview and refine their data videos within the same system, without having to switch between different creation tools. A series of evaluation results confirmed that WonderFlow is easy to use and simplifies the creation of data videos with narration-animation interplay.

Abstract:
Since point clouds acquired by scanners inevitably contain noise, recovering a clean version from a noisy point cloud is essential for further 3D geometry processing applications. Several data-driven approaches have been recently introduced to overcome the drawbacks of traditional filtering algorithms, such as less robust preservation of sharp features and tedious tuning for multiple parameters. Most of these methods achieve filtering by directly regressing the position/displacement of each point, which may blur detailed features and is prone to uneven distribution. In this article, we propose a novel data-driven method that explores the implicit fields. Our assumption is that the given noisy points implicitly define a surface, and we attempt to obtain a point's movement direction and distance separately based on the predicted signed distance fields (SDFs). Taking a noisy point cloud as input, we first obtain a consistent alignment by incorporating the global points into local patches. We then feed them into an encoder-decoder structure and predict a 7D vector consisting of SDFs. Subsequently, the distance can be obtained directly from the first element in the vector, and the movement direction can be obtained by computing the gradient descent from the last six elements (i.e., six surrounding SDFs). We finally obtain the filtered results by moving each point with its predicted distance along its movement direction. Our method can produce feature-preserving results without requiring explicit normals. Experiments demonstrate that our method visually outperforms state-of-the-art methods and generally produces better quantitative results than position-based methods (both learning and non-learning).

Abstract:
Dimensionality reduction (DR) techniques help analysts to understand patterns in high-dimensional spaces. These techniques, often represented by scatter plots, are employed in diverse science domains and facilitate similarity analysis among clusters and data samples. For datasets containing many granularities or when analysis follows the information visualization mantra, hierarchical DR techniques are the most suitable approach since they present major structures beforehand and details on demand. This work presents HUMAP, a novel hierarchical dimensionality reduction technique designed to be flexible on preserving local and global structures and preserve the mental map throughout hierarchical exploration. We provide empirical evidence of our technique’s superiority compared with current hierarchical approaches and show a case study applying HUMAP for dataset labelling.

Abstract:
Chart images, such as bar charts, pie charts, and line charts, are explosively produced due to the wide usage of data visualizations. Accordingly, knowledge mining from chart images is becoming increasingly important, which can benefit downstream tasks like chart retrieval and knowledge graph completion. However, existing methods for chart knowledge mining mainly focus on converting chart images into raw data and often ignore their visual encodings and semantic meanings, which can result in information loss for many downstream tasks. In this paper, we propose ChartKG, a novel knowledge graph (KG) based representation for chart images, which can model the visual elements in a chart image and semantic relations among them including visual encodings and visual insights in a unified manner. Further, we develop a general framework to convert chart images to the proposed KG-based representation. It integrates a series of image processing techniques to identify visual elements and relations, e.g., CNNs to classify charts, yolov5 and optical character recognition to parse charts, and rule-based methods to construct graphs. We present four cases to illustrate how our knowledge-graph-based representation can model the detailed visual elements and semantic relations in charts, and further demonstrate how our approach can benefit downstream applications such as semantic-aware chart retrieval and chart question answering. We also conduct quantitative evaluations to assess the two fundamental building blocks of our chart-to-KG framework, i.e., object recognition and optical character recognition. The results provide support for the usefulness and effectiveness of ChartKG.

Abstract:
Fuzzy clusters, where ambiguous samples belong to multiple clusters, are common in real-world applications. Analyzing such ambiguous samples in large-scale datasets is crucial for practical applications, such as diagnosing machine learning models. A promising method to support such analysis is through hierarchical cluster-aware grid visualizations, which offer high space efficiency and clear cluster perception. However, existing cluster-aware grid layout methods cannot clarify ambiguity among fuzzy clusters, which limits their effectiveness in fuzzy cluster analysis. To tackle this issue, we introduce a hierarchical fuzzy-cluster-aware grid layout method that supports hierarchical exploration of large-scale datasets. Throughout the hierarchical exploration, it is crucial to facilitate fuzzy cluster analysis while maintaining visual continuity for users. To achieve this, we propose a two-step optimization strategy for enhancing cluster perception, clarifying ambiguity, and preserving stability during the exploration. The first step is to create cluster-aware partitions, where each partition corresponds to a cluster. This step focuses on enhancing cluster perception and maintaining the previous shapes and positions of clusters to preserve stability at the cluster level. The second step is to generate a grid layout for each partition. In addition to placing similar samples together, this step also places ambiguous samples near the boundaries to clarify ambiguity and reveal the root causes of their occurrences and maintains the relative positions of the samples in the same cluster to preserve stability at the sample level. Several quantitative experiments and a use case are conducted to demonstrate the effectiveness and usefulness of our method in analyzing large-scale datasets, especially in fuzzy cluster analysis.

Abstract:
We propose a novel method to generate developable approximations for triangular meshes. Instead of fitting the Gauss image using a geodesic circle in the local neighborhood, we apply a nonlinear dimensionality reduction method, called Isomap, to use a general curve on the sphere for fitting. This brings us a larger space to represent the Gauss image in the local neighborhood as a 1D structure. Specifically, each triangle is assigned a target normal after local fitting; then, we deform the mesh to approach the target normal globally. By iteratively performing fitting and deformation, we obtain the developable approximation. We demonstrate the feasibility and effectiveness of our method over various examples. Compared to the state-of-the-art methods, our results exhibit a higher fidelity to the input mesh while possessing more prominent and visually distinct undevelopable seam curves.

Abstract:
The essence of a video lies in the dynamic motions. While text-to-video generative diffusion models have made significant strides in creating diverse content, effectively controlling specific motions through text prompts remains a challenge. By utilizing user-specified reference videos, the more precise guidance for character actions, object movements, and camera movements can be achieved. This gives rise to the task of motion customization, where the primary challenge lies in effectively decoupling the appearance and motion within a video clip. To address this challenge, we introduce MotionCrafter, a novel one-shot instance-guided motion customization method that is suitable for both pre-trained text-to-video and text-to-image diffusion models. MotionCrafter employs a parallel spatial-temporal architecture that integrates the reference motion into the temporal component of the base model, while independently adjusting the spatial module for character or style control. To enhance the disentanglement of motion and appearance, we propose an innovative dual-branch motion disentanglement approach, which includes a motion disentanglement loss and an appearance prior enhancement strategy. To facilitate more efficient learning of motions, we further propose a novel timestep-layered tuning strategy that directs the diffusion model to focus on motion-level information. Through comprehensive quantitative and qualitative experiments, along with user preference tests, we demonstrate that MotionCrafter can successfully integrate dynamic motions while maintaining the coherence and quality of the base model, providing a wide range of appearance generation capabilities. MotionCrafter can be applied to various personalized backbones in the community to generate videos with a variety of artistic styles.

Abstract:
Despite decision-making being a vital goal of data visualization, little work has been done to differentiate decision-making tasks within the field. While visualization task taxonomies and typologies exist, they often focus on more granular analytical tasks that are too low-level to describe large complex decisions, which can make it difficult to reason about and design decision-support tools. In this paper, we contribute a typology of decision-making tasks that were iteratively refined from a list of design goals distilled from a literature review. Our typology is concise and consists of only three tasks: CHOOSE, ACTIVATE, and CREATE. Although decision types originating in other disciplines exist, we provide definitions for these tasks that are suitable for the visualization community. Our proposed typology offers two benefits. First, the ability to compose and hierarchically organize the tasks enables flexible and clear descriptions of decisions with varying levels of complexities. Second, the typology encourages productive discourse between visualization designers and domain experts by abstracting the intricacies of data, thereby promoting clarity and rigorous analysis of decision-making processes. We demonstrate the benefits of our typology through four case studies, and present an evaluation of the typology from semi-structured interviews with experienced members of the visualization community who have contributed to developing or publishing decision support systems for domain experts. Our interviewees used our typology to delineate the decision-making processes supported by their systems, demonstrating its descriptive capacity and effectiveness. Finally, we present preliminary findings on the usefulness of our typology for visualization design.

Abstract:
Graph visualization techniques provide a foundational framework for offering comprehensive overviews and insights into cloud computing systems, facilitating efficient management and ensuring their availability and reliability. Despite the enhanced computational and storage capabilities of larger-scale cloud computing architectures, they introduce significant challenges to traditional graph-based visualization due to issues of hierarchical heterogeneity, scalability, and data incompleteness. This paper proposes a novel abstraction approach to visualize large multi-layer networks. Our method leverages graphons, a probabilistic representation of network layers, to encompass three core steps: an inner-layer summary to identify stable and volatile substructures, an inter-layer mixup for aligning heterogeneous network layers, and a context-aware multi-layer joint sampling technique aimed at reducing network scale while retaining essential topological characteristics. By abstracting complex network data into manageable weighted graphs, with each graph depicting a distinct network layer, our approach renders these intricate systems accessible on standard computing hardware. We validate our methodology through case studies, quantitative experiments and expert evaluations, demonstrating its effectiveness in managing large multi-layer networks, as well as its applicability to broader network types such as transportation and social networks.

Abstract:
While tooth alignment is crucial for digital dentistry, especially in orthodontic treatment, existing computer-aided methods mainly focus on the 3D dental crown but overlook the entire teeth, which is essential for applications in orthodontics. Besides, clinical orthodontic rules are not fully considered in these methods, i.e., there should be no collisions and gaps between teeth, the upper jaw and lower jaw should have correct occlusion relationships, the teeth should comply with a reasonable dental arch curve, etc. To generate optimal tooth alignment results, we propose a rule-based optimization method for solving the tooth alignment problem that takes into consideration the clinical rules functionally and aesthetically. We optimize rule-driven objective functions by adjusting the 6-DoF transformations of each tooth. Besides, our optimization formulation supports customization for different clinical scenarios by specifying the various energy terms. Extensive experiments, ablation studies, and user studies have been conducted to validate the effectiveness of our method. Quantitative and qualitative comparisons demonstrate that our method generates better tooth alignments than previous methods.

Abstract:
To address the increasing computational demands of high-resolution virtual reality headsets, foveated rendering reduces pixel sampling in the peripheral regions of the visual field. However, existing methods have not fully leveraged binocular vision, particularly the dominant eye theory. In our prior work, we proposed Dominant-Eye-Aware foveated rendering optimized with Multi-Parameter foveation (DEAMP), which divided each eye’s visual field into fixed eccentricity layers ([0, 10]^\circ ∘, [10, 22.5]^\circ ∘, [22.5, 45]^\circ ∘). The non-dominant eye received greater foveation within the same layers compared to the dominant eye. We further argue that the eccentricity ranges should vary between the eyes due to inter-eye, individual, and scene-specific differences. In this article, we introduce an enhanced method, Dominant-Eye-aware Asymmetric Foveated Rendering (DEA-FoR). This treats eccentricity as a new variable, allowing users to select eccentricity sets tailored to their eyes and supporting asymmetric configurations between the two eyes. Experimental results demonstrate significant improvements in rendering speed over our previous method while maintaining perceptual quality. Additionally, we found that individual differences and scene texture complexity significantly influence the eccentricity settings. This work offers new insights into perceptual differences in binocular vision and contributes to optimizing virtual reality experiences.

Abstract:
The neural radiance field (NeRF) achieved remarkable success in modeling 3D scenes and synthesizing high-fidelity novel views. However, existing NeRF-based methods focus more on making full use of high-resolution images to generate high-resolution novel views, but less considering the generation of high-resolution details given only low-resolution images. In analogy to the extensive usage of image super-resolution, NeRF super-resolution is an effective way to generate low-resolution-guided high-resolution 3D scenes and holds great potential applications. Up to now, such an important topic is still under-explored. In this article, we propose a NeRF super-resolution method, named Super-NeRF, to generate high-resolution NeRF from only low-resolution inputs. Given multi-view low-resolution images, Super-NeRF constructs a multi-view consistency-controlling super-resolution module to generate various view-consistent high-resolution details for NeRF. Specifically, an optimizable latent code is introduced for each input view to control the generated reasonable high-resolution 2D images satisfying view consistency. The latent codes of each low-resolution image are optimized synergistically with the target Super-NeRF representation to utilize the view consistency constraint inherent in NeRF construction. We verify the effectiveness of Super-NeRF on synthetic, real-world, and even AI-generated NeRFs. Super-NeRF achieves state-of-the-art NeRF super-resolution performance on high-resolution detail generation and cross-view consistency.

Abstract:
Current sketch extraction methods either require extensive training or fail to capture a wide range of artistic styles, limiting their practical applicability and versatility. We introduce Mixture-of-Self-Attention (MixSA), a training-free sketch extraction method that leverages strong diffusion priors for enhanced sketch perception. At its core, MixSA employs a mixture-of-self-attention technique, which manipulates self-attention layers by substituting the keys and values with those from reference sketches. This allows for the seamless integration of brushstroke elements into initial outline images, offering precise control over texture density and enabling interpolation between styles to create novel, unseen styles. By aligning brushstroke styles with the texture and contours of colored images, particularly in late decoder layers handling local textures, MixSA addresses the common issue of color averaging by adjusting initial outlines. Evaluated with various perceptual metrics, MixSA demonstrates superior performance in sketch quality, flexibility, and applicability. This approach not only overcomes the limitations of existing methods but also empowers users to generate diverse, high-fidelity sketches that more accurately reflect a wide range of artistic expressions.

Abstract:
Reconstructing real-world 3D objects has numerous applications in computer vision, such as virtual reality, video games, and animations. Ideally, 3D reconstruction methods should generate high-fidelity results with 3D consistency in real-time. Traditional methods match pixels between images using photo-consistency constraints or learned features, while differentiable rendering methods like Neural Radiance Fields (NeRF) use differentiable volume rendering or surface-based representation to generate high-fidelity scenes. However, these methods require excessive runtime for rendering, making them impractical for daily applications. To address these challenges, we present EvaSurf, an Efficient View-Aware implicit textured Surface reconstruction method on mobile devices. In our method, we first employ an efficient surface-based model with a multi-view supervision module to ensure accurate mesh reconstruction. To enable high-fidelity rendering, we learn an implicit texture embedded with view-aware encoding to capture view-dependent information. Furthermore, with the explicit geometry and the implicit texture, we can employ a lightweight neural shader to reduce the expense of computation and further support real-time rendering on common mobile devices. Extensive experiments demonstrate that our method can reconstruct high-quality appearance and accurate mesh on both synthetic and real-world datasets. Moreover, our method can be trained in just 1-2 hours using a single GPU and run on mobile devices at over 40 FPS (Frames Per Second), with a final package required for rendering taking up only 40–50 MB.

Abstract:
Deducing the 3D face from a skull is a challenging task in forensic science and archaeology. This article proposes an end-to-end 3D face reconstruction pipeline and an exploration method that can conveniently create textured, realistic faces that match the given skull. To this end, we propose a tissue-guided face creation and adaptation scheme. With the help of the state-of-the-art text-to-image diffusion model and parametric face model, we first generate an initial reference 3D face, whose biological profile aligns with the given skull. Then, with the help of tissue thickness distribution, we modify these initial faces to match the skull through a latent optimization process. The joint distribution of tissue thickness is learned on a set of skull landmarks using a collection of scanned skull-face pairs. We also develop an efficient face adaptation tool to allow users to interactively adjust tissue thickness either globally or at local regions to explore different plausible faces. Experiments conducted on a real skull-face dataset demonstrated the effectiveness of our proposed pipeline in terms of reconstruction accuracy, diversity, and stability.

Abstract:
Recently, significant progress has been made in condition-guided human motion generation. However, due to the inherent abstraction of conditional semantics like text, music and trajectory, these methods often fall short of generating precise motions that align with human intent. In contrast, free-hand sketches inherently and accurately depict human perspective intent, finding extensive applications across multiple domains. In this article, we introduce Sketch-guided human Motion Diffusion (SMD), to address a novel scenario: sketch-to-motion, aiming to generate plausible and natural human motions based on human motion sketches. Specifically, Our proposed SMD employs a Dual-branch Time-aware Transformer that utilizes both global semantic and local perspective level attention to condition 2D sketch information for 3D motion generation. At the global semantic level, we establish associations between the representation of the entire sketch and the sequential motion to ensure the generated motion aligns with the semantic content of the sketch. Meanwhile, at the local perspective level, a sketch-aware local attention is devised to correlate the sketch patches with the motion keyframes, aiming to precisely align the keyframes with the given sketches. Rooted in Diffusion model and Dual-branch Time-aware Transformer, our approach demonstrates proficiency in motion in-betweening and body part editing tasks, seamlessly generating natural motion sequences that harmonize with the provided context. Multiple experiments conducted on the curated sketch-to-motion datasets validate the efficacy of SMD, showcasing the state-of-the-art generation performances.

Abstract:
Motion retargeting for animation characters has potential applications in fields such as animation production and virtual reality. However, current methods either assume that the source and target characters have the same skeletal structure, or require designing and training specific model architectures for each structure. In this article, we aim to address the challenge of motion retargeting across previously unseen skeletal structures with a unified dynamic graph network. The proposed approach utilizes a dynamic graph transformation module to dynamically transfer latent motion features to different structures. We also take into consideration for intricate hand movements and model both torso and hand joints as graphs in a unified manner for whole-body motion retargeting. Our model allows the use of motion data from different structures to train a unified model and learns cross-structural motion retargeting in an unsupervised manner with unpaired data. Experimental results demonstrate the superiority of the proposed method in terms of data efficiency and performance on both seen and unseen structures.

Abstract:
It is common in scientific visualization for one 2D scalar field variable to be colormapped and draped onto a shaded surface representing a second variable displayed as a height field. The experiments reported here investigate the properties that make a colormap suitable for this application; specifically, how much colormap accuracy is lost because of draping and shading as well as the degree to which the colormap impacts the perception of surface shape. A new task is used to evaluate surface shape perception; it involves study participants clicking on the peaks of hills and the bottoms of valleys in the display. The results provided surprisingly little support for the hypothesis that colormaps varying greatly in luminance impacted surface shape perception more than those that are close to isoluminant. Stepped colormaps proved to be more accurately read. However, stepped colormaps also interfered much more with the perception of surface shape. The implication of these results are that when selecting a colormap to be used for draping, careful consideration should be given to which of the two variables is more important, the one represented by the shaded surface, or the one which is colormapped and draped.

Abstract:
Human mesh recovery aims to estimate all human meshes within a given image. In this article, we propose an Instance-aware Multi-person 3D Human Mesh Recovery (InstaHMR) network based on the one-stage framework. Compared to former one-stage methods, instance-aware single person feature is exploited to represent more accurate human mesh. Specifically, we propose the Contextual Instance Guidance (CIG) module which generates instance-aware single person feature by leveraging spatial and channel attention operations. In this way, it preserves more instance-specific information compared to the pixel-level feature used in some existing one-stage methods. Besides, we further introduce two auxiliary losses for better mesh recovery, namely the Human Triplet Planes (HTP) loss and the T-pose Shape (TS) loss. The HTP loss encourages the model to capture subtle differences in human joint positions, while the TS loss facilitates the learning of abstract shape parameters. By incorporating these advancements, our model achieves state-of-the-art results on four multi-person datasets.

Abstract:
Areas of interest (AOIs) are well-established means of providing semantic information for visualizing, analyzing, and classifying gaze data. However, the usual manual annotation of AOIs is time-consuming and further impaired by ambiguities in label assignments. To address these issues, we present an interactive labeling approach that combines visualization, machine learning, and user-centered explainable annotation. Our system provides uncertainty-aware visualization to build trust in classification with an increasing number of annotated examples. It combines specifically designed EyeFlower glyphs, dimensionality reduction, and selection and exploration techniques in an integrated workflow. The approach is versatile and hardware-agnostic, supporting video stimuli from stationary and unconstrained mobile eye tracking alike. We conducted an expert review to assess labeling strategies and trust building.

Abstract:
Historical visualizations are a rich resource for visualization research. While taxonomy is commonly used to structure and understand the design space of visualizations, existing taxonomies primarily focus on contemporary visualizations and largely overlook historical visualizations. To address this gap, we describe an empirical method for taxonomy development. We introduce a coding protocol and the VisTaxa system for taxonomy labeling and comparison. We demonstrate using our method to develop a historical visualization taxonomy by coding 400 images of historical visualizations. We analyze the coding result and reflect on the coding process. Our work is an initial step toward a systematic investigation of the design space of historical visualizations.

Abstract:
In medical image visualization, path tracing of volumetric medical data like computed tomography (CT) scans produces lifelike three-dimensional visualizations. Immersive virtual reality (VR) displays can further enhance the understanding of complex anatomies. Going beyond the diagnostic quality of traditional 2D slices, they enable interactive 3D evaluation of anatomies, supporting medical education and planning. Rendering high-quality visualizations in real-time, however, is computationally intensive and impractical for compute-constrained devices like mobile headsets. We propose a novel approach utilizing Gaussian Splatting (GS) to create an efficient but static intermediate representation of CT scans. We introduce a layered GS representation, incrementally including different anatomical structures while minimizing overlap and extending the GS training to remove inactive Gaussians. We further compress the created model with clustering across layers. Our approach achieves interactive frame rates while preserving anatomical structures, with quality adjustable to the target hardware. Compared to standard GS, our representation retains some of the explorative qualities initially enabled by immersive path tracing. Selective activation and clipping of layers are possible at rendering time, adding a degree of interactivity to otherwise static GS models. This could enable scenarios where high computational demands would otherwise prohibit using path-traced medical volumes.

Abstract:
Virtual Reality has been used to improve motivation and help in the visualization of complex computing topics. However, few studies directly compared immersive and non-immersive environments. To address this limitation, we developed Abacus, a programming environment that can run in both immersive and non-immersive modes. We conducted a between-subjects study (n=40), with twenty participants assigned to the desktop mode and twenty participants assigned to the VR mode. Participants used a block-based editor to complete two programming tasks: a non-spatial procedural task, and a spatial 3D math task. We found that VR led to higher gains in self-efficacy and that the gain was significant for participants with lower initial levels of experience and spatial skills.

Abstract:
In consumer research, understanding consumer behavior and experiences is vital for making informed decisions about product design, innovation and marketing. Zaltman's Metaphor Elicitation Technique (ZMET) leverages metaphors and non-verbal communication to uncover and gain deeper insights into consumers' thoughts and emotions. This paper introduces a novel system that enables consumer researchers (interviewers) to perform a modified version of metaphor elicitation interviews in virtual reality (VR). Consumers (participants) use 3D objects in the virtual environment to express their thoughts and emotions, instead of pictures conventionally used in a ZMET interview. The system features an asymmetric setup where the participant is immersed in VR using a head-mounted display (HMD), while the interviewer views the participant's perspective on a monitor. We discuss the technical and design aspects of the VR system and present the results of a user study (N=17) that we conducted to validate the effectiveness of performing the metaphor elicitation interviews in VR. This work also explores the experiences of both participants and interviewers during the interview sessions, aiming to identify potential improvements. The qualitative and quantitative analysis of the data demonstrated how immersion, presence and embodied interaction positively affect and aid in sense-making and deeper expression of the participants' thoughts, perspectives and emotions.

Abstract:
Developing convincing and realistic virtual human behavior is essential for enhancing user experiences in virtual reality (VR) and augmented reality (AR) settings. This paper introduces a novel task focused on generating long-term behaviors for virtual agents, guided by specific personality traits and contextual elements within 3D environments. We present a comprehensive framework capable of autonomously producing daily activities autoregressively. By modeling the intricate connections between personality characteristics and observable activities, we establish a hierarchical structure of Needs, Task, and Activity levels. Integrating a Behavior Planner and a World State module allows for the dynamic sampling of behaviors using large language models (LLMs), ensuring that generated activities remain relevant and responsive to environmental changes. Extensive experiments validate the effectiveness and adaptability of our approach across diverse scenarios. This research makes a significant contribution to the field by establishing a new paradigm for personalized and context-aware interactions with virtual humans, ultimately enhancing user engagement in immersive applications. Our project website is at: https://behavior.agent-x.cn/.

Abstract:
In order to serve better VR experiences to users, existing predictive methods of Redirected Walking (RDW) exploit future information to reduce the number of reset occurrences. However, such methods often impose a precondition during deployment, either in the virtual environment's layout or the user's walking direction, which constrains its universal applications. To tackle this challenge, we propose a mechanism F-RDW that is twofold: (1) forecasts the future information of a user in the virtual space without any assumptions by using the conventional method, and (2) fuse this information while maneuvering existing RDW methods. The backbone of the first step is an LSTM-based model that ingests the user's spatial and eye-tracking data to predict the user's future position in the virtual space, and the following step feeds those predicted values into existing RDW methods (such as MPCRed, S2C, TAPF, and ARC) while respecting their internal mechanism in applicable ways. The results of our simulation test and user study demonstrate the significance of future information when using RDW in small physical spaces or complex environments. We prove that the proposed mechanism significantly reduces the number of resets and increases the traveled distance between resets, hence augmenting the redirection performance of all RDW methods explored in this work.

Abstract:
Cartoon animation video is a popular visual entertainment form worldwide, however many classic animations were produced in a 4:3 aspect ratio that is incompatible with modern widescreen displays. Existing methods like cropping lead to information loss while retargeting causes distortion. Animation companies still rely on manual labor to renovate classic cartoon animations, which is tedious and labor-intensive, but can yield higher-quality videos. Conventional extrapolation or inpainting methods tailored for natural videos struggle with cartoon animations due to the lack of textures in anime, which affects the motion estimation of the objects. In this article, we propose a novel framework designed to automatically outpaint 4:3 anime to 16:9 via region-guided motion inference. Our core concept is to identify the motion correspondences between frames within a sequence in order to reconstruct missing pixels. Initially, we estimate optical flow guided by region information to address challenges posed by exaggerated movements and solid-color regions in cartoon animations. Subsequently, frames are stitched to produce a pre-filled guide frame, offering structural clues for the extension of optical flow maps. Finally, a voting and fusion scheme utilizes learned fusion weights to blend the aligned neighboring reference frames, resulting in the final outpainting frame. Extensive experiments confirm the superiority of our approach over existing methods.

Abstract:
The advances in AI-enabled techniques have accelerated the creation and automation of visualizations in the past decade. However, presenting visualizations in a descriptive and generative format remains a challenge. Moreover, current visualization embedding methods focus on standalone visualizations, neglecting the importance of contextual information for multi-view visualizations. To address this issue, we propose a new representation model, Chart2Vec, to learn a universal embedding of visualizations with context-aware information. Chart2Vec aims to support a wide range of downstream visualization tasks such as recommendation and storytelling. Our model considers both structural and semantic information of visualizations in declarative specifications. To enhance the context-aware capability, Chart2Vec employs multi-task learning on both supervised and unsupervised tasks concerning the cooccurrence of visualizations. We evaluate our method through an ablation study, a user study, and a quantitative comparison. The results verified the consistency of our embedding method with human cognition and showed its advantages over existing methods.

Abstract:
This article presents a particle-based visualization approach for finite-time analysis of the connectivity of fluid portions in multiphase flow, i.e., the evolution of the droplets in volume of fluid simulations. We address the Lagrangian inconsistency between the interpolated flow field and the interpolated volume of fluid field by a correction approach, and complement that with an uncertainty measure that provides an estimate of the involved inconsistency. We demonstrate the utility and versatility of our approach using different multiphase flow simulations, exemplify its application in physics-based assessment of droplet formation processes, and discuss its limitations and benefits.

Abstract:
Massive Open Online Courses (MOOCs) platforms are becoming increasingly popular in recent years. Online learners need to watch the whole course video on MOOC platforms to learn the underlying new knowledge, which is often tedious and time-consuming due to the lack of a quick overview of the covered knowledge and their structures. In this article, we propose ConceptThread, a visual analytics approach to effectively show the concepts and the relations among them to facilitate effective online learning. Specifically, given that the majority of MOOC videos contain slides, we first leverage video processing and speech analysis techniques, including shot recognition, speech recognition and topic modeling, to extract core knowledge concepts and construct the hierarchical and temporal relations among them. Then, by using a metaphor of thread, we present a novel visualization to intuitively display the concepts based on video sequential flow, and enable learners to perform interactive visual exploration of concepts. We conducted a quantitative study, two case studies, and a user study to extensively evaluate ConceptThread. The results demonstrate the effectiveness and usability of ConceptThread in providing online learners with a quick understanding of the knowledge content of MOOC videos.

Abstract:
The reconstruction and novel view synthesis of dynamic scenes recently gained increased attention. As reconstruction from large-scale multi-view data involves immense memory and computational requirements, recent benchmark datasets provide collections of single monocular views per timestamp sampled from multiple (virtual) cameras. We refer to this form of inputs as monocularized data. Existing work shows impressive results for synthetic setups and forward-facing real-world data, but is often limited in the training speed and angular range for generating novel views. This paper addresses these limitations and proposes a new method for full 360° inward-facing novel view synthesis of non-rigidly deforming scenes. At the core of our method are: 1) An efficient deformation module that decouples the processing of spatial and temporal information for accelerated training and inference; and 2) A static module representing the canonical scene as a fast hash-encoded neural radiance field. In addition to existing synthetic monocularized data, we systematically analyze the performance on real-world inward-facing scenes using a newly recorded challenging dataset sampled from a synchronized large-scale multi-view rig. In both cases, our method is significantly faster than previous methods, converging in less than 7 minutes and achieving real-time framerates at 1 K resolution, while obtaining a higher visual accuracy for generated novel views.

Abstract:
We present an authoring tool, called CAST+ (Canis Studio Plus), that enables the interactive creation of chart animations through the direct manipulation of keyframes. It introduces the visual specification of chart animations consisting of keyframes that can be played sequentially or simultaneously, and animation parameters (e.g., duration, delay). Building on Canis (Ge et al. 2020), a declarative chart animation grammar that leverages data-enriched SVG charts, CAST+ supports auto-completion for constructing both keyframes and keyframe sequences. It also enables users to refine the animation specification (e.g., aligning keyframes across tracks to play them together, adjusting delay) with direct manipulation. We report a user study conducted to assess the visual specification and system usability with its initial version. We enhanced the system's expressiveness and usability: CAST+ now supports the animation of multiple types of visual marks in the same keyframe group with new auto-completion algorithms based on generalized selection. This enables the creation of more expressive animations, while reducing the number of interactions needed to create comparable animations. We present a gallery of examples and four usage scenarios to demonstrate the expressiveness of CAST+. Finally, we discuss the limitations, comparison, and potentials of CAST+ as well as directions for future research.

Abstract:
Topological abstractions offer a method to summarize the behavior of vector fields, but computing them robustly can be challenging due to numerical precision issues. One alternative is to represent the vector field using a discrete approach, which constructs a collection of pairs of simplices in the input mesh that satisfies criteria introduced by Forman's discrete Morse theory. While numerous approaches exist to compute pairs in the restricted case of the gradient of a scalar field, state-of-the-art algorithms for the general case of vector fields require expensive optimization procedures. This paper introduces a fast, novel approach for pairing simplices of two-dimensional, triangulated vector fields that do not vary in time. The key insight of our approach is that we can employ a local evaluation, inspired by the approach used to construct a discrete gradient field, where every simplex in a mesh is considered by no more than one of its vertices. Specifically, we observe that for any edge in the input mesh, we can uniquely assign an outward direction of flow. We can further expand this consistent notion of outward flow at each vertex, which corresponds to the concept of a downhill flow in the case of scalar fields. Working with outward flow enables a linear-time algorithm that processes the (outward) neighborhoods of each vertex one-by-one, similar to the approach used for scalar fields. We couple our approach to constructing discrete vector fields with a method to extract, simplify, and visualize topological features. Empirical results on analytic and simulation data demonstrate drastic improvements in running time, produce features similar to the current state-of-the-art, and show the application of simplification to large, complex flows.

Abstract:
Dynamic topic modeling is useful at discovering the development and change in latent topics over time. However, present methodology relies on algorithms that separate document and word representations. This prevents the creation of a meaningful embedding space where changes in word usage and documents can be directly analyzed in a temporal context. This paper proposes an expansion of the compass-aligned temporal Word2Vec methodology into dynamic topic modeling. Such a method allows for the direct comparison of word and document embeddings across time in dynamic topics. This enables the creation of visualizations that incorporate temporal word embeddings within the context of documents into topic visualizations. In experiments against the current state-of-the-art, our proposed method demonstrates overall competitive performance in topic relevancy and diversity across temporal datasets of varying size. Simultaneously, it provides insightful visualizations focused on temporal word embeddings while maintaining the insights provided by global topic evolution, advancing our understanding of how topics evolve over time.

Abstract:
With the increase of graph size, it becomes difficult or even impossible to visualize graph structures clearly within the limited screen space. Consequently, it is crucial to design effective visual representations for large graphs. In this paper, we propose AdaMotif, a novel approach that can capture the essential structure patterns of large graphs and effectively reveal the overall structures via adaptive motif designs. Specifically, our approach involves partitioning a given large graph into multiple subgraphs, then clustering similar subgraphs and extracting similar structural information within each cluster. Subsequently, adaptive motifs representing each cluster are generated and utilized to replace the corresponding subgraphs, leading to a simplified visualization. Our approach aims to preserve as much information as possible from the subgraphs while simplifying the graph efficiently. Notably, our approach successfully visualizes crucial community information within a large graph. We conduct case studies and a user study using real-world graphs to validate the effectiveness of our proposed approach. The results demonstrate the capability of our approach in simplifying graphs while retaining important structural and community information.

Abstract:
We propose and study a novel cross-reality environment that seamlessly integrates a monoscopic 2D surface (an interactive screen with touch and pen input) with a stereoscopic 3D space (an augmented reality HMD) to jointly host spatial data visualizations. This innovative approach combines the best of two conventional methods of displaying and manipulating spatial 3D data, enabling users to fluidly explore diverse visual forms using tailored interaction techniques. Providing such effective 3D data exploration techniques is pivotal for conveying its intricate spatial structures—often at multiple spatial or semantic scales—across various application domains and requiring diverse visual representations for effective visualization. To understand user reactions to our new environment, we began with an elicitation user study, in which we captured their responses and interactions. We observed that users adapted their interaction approaches based on perceived visual representations, with natural transitions in spatial awareness and actions while navigating across the physical surface. Our findings then informed the development of a design space for spatial data exploration in cross-reality. We thus developed cross-reality environments tailored to three distinct domains: for 3D molecular structure data, for 3D point cloud data, and for 3D anatomical data. In particular, we designed interaction techniques that account for the inherent features of interactions in both spaces, facilitating various forms of interaction, including mid-air gestures, touch interactions, pen interactions, and combinations thereof, to enhance the users' sense of presence and engagement. We assessed the usability of our environment with biologists, focusing on its use for domain research. In addition, we evaluated our interaction transition designs with virtual and mixed-reality experts to gather further insights. As a result, we provide our design suggestions for the cross-reality environment, emphasizing the interaction with diverse visual representations and seamless interaction transitions between 2D and 3D spaces.

Abstract:
As a step towards improving visualization literacy, this work investigates how students approach reading visualizations differently after taking a university-level visualization course. We asked students to verbally walk through their process of making sense of unfamiliar visualizations, and conducted a qualitative analysis of these walkthroughs. Our qualitative analysis found that after taking a visualization course, students engaged with visualizations in more sophisticated ways: they were more likely to exhibit design empathy by thinking critically about the tradeoffs behind why a chart was designed in a particular way, and were better able to deconstruct a chart to make sense of it. We also gave students a quantitative assessment of visualization literacy and found no evidence of scores improving after the class, likely because the test we used focused on a different set of skills than those emphasized in visualization classes. While current measurement instruments for visualization literacy are useful, we propose developing standardized assessments for additional aspects of visualization literacy, such as deconstruction and design empathy. We also suggest that these additional aspects could be incorporated more explicitly in visualization courses. All supplemental materials are available at https://osf.io/w5pum/.

Abstract:
Visualization designers (e.g., journalists or data analysts) often rely on examples to explore the space of possible designs, yet we have little insight into how examples shape data visualization design outcomes. While the effects of examples have been studied in other disciplines, such as web design or engineering, the results are not readily applicable to visualization due to inconsistencies in findings and challenges unique to visualization design. Towards bridging this gap, we conduct an exploratory experiment involving 32 data visualization designers focusing on the influence of five factors (timing, quantity, diversity, data topic similarity, and data schema similarity) on objectively measurable design outcomes (e.g., numbers of designs and idea transfers). Our quantitative analysis shows that when examples are introduced after initial brainstorming, designers curate examples with topics less similar to the dataset they are working on and produce more designs with a high variation in visualization components. Also, designers copy more ideas from examples with higher data schema similarities. Our qualitative analysis of participants' thought processes provides insights into why designers incorporate examples into their designs, revealing potential factors that have not been previously investigated. Finally, we discuss how our results inform how designers may use examples during design ideation as well as future research on quantifying designs and supporting example-based visualization design. All supplemental materials are available in our OSF repo.

Abstract:
This paper presents discursive patinas, a technique to visualize discussions onto data visualizations, inspired by how people leave traces in the physical world. While data visualizations are widely discussed in online communities and social media, comments tend to be displayed separately from the visualization and we lack ways to relate these discussions back to the content of the visualization, e.g., to situate comments, explain visual patterns, or question assumptions. In our visualization annotation interface, users can designate areas within the visualization. Discursive patinas are made of overlaid visual marks (anchors), attached to textual comments with category labels, likes, and replies. By coloring and styling the anchors, a meta visualization emerges, showing what and where people comment and annotate the visualization. These patinas show regions of heavy discussions, recent commenting activity, and the distribution of questions, suggestions, or personal stories. We ran workshops with 90 students, domain experts, and visualization researchers to study how people use anchors to discuss visualizations and how patinas influence people's understanding of the discussion. Our results show that discursive patinas improve the ability to navigate discussions and guide people to comments that help understand, contextualize, or scrutinize the visualization. We discuss the potential of anchors and patinas to support discursive engagements, including critical readings of visualizations, design feedback, and feminist approaches to data visualization.

Abstract:
NeRF registration has gained increasing attention recently. While existing research demonstrates considerable potential for this task, most methods primarily focus on either global geometric or rendering photometric information during feature learning, overlooking the rich cross-modal information inherent in the NeRF embedding feature space. In this paper, we propose DeSC, a novel NeRF registration approach that leverages the rich cross-modal features from NeRF to learn robust semantic descriptors. In particular, we propose a Deep Semantic Aggregation module, which employs a weighted graph convolution network to capture high-frequency texture details in NeRF patches. This approach reveals the underlying semantics shared across different NeRFs of the same scene, thereby yielding more robust global feature descriptors that lead to better alignment accuracy and robustness. In addition, we design a density-aware photometric consistency loss that facilitates the learning of robust features. Extensive experimental results on Objaverse datasets demonstrate that our approach produces superior registration performance to state-of-the-art techniques.

Abstract:
Uncertainty is an intrinsic property of almost all data, regardless of the data being measured, simulated, or generated. It can significantly influence the results and reliability of subsequent analysis steps. Clearly communicating uncertainties is crucial for informed decision-making and understanding, especially in biomolecular data, where uncertainty is often difficult to infer. Uncertainty visualization (UV) is a powerful tool for this purpose. However, previously proposed uncertainty visualization (UV) methods lack sufficient empirical evaluation. We collected and categorized visualization methods for portraying positional uncertainty in biomolecular structures. We then organized the methods into metaphorical groups and extracted nine representatives: color, clouds, ensemble, hulls, sausages, contours, texture, waves, and noise. We assessed their strengths and weaknesses in a twofold approach: expert assessments with six domain experts and three perceptual evaluations involving 1,756 participants. Through the expert assessments, we aimed to highlight the advantages and limitations of the individual methods for the application domain and discussed areas for necessary improvements. Through the perceptual evaluation, we investigated whether the visualizations are intuitively associated with uncertainty and whether the directionality of the mapping is perceived as intended. We also assessed the accuracy of inferring uncertainty values from the visualizations. Based on our results, we judged the appropriateness of the metaphors for encoding uncertainty and suggest further areas for improvement.

Abstract:
This study aims to assess the impact of different visual perturbations on user balance within an immersive Virtual Environment (VE), emphasising ecological validity. Utilising a Virtual Reality headset, the researchers implemented various visual perturbations to discern their effects. The objective was to create a versatile tool featuring animations simulating diverse indoor and outdoor settings within a typical household, providing a range of ecological scenarios for evaluating user balance in the context of these visual disturbances. Participants were exposed to unpredictable visual disturbances within the VE, and their responses were captured using a combination of inertial sensors, electromyography and galvanic skin response. A comprehensive dataset replicating real-world fall scenarios on balance disturbances was collected. Throughout the trials, participants moved around the house, unaware of the timing or nature of visual perturbations. Statistical analysis identified variables with significant mean changes when visual disturbances were introduced. Muscle groups crucial for balance recovery following external disturbances exhibited the most substantial associations, akin to kinematic variables linked to loss of balance. Visual disturbances alone could induce balance disturbances, which were further categorized based on a statistical analysis. This tool offered a versatile platform for assessing balance under various challenging scenarios, facilitating research into balance control and training methods. The dataset generated through this protocol enabled a wide understanding of the impact of visual disturbances on balance and the associated compensatory reactions, shedding light on their potential applications in clinical and rehabilitative settings.

Abstract:
Partial point cloud registration plays a crucial role in computer vision and has widespread applications in 3D map construction, pose estimation, and high-precision localization. However, the collected point clouds often contain missing data due to hardware limitations and complex environments. Various partial registration algorithms have been proposed, most of which rely on estimating overlap regions. However, a significant proportion of these algorithms rely heavily on ground truth labels. Manual labeling is both time-consuming and labor-intensive, whereas algorithmic automatic labeling lacks sufficient accuracy. To tackle this issue, we present PSEudo Label learning for unsupervised partial point cloud registration (PSEL). This method utilizes complementary tasks to learn reliable pseudo labels for overlap regions and correspondences without depending on ground truth labels. The key idea is to use the complementarity between overlap estimation and registration to generate two types of pseudo labels based on the nearest points in pairs of aligned point clouds. These pseudo labels are then employed to supervise the learning of overlap regions and correspondences, gradually enhancing their accuracy throughout the learning process and ultimately establishing an unsupervised learning framework. PSEL consists of an overlap estimation module and a correspondence filtering module. The pseudo labels generated after registration are used to supervise both modules. Notably, the correspondence filtering module has two pipelines. The similarity and difference of the corresponding point features are used to eliminate false correspondences during the training and inference stages, respectively, with only the latter being optimized with pseudo labels. To validate the effectiveness of our registration method, we conducted experiments using the synthetic dataset ModelNet40, the indoor dataset 3DMatch, and the outdoor dataset KITTI.

Abstract:
Recent advancements in single-image 3D generation have produced two main categories of methods: reconstruction-based and generative methods. Reconstruction-based methods are efficient but lack uncertainty handling, leading to blurry artifacts in unseen regions. Generative approaches that based on score distillation (Poole et al. 2023), (Wang et al. 2024) are slow due to scene-specific optimization. Other methods, like InstantMesh (Xu et al. 2024), use a two-stage process - generating multi-view images with a diffusion model and then reconstructing 3D - which is inefficient due to multiple denoising steps of the diffusion model. To overcome these limitations, we introduce GECO, a feed-forward method for fast and high-quality single-image-to-3D generation within one second on a single GPU. Our approach resolves uncertainty and inefficiency issues through a two-stage distillation process. In the first stage, we distill a multi-step diffusion model (Shi et al. 2023) into a one-step model using score distillation for single-image-to-multi-view synthesis. To mitigate the synthesis quality degradation caused by the one-step model, we introduce a second distillation stage to learn to predict high-quality 3D from imperfect multi-view generated images by performing distillation directly on 3D representations. Experiments demonstrate that GECO offers significant speed improvements and comparable reconstruction quality compared to prior two-stage methods.

Abstract:
Extracting wireframes from CAD models represented by point cloud remains a significant challenge in computer graphics. This difficulty arises from two main factors: first, imperfections in the point cloud data, such as lack of orientation, noise, and sparsity; and second, the inherent complexity of geometric shapes, which often feature a high density of sharp edges in close proximity. In this paper, we propose D-FRAME, a multi-stage wireframe extraction framework that incorporates a novel direction field to improve edge detection quality and connectivity, a refinement strategy to address sparse or noisy edge points, and a final coarse-to-fine connection module to extract a robust wireframe. The direction field not only facilitates connectivity but also enhances the precision of extracted edges by mitigating the impact of misclassified points. By combining the Restricted Voronoi Diagram (RVD) with the extracted wireframes and the original point cloud, our approach also achieves highly faithful reconstruction of CAD model. Experiments conducted on synthetic and real-world scanned CAD datasets demonstrate that D-FRAME effectively manages noise, sparsity, and complex geometries, yielding high-fidelity wireframes.

Abstract:
Generating high-quality 3D scenes is a critical challenge in computer vision, driven by advances in 3D graphics and the growing demand for immersive environments. While object-centric 3D generation has achieved significant progress, scene generation remains difficult due to the scarcity of large-scale 3D scene datasets and scalability constraints of conventional 3D representations, which hinder efficient large-scale expansion. To address these challenges, we propose LucidDreamer, a novel pipeline that synthesizes diverse, high-quality, and expandable 3D scenes using a unified 3D Gaussian splatting representation. Our approach employs an iterative Navigation-Dreaming-Alignment process, leveraging 2D image generation and depth estimation to construct photorealistic, scalable 3D environments. By iteratively generating images and navigating through the scene, LucidDreamer fully utilizes the power of image generation models, enabling the creation of highly detailed and expandable 3D scenes. LucidDreamer supports various input modalities, including text, RGB, and RGBD, and enables dynamic modifications during generation. Experimental results demonstrate that LucidDreamer outperforms existing methods in generating high-quality, diverse, structurally consistent, and navigable 3D scenes.

Abstract:
Point cloud completion is a vital task focused on reconstructing complete point clouds and addressing the incompleteness caused by occlusion and limited sensor resolution. Traditional methods relying on fixed local region partitioning, such as kk-nearest neighbors, which fail to account for the highly uneven distribution of geometric complexity across different regions of a shape. This limitation leads to inefficient representation and suboptimal reconstruction, especially in areas with fine-grained details or structural discontinuities. This paper proposes a point cloud completion framework called Degree-Flexible Point Graph Completion Network (DFG-PCN). It adaptively assigns node degrees using a detail-aware metric that combines feature variation and curvature, focusing on structurally important regions. We further introduce a geometry-aware graph integration module that uses Manhattan distance for edge aggregation and detail-guided fusion of local and global features to enhance representation. Extensive experiments on multiple benchmark datasets demonstrate that our method consistently outperforms state-of-the-art approaches.

Abstract:
Free-hand sketches serve as efficient tools for creativity and communication, yet expressing ideas clearly through sketches remains challenging for untrained individuals. Optimizing sketches through text guidance can enhance individuals’ ability to effectively convey their ideas and improve overall communication efficiency. While recent advancements in Artificial Intelligence Generated Content (AIGC) have been notable, research on optimizing free-hand sketches remains relatively unexplored. In this paper, we introduce SketchRefiner, an innovative method designed to refine rough sketches from various categories into polished versions guided by text prompts. SketchRefiner utilizes a latent diffusion model with ControlNet to guide a differentiable rasterizer in optimizing a set of Bézier curves. We extend the score distillation sampling (SDS) loss and introduce a joint semantic loss to encourage sketches aligned with given text prompts and free-hand sketches. Additionally, we propose a fusion attention-map stroke initialization strategy to improve the quality of refined sketches. Furthermore, SketchRefiner provides users with fine-grained control over text guidance. Through extensive experiments, we demonstrate that our method can generate accurate and aesthetically pleasing refined sketches that closely align with input text prompts and sketches.

Abstract:
Foveated rendering exploits the non-uniform acuity of human vision to allocate computational resources more efficiently by reducing image fidelity in the peripheral field of view. While existing perceptual models for foveated rendering focus primarily on spatial resolution and contrast sensitivity, they overlook the perceptual asymmetry between direct and indirect illumination. In this work, we introduce a novel perceptual model that incorporates illuminance demodulation to account for this distinction. Our model adaptively modulates the foveation rate based on the relative contributions of direct and indirect illumination. Building on this model, we develop a practical rendering framework that separately applies tailored foveation strategies to direct and indirect illumination effects. Quantitative metrics and user studies confirm that our method maintains perceptual equivalence to full-resolution rendering. The sparse rendering stage achieves a 2.18×2.18× to 7.10×7.10× speedup, contributing to an overall acceleration of 1.71×1.71× to 3.26×3.26×.

Abstract:
Eye gaze is regarded as a promising interaction modality in extended reality (XR) environments. However, to address the challenges posed by the Midas touch problem, the determination of selection intention frequently relies on the implementation of additional manual selection techniques, such as explicit gestures (e.g., controller/hand inputs or dwell), which are inherently limited in their functionality. We hereby present a machine learning (ML) model based on the Bayesian framework, which is employed to predict user selection intention in real-time, with the unique distinction that all data used for training and prediction are obtained from gaze data alone. The model utilizes a Bayesian approach to transform gaze data into selection probabilities, which are subsequently fed into an ML model to discern selection intentions. In Study 1, a high-performance model was constructed, enabling real-time inference using solely gaze data. This approach was found to enhance performance, thereby validating the efficacy of the proposed methodology. In Study 2, a user study was conducted to validate a manual-free technique based on the prediction model. The advantages of eliminating explicit gestures and potential applications were also discussed.

Abstract:
Radiance field based methods have recently revolutionized novel view synthesis of scenes captured with multi-view photos. A significant recent advance is 3D Gaussian Splatting (3DGS), which utilizes a set of 3D Gaussians to represent a radiance field, yielding high-fidelity results in real-time rendering. However, we have observed that 3DGS struggles to capture the necessary details in sparsely observed regions, where there is not enough gradient for effective split and clone operations. In this paper, we present a novel solution to address this limitation. Our key idea is to leverage segmentation information to identify poorly optimized regions within the 3D Gaussian representation. By applying split or clone operations on the corresponding 3D Gaussians in these regions, we aim to refine the spatial distribution of Gaussians and enhance the overall quality of high-fidelity 3D scene reconstruction. To further optimize the reconstruction process, we introduce two spatial regularization terms: repulsion loss and smoothness loss. These terms effectively minimize overlap and redundancy among Gaussians, reducing outliers in the synthesized geometry. By incorporating these regularization techniques, our approach achieves state-of-the-art performance in real-time novel view synthesis and significantly improves visibility in less observed regions, leading to a more compact and accurate 3D scene representation.

Abstract:
With the explosive growth of 3D objects yet expensive annotation costs, unsupervised 3D object retrieval has become a popular but challenging research area. Existing labeled resources have been utilized to aid this task via transfer learning, which aligns the distribution of unlabeled data with the source one. However, the labeled resource are not always accessible due to the privacy disputes, limited computational capacity and other thorny restrictions. Therefore, we propose source-free model adaptation task for unsupervised 3D object management, which utilizes a pre-trained model to boost the performance with no access to source data and labels. Specifically, we compute representative prototypes to assume the source feature distribution, and design a bidirectional cumulative confidence-based adaptation strategy to adaptively align unlabeled samples towards prototypes. Subsequently, a dual-model distillation mechanism is proposed to generate source hypothesis for remedying the absence of ground-truth labels. The experiments on a cross-domain retrieval benchmark NTU-PSB (PSB-NTU) and a cross-modality retrieval benchmark MI3DOR also demonstrate the superiority of the proposed method even without access to raw data.

Abstract:
Feature level sets (FLS) have shown significant potential in the analysis of multi-field data by using traits defined in attribute space to specify features in the domain. In this work, we address key challenges in the practical use of FLS: trait design and feature selection for rendering. To simplify trait design, we propose a Cartesian decomposition of traits into simpler components, making the process more intuitive and computationally efficient. Additionally, we utilize dictionary learning results to automatically suggest point traits. To enhance feature selection, we introduce trait-induced merge trees (TIMTs), a generalization of merge trees for feature level sets, aimed at topologically analyzing tensor fields or general multi-variate data. The leaves in the TIMT represent areas in the input data that are closest to the defined trait, thereby most closely resembling the defined feature. This merge tree provides a hierarchy of features, enabling the querying of the most relevant and persistent features. Our method includes various query techniques for the tree, allowing the highlighting of different aspects. We demonstrate the cross-application capabilities of this approach through five case studies from different domains.

Abstract:
We present a photorealistic portrait style transfer approach that allows for producing high-quality results in previously challenging unconstrained conditions, e.g., large facial perspective difference between portraits, faces with complex illumination (e.g., shadow and highlight) and occlusion, and can test without portrait parsing masks. We achieve this by developing a framework to learn robust dense correspondence across portraits for semantically aligned style transfer, where a regional style contrastive learning strategy is devised to boost the effectiveness of semantic-aware style transfer while enhancing the robustness to complex illumination. Extensive experiments demonstrate the superiority of our method.

Abstract:
Unpaired shape-to-shape translation refers to the task of transforming the geometry and semantics of an input shape into a new shape domain without paired training data. Previous methods utilize GAN-based architectures to perform shape translation, employing adversarial training to transform the source shape encoding into the target domain in the low-dimensional latent feature space. However, these methods encounter difficulties in generating diverse and high-quality results, as they often suffer from issues such as “mode collapse”. This leads to limited generation diversity and makes it challenging to find an accurate latent code that adequately represents the input shape. In this article, we achieve unpaired shape-to-shape translation via a triplane diffusion model, in which we factorize 3D objects into triplane representations and conduct a diffusion process on these representations to accomplish shape domain transformation. We observe that by adding an appropriate amount of noise to an input object during the forward diffusion process, domain-specific shape structures are smoothed out while the overall structure is still preserved. Subsequently, we progressively remove the noise via an unconditional diffusion model trained on the target shape domain in the reverse diffusion process. This allows us to obtain a denoised output that retains the structural similarities of the source input while aligning with the distribution of the target shape domain. During this process, we propose two gradient-based guidance mechanisms to guide the translation process to guarantee more faithful results during the denoising process. We conduct extensive experiments on different shape domains, and the experimental results demonstrate that our method achieves superior shape fidelity with high quality compared to current state-of-the-art baselines.

Abstract:
We built a spatial hybrid system that combines a personal computer (PC) and virtual reality (VR) for visual sensemaking, addressing limitations in both environments. Although VR offers immense potential for interactive data visualization (e.g., large display space and spatial navigation), it can also present challenges such as imprecise interactions and user fatigue. At the same time, a PC offers precise and familiar interactions but has limited display space and interaction modality. Therefore, we iteratively designed a spatial hybrid system (PC+VR) to complement these two environments by enabling seamless switching between PC and VR environments. To evaluate the system's effectiveness and user experience, we compared it to using a single computing environment (i.e., PC-only and VR-only). Our study results (N=18) showed that spatial PC+VR could combine the benefits of both devices to outperform user preference for VR-only without a negative impact on performance from device switching overhead. Finally, we discussed future design implications.

Abstract:
Recent studies have demonstrated that brain synchrony can indicate the quality of social interaction in real-world communication. However, there is a lack of research on measurement of brain synchrony during social interactions in remote AR. In this study, we investigated the brain synchrony of remote augmented reality (AR; Study 1) and face-to-face (FTF; Study 2) interactions. Functional near-infrared spectroscopy was used to measure the brain synchrony during the tangram puzzle task. In a collaboration condition, participants worked together to solve the puzzle. In an individual condition, participants solved the puzzle independently. We recruited 46 participants in Study 1 and 48 participants in Study 2. Study 1 showed there was a significant difference in brain synchrony between the individual and collaboration conditions, and a positive correlation was observed between brain synchrony and the task performance in the collaboration condition. A comparison between Study 1 and 2 suggested that the difference between the collaboration and individual conditions was maintained, and some differences were observed in the brain synchrony between the AR and FTF interactions. These results suggest that measurement of brain synchrony is beneficial for social interaction in remote AR collaborations. The implications of these results on future remote interactions are discussed.

Abstract:
Prior foveated rendering methods often suffer from a limitation where the shading load escalates with increasing display resolution, leading to decreased efficiency, particularly when dealing with retinal-level resolutions. To tackle this challenge, we begin with the essence of the human visual system (HVS) perception and present visual acuity-consistent foveated rendering (VaFR), aiming to achieve exceptional rendering performance at retinal-level resolutions. Specifically, we propose a method with a novel log-polar mapping function derived from the human visual acuity model, which accommodates the natural bandwidth of the visual system. This mapping function and its associated shading rate guarantee a consistent output of rendering information, regardless of variations in the display resolution of the VR HMD. Consequently, our VaFR outperforms alternative methods, improving rendering speed while preserving perceptual visual quality, particularly when operating at retinal resolutions. We validate our approach using both the rasterization and ray-casting rendering pipelines. We also validate our approach using different binocular rendering strategies for HMD devices. In diverse testing scenarios, our approach delivers better perceptual visual quality than prior foveated rendering while achieving an impressive speedup of 6.5×−9.29× for deferred rendering of 3D scenarios and an even more powerful speedup of 10.4×−16.4× for ray-casting at retinal resolution. Additionally, our approach significantly enhances the rendering performance of binocular 8 K path tracing, achieving smooth frame rates.

Abstract:
Recent advancements in Graph Neural Networks (GNNs) show promise for various applications like social networks and financial networks. However, they exhibit fairness issues, particularly in human-related decision contexts, risking unfair treatment of groups historically subject to discrimination. While several visual analytics studies have explored fairness in machine learning (ML), few have tackled the particular challenges posed by GNNs. We propose a visual analytics framework for GNN fairness analysis, offering insights into how attribute and structural biases may introduce model bias. Our framework is model-agnostic and tailored for real-world scenarios with multiple and multinary sensitive attributes, utilizing an extended suite of fairness metrics. To operationalize the framework, we develop GNNFairViz, a visual analysis tool that integrates seamlessly into the GNN development workflow, offering interactive visualizations. Our tool enables GNN model developers, the target users, to analyze model bias comprehensively, facilitating node selection, fairness inspection, and diagnostics. We evaluate our approach through two usage scenarios and expert interviews, confirming its effectiveness and usability in GNN fairness analysis. Furthermore, we summarize two general insights into GNN fairness based on our observations on the usage of GNNFairViz, highlighting the prevalence of the “Overwhelming Effect” in highly unbalanced datasets and the importance of suitable GNN architecture selection for bias mitigation.

Abstract:
Real-time motion generation has garnered significant attention within the fields of computer animation and gaming. Existing methods typically realize motion control via isolated style or content labels, resulting in short, simply motion clips. In this paper, we propose a motion generation framework, called SPORT (“from zero-Shot Prompt tO Real-Time motion generation”), for generating real-time and ever-changing motions using zero-shot prompts. SPORT consists of three primary components: (1) a body-part phase autoencoder that ensures smooth transitions between diverse motions; (2) a body-part content encoder that mitigates semantic gap between texts and motions; (3) a diffusion-based decoder that accelerates the denoising process while enhancing the diversity and realism of motions. Moreover, we develop a prototype for real-time application in Unity, demonstrating that our approach effectively considering the semantic gap caused by abstract style texts and rapidly changing terrains. Through qualitative and quantitative comparisons, we show that SPORT outperforms other approaches in terms of motion quality, style diversity and inference speed.

Abstract:
Relightable 4D avatar reconstruction which enables high fidelity and real-time rendering continues to be a crucial but challenging problem, especially from monocular videos. Previous NeRF-based 4D avatars enable photo-realistic relighting but are too slow for rendering, while point-based or mesh-based 4D avatars are efficient but have limited rendering quality. The recent success of 3D Gaussian Splatting, i.e., 3DGS, has inspired a series of impressive 4D Gaussian avatars, however, most of which only focus on faithful appearance reconstruction but are not relightable. To address such issues, this article proposes a new Relightable 4D Gaussian Avatar, i.e., RGAvatar, tailored for high fidelity relightable rendering from monocular videos. Our key idea is to introduce a new relightable 4D Gaussian representation, based on which we can directly perform high fidelity Physically Based Rendering, and an effective joint learning mechanism for compact 4D Gaussian reconstruction with SDF regulation and accurate materials and lighting decomposition. By comparing with previous state-of-the-art approaches, RGAvatar can significantly outperform previous approaches in relightable rendering quality and speed. To our best knowledge, RGAvatar contributes a new state-of-the-art 4D Gaussian avatar from monocular videos, which enables high fidelity relightable rendering in a quite efficient manner.

Abstract:
3D scene stylization refers to generating stylized images of the scene at arbitrary novel view angles following a given set of style images while ensuring consistency when rendered from different views. Recently, several 3D style transfer methods leveraging the scene reconstruction capabilities of pre-trained neural radiance fields (NeRF) have been proposed. To successfully stylize a scene this way, one must first reconstruct a photo-realistic radiance field from collected images of the scene. However, when only sparse input views are available, pre-trained few-shot NeRFs often suffer from high-frequency artifacts, which are generated as a by-product of high-frequency details for improving reconstruction quality. Is it possible to generate more faithful stylized scenes from sparse inputs by directly optimizing encoding-based scene representation with target style? In this paper, we consider the stylization of sparse-view scenes in terms of disentangling content semantics and style textures. We propose a coarse-to-fine sparse-view scene stylization framework, where a novel hierarchical encoding-based neural representation is designed to generate high-quality stylized scenes directly from implicit scene representations. We also propose a new optimization strategy with content strength annealing to achieve realistic stylization and better content preservation. Extensive experiments demonstrate that our method can achieve high-quality stylization of sparse-view scenes and outperforms fine-tuning-based baselines in terms of stylization quality and efficiency.

Abstract:
In this article, we introduce MonoRelief, a novel method that combines the strengths of a depth map and a normal map to achieve high-quality relief recovery from a single image. By constructing a large-scale relief dataset that encompasses a diverse range of relief shapes, materials, and lighting conditions, we enable the training of a robust normal estimation network capable of handling various types of relief images. Furthermore, we leverage the state-of-the-art method, DepthAnything v2 (Yang et al. 2024), to generate depth maps from the input images. By integrating the strengths of both maps, MonoRelief recovers 2.5D reliefs with reasonable depth structures and intricate geometrical details. We validate the effectiveness and robustness of MonoRelief through comprehensive experiments, and showcase its potential in a variety of downstream applications, including Image-to-Relief, Text-to-Relief, Lines-to-Relief and relief reproduction.

Abstract:
Embedding visual representations within original hierarchical tables can mitigate additional cognitive load stemming from the division of users’ attention. The created hierarchical table visualizations can help users understand and explore complex data with multi-level attributes. However, because of many options available for transforming hierarchical tables and selecting subsets for embedding, the design space of hierarchical table visualizations becomes vast, and the construction process turns out to be tedious, hindering users from constructing hierarchical table visualizations with many data insights efficiently. We propose InsigHTable, a mixed-initiative and insight-driven hierarchical table transformation and visualization system. We first define data insights within hierarchical tables, which consider the hierarchical structure in the table headers. Since hierarchical table visualization construction is a sequential decision-making process, InsigHTable integrates a deep reinforcement learning framework incorporating an auxiliary rewards mechanism. This mechanism addresses the challenge of sparse rewards in constructing hierarchical table visualizations. Within the deep reinforcement learning framework, the agent continuously optimizes its decision-making process to create hierarchical table visualizations to uncover more insights by collaborating with analysts. We demonstrate the usability and effectiveness of InsigHTable through two case studies and sets of experiments. The results validate the effectiveness of the deep reinforcement learning framework and show that InsigHTable can facilitate users to construct hierarchical table visualizations and understand underlying data insights.

Abstract:
This article introduces constraint-based breakpoints, a technique for designing responsive visualizations for a wide variety of screen sizes and datasets. Breakpoints in responsive visualization define when different visualization designs are shown. Conventionally, breakpoints are static, pre-defined widths, and as such do not account for changes to the visualized dataset or visualization parameters. To guarantee readability and efficient use of space across datasets, these static breakpoints would require manual updates. Constraint-based breakpoints solve this by evaluating visualization-specific constraints on the size of visual elements, overlapping elements, and the aspect ratio of the visualization and available space. Once configured, a responsive visualization with constraint-based breakpoints can adapt to different screen sizes for any dataset. We describe a framework that guides designers in creating a stack of visualization designs for different display sizes and defining constraints for each of these designs. We demonstrate constraint-based breakpoints for different data types and their visualizations: geographic data (choropleth map, proportional circle map, Dorling cartogram, hexagonal grid map, bar chart, waffle chart), network data (node-link diagram, adjacency matrix, arc diagram), and multivariate data (scatterplot, heatmap).

Abstract:
The integration of visualizations and text is commonly found in data news, analytical reports, and interactive documents. For example, financial articles are presented along with interactive charts to show the changes in stock prices on Yahoo Finance. Visualizations enhance the perception of facts in the text while the text reveals insights of visual representation. However, effectively combining text and visualizations is challenging and tedious, which usually involves advanced programming skills. This paper proposes a semi-automatic pipeline that builds links between text and visualization. To resolve the relationship between text and visualizations, we present a method which structures a visualization and the underlying data as a contextual knowledge graph, based on which key phrases in the text are extracted, grouped, and mapped with visual elements. To support flexible customization of text-visualization links, our pipeline incorporates user knowledge to revise the links in a mixed-initiative manner. To demonstrate the usefulness and the versatility of our method, we replicate prior studies or cases in crafting interactive word-sized visualizations, annotating visualizations, and creating text-chart interactions based on a prototype system. We carry out two preliminary model tests and a user study and the results and user feedbacks suggest our method is effective.

Abstract:
The ability to manipulate and physically feel virtual objects without any real object being present and without equipping the user has been a long-standing goal in virtual reality (VR). Emerging ultrasound mid-air haptics (UMH) technology could potentially address this challenge, as it enables remote tactile stimulation of unequipped users. However, to date, UMH has received limited attention in the field of haptic exploration and manipulation in virtual environments. Existing work has primarily focused on interactions requiring a single hand and thus the delivery of unimanual haptic feedback. Despite being fundamental to a large part of haptic interactions with our environments, bimanual tasks have rarely been studied in the field of UMH interaction in VR. In this paper, we propose the use of non-coplanar mid-air haptic devices for providing simultaneous tactile feedback to both hands during bimanual VR manipulation. We discuss coupling schemes and haptic rendering algorithms for providing bimanual haptic feedback in bimanual interactions with virtual environments. We then present two human participant studies, assessing the benefits of bimanual ultrasound haptic feedback in a two-handed grasping and holding task and in a shape exploration task. Results suggest that the use of multiple non-coplanar UMH devices could be an interesting approach for enriching unencumbered haptic manipulation in virtual environments.

Abstract:
Bar graphs are routinely used in academic works, official reports, and mass media. Prior studies have focused on the comprehension of numerical information in bar graph design but have largely ignored the semantic information representation. Actually, along with the escalating need to convey semantic information beyond numerical data, unconventional bar graphs emerge and catch increasing eyes, highlighting the necessity of unlocking semantic information representation in bar graph design. In this paper, we attempt to address these gaps through examining the impact of three visual channels—color, shape, and orientation—on viewers' comprehension of semantic information. Drawing from prior research, we formulate a series of research hypotheses and conduct two experiments. Results show that by evoking sensorimotor experiences, conceptually relevant colors and shapes of bars facilitate the representation of semantic information. This facilitation is more pronounced in conveying concrete concepts than abstract concepts. Similarly, by evoking emotional experiences, colors and orientation aligned with the affective valence of concepts aid the representation of semantic information, with a more noticeable enhancement in conveying abstract concepts compared to concrete concepts. Additionally, we find that shape-embellished bars somewhat hinder the judgment of specific numerical values. These findings provide a renewed perspective on how semantic information is represented in bar graphs, offering valuable practical guidance for scientifically representing semantic information.

Abstract:
Creating fine-retouched portrait images is tedious and time-consuming even for professional artists. There exist automatic retouching methods, but they either suffer from over-smoothing artifacts or lack generalization ability. To address such issues, we present StyleRetoucher, a novel automatic portrait image retouching framework, leveraging StyleGAN's generation and generalization ability to improve an input portrait image's skin condition while preserving its facial details. Harnessing the priors of pretrained StyleGAN, our method shows superior robustness: a). performing stably with fewer training samples and b). generalizing well on the out-domain data. Moreover, by blending the spatial features of the input image and intermediate features of the StyleGAN layers, our method preserves the input characteristics to the largest extent. We further propose a novel blemish-aware feature selection mechanism to effectively identify and remove the skin blemishes, improving the image skin condition. Qualitative and quantitative evaluations validate the great generalization capability of our method. Further experiments show StyleRetoucher's superior performance to the alternative solutions in the image retouching task. We also conduct a user perceptive study to confirm the superior retouching performance of our method over the existing state-of-the-art alternatives.

Abstract:
The challenge of dynamic view synthesis from dynamic monocular videos, i.e., synthesizing novel views for free viewpoints given a monocular video of a dynamic scene captured by a moving camera, mainly lies in accurately modeling the dynamic objects of a scene using limited 2D frames, each with a varying timestamp and viewpoint. Existing methods usually require pre-processed 2D optical flow and depth maps by off-the-shelf methods to supervise the network, making them suffer from the inaccuracy of the pre-processed supervision and the ambiguity when lifting the 2D information to 3D. In this paper, we tackle this challenge in an unsupervised fashion. Specifically, we decouple the motion of the dynamic objects into object motion and camera motion, respectively regularized by proposed unsupervised surface consistency and patch-based multi-view constraints. The former enforces the 3D geometric surfaces of moving objects to be consistent over time, while the latter regularizes their appearances to be consistent across different viewpoints. Such a fine-grained motion formulation can alleviate the learning difficulty for the network, thus enabling it to produce not only novel views with higher quality but also more accurate scene flows and depth than existing methods requiring extra supervision.

Abstract:
Implicit neural representations (INRs) have emerged as a powerful tool for compressing large-scale volume data. This opens up new possibilities for in situ visualization. However, the efficient application of INRs to distributed data remains an underexplored area. In this work, we develop a distributed volumetric neural representation and optimize it for in situ visualization. Our technique eliminates data exchanges between processes, achieving state-of-the-art compression speed, quality and ratios. Our technique also enables the implementation of an efficient strategy for caching large-scale simulation data in high temporal frequencies, further facilitating the use of reactive in situ visualization in a wider range of scientific problems. We integrate this system with the Ascent infrastructure and evaluate its performance and usability using real-world simulations.

Abstract:
Systems with occlusion capabilities, such as those used in vision augmentation, image processing, and optical see-through head-mounted display (OST-HMD), have gained popularity. Achieving precise (hard-edge) occlusion in these systems is challenging, often requiring complex optical designs and bulky volumes. On the other hand, utilizing a single transparent liquid crystal display (LCD) is a simple approach to create occlusion masks. However, the generated mask will appear defocused (soft-edge) resulting in insufficient blocking or occlusion leakage. In our work, we delve into the perception of soft-edge occlusion by the human visual system and present a preference-based optimal expansion method that minimizes perceived occlusion leakage. In a user study involving 20 participants, we made a noteworthy observation that the human eye perceives a sharper edge blur of the occlusion mask when individuals see through it and gaze at a far distance, in contrast to the camera system's observation. Moreover, our study revealed significant individual differences in the perception of soft-edge masks in human vision when focusing. These differences may lead to varying degrees of demand for mask size among individuals. Our evaluation demonstrates that our method successfully accounts for individual differences and achieves optimal masking effects at arbitrary distances and pupil sizes.

Abstract:
Convolutional neural networks (CNNs) are widely used for embroidery feature synthesis from images. However, they are still unable to predict diverse stitch types, which makes it difficult for the CNNs to effectively extract stitch features. In this paper, we propose a multi-stitch embroidery generative adversarial network (MSEmbGAN) that uses a region-aware texture generation sub-network to predict diverse embroidery features from images. To the best of our knowledge, our work is the first CNN-based generative adversarial network to succeed in this task. Our region-aware texture generation sub-network detects multiple regions in the input image using a stitch classifier and generates a stitch texture for each region based on its shape features. We also propose a colorization network with a color feature extractor, which helps achieve full image color consistency by requiring the color attributes of the output to closely resemble the input image. Because of the current lack of labeled embroidery image datasets, we provide a new multi-stitch embroidery dataset that is annotated with three single-stitch types and one multi-stitch type. Our dataset, which includes more than 30K high-quality multi-stitch embroidery images, more than 13K aligned content-embroidered images, and more than 17K unaligned images, is currently the largest embroidery dataset accessible, as far as we know. Quantitative and qualitative experimental results, including a qualitative user study, show that our MSEmbGAN outperforms current state-of-the-art embroidery synthesis and style-transfer methods on all evaluation indicators.

Abstract:
Real-time subsurface scattering techniques are widely used in translucent material rendering. Among advanced methods that rely on the bidirectional scattering-surface reflectance distribution function (BSSRDF), screen space algorithms exhibit limited translucency, while existing large-distance methods are inefficient and yield poor illumination details. To address these limitations for better large-distance scattering, we develop a novel algorithm by extending the photon beam diffusion (PBD) model within the light view and screen space. Unlike surface irradiance in prior methods, we incorporate the refracted beam in the medium into real-time scattering estimation, presenting a new consideration for photon beam utilization. Concretely, we store all photon beam samples in light view textures and utilize an adaptive sampling pattern for beam sample selection in large filtering kernel sizes. This can reduce the sample count based on surface attributes. In screen space, virtual sources are derived from samples to estimate PBD contributions, with an approximation that preserves boundary conditions. To avoid possible overestimation, we implement correction factors that scale contributions, effectively aligning our results with path-tracing references. Through these reformulations, our efficient PBD generates results closest to references among existing methods. The experiments accurately represent better front-face illumination details and backlit translucency effects, while significantly accelerating performance compared to previous large-distance methods.

Abstract:
While numerous studies have explored NeRF-based novel view synthesis for dynamic humans, they often require training that exceeds several hours, limiting their practicality. Efforts to improve training efficiency have also encountered challenges because it is hard to optimize non-rigid transformations, thus leading to coarse renderings. In this work, we introduce an innovative approach for efficiently learning and integrating neural human representations. To achieve this, we propose a comprehensive utilization of the features stored in both canonical and observational spaces, facilitated through a collaborative refinement process that integrates canonical representations with observational details. Specifically, we initially propose decomposing high-dimensional multi-space feature volume into several feature planes, subsequently utilizing matrix multiplication to explicitly establish the correlations between different planes. This enables the simultaneous optimization of their counterparts across all dimensions by optimizing interpolated features, efficiently integrating associated details, and accelerating the rate of convergence. Additionally, we use the proposed collaborative refinement process to iteratively enhance the canonical representation. By integrating multi-space representations, we further facilitate the co-optimization of multiple frames’ time-dependent observations. Experiments demonstrate that our method can achieve high-quality free-viewpoint renderings within nearly 5 minutes of optimization. Compared to state-of-the-art approaches, our results show more realistic rendering details, marking a significant advancement in both performance and efficiency.

Abstract:
We present a lightweight system for reconstructing human geometry and appearance from sparse flashlight images. Our system produces detailed geometry including garment wrinkles and surface reflectance, which are exportable for direct rendering and relighting in traditional graphics pipelines. By capturing multi-view flashlight images using a consumer camera equipped with an co-located LED (e.g., a cell phone), we obtain view-specific shading cues that aid in the determination of surface orientation and help disambiguate between shading and material. To enable the reconstruction of geometry and appearance from sparse-view flashlight images, we integrate a pre-trained model into a differentiable physics-based rendering framework. As the learned image features from synthetic data cannot accurately reflect the shading features on real images, which is crucial for the high-quality reconstruction of geometry details and appearance, we propose to jointly optimize the image feature extractor with two MLPs for SDF and BRDF prediction using the differentiable physics-based rendering. Compared with existing methods for relightable human reconstruction, our system is able to produce high-fidelity 3D human models with more accurate geometry and appearance under the same condition. Our code and data are available at http://github.com/Jarvisss/Relightable_human_recon.

Abstract:
Creating data videos that effectively narrate stories with animated visuals requires substantial effort and expertise. A promising research trend is leveraging the easy-to-use natural language (NL) interaction to automatically synthesize data video components from narrative content like text narrations, or NL commands that specify user-required designs. Nevertheless, previous research has overlooked the integration of narrative content and specific design authoring commands, leading to generated results that lack customization or fail to seamlessly fit into the narrative context. To address these issues, we introduce a novel paradigm for creating data videos, which seamlessly integrates users’ authoring and narrative intents in a unified format called annotated narration, allowing users to incorporate NL commands for design authoring as inline annotations within the narration text. Informed by a formative study on users’ preference for annotated narration, we develop a prototype system named Data Playwright that embodies this paradigm for effective creation of data videos. Within Data Playwright, users can write annotated narration based on uploaded visualizations. The system’s interpreter automatically understands users’ inputs and synthesizes data videos with narration-animation interplay, powered by large language models. Finally, users can preview and fine-tune the video. A user study demonstrated that participants can effectively create data videos with Data Playwright by effortlessly articulating their desired outcomes through annotated narration.

Abstract:
Contour maps are an essential tool for exploring spatial features of the terrain, such as distance, directions, and surface gradient among the contour areas. User interactions in contour-based visualizations create approaches to visual analysis that are noticeably different from the perspective of human cognition. As such, various interactive approaches have been introduced to improve system usability and enhance human cognition for complex and large-scale spatial data exploration. However, what user interaction means for contour maps, its purpose, when to leverage, and design primitives have yet to be investigated in the context of analysis tasks. Therefore, further research is needed to better understand and quantify the potentials and benefits offered by user interactions in contour-based geospatial visualizations designed to support analytical tasks. In this article, we present a contour-based interactive geospatial visualization designed for analytical tasks. We conducted a crowd-sourced user study (N=62) to examine the impact of interactive features on analysis using contour-based geospatial visualizations. Our results show that the interactive features aid in their data analysis and understanding in terms of spatial data extent, map layout, task complexity, and user expertise. Finally, we discuss our findings in-depth, which will serve as guidelines for future design and implementation of interactive features in support of case-specific analytical tasks on contour-based geospatial views.

Abstract:
Novel view synthesis for dynamic 3D scenes poses a significant challenge. Many notable efforts use NeRF-based approaches to address this task and yield impressive results. However, these methods rely heavily on sufficient motion parallax in the input images or videos. When the camera motion range becomes limited or even stationary (i.e., small camera motion), existing methods encounter two primary challenges: incorrect representation of scene geometry and inaccurate estimation of camera parameters. These challenges make prior methods struggle to produce satisfactory results or even become ineffective. To address the first challenge, we propose a novel Distribution-based Depth Regularization (DDR) that ensures the rendering weight distribution to align with the true distribution. Specifically, unlike previous methods that use depth loss to calculate the error of the expectation, we calculate the expectation of the error by using Gumbel-softmax to differentiably sample points from discrete rendering weight distribution. Additionally, we introduce constraints that enforce the volume density of spatial points before the object boundary along the ray to be near zero, ensuring that our model learns the correct geometry of the scene. To demystify the DDR, we further propose a visualization tool that enables observing the scene geometry representation at the rendering weight level. For the second challenge, we incorporate camera parameter learning during training to enhance the robustness of our model to camera parameters. We conduct extensive experiments to demonstrate the effectiveness of our approach in representing scenes with small camera motion input, and our results compare favorably to state-of-the-art methods.

Abstract:
Sketch-based modeling studies reconstructing models from sketches automatically, allowing users visualize design concepts rapidly. Generating CAD models based on user sketches helps reduce the learning curve for novice users, which promotes the everyday use of CAD software, and expands its reach to non-professional groups. While various algorithms study automatically generating models from single sketch or line drawing, they often produce non-editable models or editable models limited to simple extrusion operations. To improve this issue, we propose a novel sketch-based modeling system, Sketch2Seq, which generates complex, semantic, and editable CAD models. Our system eliminates the need for additional annotations from users and produces models that support subsequent application in commercial software. The core of our method lies in understanding users’ design intent from CAD sketches. We design a novel sketch segmentation network for identifying diverse operation features in CAD sketches, which utilizes geometric features of strokes and different levels of topological connections. Additionally, to tackle the segmentation task, a dataset for CAD sketch segmentation is introduced. Comparative experiments and ablation evaluations prove the effectiveness of the proposed method. Based on segmentation result, coarse CAD sequences are generated and progressively executed. Meanwhile, the orders and parameters of the CAD sequences are optimized with context models and input sketches. All algorithms are integrated into a user interface. Experiments and evaluations validate the feasibility and superiority of our entire system which is able to reconstruct more complex features and achieve better results for longer sequence.

Abstract:
Physically-based-rendering (PBR) materials are crucial in modern rendering pipelines, and many studies have focused on acquiring these materials from reality or images. However, existing methods may result in non-tileable results, since the realistic inputs usually have seams. Compared to non-tileable materials, tileable PBR materials have more universal application scenarios. To address this issue, we introduce MaTi, a novel pipeline that converts non-tileable PBR materials into tileable ones with minimal distortion. MaTi rearranges material patches to align boundaries at the center of the image, and then uses a diffusion model to inpaint the seams. We use scaled gamma correction to reduce the occurrence of collapse when processing special material maps. The color correction and triangular blending are adopt to preserve the original material information. Additionally, we design a division and blending strategy to efficiently handle high resolution materials. Our experiments demonstrate that MaTi can seamlessly modify PBR materials while preserving the original information, outperforming existing synthesis methods.

Abstract:
With the increasing popularity and accessibility of high dynamic range (HDR) photography, tone mapping operators (TMOs) for dynamic range compression are practically demanding. In this paper, we develop a two-stage neural network-based TMO that is self-calibrated and perceptually optimized. In Stage one, motivated by the physiology of the early stages of the human visual system, we first decompose an HDR image into a normalized Laplacian pyramid. We then use two lightweight deep neural networks, taking the normalized representation as input and estimating the Laplacian pyramid of the corresponding LDR image. We optimize the tone mapping network by minimizing the normalized Laplacian pyramid distance, a perceptual metric aligning with human judgments of tone-mapped image quality. In Stage two, we input the same HDR image—self-calibrated to different maximum luminance levels—into the learned tone mapping network, and generate a pseudo-multi-exposure image stack with varying detail visibility and color saturation. We then train another fusion network to merge the LDR image stack into a desired LDR image by maximizing a variant of the structural similarity index for multi-exposure image fusion, proven perceptually relevant to fused image quality. Extensive experiments show that our method produces images with consistently better visual quality while ranking among the fastest local TMOs.

Abstract:
Financial cluster analysis allows investors to discover investment alternatives and avoid undertaking excessive risks. However, this analytical task faces substantial challenges arising from many pairwise comparisons, the dynamic correlations across time spans, and the ambiguity in deriving implications from business relational knowledge. We propose Prismatic, a visual analytics system that integrates quantitative analysis of historical performance and qualitative analysis of business relational knowledge to cluster correlated businesses interactively. Prismatic features three clustering processes: dynamic cluster generation, knowledge-based cluster exploration, and correlation-based cluster validation. Utilizing a multi-view clustering approach, it enriches data-driven clusters with knowledge-driven similarity, providing a nuanced understanding of business correlations. Through well-coordinated visual views, Prismatic facilitates a comprehensive interpretation of intertwined quantitative and qualitative features, demonstrating its usefulness and effectiveness via case studies on formulating concept stocks and extensive interviews with domain experts.

Abstract:
This paper proposes a novel approach for the low-error reconstruction of directional functions with spherical harmonics. We introduce a modified version of Spherical Gaussians with adaptive narrowness and amplitude to represent the input data in an intermediate form. This representation is then projected into spherical harmonics using a closed-form analytical solution. Because of the spectral properties of the proposed representation, the amount of ringing artifacts is reduced, and the overall precision of the reconstructed function is improved. The proposed method is more precise comparing to existing methods. The presented solution can be used in several graphical applications, as discussed in this paper. For example, the method is suitable for sparse models such as indirect illumination or reflectance functions.

Abstract:
Creating relightable and animatable avatars from multi-view or monocular videos is a challenging task for digital human creation and virtual reality applications. Previous methods rely on neural radiance fields or ray tracing, resulting in slow training and rendering processes. By utilizing Gaussian Splatting, we propose a simple and efficient method to decouple body materials and lighting from sparse-view or monocular avatar videos, so that the avatar can be rendered simultaneously under novel viewpoints, poses, and lightings at interactive frame rates (6.9 fps). Specifically, we first obtain the canonical body mesh using a signed distance function and assign attributes to each mesh vertex. The Gaussians in the canonical space then interpolate from nearby body mesh vertices to obtain the attributes. We subsequently deform the Gaussians to the posed space using forward skinning, and combine the learnable environment light with the Gaussian attributes for shading computation. To achieve fast shadow modeling, we rasterize the posed body mesh from dense viewpoints to obtain the visibility. Our approach is not only simple but also fast enough to allow interactive rendering of avatar animation under environmental light changes. Experiments demonstrate that, compared to previous works, our method can render higher quality results at a faster speed on both synthetic and real datasets.

Abstract:
Speech-driven facial animation aims to synthesize lip-synchronized 3D talking faces following the given speech signal. Prior methods to this task mostly focus on pursuing realism with deterministic systems, yet characterizing the potentially stochastic nature of facial motions has been to date rarely studied. While generative modeling approaches can easily handle the one-to-many mapping by repeatedly drawing samples, ensuring a diverse mode coverage of plausible facial motions on small-scale datasets remains challenging and less explored. In this article, we propose predicting multiple samples conditioned on the same audio signal and then explicitly encouraging sample diversity to address diverse facial animation synthesis. Our core insight is to guide our model to explore the expressive facial latent space with a diversity-promoting loss such that the desired latent codes for diversification can be ideally identified. To this end, building upon the rich facial prior learned with vector-quantized variational auto-encoding mechanism, our model temporally queries multiple stochastic codes which can be flexibly decoded into a diverse yet plausible set of speech-faithful facial motions. To further allow for control over different facial parts during generation, the proposed model is designed to predict different facial portions of interest in a sequential manner, and compose them to eventually form full-face motions. Our paradigm realizes both diverse and controllable facial animation synthesis in a unified formulation. We experimentally demonstrate that our method yields state-of-the-art performance both quantitatively and qualitatively, especially regarding sample diversity.

Abstract:
Concept decomposition is a technique for personalized text-to-image synthesis which learns textual embeddings of subconcepts from images that depicting an original concept. The learned subconcepts can then be composed to create new images. However, existing methods fail to address the issue of contextual conflicts when subconcepts from different sources are combined because contextual information remains encapsulated within the subconcept embeddings. To tackle this problem, we propose a Context-aware Concept Decomposition (C^2DC2D) framework. Specifically, we introduce a Similarity-Guided Divergent Embedding (SGDE) method to obtain subconcept embeddings. Then, we eliminate the latent contextual dependence between the subconcept embeddings and reconstruct the contextual information using an independent contextual embedding. This independent context can be combined with various subconcepts, enabling more controllable text-to-image synthesis based on subconcept recombination. Extensive experimental results demonstrate that our method outperforms existing approaches in both image quality and contextual consistency.

Abstract:
We present VOICE, a novel approach to science communication that connects large language models’ conversational capabilities with interactive exploratory visualization. VOICE introduces several innovative technical contributions that drive our conversational visualization framework. Based on the collected design requirements, we introduce a two-layer agent architecture that can perform task assignment, instruction extraction, and coherent content generation. We employ fine-tuning and prompt engineering techniques to tailor agents’ performance to their specific roles and accurately respond to user queries. Our interactive text-to-visualization method generates a flythrough sequence matching the content explanation. In addition, natural language interaction provides capabilities to navigate and manipulate 3D models in real-time. The VOICE framework can receive arbitrary voice commands from the user and respond verbally, tightly coupled with a corresponding visual representation, with low latency and high accuracy. We demonstrate the effectiveness of our approach by implementing a proof-of-concept prototype and applying it to the molecular visualization domain: analyzing three 3D molecular models with multiscale and multi-instance attributes. Finally, we conduct a comprehensive evaluation of the system, including quantitative and qualitative analyses on our collected dataset, along with a detailed public user study and expert interviews. The results confirm that our framework and prototype effectively meet the design requirements and cater to the needs of diverse target users.

Abstract:
Understanding the evolution of scholarly research is essential for many real-life decision-making processes in academia, such as research planning, frontier exploration, and award selection. Popular platforms like Google Scholar and Web of Science rely on numerical indicators that are too abstract to convey the context and content of scientific research, while most existing visualization approaches on mapping science do not consider the presentation of individual scholars’ research evolution using curated self-citation data. This paper builds on our previous work and proposes an integrated pipeline to visualize a scholar’s research evolution from multiple topic facets. A novel 3D prism-shaped visual metaphor is introduced as the overview of a scholar’s research profile, whilst their scientific evolution on each topic is displayed in a more structured manner. Additional designs by topic chord diagram, streamgraph visualization, and inter-topic flow map, optimized by an elaborate layout algorithm, assist in perceiving the scholar’s scientific evolution across topics. A new six-degree-impact glyph metaphor highlights key interdisciplinary works driving the evolution. The proposed visualization methods are evaluated through case studies analyzing the careers of prestigious Turing award laureates, one major visualization venue, and a focused user study.

Abstract:
Point cloud sampling plays a crucial role in reducing computation costs and storage requirements for various vision tasks. Traditional sampling methods, such as farthest point sampling, lack task-specific information and, as a result, cannot guarantee optimal performance in specific applications. Learning-based methods train a network to sample the point cloud for the targeted downstream task. However, they do not guarantee that the sampled points are the most relevant ones. Moreover, they may result in duplicate sampled points, which requires completion of the sampled point cloud through post-processing techniques. To address these limitations, we propose a contribution-based sampling network (CS-Net), where the sampling operation is formulated as a Top-kk operation. To ensure that the network can be trained in an end-to-end way using gradient descent algorithms, we use a differentiable approximation to the Top-kk operation via entropy regularization of an optimal transport problem. Our network consists of a feature embedding module, a cascade attention module, and a contribution scoring module. The feature embedding module includes a specifically designed spatial pooling layer to reduce parameters while preserving important features. The cascade attention module combines the outputs of three skip connected offset attention layers to emphasize the attractive features and suppress less important ones. The contribution scoring module generates a contribution score for each point and guides the sampling process to prioritize the most important ones. Experiments on the ModelNet40 and PU147 showed that CS-Net achieved state-of-the-art performance in two semantic-based downstream tasks (classification and registration) and two reconstruction-based tasks (compression and surface reconstruction). CS-Net also achieved high average precision for objection detection on the KITTI LiDAR point cloud dataset, demonstrating its effectiveness in three-dimensional object detection.

Abstract:
3D reconstruction from multi-view images is a long-standing problem in computer graphic. Neural 3D reconstruction, especially NeuS and its variants, has improved reconstruction quality compared to traditional methods. However, it is still a challenge for these methods to reconstruct fine-grained geometric details since the spherical harmonic positional encoding lacks the ability to express high-frequency signals. In this paper, we propose a multi-resolution tri-plane feature encoding that leverages the detail reconstruction capabilities of high-resolution tri-plane while using the smoothness of low-resolution tri-plane to suppress high-frequency artifacts. Additionally, a progressive training strategy is introduced, gradually merging scene details from coarse to fine granularity, enhancing reconstruction quality while maintaining training stability and reducing difficulty. Furthermore, to address reconstruction challenges arising from sparse viewpoints and inconsistent lighting in image datasets, we introduce normal priors as supervision and propose consistency verification for multi-view normal priors, which assesses the accuracy of normal priors and effectively supervise the reconstructed surfaces. Moreover, we propose a perturbing and fine-tuning strategy on regions of unreliable normal priors to further improve the quality of geometric surface reconstruction.

Abstract:
AI-based text-to-image models do not only excel at generating realistic images, they also give designers more and more fine-grained control over the image content. Consequently, these approaches have gathered increased attention within the computer graphics research community, which has been historically devoted towards traditional rendering techniques, that offer precise control over scene parameters (e.g., objects, materials, and lighting). While the quality of conventionally rendered images is assessed through well established image quality metrics, such as SSIM or PSNR, the unique challenges of text-to-image generation require other, dedicated quality metrics. These metrics must be able to not only measure overall image quality, but also how well images reflect given text prompts, whereby the control of scene and rendering parameters is interweaved. Within this survey, we provide a comprehensive overview of such text-to-image quality metrics, and propose a taxonomy to categorize these metrics. Our taxonomy is grounded in the assumption, that there are two main quality criteria, namely compositional quality and general quality, that contribute to the overall image quality. Besides the metrics, this survey covers dedicated text-to-image benchmark datasets, over which the metrics are frequently computed. Finally, we identify limitations and open challenges in the field of text-to-image generation, and derive guidelines for practitioners conducting text-to-image evaluation.

Abstract:
Single-view 3D object reconstruction (SVOR) aims to recover the 3D shape of an object from a single 2D image. Despite advances in deep learning (DL), challenges such as incomplete image information, scarce 3D data annotation, and highly variable object shapes still limit the performance of SVOR. Meanwhile, with the rapid development of novel view synthesis (NVS) techniques, the SVOR field has received significant advancements. However, existing reviews have not comprehensively covered the rapid developments in NVS-based approaches. This article aims to fill this gap by highlighting the latest progress in SVOR, particularly advancements related to NVS-based methods. Additionally, we observed discrepancies between existing quality evaluation metrics in SVOR and human visual perception. This is because some critical object parts are essential to consider during the evaluation. For example, when reconstructing airplanes, critical parts like the empennage and wings are often overlooked in evaluation metrics due to their smaller size compared to the fuselage. Consequently, poor reconstruction of these parts may not significantly affect overall evaluation scores. To address this issue, we propose a more comprehensive evaluation method that reflects human visual perception accurately. To achieve this, we introduce a weighted evaluation method that considers part saliency and proposes a novel technique for automatically perceiving reconstruction discrepancies. This study effectively enhances the accuracy and consistency of evaluations through these approaches, offering new insights and methodologies, filling a void in the existing literature, and providing valuable contributions to both research and practical applications in SVOR.

Abstract:
Point cloud completion aims to infer a complete shape from its partial observation. Many approaches utilize a pure encoder-decoder paradigm in which complete shape can be directly predicted by shape priors learned from partial scans, however, these methods suffer from the loss of details inevitably due to the feature abstraction issues. In this paper, we propose a novel framework, termed SPAC-Net, that aims to rethink the completion task under the guidance of a new structural prior, we call it interface. Specifically, our method first investigates Marginal Detector (MAD) module to localize the interface, defined as the intersection between the known observation and the missing parts. Based on the interface, our method predicts the coarse shape by learning the displacement from the points in interface move to their corresponding position in missing parts. Furthermore, we devise an additional Structure Supplement (SSP) module before the upsampling stage to enhance the structural details of the coarse shape, enabling the upsampling module to focus more on the upsampling task. Extensive experiments have been conducted on several challenging benchmarks, and the results demonstrate that our method outperforms existing state-of-the-art approaches.

Abstract:
We propose a novel framework to automatically learn to aggregate and transform photometric measurements from multiple unstructured views into spatially distinctive and view-invariant low-level features, which are subsequently fed to a multi-view stereo pipeline to enhance 3D reconstruction. The illumination conditions during acquisition and the feature transform are jointly trained on a large amount of synthetic data. We further build a system to reconstruct both the geometry and anisotropic reflectance of a variety of challenging objects from hand-held scans. The effectiveness of the system is demonstrated with a lightweight prototype, consisting of a camera and an array of LEDs, as well as an off-the-shelf tablet. Our results are validated against reconstructions from a professional 3D scanner and photographs, and compare favorably with state-of-the-art techniques.

Abstract:
For single-view clothed human reconstruction, the fashionable PIFu-like framework depends on the pixel-aligned feature essentially, while this leads to depth ambiguity and inaccuracy of representation. Additionally, this task faces the inherent problem of the lack of invisible information. To solve these two problems, we propose depth-guided pixel-aligned feature and multi-view consistency prior to constrain representation learning of the single-view reconstruction task. The difference, between the depth values of points and estimated depth map, is used to filter pixel-aligned features. Thus, the image encoder can focus on capturing the feature of the visible part which is more effective in feature representation. The method introduces contrastive learning and masked autoencoder to achieve consistency of SMPL vertex features in each view which helps model to imagine invisible information. The experimental results show that the proposed method enables the feature to represent surface details more efficiently, thus achieves more reasonable and accurate representation learning. The qualitative and quantitative evaluations on the public and commercial datasets show that the proposed method can achieve better performance than previous implicit representation based methods.

Abstract:
Although pre-trained large-scale generative models StyleGAN series have proven to be effective in various editing and translation tasks, they are limited to pre-defined fixed aspect ratio. To overcome this limitation, we propose StyleGAN-\infty∞, a model that enables pre-trained StyleGAN to perform arbitrary-ratio conditional synthesis. Our key insight is to distill the expressive StyleGAN features into a StyleBook, such that an arbitrary-ratio condition can be translated to other forms by properly assembling pre-defined StyleBook vectors. To learn and leverage the StyleBook, we employ a network with three distinct stages, each corresponding to StyleBook extraction, StyleBook correspondence learning, and arbitrary-ratio synthesis. Extensive experiments on various conditional synthesis tasks, like super-resolution, sketch synthesis, and semantic synthesis, demonstrate superior performances over state-of-the-art image-to-image translation methods. Moreover, our model can easily generate megapixel images in diverse modalities by taking advantage of different pre-trained StyleGAN models.

Abstract:
The digital construction of cultural heritage promotes communication and sharing of digital cultural resources across time and space. Digital storage serves as the foundation for the digital construction of cultural artifacts. In the digital storage of Dunhuang murals, image stitching plays a critical role in restoring the complete image of the cave murals. Traditional image stitching methods are constrained by the detection accuracy of feature points and are not fit for stitching low-texture murals. Despite deep learning-based image stitching methods, parallax misalignment and ghosting are still prevalent issues. For this reason, we perform the first Dunhuang mural stitching based on deep learning in this paper. This is in response to the need for digitizing and storing Dunhuang murals. Two mural stitching datasets are constructed, and we design a progressive regression image alignment network and a feature differential reconstruction soft-coded seam stitching network. We also introduce a soft-coded seam quality evaluation method. The algorithm presented in this paper achieves state-of-the-art alignment and stitching performance in the mural stitching task through unsupervised learning with a smaller number of model parameters, which provides technical support for the digitization and preservation of Dunhuang murals.

Abstract:
Neural radiance fields (NeRF) have demonstrated impressive performance in novel view synthesis, but are still slow to render complex scenes at a high resolution. We introduce a novel method to boost the NeRF rendering speed by utilizing the temporal coherence between consecutive frames. Rather than computing features of each frame entirely from scratch, we reuse the coherent information (e.g., density and color) computed from the previous frames to help render the current frame, which significantly boosts rendering speed. To effectively manage the coherent information of previous frames, we introduce a history buffer with a multiple-plane structure, which is built online and updated from old frames to new frames. We name this buffer as multiple plane buffer (MPB). With this MPB, a new frame can be efficiently rendered using the warped features from previous frames. Extensive experiments on the NeRF-Synthetic, LLFF, and Mip-NeRF-360 datasets demonstrate that our method significantly boosts rendering efficiency and achieves 4× speedup on real-world scenes compared to the baseline methods while preserving competitive rendering quality.

Abstract:
Creating lifelike 3D head avatars and generating compelling animations for diverse subjects remain challenging in computer vision. This paper presents GaussianHead, which models the active head based on anisotropic 3D Gaussians. Our method integrates a motion deformation field and a single-resolution tri-plane to capture the head's intricate dynamics and detailed texture. Notably, we introduce a customized derivation scheme for each 3D Gaussian, facilitating the generation of multiple “doppelgangers” through learnable parameters for precise position transformation. This approach enables efficient representation of diverse Gaussian attributes and ensures their precision. Additionally, we propose an inherited derivation strategy for newly added Gaussians to expedite training. Extensive experiments demonstrate GaussianHead's efficacy, achieving high-fidelity visual results with a remarkably compact model size (\approx 12≈12 MB). Our method outperforms state-of-the-art alternatives in tasks such as reconstruction, cross-identity reenactment, and novel view synthesis.

Abstract:
Smart contracts are the fundamental components of blockchain technology. They are programs to determine cryptocurrency transactions, and are irreversible once deployed, making it crucial for cryptocurrency investors to understand the cryptocurrency transaction behaviors of smart contracts comprehensively. However, it is a challenging (if not impossible) task for investors, as they do not necessarily have a programming background to check the complex source code. Even for investors with certain programming skills, inferring all the potential behaviors from the code alone is still difficult, since the actual behaviors can be different when different investors are involved. To address this challenge, we propose PrettiSmart, a novel visualization approach via execution simulation to achieve intuitive and reliable visual interpretation of smart contracts. Specifically, we develop a simulator to comprehensively capture most of the possible real-world smart contract behaviors, involving multiple investors and various smart contract functions. Then, we present PrettiSmart to intuitively visualize the simulation results of a smart contract, which consists of two modules: The Simulation Overview Module is a barcode-based design, providing a visual summary for each simulation, and the Simulation Detail Module is an augmented sequential design to display the cryptocurrency transaction details in each simulation, such as function call sequences, cryptocurrency flows, and state variable changes. It can allow investors to intuitively inspect and understand how a smart contract will work. We evaluate PrettiSmart through two case studies and in-depth user interviews with 12 investors. The results demonstrate the effectiveness and usability of PrettiSmart in facilitating an easy interpretation of smart contracts.

Abstract:
Achieving a high level of immersion and adaptation in virtual reality (VR) requires precise measurement and representation of user state. While extrinsic physical characteristics such as locomotion and pose can be accurately tracked in real-time, reliably capturing mental states is more challenging. Quantitative psychology allows considering more intrinsic features like emotion, attention, or cognitive load. Time perception, in particular, is strongly tied to users' mental states, including stress, focus, and boredom. However, research on objectively measuring the pace at which we perceive the passage of time is scarce. In this work, we investigate the potential of electroencephalography (EEG) as an objective measure of time perception in VR, exploring neural correlates with oscillatory responses and time-frequency analysis. To this end, we implemented a variety of time perception modulators in VR, collected EEG recordings, and labeled them with overestimation, correct estimation, and underestimation time perception states. We found clear EEG spectral signatures for these three states, that are persistent across individuals, modulators, and modulation duration. These signatures can be integrated and applied to monitor and actively influence time perception in VR, allowing the virtual environment to be purposefully adapted to the individual to increase immersion further and improve user experience. A free copy of this paper and all supplemental materials are available at https://vrarlab.uni.lu/pub/brain-signatures.

Abstract:
Foveated rendering reduces computational load by distributing resources based on the human visual system. This enables the implementation of ray tracing in virtual reality applications, where a high frame rate is essential to achieve visual immersion. However, traditional foveation methods based solely on eccentricity cannot adequately account for the complex behavior of visual attention. This is one of the main reasons that leads to lower perceived quality compared to non-foveated techniques. In this study, we introduce a novel rendering pipeline that incorporates ocular attention through the use of visual saliency. Based on foveation saliency, our approach facilitates the real-time production of high-quality images utilizing path tracing by distributing samples according to saliency metrics derived from geometric and historical data. To further augment image quality, an adaptive filtering process, aligned with the saliency metrics, is employed to reduce visible artifacts in non-foveal regions. Our experiments prove that this novel approach can demonstrate superior performance compared to previous methods, both in terms of quantitative metrics and perceived visual quality.

Abstract:
When several people are co-located and immersed in a mixed reality environment, they may feel like they share the virtual environment or not. This feeling of copresence, along with its parent dimensions of social presence and presence, has been mostly studied by relying on subjective measures gathered through questionnaires. As a way to address the drawbacks of this approach, we introduce a protocol to gather behavioral measures in the context of co-located mixed reality. As a pair of participants avoid obstacles moving towards them, their errors, gaze, interpersonal distance, and timing are measured. By combining subjective measures gathered through a questionnaire drawing from previous studies on social presence with behavioral measures, we demonstrate new ways to assess how users experience copresence. We illustrate this protocol by evaluating the effect of visual feedback on collaborators' activity. The results of this experiment suggest the capability of our protocol by revealing the effect of visual feedback on both objective and subjective measures.

Abstract:
Rendering quality and performance greatly affect the user's immersion in VR experiences. 3D Gaussian Splatting-based methods can achieve photo-realistic rendering with speeds of over 100 fps in static scenes, but the speed drops below 10 fps in monocular dynamic scenes. Foveated rendering provides a possible solution to accelerate rendering without compromising visual perceptual quality. However, 3DGS and foveated rendering are not compatible. In this paper, we propose Fov-GS, a foveated 3D Gaussian splatting method for rendering dynamic scenes in real time. We introduce a 3D Gaussian forest representation that represents the scene as a forest. To construct the 3D Gaussian forest, we propose a 3D Gaussian forest initialization method based on dynamic-static separation. Subsequently, we propose a 3D Gaussian forest optimization method based on deformation field and Gaussian decomposition to optimize the forest and deformation field. To achieve real-time dynamic scene rendering, we present a 3D Gaussian forest rendering method based on HVS models. Experiments demonstrate that our method not only achieves higher rendering quality in the foveal and salient regions compared to the SOTA methods but also dramatically improves rendering performance, achieving up to 11.33X speedup. We also conducted a user study, and the results prove that the perceptual quality of our method has a high visual similarity with the ground truth.

Abstract:
Accurate reconstruction of heterogeneous scenes for high-fidelity rendering in an efficient manner remains a crucial but challenging task in many Virtual Reality and Augmented Reality applications. The recent 3D Gaussian Splatting (3DGS) has shown impressive quality in scene rendering with real-time performance. However, for heterogeneous scenes with many weak-textured regions, the original 3DGS can easily produce numerously wrong floaters with unbalanced reconstruction using redundant 3D Gaussians, which often leads to unsatisfied scene rendering. This paper proposes a novel multi-plane Gaussian Splatting (MPGS), which aims to achieve high-fidelity rendering with compact reconstruction for heterogeneous scenes. The key insight of our MPGS is the introduction of a novel multi-plane Gaussian optimization strategy, which effectively adjusts the Gaussian distribution for both rich-textured and weak-textured regions in heterogeneous scenes. Moreover, we further propose a multi-scale geometric correction mechanism to effectively mitigate degradation of the 3D Gaussian distribution for compact scene reconstruction. Besides, we regularize the Gaussian distributions using normal information extracted from the compact scene learning. Experimental results on public datasets demonstrate that the proposed MPGS achieves much better rendering quality compared to previous methods, while using less storage and offering more efficient rendering. To our best knowledge, MPGS is a new state-of-the-art 3D Gaussian splatting method for compact reconstruction of heterogeneous scenes, enabling high-fidelity rendering in novel view synthesis, especially improving rendering quality for weak-textured regions. The code will be released at https://github.com/wanglids/MPGS.

Abstract:
We introduce multimodal neural acoustic fields for synthesizing spatial sound and enabling the creation of immersive auditory experiences from novel viewpoints and in completely unseen new environments, both virtual and real. Extending the concept of neural radiance fields to acoustics, we develop a neural network-based model that maps an environment's geometric and visual features to its audio characteristics. Specifically, we introduce a novel hybrid transformer-convolutional neural network to accomplish two core tasks: capturing the reverberation characteristics of a scene from audio-visual data, and generating spatial sound in an unseen new environment from signals recorded at sparse positions and orientations within the original scene. By learning to represent spatial acoustics in a given environment, our approach enables creation of realistic immersive auditory experiences, thereby enhancing the sense of presence in augmented and virtual reality applications. We validate the proposed approach on both synthetic and real-world visual-acoustic data and demonstrate that our method produces nonlinear acoustic effects such as reverberations, and improves spatial audio quality compared to existing methods. Furthermore, we also conduct subjective user studies and demonstrate that the proposed framework significantly improves audio perception in immersive mixed reality applications.

Abstract:
3D Gaussian splatting has recently achieved remarkable progress in dynamic scene reconstruction. However, there remain two practical challenges: (1) Existing methods typically employ a strict point-wise deformation structure to model dynamic attributes, while neglecting the uncertain motion correlation in local space, leading to inferior adaptability to complex scenes. (2) The inherent low-frequency bias properties of Gaussians often lead to blurring artifacts due to the insufficient high-frequency learning of variable motions. To address these challenges, we propose a novel Frequency-aware Uncertainty Gaussian Splatting, termed FUGS, for adaptively reconstructing dynamic scenes in the Fourier space. Specifically, we design an Uncertainty-aware Deformation Model (UDM) that explicitly models motion attributes using learnable uncertainty relations with neighboring Gaussian points. Such a paradigm is capable of facilitating temporal and spatial motion correlation learning, thereby enabling flexible Gaussian deformations. Subsequently, a Dynamic Spectrum Regularization (DSR) is developed to perform coarse-to-fine Gaussian densification through low-to-high frequency filtering. By weighting the gradient with frequency distance, the Gaussian attribute is adaptively adjusted according to the scene complexity. Benefiting from the flexible optimization, our method achieves high-fidelity reconstruction of complex scenes while enjoying real-time rendering. Extensive experiments on synthetic and real-world datasets show that our FUGS exhibits significant superiority over state-of-the-art methods. The code will be available at https://github.com/KevinJoee/GS.

Abstract:
The Solvent-Excluded Surface (SES) is an essential representation of molecules which is massively used in molecular modeling and drug discovery since it represents the interacting surface between molecules. Based on its properties, it supports the visualization of both large scale shapes and details of molecules. While several methods targeted its computation, the ability to process large molecular structures to address the introduction of big complex analysis while leveraging the massively parallel architecture of GPUs has remained a challenge. This is mostly caused by the need for consequent memory allocation or by the complexity of the parallelization of its processing. In this paper, we leverage the last theoretical advances made for the depiction of the SES to provide fast analytical computation with low impact on memory. We show that our method is able to compute the complete surface while handling large molecular complexes with competitive computation time costs compared to previous works.

Abstract:
Time-stamped event sequences (TSEQs) are time-oriented data without value information, shifting the focus of users to the exploration of temporal event occurrences. TSEQs exist in application domains, such as sleeping behavior, earthquake aftershocks, and stock market crashes. Domain experts face four challenges, for which they could use interactive and visual data analysis methods. First, TSEQs can be large with respect to both the number of sequences and events, often leading to millions of events. Second, domain experts need validated metrics and features to identify interesting patterns. Third, after identifying interesting patterns, domain experts contextualize the patterns to foster sensemaking. Finally, domain experts seek to reduce data complexity by data simplification and machine learning support. We present IVESA, a visual analytics approach for TSEQs. It supports the analysis of TSEQs at the granularities of sequences and events, supported with metrics and feature analysis tools. IVESA has multiple linked views that support overview, sort+filter, comparison, details-on-demand, and metadata relation-seeking tasks, as well as data simplification through feature analysis, interactive clustering, filtering, and motif detection and simplification. We evaluated IVESA with three case studies and a user study with six domain experts working with six different datasets and applications. Results demonstrate the usability and generalizability of IVESA across applications and cases that had up to 1,000,000 events.

Abstract:
The reset technique of Redirected Walking (RDW) forcibly reorients the user's direction overtly to avoid collisions with boundaries, obstacles, or other users in the physical space. However, excessive resetting can decrease the user's sense of immersion and presence. Several RDW studies have been conducted to address this issue. Among them, much research has been done on reset techniques that reduce the number of resets by devising reset direction rules or optimizing them for a given environment. However, existing optimization studies on reset techniques have mainly focused on a single-user environment. In a multi-user environment, the dynamic movement of other users and static obstacles in the physical space increase the possibility of resetting. In this study, we propose Multi-Agent Reinforcement Resetter (MARR), which resets the user taking into account both physical obstacles and multi-user movement to minimize the number of resets. MARR is trained using multi-agent reinforcement learning to determine the optimal reset direction in different environments. This approach allows MARR to effectively account for different environmental contexts, including arbitrary physical obstacles and the dynamic movements of other users in the same physical space. We compared MARR to other reset technologies through simulation tests and user studies, and found that MARR outperformed the existing methods. MARR improved performance by learning the optimal reset direction for each subtle technique used in training. MARR has the potential to be applied to new subtle techniques proposed in the future. Overall, our study confirmed that MARR is an effective reset technique in multi-user environments.

Abstract:
There has been a high demand for facial makeup transfer tools in fashion e-commerce and virtual avatar generation. Most of the existing makeup transfer methods are based on the generative adversarial networks. Despite their success in makeup transfer for a single image, they struggle to maintain the consistency of makeup under different poses and expressions of the same person. In this article, we propose a robust makeup transfer method which consistently transfers the makeup style of a reference image to facial images in any poses and expressions. Our method introduces the implicit 3D representation, neural radiance fields (NeRFs), to ensure the geometric and appearance consistency. It has two separate stages, including one basic NeRF module to reconstruct the geometry from the input facial image sequence, and a makeup module to learn how to transfer the reference makeup style consistently. We propose a novel hybrid makeup loss which is specially designed based on the makeup characteristics to supervise the training of the makeup module. The proposed loss significantly improves the visual quality and faithfulness of the makeup transfer effects. To better align the distribution between the transferred makeup and the reference makeup, a patch-based discriminator that works in the pose-independent UV texture space is proposed to provide more accurate control of the synthesized makeup. Extensive experiments and a user study demonstrate the superiority of our network for a variety of different makeup styles.

Abstract:
We present a general, fast, and practical solution for interpolating novel views of diverse real-world scenes given a sparse set of nearby views. Existing generic novel view synthesis methods rely on time-consuming scene geometry pre-computation or redundant sampling of the entire space for neural volumetric rendering, limiting the overall efficiency. Instead, we incorporate learned MVS priors into the neural volume rendering pipeline while improving the rendering efficiency by reducing sampling points under the guidance of depth probability distributions. Specifically, fewer but important points are sampled under the guidance of depth probability distributions extracted from the learned MVS architecture. Based on the learned probability-guided sampling, we develop a sophisticated neural volume rendering module that effectively integrates source view information with the learned scene structures. We further propose confidence-aware refinement to improve the rendering results in uncertain, occluded, and unreferenced regions. Moreover, we build a four-view camera system for holographic display and provide a real-time version of our framework for free-viewpoint experience, where novel view images of a spatial resolution of 512×512 can be rendered at around 20 fps on a single GTX 3090 GPU. Experiments show that our method achieves 15 to 40 times faster rendering compared to state-of-the-art baselines, with strong generalization capacity and comparable high-quality novel view synthesis performance.

Abstract:
In temporal (event-based) networks, time is a continuous axis, with real-valued time coordinates for each node and edge. Computing a layout for such graphs means embedding the node trajectories and edge surfaces over time in a 2D + t2D+t space, known as the space-time cube. Currently, these space-time cube layouts are visualized through animation or by slicing the cube at regular intervals. However, both techniques present problems such as below-average performance on tasks as well as loss of precision and difficulties in selecting timeslice intervals. In this article, we present TimeLighting, a novel visual analytics approach to visualize and explore temporal graphs embedded in the space-time cube. Our interactive approach highlights node trajectories and their movement over time, visualizes node “aging”, and provides guidance to support users during exploration by indicating interesting time intervals (“when”) and network elements (“where”) are located for a detail-oriented investigation. This combined focus helps to gain deeper insights into the temporal network's underlying behavior. We assess the utility and efficacy of our approach through two case studies and qualitative expert evaluation. The results demonstrate how TimeLighting supports identifying temporal patterns, extracting insights from nodes with high activity, and guiding the exploration and analysis process.

Abstract:
In soccer, player scouting aims to find players suitable for a team to increase the winning chance in future matches. To scout suitable players, coaches and analysts need to consider whether the players will perform well in a new team, which is hard to learn directly from their historical performances. Match simulation methods have been introduced to scout players by estimating their expected contributions to a new team. However, they usually focus on the simulation of match results and hardly support interactive analysis to navigate potential target players and compare them in fine-grained simulated behaviors. In this work, we propose a visual analytics method to assist soccer player scouting based on match simulation. We construct a two-level match simulation framework for estimating both match results and player behaviors when a player comes to a new team. Based on the framework, we develop a visual analytics system, Team-Scouter, to facilitate the simulative-based soccer player scouting process through player navigation, comparison, and investigation. With our system, coaches and analysts can find potential players suitable for the team and compare them on historical and expected performances. For an in-depth investigation of the players' expected performances, the system provides a visual comparison between the simulated behaviors of the player and the actual ones. The usefulness and effectiveness of the system are demonstrated by two case studies on a real-world dataset and an expert interview.

Abstract:
The numerical extraction of vortex cores from time-dependent fluid flow attracted much attention over the past decades. A commonly agreed upon vortex definition remained elusive since a proper vortex core needs to satisfy two hard constraints: it must be objective and Lagrangian. Recent methods on objectivization met the first but not the second constraint, since there was no formal guarantee that the resulting vortex coreline is indeed a pathline of the fluid flow. In this paper, we propose the first vortex core definition that is both objective and Lagrangian. Our approach restricts observer motions to follow along pathlines, which reduces the degrees of freedoms: we only need to optimize for an observer rotation that makes the observed flow as steady as possible. This optimization succeeds along Lagrangian vortex corelines and will result in a non-zero time-partial everywhere else. By performing this optimization at each point of a spatial grid, we obtain a residual scalar field, which we call vortex deviation error. The local minima on the grid serve as seed points for a gradient descent optimization that delivers sub-voxel accurate corelines. The visualization of both 2D and 3D vortex cores is based on the separation of the movement of the vortex core and the swirling flow behavior around it. While the vortex core is represented by a pathline, the swirling motion around it is visualized by streamlines in the correct frame. We demonstrate the utility of the approach on several 2D and 3D time-dependent vector fields.

Abstract:
This paper presents a practical approach for the optimization of topological simplification, a central pre-processing step for the analysis and visualization of scalar data. Given an input scalar field f and a set of “signal” persistence pairs to maintain, our approaches produces an output field g that is close to f and which optimizes (i) the cancellation of “non-signal” pairs, while (ii) preserving the “signal” pairs. In contrast to pre-existing simplification algorithms, our approach is not restricted to persistence pairs involving extrema and can thus address a larger class of topological features, in particular saddle pairs in three-dimensional scalar data. Our approach leverages recent generic persistence optimization frameworks and extends them with tailored accelerations specific to the problem of topological simplification. Extensive experiments report substantial accelerations over these frameworks, thereby making topological simplification optimization practical for real-life datasets. Our approach enables a direct visualization and analysis of the topologically simplified data, e.g., via isosurfaces of simplified topology (fewer components and handles). We apply our approach to the extraction of prominent filament structures in three-dimensional data. Specifically, we show that our pre-simplification of the data leads to practical improvements over standard topological techniques for removing filament loops. We also show how our approach can be used to repair genus defects in surface processing. Finally, we provide a C++ implementation for reproducibility purposes.

Abstract:
Points of interest on a map such as restaurants, hotels, or subway stations, give rise to categorical point data: data that have a fixed location and one or more categorical attributes. Consequently, recent years have seen various set visualization approaches that visually connect points of the same category to support users in understanding the spatial distribution of categories. Existing methods use complex and often highly irregular shapes to connect points of the same category, leading to high cognitive load for the user. In this paper we introduce SimpleSets, which uses simple shapes to enclose categorical point patterns, thereby providing a clean overview of the data distribution. SimpleSets is designed to visualize sets of points with a single categorical attribute; as a result, the point patterns enclosed by SimpleSets form a partition of the data. We give formal definitions of point patterns that correspond to simple shapes and describe an algorithm that partitions categorical points into few such patterns. Our second contribution is a rendering algorithm that transforms a given partition into a clean set of shapes resulting in an aesthetically pleasing set visualization. Our algorithm pays particular attention to resolving intersections between nearby shapes in a consistent manner. We compare SimpleSets to the state-of-the-art set visualizations using standard datasets from the literature.

Abstract:
A wide range of visualization authoring interfaces enable the creation of highly customized visualizations. However, prioritizing expressiveness often impedes the learnability of the authoring interface. The diversity of users, such as varying computational skills and prior experiences in user interfaces, makes it even more challenging for a single authoring interface to satisfy the needs of a broad audience. In this paper, we introduce a framework to balance learnability and expressivity in a visualization authoring system. Adopting insights from learnability studies, such as multimodal interaction and visualization literacy, we explore the design space of blending multiple visualization authoring interfaces for supporting authoring tasks in a complementary and flexible manner. To evaluate the effectiveness of blending interfaces, we implemented a proof-of-concept system, Blace, that combines four common visualization authoring interfaces–template-based, shelf configuration, natural language, and code editor–that are tightly linked to one another to help users easily relate unfamiliar interfaces to more familiar ones. Using the system, we conducted a user study with 12 domain experts who regularly visualize genomics data as part of their analysis workflow. Participants with varied visualization and programming backgrounds were able to successfully reproduce unfamiliar visualization examples without a guided tutorial in the study. Feedback from a post-study qualitative questionnaire further suggests that blending interfaces enabled participants to learn the system easily and assisted them in confidently editing unfamiliar visualization grammar in the code editor, enabling expressive customization. Reflecting on our study results and the design of our system, we discuss the different interaction patterns that we identified and design implications for blending visualization authoring interfaces.

Abstract:
Numerical simulation serves as a cornerstone in scientific modeling, yet the process of fine-tuning simulation parameters poses significant challenges. Conventionally, parameter adjustment relies on extensive numerical simulations, data analysis, and expert insights, resulting in substantial computational costs and low efficiency. The emergence of deep learning in recent years has provided promising avenues for more efficient exploration of parameter spaces. However, existing approaches often lack intuitive methods for precise parameter adjustment and optimization. To tackle these challenges, we introduce ParamsDrag, a model that facilitates parameter space exploration through direct interaction with visualizations. Inspired by DragGAN, our ParamsDrag model operates in three steps. First, the generative component of ParamsDrag generates visualizations based on the input simulation parameters. Second, by directly dragging structure-related features in the visualizations, users can intuitively understand the controlling effect of different parameters. Third, with the understanding from the earlier step, users can steer ParamsDrag to produce dynamic visual outcomes. Through experiments conducted on real-world simulations and comparisons with state-of-the-art deep learning-based approaches, we demonstrate the efficacy of our solution.

Abstract:
The Dunning-Kruger Effect (DKE) is a metacognitive phenomenon where low-skilled individuals tend to overestimate their competence while high-skilled individuals tend to underestimate their competence. This effect has been observed in a number of domains including humor, grammar, and logic. In this paper, we explore if and how DKE manifests in visual reasoning and judgment tasks. Across two online user studies involving (1) a sliding puzzle game and (2) a scatterplot-based categorization task, we demonstrate that individuals are susceptible to DKE in visual reasoning and judgment tasks: those who performed best underestimated their performance, while bottom performers overestimated their performance. In addition, we contribute novel analyses that correlate susceptibility of DKE with personality traits and user interactions. Our findings pave the way for novel modes of bias detection via interaction patterns and establish promising directions towards interventions tailored to an individual's personality traits. All materials and analyses are in supplemental materials: https://github.com/CAV-Lab/DKE_supplemental.git.

Abstract:
Visualizations play a critical role in validating and improving statistical models. However, the design space of model check visualizations is not well understood, making it difficult for authors to explore and specify effective graphical model checks. VMC defines a model check visualization using four components: (1) samples of distributions of checkable quantities generated from the model, including predictive distributions for new data and distributions of model parameters; (2) transformations on observed data to facilitate comparison; (3) visual representations of distributions; and (4) layouts to facilitate comparing model samples and observed data. We contribute an implementation of VMC as an R package. We validate VMC by reproducing a set of canonical model check examples, and show how using VMC to generate model checks reduces the edit distance between visualizations relative to existing visualization toolkits. The findings of an interview study with three expert modelers who used VMC highlight challenges and opportunities for encouraging exploration of correct, effective model check visualizations.

Abstract:
Despite the development of numerous visual analytics tools for event sequence data across various domains, including but not limited to healthcare, digital marketing, and user behavior analysis, comparing these domain-specific investigations and transferring the results to new datasets and problem areas remain challenging. Task abstractions can help us go beyond domain-specific details, but existing visualization task abstractions are insufficient for event sequence visual analytics because they primarily focus on multivariate datasets and often overlook automated analytical techniques. To address this gap, we propose a domain-agnostic multi-level task framework for event sequence analytics, derived from an analysis of 58 papers that present event sequence visualization systems. Our framework consists of four levels: objective, intent, strategy, and technique. Overall objectives identify the main goals of analysis. Intents comprises five high-level approaches adopted at each analysis step: augment data, simplify data, configure data, configure visualization, and manage provenance. Each intent is accomplished through a number of strategies, for instance, data simplification can be achieved through aggregation, summarization, or segmentation. Finally, each strategy can be implemented by a set of techniques depending on the input and output components. We further show that each technique can be expressed through a quartet of action-input-output-criteria. We demonstrate the framework's descriptive power through case studies and discuss its similarities and differences with previous event sequence task taxonomies.

Abstract:
Sketching is a common practice among visualization designers and serves an approachable entry to data visualization for non-experts. However, moving from a sketch to a full fledged data visualization often requires throwing away the original sketch and recreating it from scratch. Our goal is to formalize these sketches, enabling them to support iteration and systematic data mapping through a visual-first templating workflow. In this workflow, authors sketch a representative visualization and structure it into an expressive template for an envisioned or partial dataset, capturing implicit style as well as explicit data mappings. To demonstrate our proposed workflow, we implement DataGarden and evaluate it through a reproduction and a freeform study. We investigate how DataGarden supports personal expression and delve into the variety of visualizations that authors can produce with it, identifying cases that demonstrate the limitations of our approach and discussing avenues for future work.

Abstract:
How do cancer cells grow, divide, proliferate, and die? How do drugs influence these processes? These are difficult questions that we can attempt to answer with a combination of time-series microscopy experiments, classification algorithms, and data visualization. However, collecting this type of data and applying algorithms to segment and track cells and construct lineages of proliferation is error-prone; and identifying the errors can be challenging since it often requires cross-checking multiple data types. Similarly, analyzing and communicating the results necessitates synthesizing different data types into a single narrative. State-of-the-art visualization methods for such data use independent line charts, tree diagrams, and images in separate views. However, this spatial separation requires the viewer of these charts to combine the relevant pieces of data in memory. To simplify this challenging task, we describe design principles for weaving cell images, time-series data, and tree data into a cohesive visualization. Our design principles are based on choosing a primary data type that drives the layout and integrates the other data types into that layout. We then introduce Aardvark, a system that uses these principles to implement novel visualization techniques. Based on Aardvark, we demonstrate the utility of each of these approaches for discovery, communication, and data debugging in a series of case studies.

Abstract:
Generative methods for creating computer-aided design (CAD) models have gained significant attention over the past two years. However, existing methods lack fine-grained control over the generated CAD models, making it difficult to manage details such as model dimensions and the relative structure of components. To address these limitations, this study introduces Diffusion-CAD, a diffusion-based generative approach that outputs CAD construction sequences. Diffusion-CAD iteratively denoises Gaussian noise into continuous CAD vectors, which are then transformed into discrete CAD sequences. We designed classifier-free and classifier-guided methods to control the distribution of Gaussian noise, CAD sequences, and noisy CAD vectors separately, thereby achieving a variety of fine-grained control tasks. Extensive experiments demonstrated the superior performance and novel capabilities of the proposed method for conditional generation tasks.

Abstract:
We propose 3DFaceSculptor, a general-purpose framework for interactive 3D face editing. Given a source 3D face mesh with semantic materials, and a user-specified semantic image, 3DFaceSculptor can accurately edit the source mesh following the shape guidance of the semantic image, while preserving the source topology as rigid as possible. Recent studies on generating 3D faces focus on learning neural networks to predict 3D shapes, which requires high-cost 3D training datasets. These learning-based methods are limited in compatibility and can only handle face styles involved in the training datasets. Unlike these methods, our 3DFaceSculptor is a non-training and common framework, which only requires supervision from readily-available semantic images, and is compatible with producing various face styles unlimited by datasets. In 3DFaceSculptor, based on the differentiable renderer technique, we deform the source face mesh according to the correspondences between semantic images and mesh materials. However, guiding complex 3D shapes with a simple 2D image incurs extra challenges, that is, the deformation accuracy, surface smoothness, geometric rigidity, and global synchronization of the edited mesh must be guaranteed. To address these challenges, we propose a hierarchical optimization architecture to balance the global and local shape features, and further propose various strategies and losses to improve properties of accuracy, smoothness, rigidity, and so on. Extensive experiments show that our 3DFaceSculptor is able to produce impressive results and has reached the state-of-the-art level.

Abstract:
Point cloud registration is a classical topic in the field of 3D Vision and Computer Graphics. Generally, the implementation of registration is typically sensitive to similarity transformations (translation, scaling, and rotation), noisy points, and incomplete geometric structures. Especially, the non-uniform scales and defective parts of point clouds increase probability of struck local optima in registration task. In this paper, we propose a robust point cloud registration PKSS-Align that can handle various influences, including similarity transformations, non-uniform densities, random noisy points, and defective parts. The proposed method measures shape feature-based similarity between point clouds on the Pre-Kendall shape space (PKSS), which is a shape measurement-based scheme and doesn’t require point-to-point or point-to-plane metric. The employed measurement can be regarded as the manifold metric that is robust to various representations in the euclidean coordinate system. Benefited from the measurement, the transformation matrix can be directly generated for point clouds with mentioned influences at the same time. The proposed method does not require data training and complex feature encoding. Based on a simple parallel acceleration, it can achieve significant improvement for efficiency and feasibility in practice. Experiments demonstrate that our method outperforms the relevant state-of-the-art methods.

Abstract:
We propose a novel method to robustly and efficiently compute the maximum allowable step sizes so that the 3D high-order finite elements continuously preserve geometric validity when moving along the given directions with positive step sizes smaller than the computed ones. We transform the problem of finding the maximum allowable step sizes to one of solving roots of cubic polynomials. To use interval arithmetic to avoid numerical issues in cubic equation solving, we completely enumerate the roots of cubic polynomials and apply the interval version of the Newton-Raphson iteration. The effectiveness of our algorithm is demonstrated through extensive testing. Compared to the state-of-the-art method, our algorithm achieves higher efficiency.

Abstract:
Estimating dense point-to-point correspondences between two isometric shapes represented as 3D point clouds is a fundamental problem in geometry processing, with applications in texture and motion transfer. However, this task becomes particularly challenging when the shapes undergo non-rigid transformations, as is often the case with approximately isometric point clouds. Most existing algorithms address this challenge by establishing correspondences between functions defined on the shapes, rather than directly between points, because function mappings admit a linear representation in the spectral domain. State-of-the-art methods compute this linear representation using the eigenfunctions of the Laplace–Beltrami Operator (LBO) along with a small set of initial corresponding functions between the shapes. However, for approximately isometric point clouds, two key issues arise: (1) the eigenfunctions of the LBO may become misaligned, and (2) the initial corresponding functions may include outliers, both of which degrade the quality of the resulting correspondences. In this work, we propose an efficient approach to align the spectra of the LBOs of the two shapes, enabling the eigenfunctions to remain compatible even for approximately isometric 3D point clouds. Additionally, we introduce a technique to make function correspondence estimation robust to outliers. We validate our approach by comparing it with state-of-the-art 3D shape-matching algorithms on benchmark datasets, demonstrating its effectiveness.

Abstract:
Samples produced by generative models, called Generated Samples (GSs), have become a critical supplement to those collected from the real world in data-centric applications. Domain experts typically randomly collect many GSs and manually select a few of interest for applications. However, the methodology lacks guidance to locate desirable ones that exhibit specific features or adhere to application-oriented metrics among infinite generable candidates. These samples are generally concentrated in a few small regions of the generative model’s latent space, called Generative Latent Space (GLS). This paper presents Latent Space Map that projects a GLS onto a plane to help users locate regions rich in desirable GSs. Our research revolves around two challenges in constructing the map. First, many GSs in a GLS are low-quality and useless for applications. Excluding them from the projection is challenging for their irregular distribution. We employ a Monte Carlo-based method to capture a manifold for projection, where high-quality GSs are mainly distributed. Second, the GLS is high-dimensional and unbounded, complicating the projection. We design a manifold projection method that endows the map with desirable characteristics to achieve high display accuracy and effective pattern perception for users freely observing the manifold. We further develop a system integrating Latent Space Map to aid in GS selection and refinement. Real-world cases, quantitative experiments, and feedback from domain experts confirm the usability and effectiveness of our approach.

Abstract:
We present a latent talking style space with semantic meanings for speech-driven 3D facial animation. The style space is learned from 3D speech facial animations via a self-supervision paradigm without any style labeling, leading to an automatic separation of high-level attributes, i.e., different channels of the latent style code possess different semantic meanings, such as a wide/slightly open mouth, a grinning/round mouth, and frowning/raising eyebrows. The style space enables intuitive and flexible control of talking styles in speech-driven facial animation through manipulating the channels of style code. To effectively learn such a style space, we propose a two-stage approach, involving two deep neural networks, to disentangle the person identity, speech content, and talking style contained in 3D speech facial animations. The training is performed on a novel dataset of 3D talking faces of various styles, constructed from over ten hours of videos of 200 subjects collected from the Internet.

Abstract:
In this paper, we present WonderHuman to reconstruct dynamic human avatars from a monocular video for high-fidelity novel view synthesis. Previous dynamic human avatar reconstruction methods typically require the input video to have full coverage of the observed human body. However, in daily practice, one typically has access to limited viewpoints, such as monocular front-view videos, making it a cumbersome task for previous methods to reconstruct the unseen parts of the human avatar. To tackle the issue, we present WonderHuman, which leverages 2D generative diffusion model priors to achieve high-quality, photorealistic reconstructions of dynamic human avatars from monocular videos, including accurate rendering of unseen body parts. Our approach introduces a Dual-Space Optimization technique, applying Score Distillation Sampling (SDS) in both canonical and observation spaces to ensure visual consistency and enhance realism in dynamic human reconstruction. Additionally, we present a View Selection strategy and Pose Feature Injection to enforce the consistency between SDS predictions and observed data, ensuring pose-dependent effects and higher fidelity in the reconstructed avatar. In the experiments, our method achieves SOTA performance in producing photorealistic renderings from the given monocular video, particularly for those challenging unseen parts.

Abstract:
Head tracking allows users of Virtual Reality (VR) to freely rotate their heads 360 degrees while exploring virtual environments. When using VR in a limited space, the ability to physically rotate one’s head is limited to a specific range. To address this issue, previous studies have proposed methods to employ distinct rotation factors for real and imaginary rotations. However, its primary usage lies in redirected walking; thus, it is unsuitable for seated VR. In this article, we propose CaliView, which consistently adjusts the user’s perspective to always face an optimal direction in VR, all while ensuring a comfortable posture. CaliView continuously controls the rotation gain to ensure that the disparity between the present body orientation and the optimal orientation is reduced to zero, encouraging the user to assume a forward-facing position with the target orientation. To assess the suggested approach, CaliView, we experimented to compare three conditions: CaliView, snap turning only (SnapTurn), and hybrid of CaliView and snap turning (Hybrid). The research findings suggest that CaliView functions as a useful reorientation technique, enabling implicit reorientation without sacrificing the user experience. Additionally, this study showcases its compatibility with various other techniques, such as the traditional snap-turn, thus emphasizing its versatility.

Abstract:
Exploring and comprehending relevant academic literature is a vital yet challenging task for researchers, especially given the rapid expansion in research publications. This task fundamentally involves sensemaking-interpreting complex, scattered information sources to build understanding. While emerging immersive analytics tools have shown cognitive benefits like enhanced spatial memory and reduced mental load, they predominantly focus on information synthesis (e.g., organizing known documents). In contrast, the equally important information foraging phase-discovering and gathering relevant literature-remains underexplored within immersive environments, hindering a complete sensemaking workflow. To bridge this gap, we introduce LitForager, an interactive literature exploration tool designed to facilitate information foraging of research literature within an immersive sensemaking workflow using network-based visualizations and multimodal interactions. Developed with WebXR and informed by a formative study with researchers, LitForager supports exploration guidance, spatial organization, and seamless transition through a 3D literature network. An observational user study with 15 researchers demonstrated LitForager's effectiveness in supporting fluid foraging strategies and spatial sensemaking through its multimodal interface.

Abstract:
Real-time scene provisioning-reconstructing and delivering scene data to requesting XR applications during runtime-is central to enabling spatial computing in modern XR systems. However, existing solutions struggle to balance latency, power and scene fidelity under XR device constraints, and often rely on designs that are either closed, application-specific designs, or both. We present Ada, the first open distributed, power-aware, application-agnostic real-time scene provisioning system. Through computation offloading along with algorithmic and system innovations, Ada provides high-fidelity scenes with stable performance across all evaluated scene sizes and with low power consumption. To isolate the benefits of Ada's algorithmic and design innovations over the closest prior work [82], which is on-device and CPU-based, we configure a comparable on-device, CPU-based variant of Ada (AdaLocal-CPU). We show this variant achieves up to 6.8× lower scene request latency and higher scene fidelity compared to the prior work. Furthermore, Ada's final distributed GPU-accelerated implementation reduces latency by an additional 2×, highlighting the benefits of GPU acceleration and distributed computing. Additionally, Ada also lowers the incremental power cost of scene provisioning by 24% compared to the best on-device variant (AdaLocal-GPU). Finally, Ada flexibly adapts to diverse latency, power, scene fidelity, and network bandwidth requirements.

Abstract:
In extended reality (XR) applications, enhancing user perception often necessitates head-mounted display (HMD) removal. However, existing methods suffer from low time performance and suboptimal reconstruction quality. In this paper, we propose a half face mapping 3D Gaussian splatting avatar based HMD removal method (HFM-GS), which can perform real-time and high-fidelity online restoration of the complete face in HMD-occluded videos for XR applications after a short un-occluded face registration. We establish a mapping field between the upper and lower face Gaussians to enhance the adaptability to deformation. Then, we introduce correlation weight-based sampling to improve time performance and handle variations in the number of Gaussians. At last, we ensure model robustness through Gaussian Segregation Strategy. Compared to two state-of-the-art methods, our method achieves better quality and time performance. The results of the user study show that fidelity is significantly improved with our method.

Abstract:
In virtual reality (VR), collaborators often experience the same environment at different visual scales, disrupting shared attention and increasing coordination difficulty. While prior work has focused on preventing misalignment, less is known about how teams recover when alignment fails. We examine collaboration under scale asymmetry, a particularly disruptive form of perceptual divergence. In a study with 36 VR teams, we identify behavioral patterns that distinguish adaptive recovery from persistent breakdown. Successful teams flexibly shifted between user-driven and system-supported cues, while others repeated ineffective strategies. Based on these findings, we introduce the Perceptual Asymmetry Coordination Theory (PACT), a dual-pathway model that describes coordination as an evolving process shaped by cue integration and strategic responsiveness. PACT reframes recovery not as a return to alignment, but as a dynamic adaptation to misalignment. These insights inform the design of VR systems that support recovery through multi-channel, adaptive coordination in scale-divergent environments.

Abstract:
An optical-see-through near-eye display (NED) for augmented reality (AR) allows the user to perceive virtual and real imagery simultaneously. Existing technologies for optical-see-through AR NEDs involve trade-offs between key metrics such as field of view (FOV), eyebox size, form factor, etc. We have enhanced an existing compact wide-FOV pinlight AR NED design with real-time 3D pupil localization in order to dynamically steer and thus effectively enlarge the usable eyebox. This is achieved with a dual-camera rig that captures stereoscopic views of the pupils. The 3D pupil location is used to dynamically calculate a display pattern that spatio-temporally modulates the light entering the wearer's eyes. We have built a demonstrable compact prototype and have conducted a user study that indicates the effectiveness of our eyebox steering method (e.g., without eyebox steering, in 10.5% of our tests, users were unable to perceive the test pattern correctly before experiment timeout; with eyebox steering, that fraction decreased dramatically to 1.25%). This is a small yet crucial step in making simple wide-FOV pinlight NEDs usable for human users and not just as demonstration prototypes filmed with a precisely positioned camera standing in for the user's eye. Further contributions of this paper include a detailed description of display design, calibration technique, and user study design, all of which may benefit other NED research.

Abstract:
Geographically dispersed users often rely on virtual avatars as intermediaries to facilitate interactive communication and collaboration. However, existing methods for augmented reality (AR) telepresence applications exhibit limitations, including restricted movement within confined sub-areas, lack of smooth transitions, and the necessity for manually establishing object mapping between dissimilar environments. We present a novel interactive AR framework for virtual avatar locomotion adaption while preserving semantic coherence across dissimilar indoor scenes. Initially, we conduct a preliminary user study to identify key attributes influencing preferred avatar movement. These attributes are quantified as features, and a dataset of user annotations on avatar movements is created. Based on the user interaction and scene configurations, we employ a deep reinforcement learning neural network to guide the avatar to the ideal position while maximizing semantic coherence. We validate our proposed framework through simulations and user studies by implementing an AR-based 3D telepresence prototype, demonstrating the efficacy of our framework in conveying user intentions across dissimilar environments, enabling natural and immersive 3D telepresence interactions.

Abstract:
In order to reconstruct 3D clothed human with accurate fine-grained details from sparse views, we propose a deep cooperating two-level global to fine-grained reconstruction framework that constructs robust global geometry to guide fine-grained geometry learning. The core of the framework is a novel visibility aware Transformer VAT, which bridges the two-level reconstruction architecture by connecting its global encoder and fine-grained decoder with two pixel-aligned implicit functions, respectively. The global encoder fuses semantic features of multiple views to integrate global geometric features. In the fine-grained decoder, visibility aware attention mechanism is designed to efficiently fuse multi-view and multi-scale features for mining fine-grained geometric features. The global encoder and fine-grained decoder are connected by a global embeding module to form a deep cooperation in the two-level framework, which provides global geometric embedding as a query guidance for calculating visibility aware attention in the fine-grained decoder. In addition, to extract highly aligned multi-scale features for the two-level reconstruction architecture, we design an image feature extractor MSUNet, which establishes strong semantic connections between different scales at minimal cost. Our proposed framework is end-to-end trainable, with all modules jointly optimized. We validate the effectiveness of our framework on public benchmarks, and experimental results demonstrate that our method has significant advantages over state-of-the-art methods in terms of both fine-grained performance and generalization.

Abstract:
Various SDF-based neural implicit surface reconstruction methods have been proposed recently, and have demonstrated remarkable modeling capabilities. However, due to the global nature and limited representation ability of a single network, existing methods still suffer from many drawbacks, such as limited accuracy and scale of the reconstruction. In this paper, we propose a versatile, scalable and high-quality neural implicit representation to address these issues. We integrate a divide-and-conquer approach into the neural SDF-based reconstruction. Specifically, we model the object or scene as a fusion of multiple independent local neural SDFs with overlapping regions. The construction of our representation involves three key steps: (1) constructing the distribution and overlap relationship of the local radiance fields based on object structure or data distribution, (2) relative pose registration for adjacent local SDFs, and (3) SDF blending. Thanks to the independent representation of each local region, our approach can not only achieve high-fidelity surface reconstruction, but also enable scalable scene reconstruction. Extensive experimental results demonstrate the effectiveness and practicality of our proposed method.

Abstract:
Spatial time series visualization offers scientific research pathways and analytical decision-making tools across various spatiotemporal domains. Despite many advanced methodologies, the seamless integration of temporal and spatial information remains a challenge. The space-time cube (STC) stands out as a promising approach for the synergistic presentation of spatial and temporal information, with successful applications across various spatiotemporal datasets. However, the STC is plagued by well-known issues such as visual occlusion and depth ambiguity, which are further exacerbated when dealing with large-scale spatial time series data. In this study, we introduce a novel technical framework termed VolumeSTCube, designed for continuous spatiotemporal phenomena. It first leverages the concept of the STC to transform discretely distributed spatial time series data into continuously volumetric data. Subsequently, volume rendering and surface rendering techniques are employed to visualize the transformed volumetric data. Volume rendering is utilized to mitigate visual occlusion, while surface rendering provides pattern details by enhanced lighting information. Lastly, we design interactions to facilitate the exploration and analysis from temporal, spatial, and spatiotemporal perspectives. VolumeSTCube is evaluated through a computational experiment, a real-world case study with one expert, and a controlled user study with twelve non-experts, compared against a baseline from prior work, showing its superiority and effectiveness in large-scale spatial time series analysis.

Abstract:
Floor plan restoration aims to recover vector and semantic information from raster floor plan images, which is significant for advanced applications including interior design, interative walkthroughs, and layout planning. Existing methods generally adopt a two-stage paradigm: a parsing stage to extract semantic elements such as rooms, walls, doors, and windows from raster images; and then a vectorization stage to convert them into vector graphics. However, these methods are deficient in both accuracy and efficiency due to the neglect of the unique cues of floor plans compared to natural images. To address the above issues, we propose MMParseNet that yields accurate parsing results by incorporating multimodal cues unique to floor plans, such as room names, furniture icons, and room boundaries. Moreover, we implement an efficiency-optimized vectorization method based on PCA that avoids unnecessary iterative solutions. We conduct both quantitative and qualitative experiments on three public and one self-built dataset. The results exhibit consistent improvements in accuracy and sub-second overall restoration time across various datasets.

Abstract:
Each scanner possesses its unique characteristics and exhibits its distinct sampling error distribution. Training a network on a dataset that includes data collected from different scanners is less effective than training it on data specific to a single scanner. Therefore, we present a novel one-shot learning method allowing for edge extraction on point clouds, by learning the specific data distribution of the target point cloud, and thus achieve superior results compared to networks that were trained on general data distributions. More specifically, we present how to train a lightweight network named OSFENet (One-Shot edge Feature Extraction Network), by designing a filtered-KNN-based surface patch representation that supports a one-shot learning framework. Additionally, we introduce an RBF_DoS module, which integrates Radial Basis Function-based Descriptor of the Surface patch, highly beneficial for the edge extraction on point clouds. The advantage of the proposed OSFENet is demonstrated through comparative analyses against 7 baselines on the ABC dataset, and its practical utility is validated by results across diverse real-scanned datasets, including indoor scenes like S3DIS dataset, and outdoor scenes such as the Semantic3D dataset and UrbanBIS dataset.

Abstract:
Parametric edge reconstruction for point cloud data is a fundamental problem in computer graphics. Existing methods first classify points as either edge points (including corners) or non-edge points, and then fit parametric edges to the edge points. However, few points are exactly sampled on edges in practical scenarios, leading to significant fitting errors in the reconstructed edges. Prominent deep learning-based methods also primarily emphasize edge points, overlooking the potential of non-edge areas. Given that sparse and non-uniform edge points cannot provide adequate information, we address this challenge by leveraging neighboring segmented patches to supply additional cues. We introduce a novel two-stage framework that reconstructs edges precisely and completely via surface patch segmentation. First, we propose PCER-Net, a Point Cloud Edge Reconstruction Network that segments surface patches, detects edge points, and predicts normals simultaneously. Second, a joint optimization module is designed to reconstruct a complete and precise 3D wireframe by fully utilizing the predicted results of the network. Concretely, the segmented patches enable accurate fitting of parametric edges, even when sparse points are not precisely distributed along the model's edges. Corners can also be naturally detected from the segmented patches. Benefiting from fitted edges and detected corners, a complete and precise 3D wireframe model with topology connections can be reconstructed by geometric optimization. Finally, we present a versatile patch-edge dataset, including CAD and everyday models (furniture), to generalize our method. Extensive experiments and comparisons against previous methods demonstrate our effectiveness and superiority. We will release the code and dataset to facilitate future research.

Abstract:
Rendering realistic appearances of homogeneous translucent materials, such as milk and marble, poses challenges due to the complexity of subsurface scattering. In this paper, we present a neural method for real-time rendering of homogeneous translucent objects. Based on the observation that light propagation inside a highly scattered media is like a diffusion process (Stam 1995), we propose a neural data structure named diffusion block to mimic the behavior of the diffusion process. The diffusion block is built upon a recent network structure named DiffusionNet (Sharp et al. 2022) with a few modifications to adapt to our problem of translucent rendering. Our network is lightweight and efficient, leading to a real-time rendering method. Furthermore, our method supports dynamic material properties and diverse lighting conditions. Comparisons with state-of-the-art real-time translucent rendering methods demonstrate the superiority of our method in rendering quality.

Abstract:
This study explores the design of a selection technique in virtual environments leveraging kinematic data derived from hand movements. We first identified the intrinsic challenges of virtual hand selection techniques, particularly in complex settings, including Accidental Selection, Slow Selection, Failed Selection, and Fragmented Selection. To mitigate these issues, we introduce Magic-Tap, a selection technique that ascertains the trigger of an object based on real-time variations in virtual hand acceleration and speed, seamlessly integrating the pointing and triggering processes without requiring explicit triggering signals. The parameter settings of Magic-Tap were fine-tuned through Study One, ameliorating its trigger rate, error rate, and trigger time. Furthermore, we compared Magic-Tap with three conventional virtual hand selection techniques (Touch, Dwell-Time, and Pinch) in Study Two. The results indicate that the task completion time of Magic-Tap is comparable to Touch in all situations while exhibiting an error rate as low as Dwell-Time and Pinch.

Abstract:
Many current image restoration approaches utilize neural networks to acquire robust image-level priors from extensive datasets, aiming to reconstruct missing details. Nevertheless, these methods often falter with images that exhibit significant information gaps. While incorporating external priors or leveraging reference images can provide supplemental information, these strategies are limited in their practical scope. Alternatively, textual inputs offer greater accessibility and adaptability. In this study, we develop a sophisticated framework enabling users to guide the restoration of deteriorated images via textual descriptions. Utilizing the text-image compatibility feature of CLIP enhances the integration of textual and visual data. Our versatile framework supports multiple restoration activities such as image inpainting, super-resolution, and colorization. Comprehensive testing validates our technique's efficacy.

Abstract:
In a formal data analysis workflow, data validation is a necessary step that helps data analysts verify the quality of the data and ensure the reliability of the results. Data analysts usually need to validate the result when encountering an unexpected result, such as an abnormal record in a table. In order to understand how a specific record is derived, they would backtrace it in the pipeline step by step via checking the code lines, exposing the intermediate tables, and finding the data records from which it is derived. However, manually reviewing code and backtracing data requires certain expertise, while inspecting the traced records in multiple tables and interpreting their relationships is tedious. In this work, we propose HYPNOS, a visualization system that supports interactive data lineage tracing for data transformation scripts. HYPNOS uses a lineage module for parsing and adapting code to capture both schema-level and instance-level data lineage from data transformation scripts. Then, it provides users with a lineage view for obtaining an overview of the data transformation process and a detail view for tracing instance-level data lineage and inspecting details. HYPNOS reveals different levels of data relationships and helps users with data lineage tracing. We demonstrate the usability and effectiveness of HYPNOS through a use case, interviews of four expert users, and a user study.

Abstract:
Designing synthetic routes for novel molecules is pivotal in various fields like medicine and chemistry. In this process, researchers need to explore a set of synthetic reactions to transform starting molecules into intermediates step by step until the target novel molecule is obtained. However, designing synthetic routes presents challenges for researchers. First, researchers need to make decisions among numerous possible synthetic reactions at each step, considering various criteria (e.g., yield, experimental duration, and the count of experimental steps) to construct the synthetic route. Second, they must consider the potential impact of one choice at each step on the overall synthetic route. To address these challenges, we proposed SynthLens, a visual analytics system to facilitate the iterative construction of synthetic routes by exploring multiple possibilities for synthetic reactions at each step of construction. Specifically, we have introduced a tree-form visualization in SynthLensto compare and evaluate all the explored routes at various exploration steps, considering both the exploration step and multiple criteria. Our system empowers researchers to consider their construction process comprehensively, guiding them toward promising exploration directions to complete the synthetic route. We validated the usability and effectiveness of SynthLensthrough a quantitative evaluation and expert interviews, highlighting its role in facilitating the design process of synthetic routes. Finally, we discussed the insights of SynthLensto inspire other multi-criteria decision-making scenarios with visual analytics.

Abstract:
Neural radiance fields (NeRFs) have achieved impressive view synthesis results by learning an implicit volumetric representation from multi-view images. To project the implicit representation into an image, NeRF employs volume rendering that approximates the continuous integrals of rays as an accumulation of the colors and densities of the sampled points. Although this approximation enables efficient rendering, it ignores the direction information in point intervals, resulting in ambiguous features and limited reconstruction quality. In this paper, we propose a learning method that utilizes learnable view-dependent features to improve scene representation and reconstruction. We model the volume rendering integral with a piecewise constant volume density and spherical harmonic-guided view-dependent features, facilitating ambiguity elimination while preserving the rendering efficiency. In addition, we introduce a regularization term that restricts the anisotropic representation effect to be local, with negligible effect on geometry representations, and that encourages recovering the correct geometry. Our method is flexible and can be plugged into NeRF-based frameworks. Extensive experiments show that the proposed representation can boost the rendering quality of various NeRFs and achieve state-of-the-art rendering performance on both synthetic and real-world scenes.

Abstract:
Traditional deep functional map frameworks are widely used for 3D shape matching; however, many methods fail to adaptively capture the relevant frequency information required for functional map estimation in complex scenarios, leading to poor performance, especially under significant deformations. To address these challenges, we propose a novel unsupervised learning-based framework, Deep Frequency Awareness Functional Maps (DFAFM), specifically designed to tackle diverse shape-matching problems. Our approach introduces the Spectral Filter Operator Preservation constraint, which ensures the preservation of critical frequency information. These constraints promote frequency awareness by learning a set of spectral filters and incorporating them as a loss function to jointly supervise the functional maps, pointwise maps, and spectral filters. The spectral filters are constructed using orthonormal Jacobi polynomials with learnable coefficients, enabling adaptive and efficient frequency representation. Furthermore, we propose a refinement strategy that leverages the learned spectral filters and constraints to enhance the accuracy of the final pointwise map. Extensive experiments conducted on multiple benchmark datasets demonstrate that our method outperforms state-of-the-art approaches, particularly in challenging scenarios involving non-isometric deformations and inconsistent topology.

Abstract:
In real-world scenarios, scanned point clouds are often incomplete due to occlusion issues. The tasks of self-supervised and weakly-supervised point cloud completion involve reconstructing missing regions of these incomplete objects without the supervision of complete ground truth. Current methods either rely on multiple views of partial observations for supervision or overlook the intrinsic geometric similarity that can be identified and utilized from the given partial point clouds. In this paper, we propose MAL-UPC, a framework that effectively leverages both region-level and category-specific geometric similarities to complete missing structures. Our MAL-UPC does not require any 3D complete supervision and only necessitates single-view partial observations in the training set. Specifically, we first introduce a Pattern Retrieval Network to retrieve similar position and curvature patterns between the partial input and the predicted shape, then leverage these similarities to densify and refine the reconstructed results. Additionally, we render the reconstructed complete shape into multi-view depth maps and design an adversarial learning module to learn the geometry of the target shape from category-specific single-view depth images of the partial point clouds in the training set. To achieve anisotropic rendering, we design a density-aware radius estimation algorithm to improve the quality of the rendered images. Our MAL-UPC outperforms current state-of-the-art self-supervised methods and even some unpaired approaches.

Abstract:
The boundary-constrained floor plan generation problem aims to generate the topological and geometric properties of a set of rooms within a given boundary. Recently, learning-based methods have made significant progress in generating realistic floor plans. However, these methods involve a workflow of converting vector data into raster images, using image-based generative models, and then converting the results back into vector data. This process is complex and redundant, often resulting in information loss. Raster images, unlike vector data, cannot scale without losing detail and precision. To address these issues, we propose a novel deep learning framework called DiffPlanner for boundary-constrained floor plan generation, which operates entirely in vector space. Our framework is a Transformer-based conditional diffusion model that integrates an alignment mechanism in training, aligning the optimization trajectory of the model with the iterative design processes of designers. This enables our model to handle complex vector data, better fit the distribution of the predicted targets, accomplish the challenging task of floor plan layout design, and achieve user-controllable generation. We conduct quantitative comparisons, qualitative evaluations, ablation experiments, and perceptual studies to evaluate our method. Extensive experiments demonstrate that DiffPlanner surpasses existing state-of-the-art methods in generating floor plans and bubble diagrams in the creative stages, offering more controllability to users and producing higher-quality results that closely match the ground truths.

Abstract:
In this paper, we present a flexible and probabilistic framework for tracking topological features in time-varying scalar fields using merge trees and partial optimal transport. Merge trees are topological descriptors that record the evolution of connected components in the sublevel sets of scalar fields. We present a new technique for modeling and comparing merge trees using tools from partial optimal transport. In particular, we model a merge tree as a measure network, that is, a network equipped with a probability distribution, and define a notion of distance on the space of merge trees inspired by partial optimal transport. Such a distance offers a new and flexible perspective for encoding intrinsic and extrinsic information in the comparative measures of merge trees. More importantly, it gives rise to a partial matching between topological features in time-varying data, thus enabling flexible topology tracking for scientific simulations. Furthermore, such partial matching may be interpreted as probabilistic coupling between features at adjacent time steps, which gives rise to probabilistic tracking graphs. We derive a stability result for our distance and provide numerous experiments indicating the efficacy of our framework in extracting meaningful feature tracks.

Abstract:
Recently, Light Field (LF) shows great potential in removing occlusion since the objects occluded in some views may be visible in other views. However, existing LF-based methods implicitly model each scene and can only remove objects that have positive disparities in one central views. In this article, we propose a novel Progressive Multi-Plane Images (MPI) Construction method specifically designed for LF-based occlusion removal. Different from the previous MPI construction methods, we progressively construct MPIs layer by layer in order from near to far. In order to accurately model the current layer, the positions of foreground occlusions in the nearer layers are taken as occlusion prior. Specifically, we propose an Occlusion-Aware Attention Network to generate each layer of MPIs with reliable information in occluded regions. For each layer, occlusions in the current layer are filtered out so that the background is better recovered just using the visible views instead of the other occluded views. Then, by simply removing the layers containing occlusions and rendering MPIs in kinds of viewpoints, the occlusion removal results for different views are generated. Experiments on synthetic and real-world scenes show that our method outperforms state-of-the-art LF occlusion removal methods in quantitative and visual comparisons. Moreover, we also apply the proposed progressive MPI construction method to the view synthesis task. The occlusion edges in our synthesized views achieve significantly better quality, which also verifies that our method can better model the occluded regions.

Abstract:
Positioning clothing parts (\mathcal SSs) such as sleeves and collars has been in the realm of manual task that had to be done meticulously in order to prevent unnecessary tanglements during the simulation. This article proposes an optimization-based method to computerize the above \mathcal SS-positioning task. For that, we embed each \mathcal SS to an abstracting cylinder \mathcal CC such that \mathcal SS-positioning can be done by adjusting only 3～∼4 DOFs (e.g., translating/rotating \mathcal CC or adjusting its radius) instead of per-vertex-full-DOFs. Then, we formulate an objective function EE by scoring undesirableness of \mathcal SS's position (e.g., \mathcal SS penetrating the body, \mathcal SS making cloth-to-cloth intersection). In organizing EE into the loop of the Newton's method, the main challenge was to calculate the symbolic gradient and hessian, for which this article makes several novel contributions. The resultant \mathcal SS-positioning method works quite successfully; \mathcal S^Ss (the output of the \mathcal SS-positioning method) are tanglement-free thus running the simulator to that configuration produces acceptable draping quickly; Experiments show that, in obtaining acceptable draping, the proposed method produces about × 9.7×9.7 speed up compared to when not using it.

Abstract:
Volume visualization plays a significant role in revealing important intrinsic patterns of 3D scientific datasets. However, these datasets are often large, making it challenging for interactive visualization systems to deliver a seamless user experience because of high input latency that arises from I/O bottlenecks and limited fast memory resources with high miss rates. To address this issue, we have proposed a deep learning-based prefetching method called RmdnCache, which optimizes the data flow across the memory hierarchy to reduce the input latency of large-scale volume visualization. Our approach accurately prefetches the content of the next view to fast memory using learning-based prediction while rendering the current view. The proposed deep learning architecture consists of two networks, RNN and MDN in respective spaces, which work together to predict both the location and likelihood distribution of the next view for defining an optimal prefetching range. Our method outperforms existing state-of-the-art prefetching algorithms in reducing overall input latency for visualizing real-world large-scale volumetric datasets.

Abstract:
It is a challenging task to create realistic 3D avatars that accurately replicate individuals’ speech and unique talking styles for speech-driven facial animation. Existing techniques have made remarkable progress but still struggle to achieve lifelike mimicry. This article proposes “TalkingStyle”, a novel method to generate personalized talking avatars while retaining the talking style of the person. Our approach uses a set of audio and animation samples from an individual to create new facial animations that closely resemble their specific talking style, synchronized with speech. We disentangle the style codes from the motion patterns, allowing our method to associate a distinct identifier with each person. To manage each aspect effectively, we employ three separate encoders for style, speech, and motion, ensuring the preservation of the original style while maintaining consistent motion in our stylized talking avatars. Additionally, we propose a new style-conditioned transformer decoder, offering greater flexibility and control over the facial avatar styles. We comprehensively evaluate TalkingStyle through qualitative and quantitative assessments, as well as user studies demonstrating its superior realism and lip synchronization accuracy compared to current state-of-the-art methods.

Abstract:
The field of 3D tooth segmentation has made considerable advances thanks to deep learning, but challenges remain with coarse segmentation boundaries and prediction errors. In this article, we introduce a novel learnable method to refine coarse results obtained from existing 3D tooth segmentation algorithms. The refinement framework features a dual-stream network called TSRNet (Tooth Segmentation Refinement Network) to rectify defective boundary and distance maps extracted from the coarse segmentation. The boundary map provides explicit boundary information, while the distance map provides gradient information in the form of the shortest geodesic distance between the vertex and the segmentation boundary. Following well-designed rules, the two refined maps are utilized to move the coarse tooth boundaries toward their correct positions through an iterative refinement process. The two-stage refinement method is validated on both 3D tooth and segmentation benchmark datasets. Extensive experiments demonstrate that our method significantly improves upon the coarse results from baseline methods and achieves state-of-the-art performance.

Abstract:
High-fidelity online 3D scene reconstruction from monocular videos continues to be challenging, especially for coherent and fine-grained geometry reconstruction. The previous learning-based online 3D reconstruction approaches with neural implicit representations have shown a promising ability for coherent scene reconstruction, but often fail to consistently reconstruct fine-grained geometric details during online reconstruction. This paper presents a new on-the-fly monocular 3D reconstruction approach, named GP-Recon, to perform high-fidelity online neural 3D reconstruction with fine-grained geometric details. We incorporate geometric prior (GP) into a scene's neural geometry learning to better capture its geometric details and, more importantly, propose an online volume rendering optimization to reconstruct and maintain geometric details during the online reconstruction task. The extensive comparisons with state-of-the-art approaches show that our GP-Recon consistently generates more accurate and complete reconstruction results with much better fine-grained details, both quantitatively and qualitatively.

Affiliations: State Key Lab of CAD&CG, Zhejiang University, Zhejiang, China; Zhejiang University-University of Illinois at Urbana Champaign Institute, Zhejiang University, Zhejiang, China; Angelalign Research Institute, Angelalign Technology Inc., Shanghai, China; ChohoTech Company Ltd., Hangzhou, China; Harvard School of Dental Medicine, Harvard University, Boston, MA, USA; Department of Orthodontics, Shanghai Ninth People's Hospital, College of Stomatology, Shanghai Jiao Tong University School of Medicine, Shanghai, China

Abstract:
Accurately determining the clinical positions for each tooth is essential in orthodontics, while most existing solutions heavily rely on inefficient manual design. In this paper, we present the LETA, a dual-branch Latent Encoding based 3D Tooth Alignment. Our system takes as input the segmented individual 3D tooth meshes in the Intra-oral Scanner (IOS) dental surfaces, and automatically predicts the proper 3D pose transformation for each tooth. LETA includes three components: an Encoder that learns a latent code of dental pointcloud, a Projector that transforms the latent code of misaligned teeth to predicted aligned ones, and a Solver to estimate the transformation between different dental latent codes. A key novelty of LETA is that we extract the features from the ground truth (GT) aligned teeth to guide network learning during training. To effectively learn tooth features, our Encoder employs an improved point-wise convolutional operation and an attention-based network to extract local shape features and global context features respectively. Extensive experimental results on a large-scale dataset with 9,868 IOS surfaces demonstrate that LETA can achieve state-of-the-art performance. A further clinical applicability study reveals that our method can reduce orthodontists’ workload over 60% compared to starting tooth alignment from scratch, demonstrating the strong potential of deep learning for future digital dentistry.

Abstract:
Recently, single-image SVBRDF capture is formulated as a regression problem, which uses a network to infer four SVBRDF maps from a flash-lit image. However, the accuracy is still not satisfactory since previous approaches usually adopt end-to-end inference strategies. To mitigate the challenge, we propose “auxiliary renderings” as the intermediate regression targets, through which we divide the original end-to-end regression task into several easier sub-tasks, thus achieving better inference accuracy. Our contributions are threefold. First, we design three (or two pairs of) auxiliary renderings and summarize the motivations behind the designs. By our design, the auxiliary images are bumpiness-flattened or highlight-removed, containing disentangled visual cues about the final SVBRDF maps and can be easily transformed to the final maps. Second, to help estimate the auxiliary targets from the input image, we propose two mask images including a bumpiness mask and a highlight mask. Our method thus first infers mask images, then with the help of the mask images infers auxiliary renderings, and finally transforms the auxiliary images to SVBRDF maps. Third, we propose backbone UNets to infer mask images, and gated deformable UNets for estimating auxiliary targets. Thanks to the well-designed networks and intermediate images, our method outputs better SVBRDF maps than previous approaches, validated by the extensive comparisonal and ablation experiments.

Abstract:
Motion retargeting is an active research area in computer graphics and animation, allowing for the transfer of motion from one character to another, thereby creating diverse animated character data. While this technology has numerous applications in animation, games, and movies, current methods often produce unnatural or semantically inconsistent motion when applied to characters with different shapes or joint counts. This is primarily due to a lack of consideration for the geometric and spatial relationships between the body parts of the source and target characters. To tackle this challenge, we introduce a novel spatially-preserving Skinned Motion Retargeting Network (SMRNet) capable of handling motion retargeting for characters with varying shapes and skeletal structures while maintaining semantic consistency. By learning a hybrid representation of the character's skeleton and shape in a rest pose, SMRNet transfers the rotation and root joint position of the source character's motion to the target character through embedded rest pose feature alignment. Additionally, it incorporates a differentiable loss function to further preserve the spatial consistency of body parts between the source and target. Comprehensive quantitative and qualitative evaluations demonstrate the superiority of our approach over existing alternatives, particularly in preserving spatial relationships more effectively.

Abstract:
Many 3D mesh processing tasks revolve around generating and manipulating curves on surface meshes. While it is intuitive to explicitly model these curves using mesh edges or parametric curves in the ambient space, these methods often suffer from numerical instability or inaccuracy due to the projection operation. Another natural strategy is to adapt spline based tools, these methods are quite fast but are hard to be extended to more versatile constraints and need heavy manual interactions. In this article, we present an efficient and versatile approach to curve design based on an implicit representation known as the level set. While previous works have explored the use of the level set to generate curves with minimal length, they typically have limitations in accommodating additional conditions for rich and robust control. To address these challenges, we formulate curve editing with constraints like smoothness, interpolation, tangent control, etc., via a level set based variational problem by constraining the values or derivatives of the level set function. However, the widely used gradient flow strategy converges very slowly for this complicated variational problem compared to the classical geodesic one. Thus, we propose to solve it via Newton's method enhanced by local Hessian correction and a trust-region strategy. As a result, our method not only enables versatile control, but also excels in terms of performance due to nearly quadratic convergence and almost linear complexity in each iteration via narrow band acceleration. In practice, these advantages effectively benefit various applications, such as interactive curve manipulation, boundary smoothing for surface segmentation and path planning with obstacles as demonstrated.

Abstract:
Neural radiance fields (NeRF) have achieved great success in novel view synthesis and 3D representation for static scenarios. Existing dynamic NeRFs usually exploit a locally dense grid to fit the deformation fields; however, they fail to capture the global dynamics and concomitantly yield models of heavy parameters. We observe that the 4D space is inherently sparse. First, the deformation fields are sparse in spatial but dense in temporal due to the continuity of motion. Second, the radiance fields are only valid on the surface of the underlying scene, usually occupying a small fraction of the whole space. We thus represent the 4D scene using a learnable sparse latent space, a.k.a. SLS4D. Specifically, SLS4D first uses dense learnable time slot features to depict the temporal space, from which the deformation fields are fitted with linear multi-layer perceptions (MLP) to predict the displacement of a 3D position at any time. It then learns the spatial features of a 3D position using another sparse latent space. This is achieved by learning the adaptive weights of each latent feature with the attention mechanism. Extensive experiments demonstrate the effectiveness of our SLS4D: It achieves the best 4D novel view synthesis using only about 6% parameters of the most recent work.

Abstract:
Surface reconstruction has traditionally relied on the Multi-View Stereo (MVS)-based pipeline, which often suffers from noisy and incomplete geometry. This is due to that although MVS has been proven to be an effective way to recover the geometry of the scenes, especially for locally detailed areas with rich textures, it struggles to deal with areas with low texture and large variations of illumination where the photometric consistency is unreliable. Recently, Neural Implicit Surface Reconstruction (NISR) combines surface rendering and volume rendering techniques and bypasses the MVS as an intermediate step, which has emerged as a promising alternative to overcome the limitations of traditional pipelines. While NISR has shown impressive results on simple scenes, it remains challenging to recover delicate geometry from uncontrolled real-world scenes which is caused by its underconstrained optimization. To this end, the framework PSDF is proposed which resorts to external geometric priors from a pretrained MVS network and internal geometric priors inherent in the NISR model to facilitate high-quality neural implicit surface learning. Specifically, the visibility-aware feature consistency loss and depth prior-assisted sampling based on external geometric priors are introduced. These proposals provide powerfully geometric consistency constraints and aid in locating surface intersection points, thereby significantly improving the accuracy and delicate reconstruction of NISR. Meanwhile, the internal prior-guided importance rendering is presented to enhance the fidelity of the reconstructed surface mesh by mitigating the biased rendering issue in NISR. Extensive experiments on Tanks and Temples datasets show that PSDF achieves state-of-the-art performance on complex uncontrolled scenes.

Abstract:
Existing research on sensemaking in immersive analytics systems primarily focuses on understanding how users complete analysis within these systems with quantitative and qualitative datasets. However, these user studies mainly concentrate on understanding analysis styles and methodologies from a predominantly novice user study population. While this approach provides excellent initial insights into what users may do within IA systems, it fails to address how professionals may utilize an immersive analytic system for analysis tasks. In our work, we build upon an existing immersive analytics concept - “Immersive Space to Think” to understand how professional user populations differ from novice users in immersive analytic system usage. We conducted a user study with 11 professional intelligence analysts who completed three analysis sessions each. Using our results from this study, we provide deep analysis into how professional users complete sensemaking within immersive analytic systems, compare our findings to previously published findings with a novice user population, and provide insights into how to develop better IA systems to support the professional analyst's strategies within these systems.

Abstract:
Per-view disparity estimation for light field (LF) is critical for various applications such as light field editing, but previous work mostly focuses on estimating disparity for the center view. In this paper, we propose a view-guided cost volume (VGCV), which successfully generates high-quality disparity maps for LF arbitrary view. Unlike previous methods that construct a static cost for center view only, VGCV is designed with view information and can be applicable to arbitrary-view estimation. In particular, since the key to achieving it is to condition cost on view, we extend previous static cost to a conditional one by introducing the spatial and angular information of target view into cost construction and aggregation, experiments show that this way can effectively adapt VGCV to arbitrary-view task. For construction, previous stereo-matching methods usually adopt correlation (e.g., variance) for dynamic estimation, but just using correlation can lose image structure information, which is essential for scene detail recovery, therefore we design an image-guided construction module and use cross-view attention to adapt cost for conditional construction while keeping its spatial information. Then for aggregation, we present a coordinate-guided aggregation module for VGCV regularization, which is specially designed to solve the problem of LF view deviation. Finally, we implement a Light Field Arbitrary-View Disparity Estimation Network (LFAVNet), then perform it on both synthetic and real LFs. Experiments demonstrate that LFAVNet can generate a higher-quality disparity map for arbitrary view in LF. We also extend our method to center-view estimation and light field editing tasks, which all achieve advanced performance.

Abstract:
3D shapes captured by scanning devices are often incomplete due to occlusion. 3D shape completion methods have been explored to tackle this limitation. However, most of these methods are only trained and tested on a subset of categories, resulting in poor generalization to unseen categories. In this article, we propose a novel weakly-supervised framework to reconstruct the complete shapes from unseen categories. We first propose an end-to-end prior-assisted shape learning network that leverages data from the seen categories to infer a coarse shape. Specifically, we construct a prior bank consisting of representative shapes from the seen categories. Then, we design a multi-scale pattern correlation module for learning the complete shape of the input by analyzing the correlation between local patterns within the input and the priors at various scales. In addition, we propose a self-supervised shape refinement model to further refine the coarse shape. Considering the shape variability of 3D objects across categories, we construct a category-specific prior bank to facilitate shape refinement. Then, we devise a voxel-based partial matching loss and leverage the partial scans to drive the refinement process. Extensive experimental results show that our approach is superior to state-of-the-art methods by a large margin.

Abstract:
How much can we infer about human shape using an event camera that only detects the pixel position where the luminance changed and its timestamp? This neuromorphic vision technology captures changes in pixel values at ultra-high speeds, regardless of the variations in environmental lighting brightness. Existing methods for human mesh recovery (HMR) from event data need to utilize intensity images captured with a generic frame-based camera, rendering them vulnerable to low-light conditions, energy/memory constraints, and privacy issues. In contrast, we explore the potential of solely utilizing event data to alleviate these issues and ascertain whether it offers adequate cues for HMR, as illustrated in Fig. 1. This is a quite challenging task due to the substantially limited information ensuing from the absence of intensity images. To this end, we propose EventPointMesh, a framework which treats event data as a three-dimensional (3D) spatio-temporal point cloud for reconstructing the human mesh. By employing a coarse-to-fine pose feature extraction strategy, we extract both global features and local features. The local features are derived by processing the spatio-temporally dispersed event points into groups associated with individual body segments. This combination of global and local features allows the framework to achieve a more accurate HMR, capturing subtle differences in human movements. Experiments demonstrate that our method with only sparse event data outperforms baseline methods.

Abstract:
This article presents a systematic study of the generalization of convolutional neural networks (CNNs) and humans on relational reasoning tasks with bar charts. We first revisit previous experiments on graphical perception and update the benchmark performance of CNNs. We then test the generalization performance of CNNs on a classic relational reasoning task: estimating bar length ratios in a bar chart, by progressively perturbing the standard visualizations. We further conduct a user study to compare the performance of CNNs and humans. Our results show that CNNs outperform humans only when the training and test data have the same visual encodings. Otherwise, they may perform worse. We also find that CNNs are sensitive to perturbations in various visual encodings, regardless of their relevance to the target bars. Yet, humans are mainly influenced by bar lengths. Our study suggests that robust relational reasoning with visualizations is challenging for CNNs. Improving CNNs’ generalization performance may require training them to better recognize task-related visual properties.

Abstract:
Despite the remarkable process in the field of arbitrary image style transfer (AST), inconsistent evaluation continues to plague style transfer research. Existing methods often suffer from limited objective evaluation and inconsistent subjective feedback, hindering reliable comparisons among AST variants. In this study, we propose a multi-granularity assessment system that combines standardized objective and subjective evaluations. We collect a fine-grained dataset considering a range of image contexts such as different scenes, object complexities, and rich parsing information from multiple sources. Objective and subjective studies are conducted using the collected dataset. Specifically, we innovate on traditional subjective studies by developing an online evaluation system utilizing a combination of point-wise, pair-wise, and group-wise questionnaires. Finally, we bridge the gap between objective and subjective evaluations by examining the consistency between the results from the two studies. We experimentally evaluate CNN-based, flow-based, transformer-based, and diffusion-based AST methods by the proposed multi-granularity assessment system, which lays the foundation for a reliable and robust evaluation. Providing standardized measures, objective data, and detailed subjective feedback empowers researchers to make informed comparisons and drive innovation in this rapidly evolving field.

Abstract:
In today's data-rich environment, visualization literacy—the ability to understand and communicate information through charts—is increasingly important. However, constructing effective charts can be challenging due to the numerous design choices involved. Off-the-shelf systems and libraries produce charts with carefully selected defaults that users may not be aware of, making it hard to increase their visualization literacy with those systems. In addition, traditional ways of improving visualization literacy, such as textbooks and tutorials, can be burdensome as they require sifting through a plethora of resources. To address this challenge, we designed Iguanodon, an easy-to-use game application that complements the traditional methods of improving visualization construction literacy. In our game application, users interactively choose whether to apply design choices, which we assign to sub-tasks that must be optimized to create an effective chart. The application offers multiple game variations to help users learn how different design choices should be applied to construct effective charts. Furthermore, our approach easily adapts to different visualization design guidelines. We describe the application's design and present the results of a user study with 37 participants. Our findings indicate that our game-based approach supports users in improving their visualization literacy.

Abstract:
The speech-driven facial animation technology is generally categorized into two main types: 3D and 2D talking face. Both of these have garnered considerable research attention in recent years. However, to our knowledge, the research into 3D talking face has not progressed as deeply as that of 2D talking face, particularly in terms of lip-sync and perceptual mouth movements. The lip-sync necessitates an impeccable synchronization between mouth motion and speech audio. The speech perception derived from the perceptual mouth movements should resemble that of the driving audio. To mind the gap between the two sub-fields, we propose Learn2Talk, a learning framework that enhances 3D talking face network by integrating two key insights from the field of 2D talking face. First, drawing inspiration from the audio-video sync network, we develop a 3D sync-lip expert model for the pursuit of lip-sync between audio and 3D facial motions. Second, we utilize a teacher model, carefully chosen from among 2D talking face methods, to guide the training of the audio-to-3D motions regression network, thereby increasing the accuracy of 3D vertex movements. Extensive experiments demonstrate the superiority of our proposed framework over state-of-the-art methods in terms of lip-sync, vertex accuracy and perceptual movements. Finally, we showcase two applications of our framework: audio-visual speech recognition and speech-driven 3D Gaussian Splatting-based avatar animation.

Abstract:
3D style transfer aims to generate stylized views of 3D scenes with specified styles, which requires high-quality generating and keeping multi-view consistency. Existing methods still suffer the challenges of high-quality stylization with texture details and stylization with multimodal guidance. In this paper, we reveal that the common training method of stylization with NeRF, which generates stylized multi-view supervision by 2D style transfer models, causes the same object in supervision to show various states (color tone, details, etc.) in different views, leading NeRF to tend to smooth the texture details, further resulting in low-quality rendering for 3D multi-style transfer. To tackle these problems, we propose a novel Multimodal-guided 3D Multi-style transfer of NeRF, termed MM-NeRF. First, MM-NeRF projects multimodal guidance into a unified space to keep the multimodal styles consistency and extracts multimodal features to guide the 3D stylization. Second, a novel multi-head learning scheme is proposed to relieve the difficulty of learning multi-style transfer, and a multi-view style consistent loss is proposed to track the inconsistency of multi-view supervision data. Finally, a novel incremental learning mechanism is proposed to generalize MM-NeRF to any new style with small costs. Extensive experiments on several real-world datasets show that MM-NeRF achieves high-quality 3D multi-style stylization with multimodal guidance, and keeps multi-view consistency and style consistency between multimodal guidance.

Abstract:
Reconstructing CSG trees from CAD models is a critical subject in reverse engineering. While there have been notable advancements in CSG reconstruction, challenges persist in capturing geometric details and achieving efficiency. Additionally, since non-axis-aligned volumetric primitives cannot maintain coplanar characteristics due to discretization errors, existing Boolean operations often lead to zero-volume surfaces and suffer from topological errors during the CSG modeling process. To address these issues, we propose a novel workflow to achieve fast CSG reconstruction and reliable forward modeling. First, we employ feature removal and model subdivision techniques to decompose models into sub-components. This significantly expedites the reconstruction by simplifying the complexity of the models. Then, we introduce a more reasonable method for primitive generation and filtering, and utilize a size-related optimization approach to reconstruct CSG trees. By re-adding features as additional nodes in the CSG trees, our method not only preserves intricate details but also ensures the conciseness, semantic integrity, and editability of the resulting CSG tree. Finally, we develop a coplanar primitive discretization method that represents primitives as large planes and extracts the original triangles after intersection. We extend the classification of triangles and incorporate a coplanar-aware Boolean tree assessment technique, allowing us to achieve manifold and watertight modeling results without zero-volume surfaces, even in extreme degenerate cases. We demonstrate the superiority of our method over state-of-the-art approaches. Moreover, the reconstructed CSG trees generated by our method contain extensive semantic information, enabling diverse model editing tasks.

Abstract:
Intrinsic decomposition separates an image into reflectance and shading, which contributes to image editing, augmented reality, etc. Despite recent efforts dedicated to this field, effectively separating colored illumination from reflectance and correctly restoring it into shading remains an challenge. We propose a deep intrinsic decomposition method to address this issue. Specifically, by transforming intrinsic decomposition process in RGB image domains into the combination of intensity and chromaticity domains, we propose a novel macro intrinsic decomposition network framework. This framework enables the generation of finer intrinsic components through more relevant features propagation and more detailed sub-constraints guidance. In order to expand the macro network, we integrate multiple attention mechanism modules in key positions of encoders, which enhances the extraction of distinct features. We also propose a skip connection module based on specific deep features guidance, which can filter out features that are physically irrelevant to each intrinsic component. Our method not only outperforms state-of-the-art methods across multiple datasets, but also robustly separates illumination from reflectance and restores it into shading in various types of images. By leveraging our intrinsic images, we achieve visually superior image editing effects compared to other methods, while also being able to manipulate the inherent lighting of the original scene.

Abstract:
Simulating the coupled dynamics between rigid bodies and two-phase fluids, especially those with a large density ratio and a high Reynolds number, is computationally demanding but visually compelling with a broad range of applications. Traditional approaches that directly solve the Navier-Stokes equations often struggle to reproduce these flow phenomena due to stronger numerical diffusion, resulting in lower accuracy. While recent advancements in kinetic lattice Boltzmann methods for two-phase flows have notably enhanced efficiency and accuracy, challenges remain in correctly managing fluid-rigid boundaries, resulting in physically inconsistent results. In this article, we propose a novel kinetic framework for fluid-rigid interaction involving two fluid phases. Our approach leverages the idea of an overset grid, and proposes a novel formulation in the two-phase flow context with multiple improvements to handle complex scenarios and support moving multi-resolution domains with boundary layer control. These new contributions successfully resolve many issues inherent in previous methods and enable physically more consistent simulations of two-phase flow phenomena. We have conducted both quantitative and qualitative evaluations, compared our method to previous techniques, and validated its physical consistency through real-world experiments. Additionally, we demonstrate the versatility of our method across various scenarios.

Abstract:
Although remarkable progress has been made in image style transfer, style is just one of the components of artistic paintings. Directly transferring extracted style features to natural images often results in outputs with obvious synthetic traces. This is because key painting attributes including layout, perspective, shape, and semantics often cannot be conveyed and expressed through style transfer. Large-scale pretrained text-to-image generation models have demonstrated their capability to synthesize a vast amount of high-quality images. However, even with extensive textual descriptions, it is challenging to fully express the unique visual properties and details of paintings. Moreover, generic models often disrupt the overall artistic effect when modifying specific areas, making it more complicated to achieve a unified aesthetic in artworks. Our main novel idea is to integrate multimodal semantic information as a synthesis guide into artworks, rather than transferring style to the real world. We also aim to reduce the disruption to the harmony of artworks while simplifying the guidance conditions. Specifically, we propose an innovative multi-task unified framework called CreativeSynth, based on the diffusion model with the ability to coordinate multimodal inputs. CreativeSynth combines multimodal features with customized attention mechanisms to seamlessly integrate real-world semantic content into the art domain through Cross-Art-Attention for aesthetic maintenance and semantic fusion. We demonstrate the results of our method across a wide range of different art categories, proving that CreativeSynth bridges the gap between generative models and artistic expression.

Abstract:
We introduce the Stress-Guided Lightweight Design Benchmark (SGLDBench), a comprehensive benchmark suite for applying and evaluating material layout strategies to generate stiff, lightweight designs in 3D domains. SGLDBench provides a seamlessly integrated simulation and analysis framework, including six reference strategies and a scalable multigrid elasticity solver to efficiently execute these strategies and validate the stiffness of their results. This facilitates the systematic analysis and comparison of design strategies based on the mechanical properties they achieve. SGLDBench enables the evaluation of diverse load conditions and, through the tight integration of the solver, supports high-resolution designs and stiffness analysis. Additionally, SGLDBench emphasizes visual analysis to explore the relationship between the geometric structure of a design and the distribution of stresses, offering insights into the specific properties and behaviors of different design strategies. SGLDBench’s specific features are highlighted through several experiments, comparing the results of reference strategies with respect to geometric and mechanical properties.

Abstract:
Structural representation is crucial for reconstructing and generating editable 3D shapes with part semantics. Recent 3D shape generation works employ complicated networks and structure definitions relying on hierarchical annotations and pay less attention to the details inside parts. In this paper, we propose the method that parameterizes the shared structure in the same category using a differentiable template and corresponding fixed-length parameters. Specific parameters are fed into the template to calculate cuboids that indicate a concrete shape. We utilize the boundaries of three-view renderings of each cuboid to further describe the inside details. Shapes are represented with the parameters and three-view details inside cuboids, from which the SDF can be calculated to recover the object. Benefiting from our fixed-length parameters and three-view details, our networks for reconstruction and generation are simple and effective to learn the latent space. Our method can reconstruct or generate diverse shapes with complicated details, and interpolate them smoothly. Extensive evaluations demonstrate the superiority of our method on reconstruction from point cloud, generation, and interpolation.

Abstract:
The success of deep learning heavily relies on the large amount of training samples. However, in scientific visualization, due to the high computational cost, only few data are available during training, which limits the performance of deep learning. A common technique to address the data sparsity issue is data augmentation. In this paper, we present a comprehensive study on nine data augmentation techniques (i.e., noise injection, interpolation, scale, flip, rotation, variational auto-encoder, generative adversarial network, diffusion model, and implicit neural representation) for understanding their effectiveness on two scientific visualization tasks, i.e., spatial super-resolution and ambient occlusion prediction. We compare the data quality, rendering fidelity, optimization time, and memory consumption of these data augmentation techniques using several scientific datasets with various characteristics. We investigate the effects of data augmentation on the method, quantity, and diversity for these tasks with various deep learning models. Our study shows that increasing the quantity and single-domain diversity of augmented data can boost model performance, while the method and cross-domain diversity of the augmented data do not have the same impact. Based on our findings, we discuss the opportunities and future directions for scientific data augmentation.

Abstract:
Human head-eye coordination is a complex behavior, shaped by physiological constraints, psychological context, and gaze intent. Current context-specific gaze models in both psychology and graphics fail to produce plausible head-eye coordination for general patterns of human gaze behavior. In this paper, we: 1) propose and validate an experimental protocol to collect head-eye motion data during sequential look-at tasks in Virtual Reality; 2) identify factors influencing head-eye coordination using this data; and 3) introduce a head-eye coordinated Inverse Kinematic gaze model Head-EyeK that integrates these insights. Our evaluation of Head-EyeK is three-fold: we show the impact of algorithmic parameters on gaze behavior; we show a favorable comparison to prior art both quantitatively against ground-truth data, and qualitatively using a perceptual study; and we show multiple scenarios of complex gaze behavior credibly animated using Head-EyeK.

Abstract:
Tooth arrangement is an essential step in the digital orthodontic planning process. Existing learning-based methods use hidden teeth features to directly regress teeth motions, which couples target pose perception and motion regression. It could lead to poor perceptions of three-dimensional transformation. They also ignore the possible overlaps or gaps between teeth of predicted dentition, which is generally unacceptable. Therefore, we propose DTAN, a differentiable collision-supervised tooth arrangement network, decoupling predicting tasks and feature modeling. DTAN decouples the tooth arrangement task by first predicting the hidden features of the final teeth poses and then using them to assist in regressing the motions between the beginning and target teeth. To learn the hidden features better, DTAN also decouples the teeth-hidden features into geometric and positional features, which are further supervised by feature consistency constraints. Furthermore, we propose a novel differentiable collision loss function for point cloud data to constrain the related gestures between teeth, which can be easily extended to other 3D point cloud tasks. We propose an arch-width guided tooth arrangement network, named C-DTAN, to make the results controllable. We construct three different tooth arrangement datasets and achieve drastically improved performance on accuracy and speed compared with existing methods.

Abstract:
We present a novel, physically-based morphing technique for elastic shapes, leveraging the differentiable material point method (MPM) with space-time control through per-particle deformation gradients to accommodate complex topology changes. This approach, grounded in MPM’s natural handling of dynamic topologies, is enhanced by a chained iterative optimization technique, allowing for the creation of both succinct and extended morphing sequences that maintain coherence over time. Demonstrated across various challenging scenarios, our method is able to produce detailed elastic deformation and topology transitions, all grounded within our physics-based simulation framework.

Abstract:
This paper presents Qffusion, a dual-frame-guided framework for portrait video editing. Specifically, we consider a design principle of “animation for editing”, and train Qffusion as a general animation framework from two still reference images while we can use it for portrait video editing easily by applying modified start and end frames as references during inference. Leveraging the powerful generative power of Stable Diffusion, we propose a Quadrant-grid Arrangement (QGA) scheme for latent re-arrangement, which arranges the latent codes of two reference images and that of four facial conditions into a four-grid fashion, separately. Then, we fuse features of these two modalities and use self-attention for both appearance and temporal learning, where representations at different times are jointly modeled under QGA. Our Qffusion can achieve stable video editing without additional networks or complex training stages, where only the input format of Stable Diffusion is modified. Further, we propose a Quadrant-grid Propagation (QGP) inference strategy, which enjoys a unique advantage on stable arbitrary-length video generation by processing reference and condition frames recursively. Through extensive experiments, Qffusion consistently outperforms state-of-the-art techniques on portrait video editing.

Abstract:
Complex surfaces are commonly observed in various applications and have significant value in enhancing comfort, aesthetics, and functionality. However, their fabrication often involves complex and costly processes. To simplify the fabrication difficulty, significant research has focused on using 3D developable surfaces to approximate target 3D surfaces. This process involves converting target 3D surfaces into developable surfaces and then flattening them into 2D patterns. Since the geometric and topological diversity of target surfaces, this task is both comprehensive and intricate, encompassing multiple aspects from design to fabrication. In this paper, we review relevant technologies and methods in fabrication processes, classify them, and summarize a pipeline from design to fabrication. This provides a comprehensive introduction to the field for researchers and practitioners. Through the analysis of relevant literature, we also discuss some of the research challenges and future research opportunities.

Abstract:
Narrative visualization transforms data into engaging stories, making complex information accessible to a broad audience. Foundation models, with their advanced capabilities such as natural language processing, content generation, and multimodal integration, hold substantial potential for enriching narrative visualization. Recently, a collection of techniques have been introduced for crafting narrative visualizations based on foundation models from different aspects. We build our survey upon 66 articles to study how foundation models can progressively engage in this process and then propose a reference model categorizing the reviewed literature into four essential phases: Analysis, Narration, Visualization, and Interaction. Furthermore, we identify eight specific tasks (e.g., Insight Extraction and Authoring) where foundation models are applied across these stages to facilitate the creation of visual narratives. Detailed descriptions, related literature, and reflections are presented for each task. To make it a more impactful and informative experience for diverse readers, we discuss key research problems and provide the strengths and weaknesses in each task to guide people in identifying and seizing opportunities while navigating challenges in this field.

Abstract:
Diverging from conventional explicit geometric representations, neural implicit representations utilize continuous function approximators to encode 3D surfaces through parametric formulations including signed distance fields (SDF), unsigned distance fields (UDF), occupancy fields (OF), and neural radiance fields (NeRF). These approaches demonstrate superior multi-view reconstruction fidelity by inherently supporting non-manifold geometries and complex topological variations, establishing themselves as foundational tools in 3D reconstruction. Neural implicit representations can be applied to a diverse array of reconstruction tasks, including object-level reconstruction, scene-level reconstruction, open-surface reconstruction and dynamic reconstruction. The exponential advancement of neural implicit representations in 3D reconstruction necessitates systematic analysis of their evolving methodologies and applications. This survey presents a structured synthesis of cutting-edge research from 2020-2025, establishing a dual-axis taxonomy that categorizes techniques by geometric representation types and application scenarios. Through this survey, we aim to familiarize emerging researchers with the current landscape of neural implicit representation in surface reconstruction, assess innovative contributions and limitations in existing research, and encourage prospective research directions.

Abstract:
We present a novel data-driven Parametric Linear Blend Skinning (PLBS) model meticulously crafted for generalized 3D garment dressing and animation. Previous data-driven methods are impeded by certain challenges including overreliance on human body modeling and limited adaptability across different garment shapes. Our method resolves these challenges via two goals: 1) Develop a model based on garment modeling rather than human body modeling. 2) Separately construct low-dimensional sub-spaces for modeling in-plane deformation (such as variation in garment shape and size) and out-of-plane deformation (such as deformation due to varied body size and motion). Therefore, we formulate garment deformation as a PLBS model controlled by canonical 3D garment mesh, vertex-based skinning weights and associated local patch transformation. Unlike traditional LBS models specialized for individual objects, PLBS model is capable of uniformly expressing varied garments and bodies, the in-plane deformation is encoded on the canonical 3D garment and the out-of-plane deformation is controlled by the local patch transformation. Besides, we propose novel 3D garment registration and skinning weight decomposition strategies to obtain adequate data to build PLBS model under different garment categories. Furthermore, we employ dynamic fine-tuning to complement high-frequency signals missing from LBS for unseen testing data. Experiments illustrate that our method is capable of modeling dynamics for loose-fitting garments, outperforming previous data-driven modeling methods using different sub-space modeling strategies. We showcase that our method can factorize and be generalized for varied body sizes, garment shapes, garment sizes and human motions under different garment categories.

Abstract:
We offer a new model of the sensemaking process for data analysis and visualization. Whereas past sensemaking models have been grounded in positivist assumptions about the nature of knowledge, we reframe data sensemaking in critical, humanistic terms by approaching it through an interpretivist lens. Our three-phase process model uses the analogy of an iceberg, where data is the visible tip of underlying schemas. In the Add phase, the analyst acquires data, incorporates explicit schemas from the data, and absorbs the tacit schemas of both data and people. In the Check phase, the analyst interprets the data with respect to the current schemas and evaluates whether the schemas match the data. In the Refine phase, the analyst considers the role of power, articulates what was tacit into explicitly stated schemas, updates data, and formulates findings. Our model has four important distinguishing features: Tacit and Explicit Schemas, Schemas First and Always, Data as a Schematic Artifact, and Schematic Multiplicity. We compare the roles of schemas in past sensemaking models and draw conceptual distinctions based on a historical review of schemas in different academic traditions. We validate the descriptive and prescriptive power of our model through four analysis scenarios: noticing uncollected data, learning to wrangle data, downplaying inconvenient data, and measuring with sensors. We conclude by discussing the value of interpretivism, the virtue of epistemic humility, and the pluralism this sensemaking model can foster.

Abstract:
Identifying causality behind complex systems plays a significant role in different domains, such as decision-making, policy implementations, and management recommendations. However, existing causality studies on temporal event sequence data mainly focus on individual causal discovery, which is incapable of capturing combined causality. To address the gap in combined causality discovery on temporal event sequence data, eliminating and recruiting principles are defined to balance the effectiveness and controllability of cause combinations. We also leverage the Granger causality algorithm based on the Reactive point processes to describe impelling or inhibiting behavior patterns among entities. In addition, we design an informative and aesthetic visual metaphor of “electrocircuit” to encode aggregated causality for ensuring that our causality visualization exhibits no node-overlap, no edge-intersection, and no link-ambiguity. Aggregation layout, diverse sorting strategies, and smooth interactions are also integrated into our directed, weighted, and parallel-based hypergraph for illustrating combined causality. Our developed combined causality visual analysis system, namely VAC^22, can help users effectively explore combined causes as well as individual causes. This interactive system supports multi-level causality exploration with diverse ordering strategies and a focus+context technique to help users obtain different levels of information abstraction. The usefulness and effectiveness of our work are further evaluated by conducting two case studies and a controlled user study on event sequence data.

Abstract:
We present a novel learning-based method for predicting tooth movements in orthodontic treatment path planning (orthodontic staging). Recognizing the multi-solution nature of orthodontic staging, our approach involves generating the staging sequence progressively with a dedicated Transformer model. This model predicts teeth movements within a predefined number of steps (e.g., 10 or 20), targeting alignment in problematic dentition. The Transformer refines its predictions iteratively, building on previous outcomes until reaching a state that aligns with the target within an acceptable distance. This mirrors real-life scenarios where orthodontists dynamically adjust staging plans based on treatment outcomes. Our Transformer model is tailored to incorporate spatial and temporal attentions, addressing inter-tooth and inter-step interactions, respectively. These attentions are further refined with relative positional encoding. Recognizing the significant influence of tooth shape on the alignment process, we propose integrating a tooth-wise shape encoder to extract morphological features from the 3D teeth point cloud. These features are then fused into the Transformer, facilitating the capture of inter-tooth dynamics during staging, in collaboration with spatial attention. We validate the proposed method on a large-scale dataset that contains 10K real-life orthodontic cases. The results show that our method outperforms the state-of-the-art, and orthodontists favor its predictions.

Abstract:
The accurate and diversified generation of motion sequences for virtual characters poses both an enticing and challenging task within the domain of 3D animation and game content production. To achieve a natural and realistic full-body motion, the movements of virtual characters must adhere to a set of constraints, promoting reliable and seamless pose-changing. This study presents a two-stage model specifically designed to learn Inverse Kinematics (IK) constraints from the representative quadruped character poses. In the first stage, we employ frequency analysis to decompose motion poses into the base-level and style-level components. The base-level content encapsulates the global correlations in the dataset, while the style-level variation centers on distinguishing the local attributes in similar data elements. In order to construct data correlations among poses, we embed the decomposed pose feature into a latent space in the second stage. The kernel matrix of the embedding, which is refined from the original joint angles to the decomposed representation and the IK constraints, creates a more compact distribution of the pose similarity and also guarantees a plausible sampling result with certain IK constraints. Moreover, new motions from the edited IK constraints can also be generated by proposing a searching strategy to adapt to our latent embedding. Experimental results reveal that our method is competitive with the state-of-the-art synthetic approaches in terms of accuracy, highlighting our considerable potential for high efficiency in the animation production.

Abstract:
With the prevalence of smart contracts, smart Ponzi schemes have become a common fraud on blockchain and have caused significant financial loss to cryptocurrency investors in the past few years. Despite the critical importance of detecting smart Ponzi schemes, a reliable and transparent identification approach adaptive to various smart Ponzi schemes is still missing. To fill the research gap, we first extract semantic-meaningful actions to represent the execution behaviors specified in smart contract bytecodes, which are derived from a literature review and in-depth interviews with domain experts. We then propose PonziLens+, a novel visual analytic approach that provides an intuitive and reliable analysis of Ponzi-scheme-related features within these execution behaviors. PonziLens+ has three visualization modules that intuitively reveal all potential behaviors of a smart contract, highlighting fraudulent features across three levels of detail. It can help smart contract investors and auditors achieve confident identification of any smart Ponzi schemes. We conducted two case studies and in-depth user interviews with 12 domain experts and common investors to evaluate PonziLens+. The results demonstrate the effectiveness and usability of PonziLens+ in achieving an effective identification of smart Ponzi schemes.

Abstract:
Meshless simulation of elasticity is important for deformable simulation in computer graphics. While shape matching is a popular meshless solution, it is limited to a subset of elastic constitutive models, challenging the simulation of generic elastic constitutive models using meshless integration. In contrast, peridynamics offers a more versatile capacity and can describe various material behavior through non-local interactions between vertices. However, the size of each stencil Hessian matrix varies with the number of nearby integration points, leading to inefficiency and accuracy loss. To address these challenges, we present an efficient and robust solver for generic elastic models based on peridynamics. We propose an efficient first-order Hessian proxy derived from the positive-negative decomposition of the stress tensor. The proposed symmetric positive definite proxies ensure convergence within a reasonable number of iterations while also being easy to parallelize on GPU. To further enhance stability, particularly for hyperelastic models, we propose enforcing strain limiting between peridynamics bonds to prevent tensile instability in meshless integration. Our algorithm includes two iteration loops of strain limiting and elastic Jacobis, and the pipeline is well-suited for GPU implementation. We evaluated the performance of our approach with a wide range of elastic constitutive models in diverse testing scenarios against the alternative numerical solvers. Our method features superior efficiency and faster convergence compared to existing numerical solvers. These compelling results underscore the practicality and effectiveness of our method for simulating elasticity via meshless integration.

Abstract:
Rendering animatable and realistic hand avatars is pivotal for enhancing user experiences in human-centered AR/VR applications. While recent initiatives have utilized neural radiance fields to forge hand avatars with lifelike appearances, these methods are often hindered by high computational demands and the necessity for extensive training views. In this paper, we introduce GaussianHand, the first Gaussian-based real-time 3D rendering approach that enables efficient free-view and free-pose hand avatar animation from sparse view images. Our approach encompasses two key innovations. We first propose Hand Gaussian Blend Shapes that effectively models hand surface geometry while ensuring consistent appearance across various poses. Second, we introduce the Neural Residual Skeleton, equipped with Residual Skinning Weights, designed to rectify inaccuracies involved in Linear Blend Skinning deformations due to geometry offsets. Experiments demonstrate that our method not only achieves far more realistic rendering quality with as few as 5 or 20 training views, compared to the 139 views required by existing methods, but also excels in efficiency, achieving up to 125 frames per second for real-time rendering and remarkably surpassing recent methods.

Abstract:
Passive haptics utilize the shape of a physical object to convey feedback to the user and enhance immersion in virtual reality. Haptic retargeting is a passive haptic interaction method. Its mapping of physical objects to virtual objects solves the matching problem between virtual and physical objects in the passive haptic method. However, most existing haptic retargeting methods improve efficiency without considering the important factor of user comfort. In this article, we propose an efficient and comfortable haptic retargeting method based on reset point optimization. First, we construct two maps indicating user interaction comfort: the RULA score map and the dominant hand gain map. Subsequently, we propose a reset point optimization algorithm based on these two maps. Moreover, we also optimize the selection of the physical proxy and the placement location when the reset occurs. The user study results show a significant improvement in the efficiency and comfort of our method compared to state-of-the-art methods.

Abstract:
Text-to-image generation models have significantly broadened the horizons of creative expression through the power of natural language. However, navigating these models to generate unique concepts, alter their appearance, or reimagine them in unfamiliar roles presents an intricate challenge. For instance, how can we exploit language-guided models to transpose an anime character into a different art style, or envision a beloved character in a radically different setting or role? This paper unveils a novel approach named DreamAnime, designed to provide this level of creative freedom. Using a minimal set of 2–3 images of a user-specified concept such as an anime character or an art style, we teach our model to encapsulate its essence through novel “words” in the embedding space of a pre-existing text-to-image model. Crucially, we disentangle the concepts of style and identity into two separate “words”, thus providing the ability to manipulate them independently. These distinct “words” can then be pieced together into natural language sentences, promoting an intuitive and personalized creative process. Empirical results suggest that this disentanglement into separate word embeddings successfully captures a broad range of unique and complex concepts, with each word focusing on style or identity as appropriate. Comparisons with existing methods illustrate DreamAnime's superior capacity to accurately interpret and recreate the desired concepts across various applications and tasks.

Abstract:
Exemplar-based colorization aims to generate plausible colors for a grayscale image with the guidance of a color reference image. The main challenging problem is finding the correct semantic correspondence between the target image and the reference image. However, the colors of the object and background are often confused in the existing methods. Besides, these methods usually use simple encoder-decoder architectures or pyramid structures to extract features and lack appropriate fusion mechanisms, which results in the loss of high-frequency information or high complexity. To address these problems, this article proposes a lightweight semantic attention-guided Laplacian pyramid network (SAGLP-Net) for deep exemplar-based colorization, exploiting the inherent multi-scale properties of color representations. They are exploited through a Laplacian pyramid, and semantic information is introduced as high-level guidance to align the object and background information. Specially, a semantic guided non-local attention fusion module is designed to exploit the long-range dependency and fuse the local and global features. Moreover, a Laplacian pyramid fusion module based on criss-cross attention is proposed to fuse high frequency components in the large-scale domain. An unsupervised multi-scale multi-loss training strategy is further introduced for network training, which combines pixel loss, color histogram loss, total variance regularisation, and adversarial loss. Experimental results demonstrate that our colorization method achieves better subjective and objective performance with lower complexity than the state-of-the-art methods.

Abstract:
In volume rendering, transfer functions are used to classify structures of interest, and to assign optical properties such as color and opacity. They are commonly defined as 1D or 2D functions that map simple features to these optical properties. As the process of designing a transfer function is typically tedious and unintuitive, several approaches have been proposed for their interactive specification. In this article, we present a novel method to define transfer functions for volume rendering by leveraging the feature extraction capabilities of self-supervised pre-trained vision transformers. To design a transfer function, users simply select the structures of interest in a slice viewer, and our method automatically selects similar structures based on the high-level features extracted by the neural network. Contrary to previous learning-based transfer function approaches, our method does not require training of models and allows for quick inference, enabling an interactive exploration of the volume data. Our approach reduces the amount of necessary annotations by interactively informing the user about the current classification, so they can focus on annotating the structures of interest that still require annotation. In practice, this allows users to design transfer functions within seconds, instead of minutes. We compare our method to existing learning-based approaches in terms of annotation and compute time, as well as with respect to segmentation accuracy. Our accompanyingvideo showcases the interactivity and effectiveness of our method.

Abstract:
3D Gaussian Splatting (3D-GS) has emerged as a significant advancement in the field of computer graphics and 3D vision, offering explicit scene representation and novel view synthesis without the reliance on neural networks. This technique has found diverse applications in areas such as robotics, urban mapping, autonomous navigation, and virtual reality/augmented reality, just name a few. Given the growing popularity and expanding research in 3D-GS, this paper presents a comprehensive survey of relevant papers from the past year. We organize the survey into taxonomies based on characteristics and applications, providing an introduction to the theoretical underpinnings of 3D-GS. The survey aims to introduce the theoretical foundations of 3D Gaussian Splatting and provide a reference for new researchers while inspiring future research directions.

Abstract:
The article presents a new method for constructing self-supporting surfaces using arch beams that are designed to convert their thrust into supporting force, thereby eliminating shear stress and bending moments. Our method allows for the placement of the arch beams on the boundary or within a surface and partitions the surface into multiple self-supporting parts. The use of arch beams enhances stability and durability, adds aesthetic appeal, and allows for greater flexibility in the design process. We develop an iterative algorithm for designing self-supporting surfaces with arch beams that enables the user to control the shape of the beams and surface through intuitive parameters and specify the desired location of the arch beams. We verify the physical stability of the structure using finite element analysis. Experimental results show that our method can produce visually pleasing self-supporting surfaces that satisfy the equilibrium equation with high accuracy.

Abstract:
We contribute a design study on using visual analytics for AI-assisted music composition. The main result is the interface MAICO (Music AI Co-creativity), which allows composers and other music creators to interactively generate, explore, select, edit, and compare samples from generative music models. MAICO is based on the idea of visual parameter space analysis and supports the simultaneous analysis of hundreds of short samples of symbolic music from multiple models, displaying them in different metric- and similarity-based layouts. We developed and evaluated MAICO together with a professional composer who actively used it for five months to create, among other things, a composition for the Biennale Arte 2024 in Venice, which was recorded by the Munich Symphonic Orchestra. We discuss our design choices and lessons learned from this endeavor to support Human-AI co-creativity with visual analytics.

Abstract:
Degree centrality (DC) is a widely used metric that measures node importance in data space. A node–link diagram is a commonly used graph visualization to help viewers identify important nodes in visual space. Previous graph perception studies largely concentrated on revealing perception principles in visual space. However, they rarely investigated the intrinsic relations between computed and perceived important nodes by jointly using data and visual spaces, thereby hindering a deep integration of computational and interactive graph analytics. To address this gap, we adopted the visual perception of DC as a representative object to conduct a graph perception study by jointly using data and visual spaces. Two research questions were defined. (RQ1) Can viewers accurately estimate the relative DCs of the given nodes in a node–link diagram through visual perception? (RQ2) What visual factors influence viewers’ visual estimation of relative DCs? A controlled user experiment was conducted to answer the questions. Results showed that: (1) The participants failed to estimate the relative DCs accurately, particularly when the DC differences between nodes were not great. (2) Seven visual factors influencing the tasks were summarized, such as the size of the visual receptive region of a node, the link and node densities in the visual receptive region of the node, and the wrapping angle of the node’s neighbors. (3) The factors presented certain priorities in complicated situations. These findings provide rich implications for graph analytics, such as utilizing the findings to optimize graph visualizations to achieve the desired consistency between computed and perceived important nodes.

Abstract:
Unlike their line-based counterparts, surface-based techniques have yet to be thoroughly investigated in flow visualization due to their significant placement, speed, perception, and evaluation challenges. This article presents SurfPatch, a novel framework supporting exploratory stream surface visualization. To begin with, we translate the issue of surface placement to surface selection and trace a large number of stream surfaces from a given flow field dataset. Then, we introduce a three-stage process: vertex-level classification, patch-level matching, and surface-level clustering that hierarchically builds the connection between vertices and patches and between patches and surfaces. This bottom-up approach enables fine-grained, multiscale patch-level matching, sharply contrasts surface-level matching offered by existing works, and provides previously unavailable flexibility during querying. We design an intuitive visual interface for users to conveniently visualize and analyze the underlying collection of stream surfaces in an exploratory manner. SurfPatch is not limited to stream surfaces traced from steady flow datasets. We demonstrate its effectiveness through experiments on stream surfaces produced from steady and unsteady flows as well as isosurfaces extracted from scalar fields.

Abstract:
The Digital Twin Brain (DTB) is an advanced artificial intelligence framework that integrates spiking neurons to simulate complex cognitive functions and collaborative behaviors. For domain experts, visualizing the DTB’s simulation outcomes is essential to understanding complex cognitive activities. However, this task poses significant challenges due to DTB data’s inherent characteristics, including its high-dimensionality, temporal dynamics, and spatial complexity. To address these challenges, we developed DTBIA, an Immersive Visual Analytics System for Brain-Inspired Research. In collaboration with domain experts, we identified key requirements for effectively visualizing spatiotemporal and topological patterns at multiple levels of detail. DTBIA incorporates a hierarchical workflow — ranging from brain regions to voxels and slice sections — along with immersive navigation and a 3D edge bundling algorithm to enhance clarity and provide deeper insights into both functional (BOLD) and structural (DTI) brain data. The utility and effectiveness of DTBIA are validated through two case studies involving with brain research experts. The results underscore the system’s role in enhancing the comprehension of complex neural behaviors and interactions.

Abstract:
The performance of computer vision models in certain real-world applications (e.g., rare wildlife observation) is limited by the small number of available images. Expanding datasets using pre-trained generative models is an effective way to address this limitation. However, since the automatic generation process is uncontrollable, the generated images are usually limited in diversity, and some of them are undesired. In this paper, we propose a human-guided image generation method for more controllable dataset expansion. We develop a multi-modal projection method with theoretical guarantees to facilitate the exploration of both the original and generated images. Based on the exploration, users refine the prompts and re-generate images for better performance. Since directly refining the prompts is challenging for novice users, we develop a sample-level prompt refinement method to make it easier. With this method, users only need to provide sample-level feedback (e.g., which samples are undesired) to obtain better prompts. The effectiveness of our method is demonstrated through the quantitative evaluation of the multi-modal projection method, improved model performance in the case study for both classification and object detection tasks, and positive feedback from the experts.

Abstract:
With the development of eXtended Reality (XR), photo capturing and display technology based on head-mounted displays (HMDs) have experienced significant advancements and gained considerable attention. Egocentric spatial images and videos are emerging as a compelling form of stereoscopic XR content. The assessment for the Quality of Experience (QoE) of XR content is important to ensure a high-quality viewing experience. Different from traditional 2D images, egocentric spatial images present challenges for perceptual quality assessment due to their special shooting, processing methods, and stereoscopic characteristics. However, the corresponding image quality assessment (IQA) research for egocentric spatial images is still lacking. In this paper, we establish the Egocentric Spatial Images Quality Assessment Database (ESIQAD), the first IQA database dedicated for egocentric spatial images as far as we know. Our ESIQAD includes 500 egocentric spatial images and the corresponding mean opinion scores (MOSs) under three display modes, including 2D display, 3D-window display, and 3D-immersive display. Based on our ESIQAD, we propose a novel mamba2-based multi-stage feature fusion model, termed ESIQAnet, which predicts the perceptual quality of egocentric spatial images under the three display modes. Specifically, we first extract features from multiple visual state space duality (VSSD) blocks, then apply cross attention to fuse binocular view information and use transposed attention to further refine the features. The multi-stage features are finally concatenated and fed into a quality regression network to predict the quality score. Extensive experimental results demonstrate that the ESIQAnet outperforms 22 state-of-the-art IQA models on the ESIQAD under all three display modes. The database and code are available at https://github.com/IntMeGroup/ESIQA.

Abstract:
Current mixed reality (MR) designs predominantly prioritise functionality and usability, often overlooking individual's diverse value needs. To create more meaningful MR experiences, this paper aims to address this gap by exploring how human values can be integrated into MR design. We propose a values-based design process and evaluate it through three design workshops considering a remote collaborative learning scenario. By comparing our approach to an existing MR application for collaborative learning, we demonstrate how embedding human values into MR design can lead to more ethical and human-centred outcomes. Our findings contribute to advancing the design and application of MR technologies to be more aligned with people's diverse needs and values.

Abstract:
360° images are a popular medium for bringing photography into virtual reality. While users can look in any direction by rotating their heads, 360° images ultimately look flat. That is because they lack depth information and thus cannot create motion parallax when translating the head. To achieve a fully immersive VR experience from a single 360° image, we introduce a novel method to upgrade 360° images to free-viewpoint renderings with 6 degrees of freedom. Alternative approaches reconstruct textured 3D geometry, which is fast to render but suffers from visible reconstruction artifacts, or use neural radiance fields that produce high-quality novel views but too slowly for VR applications. Our 360°3D photos build on 3D Gaussian splatting as the underlying scene representation to simultaneously achieve high visual quality and real-time rendering speed. To fill plausible content in previously unseen regions, we introduce a novel combination of latent diffusion inpainting and monocular depth estimation with Poisson-based blending. Our results demonstrate state-of-the-art visual and depth quality at rendering rates of 105 FPS per megapixel on a commodity GPU.

Abstract:
Achieving immersive virtual reality (VR) experiences typically requires extensive computational resources to ensure high-definition visuals, high frame rates, and low latency in stereoscopic rendering. This challenge is particularly pronounced for lower-tier and standalone VR devices with limited processing power. To accelerate rendering, existing supersampling and image reprojection techniques have shown significant potential, yet to date, no previous work has explored their combination to minimize stereo rendering overhead. In this paper, we introduce a lightweight supersampling framework that integrates image projection with spatio-temporal supersampling to accelerate stereo rendering. Our approach effectively leverages the temporal and spatial redundancies inherent in stereo videos, enabling rapid image generation for unshaded viewpoints and providing resolution-enhanced and anti-aliased images for binocular viewpoints. We first blend a rendered low-resolution (LR) frame with accumulated temporal samples to construct an high-resolution (HR) frame. This HR frame is then reprojected to the other viewpoint to directly synthesize a new image. To address disocclusions in reprojected images, we utilize accumulated history data and low-pass filtering for filling, ensuring high-quality results with minimal delay. Extensive evaluations on both the PC and the standalone device confirm that our framework requires short runtime to generate high-fidelity images, making it an effective solution for stereo rendering across various VR platforms.

Abstract:
The viewing experience of 3D artifacts in Virtual Reality (VR) museums is constrained and affected by various factors, such as pedestal height, viewing distance, and object scale. User experiences regarding these factors can vary subjectively, making it difficult to identify a universal optimal solution. In this paper, we collect empirical data on user-determined parameters for the optimal viewing experience in VR museums. By modeling users' viewing behaviors in VR museums, we derive predictive functions that configure the pedestal height, calculate the optimal viewing distance, and adjust the appropriate handheld scale for the optimal viewing experience. This led to our novel 3D responsive design, ResponsiveView. Similar to the responsive web design that automatically adjusts for different screen sizes, ResponsiveView automatically adjusts the parameters in the VR environment to facilitate users' viewing experience. The design has been validated with two popular inputs available in current commercial VR devices: controller-based interactions and hand tracking, demonstrating enhanced viewing experience in VR museums.

Abstract:
Cross-reality is an emerging research area concerned with systems operating across different points on the reality-virtuality continuum. These systems are often complex, involving multiple realities and users, and thus there is a need for an overarching design framework, which, despite growing interest has yet to be developed. This paper addresses this need by presenting eleven design patterns for cross-reality applications across the following four categories: fundamental, origin, display, and interaction patterns. To develop these design patterns we analysed a sample of 60 papers, with the goal of identifying recurring solutions. These patterns were then described in form of intent, solution, and application examples, accompanied by a diagram and archetypal example. This paper provides designers with a comprehensive set of patterns that they can use and draw inspiration from when creating cross-reality systems.

Abstract:
We study the correlations between redirected walking (RDW) rotation gains and patterns in users' posture and gaze data during locomotion in virtual reality (VR). To do this, we conducted a psychophysical experiment to measure users' sensitivity to RDW rotation gains and collect gaze and posture data during the experiment. Using multilevel modeling, we studied how different factors of the VR system and user affected their physiological signals. In particular, we studied the effects of redirection gain, trial duration, trial number (i.e., time spent in VR), and participant gender on postural sway, gaze velocity (a proxy for gaze stability), and saccade and blink rate. Our results showed that, in general, physiological signals were significantly positively correlated with the strength of redirection gain, the duration of trials, and the trial number. Gaze velocity was negatively correlated with trial duration. Additionally, we measured users' sensitivity to rotation gains in well-lit (photopic) and dimly-lit (mesopic) virtual lighting conditions. Results showed that there were no significant differences in RDW detection thresholds between the photopic and mesopic luminance conditions.

Abstract:
Aiming tasks are common in VR, but are challenging to perform without vision. They require identifying a target's location and then precisely aiming and selecting it. In this paper, we explore how to support blind people in aiming tasks using a VR Archery scenario. We implemented three techniques: 1) Spatialized Audio, a baseline where the target emits a specific 3D sound to convey its location; 2) Target Confirmation, where the previous condition is augmented with secondary Beep sounds to indicate proximity to the target; and 3) Reticle-Target perspective, where the auditory feedback conveys the relation between the target and the user's aiming reticle. A study with 15 blind participants compared the three techniques under two scenarios: stationary and moving targets. Target Confirmation and Reticle-Target Perspective clearly outperformed Spatialized Audio, but user preferences were evenly split between these two techniques. We discuss how our findings may support the development of VR experiences that are more accessible and enjoyable to a broader range of users.

Abstract:
Sports performance is often hindered by unintentional habits, particularly in golf, where achieving a consistent and correct swing is crucial yet challenging due to ingrained swing path habits. This study explores redirection approaches in virtual reality (VR) to correct golfers' swing paths through strategic ball shifting. By initiating a forward ball shift just before impact, we aim to prompt golfers to react and modify their swing motion, thereby eliminating undesirable swing habits. Building on recent research, our VR-based methods incorporate a gradual transformation of visuomotor associations to enhance motor skill learning. In this study, we develop three ball shift patterns, including a novel pattern that employs gradual ball shifts with interspersed normal conditions, designed to retain learning effects post-training. A preliminary study, including expert interviews, assesses the feasibility of various ball-shifting directions. Subsequently, a comprehensive user study measures the learning effects across different ball shift modes. The results indicate that our proposed redirection mode effectively corrects swing paths and yields a sustained learning effect.

Abstract:
Visual localization plays an important role in the applications of Augmented Reality (AR), which enable AR devices to obtain their 6-DoF pose in the pre-build map in order to render virtual content in real scenes. However, most existing approaches can not perform novel view rendering and require large storage capacities for maps. To overcome these limitations, we propose an efficient visual localization method capable of high-quality rendering with fewer parameters. Specifically, our approach leverages 3D Gaussian primitives as the scene representation. To ensure precise 2D–3D correspondences for pose estimation, we develop an unbiased 3D scene-specific descriptor decoder for Gaussian primitives, distilled from a constructed feature volume. Additionally, we introduce a salient 3D landmark selection algorithm that selects a suitable primitive subset based on the saliency score for localization. We further regularize key Gaussian primitives to prevent anisotropic effects, which also improves localization performance. Extensive experiments on two widely used datasets demonstrate that our method achieves superior or comparable rendering and localization performance to state-of-the-art implicit-based visual localization approaches. Code and data are available at project page: https://zju3dv.github.io/splatloc.

Abstract:
Mesh denoising is a crucial technology that aims to recover a high-fidelity 3D mesh from a noise-corrupted one. Deep learning methods, particularly graph convolutional networks (GCNs) based mesh denoisers, have demonstrated their effectiveness in removing various complex real-world noises while preserving authentic geometry. However, it is still a quite challenging work to faithfully regress uncontaminated normals and vertices on meshes with irregular topology. In this article, we propose a novel pipeline that incorporates two parallel normal-aware and vertex-aware branches to achieve a balance between smoothness and geometric details while maintaining the flexibility of surface topology. We introduce ResGEM, a new GCN, with multi-scale embedding modules and residual decoding structures to facilitate normal regression and vertex modification for mesh denoising. To effectively extract multi-scale surface features while avoiding the loss of topological information caused by graph pooling or coarsening operations, we encode the noisy normal and vertex graphs using four edge-conditioned embedding modules (EEMs) at different scales. This allows us to obtain favorable feature representations with multiple receptive field sizes. Formulating the denoising problem into a residual learning problem, the decoder incorporates residual blocks to accurately predict true normals and vertex offsets from the embedded feature space. Moreover, we propose novel regularization terms in the loss function that enhance the smoothing and generalization ability of our network by imposing constraints on normal fidelity and consistency. Comprehensive experiments have been conducted to demonstrate the superiority of our method over the state-of-the-art on both synthetic and real-scanned datasets.

Abstract:
Due to the unsatisfactory performance of supervised methods on unpaired real-world scans, point cloud completion via cross-domain adaptation has recently drawn growing attention. Nevertheless, previous approaches only focus on alleviating the distribution shift through domain alignment, resulting in massive information loss of real-world domain data. To tackle this issue, we propose a dual mixup-induced consistency regularization to integrate both source and target domain to improve robustness and generalization capability. Specifically, we mix up virtual and real-world shapes in the input and latent feature space respectively, and then regularize the completion network by forcing two kinds of mixed completion predictions to be consistent. To further adapt to each instance within the real-world domain, we design a novel density-aware refiner to utilize local context information to preserve the fine-grained details and remove noise or outliers for coarse completion. Extensive experiments on real-world scans and our synthetic unpaired datasets demonstrate the superiority of our method over existing state-of-the-art approaches.

Abstract:
A novel neural network called the isomorphic mesh generator (iMG) is proposed to generate isomorphic meshes from point clouds containing noise and missing parts. Isomorphic meshes of arbitrary objects exhibit a unified mesh structure, despite objects belonging to different classes. This unified representation enables various modern deep neural networks (DNNs) to easily handle surface models without requiring additional pre-processing. Additionally, the unified mesh structure of isomorphic meshes enables the application of the same process to all isomorphic meshes, unlike general mesh models, where processes need to be tailored depending on their mesh structures. Therefore, the use of isomorphic meshes can ensure efficient memory usage and reduce calculation time. Apart from the point cloud of the target object used as input for the iMG, point clouds and mesh models need not be prepared in advance as training data because the iMG is a data-free method. Furthermore, the iMG outputs an isomorphic mesh obtained by mapping a reference mesh to a given input point cloud. To stably estimate the mapping function, a step-by-step mapping strategy is introduced. This strategy enables flexible deformation while simultaneously maintaining the structure of the reference mesh. Simulations and experiments conducted using a mobile phone have confirmed that the iMG reliably generates isomorphic meshes of given objects, even when the input point cloud includes noise and missing parts.

Abstract:
Most of the existing 3D talking face synthesis methods suffer from the lack of detailed facial expressions and realistic head poses, resulting in unsatisfactory experiences for users. In this article, we propose a novel pose-aware 3D talking face synthesis method with a novel geometry-guided audio-vertices attention. To capture more detailed expression, such as the subtle nuances of mouth shape and eye movement, we propose to build hierarchical audio features including a global attribute feature and a series of vertex-wise local latent movement features. Then, in order to fully exploit the topology of facial models, we further propose a novel geometry-guided audio-vertices attention module to predict the displacement of each vertex by using vertex connectivity relations to take full advantage of the corresponding hierarchical audio features. Finally, to accomplish pose-aware animation, we expand the existing database with an additional pose attribute, and a novel pose estimation module is proposed by paying attention to the whole head model. Numerical experiments demonstrate the effectiveness of the proposed method on realistic expression and head movements against state-of-the-art methods.

Abstract:
Application developers frequently augment their code to produce event logs of specific operations performed by their users. Subsequent analysis of these event logs can help provide insight about the users’ behavior relative to its intended use. The analysis process typically includes both event organization and pattern discovery activities. However, most existing visual analytics systems for interaction log analysis excel at supporting pattern discovery and overlook the importance of flexible event organization. This omission limits the practical application of these systems. Therefore, we developed a novel visual analytics system called IntiVisor that implements the entire end-to-end interaction analysis approach. An evaluation of the system with interaction data from four visualization applications showed the value and importance of supporting event organization in interaction log analysis.

Abstract:
In this study, we devise a framework for volumetrically reconstructing fluid from observable, measurable free surface motion. Our innovative method amalgamates the benefits of deep learning and conventional simulation to preserve the guiding motion and temporal coherence of the reproduced fluid. We infer surface velocities by encoding and decoding spatiotemporal features of surface sequences, and a 3D CNN is used to generate the volumetric velocity field, which is then combined with 3D labels of obstacles and boundaries. Concurrently, we employ a network to estimate the fluid's physical properties. To progressively evolve the flow field over time, we input the reconstructed velocity field and estimated parameters into the physical simulator as the initial state. Our approach yields promising results for both synthetic fluid generated by different fluid solvers and captured real fluid. The developed framework naturally lends itself to a variety of graphics applications, such as 1) effective reproductions of fluid behaviors visually congruent with the observed surface motion, and 2) physics-guided re-editing of fluid scenes. Extensive experiments affirm that our novel method surpasses state-of-the-art approaches for 3D fluid inverse modeling and animation in graphics.

Abstract:
Visual analytics supports data analysis tasks within complex domain problems. However, due to the richness of data types, visual designs, and interaction designs, users need to recall and process a significant amount of information when they visually analyze data. These challenges emphasize the need for more intelligent visual analytics methods. Large language models have demonstrated the ability to interpret various forms of textual data, offering the potential to facilitate intelligent support for visual analytics. We propose LEVA, a framework that uses large language models to enhance users’ VA workflows at multiple stages: onboarding, exploration, and summarization. To support onboarding, we use large language models to interpret visualization designs and view relationships based on system specifications. For exploration, we use large language models to recommend insights based on the analysis of system status and data to facilitate mixed-initiative exploration. For summarization, we present a selective reporting strategy to retrace analysis history through a stream visualization and generate insight reports with the help of large language models. We demonstrate how LEVA can be integrated into existing visual analytics systems. Two usage scenarios and a user study suggest that LEVA effectively aids users in conducting visual analytics.

Abstract:
Visualization Recommendation Systems (VRSs) are a novel and challenging field of study aiming to help generate insightful visualizations from data and support non-expert users in information discovery. Among the many contributions proposed in this area, some systems embrace the ambitious objective of imitating human analysts to identify relevant relationships in data and make appropriate design choices to represent these relationships with insightful charts. We denote these systems as “agnostic” VRSs since they do not rely on human-provided constraints and rules but try to learn the task autonomously. Despite the high application potential of agnostic VRSs, their progress is hindered by several obstacles, including the absence of standardized datasets to train recommendation algorithms, the difficulty of learning design rules, and defining quantitative criteria for evaluating the perceptual effectiveness of generated plots. This article summarizes the literature on agnostic VRSs and outlines promising future research directions.

Abstract:
Endovascular intervention is a minimally invasive method for treating cardiovascular diseases. Although fluoroscopy, known for real-time catheter visualization, is commonly used, it exposes patients and physicians to ionizing radiation and lacks depth perception due to its 2D nature. To address these limitations, a study was conducted using teleoperation and 3D visualization techniques. This in-vitro study involved the use of a robotic catheter system and aimed to evaluate user performance through both subjective and objective measures. The focus was on determining the most effective modes of interaction. Three interactive modes for guiding robotic catheters were compared in the study: 1) Mode GM, using a gamepad for control and a standard 2D monitor for visual feedback; 2) Mode GH, with a gamepad for control and HoloLens providing 3D visualization; and 3) Mode HH, where HoloLens serves as both control input and visualization device. Mode GH outperformed other modalities in subjective metrics, except for mental demand. It exhibited a median tracking error of 4.72 mm, a median targeting error of 1.01 mm, a median duration of 82.34 s, and a median natural logarithm of dimensionless squared jerk of 40.38 in the in-vitro study. Mode GH showed 8.5%, 4.7%, 6.5%, and 3.9% improvements over Mode GM and 1.5%, 33.6%, 34.9%, and 8.1% over Mode HH for tracking error, targeting error, duration, and dimensionless squared jerk, respectively. To sum up, the user study emphasizes the potential benefits of employing HoloLens for enhanced 3D visualization in catheterization. The user study also illustrates the advantages of using a gamepad for catheter teleoperation, including user-friendliness and passive haptic feedback, compared to HoloLens. To further gauge the potential of using a more traditional joystick as a control input device, an additional study utilizing the Haption Virtuose robot was conducted. It reveals the potential for achieving smoother trajectories, with a 38.9% reduction in total path length compared to a gamepad, potentially due to its larger range of motion and single-handed control.

Abstract:
Fluid flows in spherical coordinates have raised the interest of the graphics community in recent years. The majority of existing works focus on 2D manifold flows on a spherical shell, and there are still many unresolved problems for 3D simulations in spherical coordinates, such as boundary conditions for arbitrary obstacles and flexible artistic controls. In this article, we propose a practical spherical-coordinate simulator for flow motions in 3D domains. Based on a layer-by-layer structure and a boundary-aware pressure solving scheme, we are able to recover horizontal and vertical flow motions in the presence of arbitrary terrain shapes within a spherical shell of finite thickness. Our proposed method straightforwardly builds on the conventions of previous 2D-manifold spherical-coordinate simulations and provides flexible artistic control strategies for art design.

Abstract:
Throughout history, static paintings have captivated viewers within display frames, yet the possibility of making these masterpieces vividly interactive remains intriguing. This research paper introduces 3DArtmator, a novel approach that aims to represent artforms in a highly interpretable stylized space, enabling 3D-aware animatable reconstruction and editing. Our rationale is to transfer the interpretability and 3D controllability of the latent space in a 3D-aware GAN to a stylized sub-space of a customized GAN, revitalizing the original artforms. To this end, the proposed two-stage optimization framework of 3DArtmator begins with discovering an anchor in the original latent space that accurately mimics the pose and content of a given art painting. This anchor serves as a reliable indicator of the original latent space local structure, therefore sharing the same editable predefined expression vectors. In the second stage, we train a customized 3D-aware GAN specific to the input artform, while enforcing the preservation of the original latent local structure through a meticulous style-directional difference loss. This approach ensures the creation of a stylized sub-space that remains interpretable and retains 3D control. The effectiveness and versatility of 3DArtmator are validated through extensive experiments across a diverse range of art styles. With the ability to generate 3D reconstruction and editing for artforms while maintaining interpretability, 3DArtmator opens up new possibilities for artistic exploration and engagement.

Abstract:
In this article, we introduce Neural-ABC, a novel parametric model based on neural implicit functions that can represent clothed human bodies with disentangled latent spaces for identity, clothing, shape, and pose. Traditional mesh-based representations struggle to represent articulated bodies with clothes due to the diversity of human body shapes and clothing styles, as well as the complexity of poses. Our proposed model provides a unified framework for parametric modeling, which can represent the identity, clothing, shape and pose of the clothed human body. Our proposed approach utilizes the power of neural implicit functions as the underlying representation and integrates well-designed structures to meet the necessary requirements. Specifically, we represent the underlying body as a signed distance function and clothing as an unsigned distance function, and they can be uniformly represented as unsigned distance fields. Different types of clothing do not require predefined topological structures or classifications, and can follow changes in the underlying body to fit the body. Additionally, we construct poses using a controllable articulated structure. The model is trained on both open and newly constructed datasets, and our decoupling strategy is carefully designed to ensure optimal performance. Our model excels at disentangling clothing and identity in different shape and poses while preserving the style of the clothing. We demonstrate that Neural-ABC fits new observations of different types of clothing. Compared to other state-of-the-art parametric models, Neural-ABC demonstrates powerful advantages in the reconstruction of clothed human bodies, as evidenced by fitting raw scans, depth maps and images. We show that the attributes of the fitted results can be further edited by adjusting their identities, clothing, shape and pose codes.

Abstract:
Creating a vivid video from the event or scenario in our imagination is a truly fascinating experience. Recent advancements in text-to-video synthesis have unveiled the potential to achieve this with prompts only. While text is convenient in conveying the overall scene context, it may be insufficient to control precisely. In this paper, we explore customized video generation by utilizing text as context description and motion structure (e.g., frame-wise depth) as concrete guidance. Our method, dubbed Make-Your-Video, involves joint-conditional video generation using a Latent Diffusion Model that is pre-trained for still image synthesis and then promoted for video generation with the introduction of temporal modules. This two-stage learning scheme not only reduces the computing resources required, but also improves the performance by transferring the rich concepts available in image datasets solely into video generation. Moreover, we use a simple yet effective causal attention mask strategy to enable longer video synthesis, which mitigates the potential quality degradation effectively. Experimental results show the superiority of our method over existing baselines, particularly in terms of temporal coherence and fidelity to users’ guidance. In addition, our model enables several intriguing applications that demonstrate potential for practical usage.

Abstract:
Current virtual reality (VR) system takes gesture interaction based on camera, handle and touch screen as one of the mainstream interaction methods, which can provide accurate gesture input for it. However, limited by application forms and the volume of devices, these methods cannot extend the interaction area to such surfaces as walls and tables. To address the above challenge, we propose AudioGest, a portable, plug-and-play system that detects the audio signal generated by finger tapping and sliding on the surface through a set of microphone devices without extensive calibration. First, an audio synthesis-recognition pipeline based on micro-contact dynamics simulation is constructed to generate modal audio synthesis from different materials and physical properties. Then the accuracy and effectiveness of the synthetic audio are verified by mixing the synthetic audio with real audio proportionally as the training sets. Finally, a series of desktop office applications are developed to demonstrate the application potential of AudioGest's scalability and versatility in VR scenarios.

Abstract:
Tactics play an important role in team sports by guiding how players interact on the field. Both sports fans and experts have a demand for analyzing sports tactics. Existing approaches allow users to visually perceive the multivariate tactical effects. However, these approaches require users to experience a complex reasoning process to connect the multiple interactions within each tactic to the final tactical effect. In this work, we collaborate with basketball experts and propose a progressive approach to help users gain a deeper understanding of how each tactic works and customize tactics on demand. Users can progressively sketch on a tactic board, and a coach agent will simulate the possible actions in each step and present the simulation to users with facet visualizations. We develop an extensible framework that integrates large language models (LLMs) and visualizations to help users communicate with the coach agent with multimodal inputs. Based on the framework, we design and develop Smartboard, an agent-based interactive visualization system for fine-grained tactical analysis, especially for play design. Smartboard provides users with a structured process of setup, simulation, and evolution, allowing for iterative exploration of tactics based on specific personalized scenarios. We conduct case studies based on real-world basketball datasets to demonstrate the effectiveness and usefulness of our system.

Abstract:
Visualization linters are end-user facing evaluators that automatically identify potential chart issues. These spell-checker like systems offer a blend of interpretability and customization that is not found in other forms of automated assistance. However, existing linters do not model context and have primarily targeted users who do not need assistance, resulting in obvious—even annoying—advice. We investigate these issues within the domain of color palette design, which serves as a microcosm of visualization design concerns. We contribute a GUI-based color palette linter as a design probe that covers perception, accessibility, context, and other design criteria, and use it to explore visual explanations, integrated fixes, and user defined linting rules. Through a formative interview study and theory-driven analysis, we find that linters can be meaningfully integrated into graphical contexts thereby addressing many of their core issues. We discuss implications for integrating linters into visualization tools, developing improved assertion languages, and supporting end-user tunable advice—all laying the groundwork for more effective visualization linters in any context.

Abstract:
We present Cell2Cell, a novel visual analytics approach for quantifying and visualizing networks of cell-cell interactions in three-dimensional (3D) multi-channel cancerous tissue data. By analyzing cellular interactions, biomedical experts can gain a more accurate understanding of the intricate relationships between cancer and immune cells. Recent methods have focused on inferring interaction based on the proximity of cells in low-resolution 2D multi-channel imaging data. By contrast, we analyze cell interactions by quantifying the presence and levels of specific proteins within a tissue sample (protein expressions) extracted from high-resolution 3D multi-channel volume data. Such analyses have a strong exploratory nature and require a tight integration of domain experts in the analysis loop to leverage their deep knowledge. We propose two complementary semi-automated approaches to cope with the increasing size and complexity of the data interactively: On the one hand, we interpret cell-to-cell interactions as edges in a cell graph and analyze the image signal (protein expressions) along those edges, using spatial as well as abstract visualizations. Complementary, we propose a cell-centered approach, enabling scientists to visually analyze polarized distributions of proteins in three dimensions, which also captures neighboring cells with biochemical and cell biological consequences. We evaluate our application in three case studies, where biologists and medical experts use Cell2Cell to investigate tumor micro-environments to identify and quantify T-cell activation in human tissue data. We confirmed that our tool can fully solve the use cases and enables a streamlined and detailed analysis of cell-cell interactions.

Abstract:
In understanding and redesigning the function of proteins in modern biochemistry, protein engineers are increasingly focusing on exploring regions in proteins called loops. Analyzing various characteristics of these regions helps the experts design the transfer of the desired function from one protein to another. This process is denoted as loop grafting. We designed a set of interactive visualizations that provide experts with visual support through all the loop grafting pipeline steps. The workflow is divided into several phases, reflecting the steps of the pipeline. Each phase is supported by a specific set of abstracted 2D visual representations of proteins and their loops that are interactively linked with the 3D View of proteins. By sequentially passing through the individual phases, the user shapes the list of loops that are potential candidates for loop grafting. Finally, the actual in-silico insertion of the loop candidates from one protein to the other is performed, and the results are visually presented to the user. In this way, the fully computational rational design of proteins and their loops results in newly designed protein structures that can be further assembled and tested through in-vitro experiments. We showcase the contribution of our visual support design on a real case scenario changing the enantiomer selectivity of the engineered enzyme. Moreover, we provide the readers with the experts' feedback.

Abstract:
Feature grid Scene Representation Networks (SRNs) have been applied to scientific data as compact functional surrogates for analysis and visualization. As SRNs are black-box lossy data representations, assessing the prediction quality is critical for scientific visualization applications to ensure that scientists can trust the information being visualized. Currently, existing architectures do not support inference time reconstruction quality assessment, as coordinate-level errors cannot be evaluated in the absence of ground truth data. By employing the uncertain neural network architecture in feature grid SRNs, we obtain prediction variances during inference time to facilitate confidence-aware data reconstruction. Specifically, we propose a parameter-efficient multi-decoder SRN (MDSRN) architecture consisting of a shared feature grid with multiple lightweight multilayer perceptron decoders. MDSRN can generate a set of plausible predictions for a given input coordinate to compute the mean as the prediction of the multi-decoder ensemble and the variance as a confidence score. The coordinate-level variance can be rendered along with the data to inform the reconstruction quality, or be integrated into uncertainty-aware volume visualization algorithms. To prevent the misalignment between the quantified variance and the prediction quality, we propose a novel variance regularization loss for ensemble learning that promotes the Regularized multi-decoder SRN (RMDSRN) to obtain a more reliable variance that correlates closely to the true model error. We comprehensively evaluate the quality of variance quantification and data reconstruction of Monte Carlo Dropout (MCD), Mean Field Variational Inference (MFVI), Deep Ensemble (DE), and Predicting Variance (PV) in comparison with our proposed MDSRN and RMDSRN applied to state-of-the-art feature grid SRNs across diverse scalar field datasets. We demonstrate that RMDSRN attains the most accurate data reconstruction and competitive variance-error correlation among uncertain SRNs under the same neural network parameter budgets. Furthermore, we present an adaptation of uncertainty-aware volume rendering and shed light on the potential of incorporating uncertain predictions in improving the quality of volume rendering for uncertain SRNs. Through ablation studies on the regularization strength and decoder count, we show that MDSRN and RMDSRN are expected to perform sufficiently well with a default configuration without requiring customized hyperparameter settings for different datasets.

Abstract:
Trained on vast corpora, Large Language Models (LLMs) have the potential to encode visualization design knowledge and best practices. However, if they fail to do so, they might provide unreliable visualization recommendations. What visualization design preferences, then, have LLMs learned? We contribute DracoGPT, a method for extracting, modeling, and assessing visualization design preferences from LLMs. To assess varied tasks, we develop two pipelines—DracoGPT-Rank and DracoGPT-Recommend—to model LLMs prompted to either rank or recommend visual encoding specifications. We use Draco as a shared knowledge base in which to represent LLM design preferences and compare them to best practices from empirical research. We demonstrate that DracoGPT can accurately model the preferences expressed by LLMs, enabling analysis in terms of Draco design constraints. Across a suite of backing LLMs, we find that DracoGPT-Rank and DracoGPT-Recommend moderately agree with each other, but both substantially diverge from guidelines drawn from human subjects experiments. Future work can build on our approach to expand Draco's knowledge base to model a richer set of preferences and to provide a robust and cost-effective stand-in for LLMs.

Abstract:
The integration of Large Language Models (LLMs), especially ChatGPT, into education is poised to revolutionize students' learning experiences by introducing innovative conversational learning methodologies. To empower students to fully leverage the capabilities of ChatGPT in educational scenarios, understanding students' interaction patterns with ChatGPT is crucial for instructors. However, this endeavor is challenging due to the absence of datasets focused on student-ChatGPT conversations and the complexities in identifying and analyzing the evolutional interaction patterns within conversations. To address these challenges, we collected conversational data from 48 students interacting with ChatGPT in a master's level data visualization course over one semester. We then developed a coding scheme, grounded in the literature on cognitive levels and thematic analysis, to categorize students' interaction patterns with ChatGPT. Furthermore, we present a visual analytics system, StuGPTViz, that tracks and compares temporal patterns in student prompts and the quality of ChatGPT's responses at multiple scales, revealing significant pedagogical insights for instructors. We validated the system's effectiveness through expert interviews with six data visualization instructors and three case studies. The results confirmed StuGPTViz's capacity to enhance educators' insights into the pedagogical value of ChatGPT. We also discussed the potential research opportunities of applying visual analytics in education and developing AI-driven personalized learning solutions.

Abstract:
Comics are an effective method for sequential data-driven storytelling, especially for dynamic graphs—graphs whose vertices and edges change over time. However, manually creating such comics is currently time-consuming, complex, and error-prone. In this paper, we propose DG COMICS, a novel comic authoring tool for dynamic graphs that allows users to semi-automatically build and annotate comics. The tool uses a newly developed hierarchical clustering algorithm to segment consecutive snapshots of dynamic graphs while preserving their chronological order. It also presents rich information on both individuals and communities extracted from dynamic graphs in multiple views, where users can explore dynamic graphs and choose what to tell in comics. For evaluation, we provide an example and report the results of a user study and an expert review.

Abstract:
We present a path-based design model and system for designing and creating visualisations. Our model represents a systematic approach to constructing visual representations of data or concepts following a predefined sequence of steps. The initial step involves outlining the overall appearance of the visualisation by creating a skeleton structure, referred to as a flowpath. Subsequently, we specify objects, visual marks, properties, and appearance, storing them in a gene. Lastly, we map data onto the flowpath, ensuring suitable morphisms. Alternative designs are created by exchanging values in the gene. For example, designs that share similar traits, are created by making small incremental changes to the gene. Our design methodology fosters the generation of diverse creative concepts, space-filling visualisations, and traditional formats like bar charts, circular plots and pie charts. Through our implementation we showcase the model in action. As an example application, we integrate the output visualisations onto a smartwatch and visualisation dashboards. In this article we (1) introduce, define and explain the path model and discuss possibilities for its use, (2) present our implementation, results, and evaluation, and (3) demonstrate and evaluate an application of its use on a mobile watch.

Abstract:
Over the past decade, several urban visual analytics systems and tools have been proposed to tackle a host of challenges faced by cities, in areas as diverse as transportation, weather, and real estate. Many of these tools have been designed through collaborations with urban experts, aiming to distill intricate urban analysis workflows into interactive visualizations and interfaces. However, the design, implementation, and practical use of these tools still rely on siloed approaches, resulting in bespoke systems that are difficult to reproduce and extend. At the design level, these tools undervalue rich data workflows from urban experts, typically treating them only as data providers and evaluators. At the implementation level, they lack interoperability with other technical frameworks. At the practical use level, they tend to be narrowly focused on specific fields, inadvertently creating barriers to cross-domain collaboration. To address these gaps, we present Curio, a framework for collaborative urban visual analytics. Curio uses a dataflow model with multiple abstraction levels (code, grammar, GUI elements) to facilitate collaboration across the design and implementation of visual analytics components. The framework allows experts to intertwine data preprocessing, management, and visualization stages while tracking the provenance of code and visualizations. In collaboration with urban experts, we evaluate Curio through a diverse set of usage scenarios targeting urban accessibility, urban microclimate, and sunlight access. These scenarios use different types of data and domain methodologies to illustrate Curio's flexibility in tackling pressing societal challenges. Curio is available at urbantk.org/curio.

Abstract:
A growing body of work draws on feminist thinking to challenge assumptions about how people engage with and use visualizations. This work draws on feminist values, driving design and research guidelines that account for the influences of power and neglect. This prior work is largely prescriptive, however, forgoing articulation of how feminist theories of knowledge — or feminist epistemology — can alter research design and outcomes. At the core of our work is an engagement with feminist epistemology, drawing attention to how a new framework for how we know what we know enabled us to overcome intellectual tensions in our research. Specifically, we focus on the theoretical concept of entanglement, central to recent feminist scholarship, and contribute: a history of entanglement in the broader scope of feminist theory; an articulation of the main points of entanglement theory for a visualization context; and a case study of research outcomes as evidence of the potential of feminist epistemology to impact visualization research. This work answers a call in the community to embrace a broader set of theoretical and epistemic foundations and provides a starting point for bringing feminist theories into visualization research.

Abstract:
Semantic segmentation of 3D point clouds is important for many applications, such as autonomous driving. To train semantic segmentation models, labeled point cloud segmentation datasets are essential. Meanwhile, point cloud labeling is time-consuming for annotators, which typically involves tuning the camera viewpoint and selecting points by lasso. To reduce the time cost of point cloud labeling, we propose a viewpoint recommendation approach to reduce annotators’ labeling time costs. We adapt Fitts’ law to model the time cost of lasso selection in point clouds. Using the modeled time cost, the viewpoint that minimizes the lasso selection time cost is recommended to the annotator. We build a data labeling system for semantic segmentation of 3D point clouds that integrates our viewpoint recommendation approach. The system enables users to navigate to recommended viewpoints for efficient annotation. Through an ablation study, we observed that our approach effectively reduced the data labeling time cost. We also qualitatively compare our approach with previous viewpoint selection approaches on different datasets.

Abstract:
Faithfully reconstructing textured meshes is crucial for many applications. Compared to text or image modalities, leveraging 3D colored point clouds as input (colored-PC-to-mesh) offers inherent advantages in comprehensively and precisely replicating the target object’s 360^\circ ∘ characteristics. While most existing colored-PC-to-mesh methods suffer from blurry textures or require hard-to-acquire 3D training data, we propose PointDreamer, a novel framework that harnesses 2D diffusion prior for superior texture quality. Crucially, unlike prior 2D-diffusion-for-3D works driven by text or image inputs, PointDreamer successfully adapts 2D diffusion models to 3D point cloud data by a novel project-inpaint-unproject pipeline. Specifically, it first projects the point cloud into sparse 2D images and then performs diffusion-based inpainting. After that, diverging from most existing 3D reconstruction or generation approaches that predict texture in 3D/UV space thus often yielding blurry texture, PointDreamer achieves high-quality texture by directly unprojecting the inpainted 2D images to the 3D mesh. Furthermore, we identify for the first time a typical kind of unprojection artifact appearing in occlusion borders, which is common in other multiview-image-to-3D pipelines but less-explored. To address this, we propose a novel solution named the Non-Border-First (NBF) unprojection strategy. Extensive qualitative and quantitative experiments on various synthetic and real-scanned datasets demonstrate that PointDreamer, though zero-shot, exhibits SoTA performance (30% improvement on LPIPS score from 0.118 to 0.068), and is robust to noisy, sparse, or even incomplete input data.

Abstract:
In the field of 3D point cloud generation, numerous 3D generative models have demonstrated the ability to generate diverse and realistic 3D shapes. However, the majority of these approaches struggle to generate controllable 3D point cloud shapes that meet user-specific requirements, hindering the large-scale application of 3D point cloud generation. To address the challenge of lacking control in 3D point cloud generation, we are the first to propose controlling the generation of point clouds by shape structures that comprise part existences and part adjacency relationships. We manually annotate the adjacency relationships between the segmented parts of point cloud shapes, thereby constructing a StructureGraph representation. Based on this StructureGraph representation, we introduce StrucADT, a novel structure-controllable point cloud generation model, which consists of StructureGraphNet module to extract structure-aware latent features, cCNF Prior module to learn the distribution of the latent features controlled by the part adjacency, and Diffusion Transformer module conditioned on the latent features and part adjacency to generate structure-consistent point cloud shapes. Experimental results demonstrate that our structure-controllable 3D point cloud generation method produces high-quality and diverse point cloud shapes, enabling the generation of controllable point clouds based on user-specified shape structures and achieving state-of-the-art performance in controllable point cloud generation on the ShapeNet dataset.

Abstract:
Neural rendering algorithms have revolutionized computer graphics, yet their impact on real-time rendering under arbitrary lighting conditions remains limited due to strict latency constraints in practical applications. The key challenge lies in formulating a compact yet expressive material representation. To address this, we propose TransGI, a novel neural rendering method for real-time, high-fidelity global illumination. It comprises an object-centric neural transfer model for material representation and a radiance-sharing lighting system for efficient illumination. Traditional BSDF representations and spatial neural material representations lack expressiveness, requiring thousands of ray evaluations to converge to noise-free colors. Conversely, real-time methods trade quality for efficiency by supporting only diffuse materials. In contrast, our object-centric neural transfer model achieves compactness and expressiveness through an MLP-based decoder and vertex-attached latent features, supporting glossy effects with low memory overhead. For dynamic, varying lighting conditions, we introduce local light probes capturing scene radiance, coupled with an across-probe radiance-sharing strategy for efficient probe generation. We implemented our method in a real-time rendering engine, combining compute shaders and CUDA-based neural networks. Experimental results demonstrate that our method achieves real-time performance of less than 10 ms to render a frame and significantly improved rendering quality compared to baseline methods.

Abstract:
Hierarchical time series data comprises a collection of time series aggregated at multiple levels based on categorical, geographical, or physical constraints, the analysis of which aids analysts across various domains like retail, finance, and energy, in gaining valuable insights and making informed decisions. However, existing interactive exploratory analysis approaches for hierarchical time series data fall short in analyzing time series across different aggregation levels and supporting more complex analytical tasks beyond common ones like summarize and compare. These limitations motivate us to develop a new visual analytics approach. We first generalize a taxonomy to delineate various tasks in hierarchical time series analysis, derived from literature survey and expert interviews. Based on this taxonomy, we develop ChronoDeck, an interactive system that incorporates a multi-column hierarchical time series visualization for implementing various analytical tasks and distilling insights from the data. ChronoDeck visualizes each aggregation level of hierarchical time series with a combination of coordinated dimensionality reduction and small multiples visualizations, alongside interactions including highlight, align, filter, and select, assisting users in the visualization, comparison, and transformation of hierarchical time series, as well as identifying the entities of interest. The effectiveness of ChronoDeck is demonstrated by case studies on three real-world datasets and expert interviews.

Abstract:
Automatic 3D shape analysis is heavily influenced by the pose of input 3D models, as the continuous nature of pose space introduces complexities that usually exceed the encoding capacities of standard deep learning frameworks. To tackle this challenge, we present Upright-Net+, an enhancement of our previous model, Upright-Net, specifically developed for estimating upright orientation in 3D point clouds. Our approach is grounded in the design principle that ”form ever follows function,” treating the natural base of an object as a functional structure that stabilizes it in its typical pose, influenced by physical laws and geometric properties. We reformulate the continuous orientation problem into a discrete classification task, focusing on learning the points that constitute the natural base of a 3D model. The upright orientation is determined by aligning the normal orientation of this base towards the mass center. To mitigate over-smoothing in the global feature embeddings from stacked graph convolutional layers, we introduce a Global Positional Encoding Module using Relative Distance Histogram Statistics Embedding (GPE-RDHS), which reduces structural ambiguity and enhances orientation estimation. We also enhanced a weighted residual loss term to penalize false positive predictions, enhancing overall model performance. Our method demonstrates exceptional performance in upright orientation estimation and reveals that the learned orientation-aware features significantly benefit downstream tasks, particularly in classification.

Abstract:
High-resolution images come with rich color information and texture details. Due to the rapid upgrading of display devices and rendering technologies, high-resolution real-time rendering faces the computational overhead challenge. To address this, the current mainstream solution is to render at a lower resolution and then upsample to the target resolution by supersampling techniques. However, while many prior supersampling approaches have attempted to exploit rich rendered data such as color, depth, motion vectors at low resolution, there is little discussion on how to harness high-frequency information that is readily available in the high-resolution (HR) G-buffers of modern renders. In this article, we seek to investigate how to fully leverage information from HR G-buffers to maximize the visual quality of supersampling results. We propose a neural network for real-time supersampling of rendered content, which is based on several core designs, including gated G-buffers encoder, G-buffers attended encoder and reflection-aware loss. These designs are especially made for the sake of effectively using HR G-buffers, enabling faithful recovery of a variety of high-frequency scene details from low-resolution, highly aliased inputs. Furthermore, a simple occlusion-aware blender is proposed to efficiently rectify dis-occluded features in the warped previous frame, allowing us to better exploit history information to improve temporal stability. The experiments show that our method, equipped with strong ability to harness HR G-buffer information, significantly improves the visual fidelity of high-resolution reconstructions upon previous state-of-the-art methods, even for challenging 4 × 44×4 upsampling, while still being compute-efficient.

Abstract:
Classically, affordance research investigates how the shape of objects communicates actions to potential users. Cognitive affordances, a subset of this research, characterize how the design of objects influences cognitive actions, such as information processing. Within visualization, cognitive affordances inform how graphs’ design decisions communicate information to their readers. Although several related concepts exist in visualization, a formal translation of affordance theory to visualization is still lacking. In this paper, we review and translate affordance theory to visualization by formalizing how cognitive affordances operate within a visualization context. We also review common methods and terms, and compare related constructs to cognitive affordances in visualization. Based on a synthesis of research from psychology, human-computer interaction, and visualization, we propose a framework of cognitive affordances in visualization that enumerates design decisions and reader characteristics that influence a visualization’s hierarchy of communicated information. Finally, we demonstrate how this framework can guide the evaluation and redesign of visualizations.

Abstract:
We present a Lagrangian dynamic importance Monte Carlo method without non-trivial random walks for solving the Velocity-Vorticity Poisson Equation (VVPE) in Smoothed Particle Hydrodynamics (SPH) for vortical flows. Key to our approach is the use of the Kinematic Vorticity Number (KVN) to detect vortex cores and to compute the KVN-based importance of each particle when solving the VVPE. We use Adaptive Kernel Density Estimation (AKDE) to extract a probability density distribution from the KVN for the the Monte Carlo calculations. Even though the distribution of the KVN can be non-trivial, AKDE yields a smooth and normalized result which we dynamically update at each time step. As we sample actual particles directly, the Lagrangian attributes of particle samples ensure that the continuously evolved KVN-based importance, modeled by the probability density distribution extracted from the KVN by AKDE, can be closely followed. Our approach enables effective vortical flow simulations with significantly reduced computational overhead and comparable quality to the classic Biot-Savart law that in contrast requires expensive global particle querying.

Abstract:
Textured mesh quality assessment (TMQA) is critical for various 3D mesh applications. However, existing TMQA methods often struggle to provide accurate and robust evaluations. Motivated by the effectiveness of fields in representing both 3D geometry and color information, we propose a novel point-based TMQA method called field mesh quality metric (FMQM). FMQM utilizes signed distance fields and a newly proposed color field named nearest surface point color field to realize effective mesh feature description. Four features related to visual perception are extracted from the geometry and color fields: geometry similarity, geometry gradient similarity, space color distribution similarity, and space color gradient similarity. Experimental results on three benchmark datasets demonstrate that FMQM outperforms state-of-the-art (SOTA) TMQA metrics. Furthermore, FMQM exhibits low computational complexity, making it a practical and efficient solution for real-world applications in 3D graphics and visualization.

Abstract:
3D Gaussian Splatting (3DGS) achieves remarkable speed and performance in novel view synthesis but suffers from overfitting and degraded reconstruction when handling sparse-view inputs. This paper innovatively addresses this challenge from a learning-to-optimize perspective by leveraging a learned optimizer (i.e., a multi-layer perceptron, MLP) to update the relevant parameters of 3DGS during the optimization process. Evidently, using a single MLP to handle all optimization variables, whose numbers may even vary during the optimization process, is impossible. Therefore, we present a point-wise position-aware optimizer that updates the parameters for each 3DGS point individually. Specifically, it takes the point coordinates and corresponding parameter values as input to predict the updates, thereby allowing efficient and adaptive optimization. In the case of sparse view modeling, the learned optimizer imposes position-aware constraints on the parameter updates during optimization. This effectively encourages the relevant parameters to converge stably to better solutions. To update the optimizer’s parameters, we propose a dynamic gradient update strategy based on spatial perturbation and weighted fusion, enabling the optimizer to capture broader contextual information. Experiments demonstrate that our method effectively addresses the problem of modeling 3DGS from sparse training views, achieving state-of-the-art results across multiple datasets.

Abstract:
Previous acoustic transfer methods rely on extensive precomputation and storage of data to enable real-time interaction and auditory feedback. However, these methods struggle with complex scenes, especially when dynamic changes in object position, material, and size significantly alter sound effects. These continuous variations lead to fluctuating acoustic transfer distributions, making it challenging to represent with basic data structures and render efficiently in real time. To address this challenge, we present Neural Acoustic Transfer, a novel approach that leverages implicit neural representations to encode acoustic transfer functions and their variations. This enables real-time prediction of dynamically evolving sound fields and their interactions with the environment under varying conditions. To efficiently generate high-quality training data for the neural acoustic field while avoiding reliance on mesh quality of a model, we develop a fast and efficient Monte-Carlo-based boundary element method (BEM) approximation, suitable for general scenarios with smooth Neumann boundary conditions. In addition, we devise strategies to mitigate potential singularities during the synthesis of training data, thereby enhancing its reliability. Together, these methods provide robust and accurate data that empower the neural network to effectively model complex sound radiation space. We demonstrate our method’s numerical accuracy and runtime efficiency (within several milliseconds for 30 s audio) through comprehensive validation and comparisons in diverse acoustic transfer scenarios. Our approach allows for efficient and accurate modeling of sound behavior in dynamically changing environments, which can benefit a wide range of interactive applications such as virtual reality, augmented reality, and advanced audio production.

Abstract:
Stroke-based Rendering (SBR) aims to decompose an input image into a sequence of parameterized strokes, which can be rendered into a painting that resembles the input image. Recently, Neural Painting methods that utilize deep learning and reinforcement learning models to predict the stroke sequences have been developed, but suffer from longer inference time or unstable training. To address these issues, we propose AttentionPainter, an efficient and adaptive model for single-step neural painting. First, we propose a novel scalable stroke predictor, which predicts a large number of stroke parameters within a single forward process, instead of the iterative prediction of previous Reinforcement Learning or auto-regressive methods, which makes AttentionPainter faster than previous neural painting methods. To further increase the training efficiency, we propose a Fast Stroke Stacking algorithm, which brings 13 times acceleration for training. Moreover, we propose Stroke-density Loss, which encourages the model to use small strokes for detailed information, to help improve the reconstruction quality. Finally, we design a Stroke Diffusion Model as an application of AttentionPainter, which conducts the denoising process in the stroke parameter space and facilitates stroke-based inpainting and editing applications helpful for human artists’ design. Extensive experiments show that AttentionPainter outperforms the state-of-the-art neural painting methods.

Abstract:
Rendering huge biological scenes with atomistic detail presents a significant challenge in molecular visualization due to the memory limitations inherent in traditional rendering approaches. In this paper, we propose a novel method for the interactive rendering of massive molecular scenes based on hardware-accelerated ray tracing. Our approach circumvents GPU memory constraints by introducing virtual instantiation of full-detail scene elements. Using instancing significantly reduces memory consumption while preserving the full atomistic detail of scenes comprising trillions of atoms, with interactive rendering performance and completely free user exploration. We utilize coarse meshes as proxy geometries to approximate the overall shape of biological compartments, and access all atomistic detail dynamically during ray tracing. We do this via a novel adaptive technique utilizing a volumetric shell layer of prisms extruded around proxy geometry triangles, and a virtual volume grid for the interior of each compartment. Our algorithm scales to enormous molecular scenes with minimal memory consumption and the potential to accommodate even larger scenes. Our method also supports advanced effects such as clipping planes and animations. We demonstrate the efficiency and scalability of our approach by rendering tens of instances of Red Blood Cell and SARS-CoV-2 models theoretically containing more than 20 trillion atoms.

Abstract:
Large transparent touch screens (LTTS) have recently become commercially available. These displays have the potential for engaging Augmented Reality (AR) applications, especially in public and shared spaces. However, the interaction with objects in the real environment behind the display remains challenging: Users must combine pointing and touch input if they want to select objects at varying distances. There is a lot of work on wearable or mobile AR displays, but little on how users interact with LTTS. Our goal is to contribute to a better understanding of natural user interaction for these AR displays. To this end, we developed a prototype and evaluated different pointing techniques for selecting 12 physical targets behind an LTTS, with distances ranging from 6 to 401 cm. We conducted a user study with 16 participants and measured user preferences, performance, and behavior. We analyzed the change in accuracy depending on the target position and the selection technique used. Our findings include: (a) Users naturally align the touch point with their line of sight for targets farther than 36 cm behind the LTTS. (b) This technique provides the lowest angular deviation compared to other techniques. (c) Some user close one eye to improve their performance. Our results help to improve future AR scenarios using LTTS systems.

Abstract:
With the advent of consumer-targeted, low-cost virtual reality devices and facile authoring technologies, the development and design of experiences in virtual reality are also becoming more accessible to non-expert authors. However, the inherent freedom of exploration in these virtual spaces presents a significant challenge for designers seeking to guide user attention toward points and objects of interest. This paper proposes the new technique WarpVision, which utilizes spatial curvature to subtly guide the user's attention in virtual reality. WarpVision distorts an area around the point of interest, thus changing the size, form, and location of all objects and the space around them. In this way, the user's attention can be guided even when the point of interest is not in the immediate field of vision. WarpVision is evaluated in a user study based on a within-subjects design, comparing it to the state-of-the-art technique Deadeye. Participants completed visual search tasks across two virtual environments being supported with WarpVision at four different intensities. Results show that WarpVision significantly reduces search times compared to Deadeye. While both techniques introduce comparable levels of immersion disruption, WarpVision has a lower reported impact on the user's well-being.

Abstract:
Window management in virtual reality (VR) remains a challenging task due to the spatial complexity and physical demands of current interaction methods. We introduce Handows, a palm-based interface that enables direct manipulation of spatial windows through familiar smartphone-inspired gestures on the user's non-dominant hand. Combining ergonomic layout design with body-centric input and passive haptics, Handows supports four core operations: window selection, closure, positioning, and scaling. We evaluate Handows in a user study (N = 15) against two common VR techniques (virtual hand and controller) across four core window operations. Results show that Handows significantly reduces physical effort and head movement while improving task efficiency and interaction precision. A follow-up case study (N = 8) demonstrates Handows' usability in realistic multitasking scenarios, highlighting user-adapted workflows and spontaneous layout strategies. Our findings also suggest the potential of embedding mobile-inspired metaphors into proprioceptive body-centric interfaces to support low-effort and spatially coherent interaction in VR.

Abstract:
Redirection in virtual reality (VR) enhances haptic feedback versatility by relaxing the need for precise alignment between virtual and physical objects. In mixed reality (MR), where users see the real world and their own hands, haptic redirection enables a physical interaction with virtual objects but poses greater challenges due to altering real-world perception. This paper investigates the effect of the realism of the user's surroundings and of the user's hand on haptic redirection. The user's familiarity with their actual physical surroundings and their actual hand could make the redirection manipulations easier-or harder-to detect. In a user study (N = 30) participants saw either a virtual environment or their actual physical surroundings, and saw their hand rendered either with a generic 3D model or with a live 2D video sprite of their actual hand. The study used a two-alternative forced choice (2AFC) design asking participants to detect hand redirections that bridged physical to virtual offsets of varying magnitudes. The results show that participants were not more sensitive to 2D video sprite hand redirection than to VR hand redirection, which supports the use of haptic redirection in MR.

Abstract:
Our sense of touch plays a crucial role in physical collaboration, yet rendering realistic haptic feedback in collaborative extended reality (XR) remains a challenge. Co-located XR systems predominantly rely on prefabricated passive props that provide high-fidelity interaction but offer limited adaptability. Haptic Illusions (HIs), which leverage multisensory integration, have proven effective in expanding haptic experiences in single-user contexts. However, their role in XR collaboration has not been explored. To examine the applicability of HIs in multi-user scenarios, we conducted an experimental user study (N=30) investigating their effect on a collaborative object handover task in virtual reality. We manipulated visual shape and size individually and analyzed their impact on users' performance, experience, and behavior. Results show that while participants adapted to the illusions by shifting sensory reliance and employing specific sensorimotor strategies, visuo-haptic mismatches reduced both performance and experience. Moreover, mismatched visualizations in asymmetric user roles negatively impacted performance. Drawing from these findings, we provide practical guidelines for incorporating HIs into collaborative XR, marking a first step toward richer haptic interactions in shared virtual spaces.

Abstract:
The core of self-supervised point cloud learning lies in setting up appropriate pretext tasks, to construct a pre-training framework that enables the encoder to perceive 3D objects effectively. In this article, we integrate two prevalent methods, masked point modeling (MPM) and 3D-to-2D generation, as pretext tasks within a pre-training framework. We leverage the spatial awareness and precise supervision offered by these two methods to address their respective limitations: ambiguous supervision signals and insensitivity to geometric information. Specifically, the proposed framework, abbreviated as PointCG, consists of a Hidden Point Completion (HPC) module and an Arbitrary-view Image Generation (AIG) module. We first capture visible points from arbitrary views as inputs by removing hidden points. Then, HPC extracts representations of the inputs with an encoder and completes the entire shape with a decoder, while AIG is used to generate rendered images based on the visible points’ representations. Extensive experiments demonstrate the superiority of the proposed method over the baselines in various downstream tasks. Our code will be made available upon acceptance.

Abstract:
In boundary representation (B-rep) reconstruction for computer aided design (CAD) applications, it is still challenging with existing methods to distinguish primitives in the smoothly blended regions reasonably. Thus, intensive manual post-processing is always required for correcting the primitives and their neighboring relationships to obtain a valid B-rep solid, seriously preventing the efficiency. In this paper, we address these challenges by presenting two novel techniques. The first is to robustly extract primitives by iteratively estimating the probability distribution of the noise to eliminate outliers. The second is to present intersection-aware constraints, like tangency and collinearity constraints, to correctly obtain intersections between primitives, which have not been explored in existing methods to our knowledge. Therefore, we can effectively extract primitives, especially those blended smoothly, and obtain high-quality relationships between them. As a result, a valid B-rep model can be constructed without a lot of manual post-processing on topology correction, while not with existing methods. As a benefit, with our constructed B-rep models, their corresponding meshes can be intuitively and conveniently edited, which is quite useful in CAD applications. Experimental results show that our proposed B-rep construction method outperforms both classical and recent learning-based methods in terms of reconstruction efficiency and accuracy.

Abstract:
Temporal graphs are commonly used to represent complex systems and track the evolution of their constituents over time. Visualizing these graphs is crucial as it allows one to quickly identify anomalies, trends, patterns, and other properties that facilitate better decision-making. In this context, selecting an appropriate temporal resolution is essential for constructing and visually analyzing the layout. The choice of resolution is particularly important, especially when dealing with temporally sparse graphs. In such cases, changing the temporal resolution by grouping events (i.e., edges) from consecutive timestamps — a technique known as timeslicing — can aid in the analysis and reveal patterns that might not be discernible otherwise. However, selecting an appropriate temporal resolution is a challenging task. In this paper, we propose ZigzagNetVis, a methodology that suggests temporal resolutions potentially relevant for analyzing a given graph, i.e., resolutions that lead to substantial topological changes in the graph structure. ZigzagNetVis achieves this by leveraging zigzag persistent homology, a well-established technique from Topological Data Analysis (TDA). To improve visual graph analysis, ZigzagNetVis incorporates the colored barcode, a novel timeline-based visualization inspired by persistence barcodes commonly used in TDA. We also contribute with a web-based system prototype that implements suggestion methodology and visualization tools. Finally, we demonstrate the usefulness and effectiveness of ZigzagNetVis through a usage scenario, a user study with 27 participants, and a detailed quantitative evaluation.

Abstract:
The design of urban road networks significantly influences traffic conditions, underscoring the importance of informed traffic planning. Traffic planning experts rely on specialized platforms to simulate traffic systems, assessing the efficacy of the road network across various states of modifications. Nevertheless, a prevailing issue persists: many existing traffic planning platforms exhibit inefficiencies in flexibly interacting with the road network’s structure and attributes and intuitively comparing multiple states during the iterative planning process. This paper introduces TraSculptor, an interactive planning decision-making system. To develop TraSculptor, we identify and address two challenges: interactive modification of road networks and intuitive comparison of multiple network states. For the first challenge, we establish flexible interactions to enable experts to easily and directly modify the road network on the map. For the second challenge, we design a comparison view with a history tree of multiple states and a road-state matrix to facilitate intuitive comparison of road network states. To evaluate TraSculptor, we provided a usage scenario where the Braess’s paradox was showcased, invited experts to perform a case study on the Sioux Falls network, and collected expert feedback through interviews.

Abstract:
We introduce DashSpace, a live collaborative immersive and ubiquitous analytics (IA/UA) platform designed for handheld and head-mounted Augmented/Extended Reality (AR/XR) implemented using WebXR and open standards. To bridge the gap between existing web-based visualizations and the immersive analytics setting, DashSpace supports visualizing both legacy D3 and Vega-Lite visualizations on 2D planes, and extruding Vega-Lite specifications into 2.5D. It also supports fully 3D visual representations using the Optomancy grammar. To facilitate authoring new visualizations in immersive XR, the platform provides a visual authoring mechanism where the user groups specification snippets to construct visualizations dynamically. The approach is fully persistent and collaborative, allowing multiple participants—whose presence is shown using 3D avatars and webcam feeds—to interact with the shared space synchronously, both co-located and remotely. We present three examples of DashSpace in action: immersive data analysis in 3D space, synchronous collaboration, and immersive data presentations.

Abstract:
Layered network visualizations assign each node to one of several parallel axes. They can convey sequence or flow data, hierarchies, or multiple data classes, but edge crossings and long edges often impair readability. Layout algorithms can reduce edge crossings and shorten edges using quick heuristics or optimal methods that prioritize human readability over computation speed. This work uses an optimization metaheuristic to provide the best of both worlds: high-quality layouts within a predetermined execution time. Our adaptation of the large neighborhood search (LNS) metaheuristic repeatedly selects fixed-sized subgraphs to lay out optimally. We conducted a computational evaluation using 450 synthetic networks to compare five ways of selecting candidate nodes, four ways of selecting their neighboring subgraph, and three criteria for determining subgraph size. LNS generally halved the number of crossings versus the barycentric heuristic while maintaining a reasonable runtime. Our best approach randomly selected candidate nodes, used degree centrality to pick cluster-like neighborhoods, and chose smaller neighborhoods that could be optimally laid out in 0.6 or 1.2 seconds (versus 6 seconds). In a case study visualizing 13 control flow graphs, most with over 1000 nodes, we show that our method can be employed to create visualizations with fewer crossings than Tabu Search, another metaheuristic, and vastly outperforms an ILP solver when runtime is bounded.

Abstract:
We present drillboards, a technique for adaptive visualization dashboards consisting of a hierarchy of coordinated charts that the user can drill down to reach a desired level of detail depending on their expertise, interest, and desired effort. This functionality allows different users to personalize the same dashboard to their specific needs and expertise. The technique is based on a formal vocabulary of chart representations and rules for merging multiple charts of different types and data into single composite representations. The drillboard hierarchy is created by iteratively applying these rules starting from a baseline dashboard, with each consecutive operation yielding a new dashboard with fewer charts and progressively more abstract and simplified views. We also present an authoring tool for building drillboards and show how it can be applied to an agricultural dataset with hundreds of expert users. Our evaluation asked three domain experts to author drillboards for their own datasets, which we then showed to casual end-users with favorable outcomes.

Abstract:
The task of reconstructing indoor floorplans has become an increasingly popular subject, offering substantial benefits across various applications such as interior design, virtual reality, and robotics. Despite the growing interest, existing approaches frequently encounter challenges due to high computational costs and sensitivity to errors in primitive detection. In this article, we introduce PolyGraph, a new computational framework that combines a deep-learning based primitive detection network with an optimization-based reconstruction algorithm to facilitate high-quality reconstruction results. Specifically, we develop a novel guided wall point primitive estimation network capable of generating dense samples along wall boundaries. This network not only retains structural detail but also shows improved robustness in the detection phase. Then, PolyGraph utilizes wall points to establish a graph-based representation, formulating indoor floorplan reconstruction as a subgraph optimization problem. This approach significantly reduces the search space comparing to existing pixel-level optimization approaches. By utilizing “structural weight”, we seamlessly integrate the structural information of walls and rooms into graph representations, ensuring high-quality reconstruction results. Experimental results demonstrate PolyGraph's effectiveness and its advantages compared to other optimization-based approaches, showcasing its computational efficiency, and its ability to preserve structural integrity and capture fine details, as quantified by the structure metrics.

Abstract:
Partial dependence plots (PDPs) and individual conditional expectation (ICE) plots are visualizations used for explaining the behavior of machine learning (ML) models trained on tabular datasets. They show how the values of a feature or pair of features impact a model’s predictions. However, in models with a large number of features, it is impractical for an ML practitioner to analyze all possible plots. To address this, we present new techniques for ranking and filtering PDP and ICE plots and build upon existing strategies for clustering the lines in ICE plots. Together, these techniques aim to help ML practitioners efficiently explore PDP and ICE plots and identify interesting model behavior. We integrate these techniques into PDPilot, a visual analytics tool that runs in Jupyter notebooks. We use PDPilot to study how 7 ML practitioners utilize the ranking, filtering, and clustering techniques to analyze an ML model.

Abstract:
Neural Radiance Fields (NeRF) use multi-view images for 3D scene representation, demonstrating remarkable performance. As one of the primary sources of multi-view images, multi-camera systems encounter challenges such as varying intrinsic parameters and frequent pose changes. Most previous NeRF-based methods assume a unique camera and rarely consider multi-camera scenarios. Besides, some NeRF methods that can optimize intrinsic and extrinsic parameters still remain susceptible to suboptimal solutions when these parameters are poor initialized. In this paper, we propose MC-NeRF, a method for joint optimization of both intrinsic and extrinsic parameters alongside NeRF, allowing individual camera parameters for each image. First, we analyze the coupling issue that arises from the joint optimization between intrinsics and extrinsics, and propose a decoupling constraint utilizing auxiliary images. To further address the degenerate cases in the decoupling process, we introduce an efficient auxiliary image acquisition scheme to mitigate these effects. Furthermore, recognizing that most existing datasets are designed for a unique camera, we provided a new dataset that includes both simulated data and real-world data. Experiments demonstrate the effectiveness of our method in scenarios where each image corresponds to different camera parameters. Specifically, our approach outperforms the baselines favorably in terms of intrinsics estimation, extrinsics estimation, scale estimation, and rendering quality.

Abstract:
Immersive analytics is gaining attention across multiple domains due to its capability to facilitate intuitive data analysis in expansive environments through user interaction with data. However, creating immersive analytics systems for specific tasks is challenging due to the need for programming expertise and significant development effort. Despite the introduction of various immersive visualization authoring toolkits, domain experts still face hurdles in adopting immersive analytics into their workflow, particularly when faced with dynamically changing tasks and data in real time. To lower such technical barriers, we introduce XROps, a web-based authoring system that allows users to create immersive analytics applications through interactive visual programming, without the need for low-level scripting or coding. XROps enables dynamic immersive analytics authoring by allowing users to modify each step of the data visualization process with immediate feedback, enabling them to build visualizations on-the-fly and adapt to changing environments. It also supports the integration and visualization of real-time sensor data from XR devices—a key feature of immersive analytics—facilitating the creation of various analysis scenarios. We evaluated the usability of XROps through a user study and demonstrate its efficacy and usefulness in several example scenarios.

Abstract:
Reconstructing objects from posed images is a crucial and complex task in computer graphics and computer vision. While NeRF-based neural reconstruction methods have exhibited impressive reconstruction ability, they tend to be time-comsuming. Recent strategies have adopted 3D Gaussian Splatting (3D-GS) for inverse rendering, which have led to quick and effective outcomes. However, these techniques generally have difficulty in producing believable geometries and materials for glossy objects, a challenge that stems from the inherent ambiguities of inverse rendering. To address this, we introduce GlossyGS, an innovative 3D-GS-based inverse rendering framework that aims to precisely reconstruct the geometry and materials of glossy objects by integrating material priors. The key idea is the use of micro-facet geometry segmentation prior, which helps to reduce the intrinsic ambiguities and improve the decomposition of geometries and materials. Additionally, we introduce a normal map prefiltering strategy to more accurately simulate the normal distribution of reflective surfaces. These strategies are integrated into a hybrid geometry and material representation that employs both explicit and implicit methods to depict glossy objects. We demonstrate through quantitative analysis and qualitative visualization that the proposed method is effective to reconstruct high-fidelity geometries and materials of glossy objects, and performs favorably against State-of-the-Arts.

Abstract:
This paper presents GaussEdit, a framework for adaptive 3D scene editing guided by text and image prompts. GaussEdit leverages 3D Gaussian Splatting as its backbone for scene representation, enabling convenient Region of Interest selection and efficient editing through a three-stage process. The first stage involves initializing the 3D Gaussians to ensure high-quality edits. The second stage employs an Adaptive Global-Local Optimization strategy to balance global scene coherence and detailed local edits and a category-guided regularization technique to alleviate the Janus problem. The final stage enhances the texture of the edited objects using a sophisticated image-to-image synthesis technique, ensuring that the results are visually realistic and align closely with the given prompts. Our experimental results demonstrate that GaussEdit surpasses existing methods in editing accuracy, visual fidelity, and processing speed. By successfully embedding user-specified concepts into 3D scenes, GaussEdit is a powerful tool for detailed and user-driven 3D scene editing, offering significant improvements over traditional methods.

Abstract:
Traditional human neural radiance fields often overlook crucial body semantics, resulting in ambiguous reconstructions, particularly in occluded regions. To address this problem, we propose the Super-Semantic Disentangled Neural Renderer (SSD-NeRF), which employs rich regional semantic priors to enhance human rendering accuracy. This approach initiates with a Visible-Invisible Semantic Propagation module, ensuring coherent semantic assignment to occluded parts based on visible body segments. Furthermore, a Region-Wise Texture Propagation module independently extends textures from visible to occluded areas within semantic regions, thereby avoiding irrelevant texture mixtures and preserving semantic consistency. Additionally, a view-aware curricular learning approach is integrated to bolster the model's robustness and output quality across different viewpoints. Extensive evaluations confirm that SSD-NeRF surpasses leading methods, particularly in generating quality and structurally semantic reconstructions of unseen or occluded views and poses.

Abstract:
Geometry- and appearance-controlled full-body human image generation is an interesting but challenging task. Existing solutions are either unconditional or dependent on coarse conditions (e.g., pose, text), thus lacking explicit geometry and appearance control of body and garment. Sketching offers such editing ability and has been adopted in various sketch-based face generation and editing solutions. However, directly adapting sketch-based face generation to full-body generation often fails to produce high-fidelity and diverse results due to the high complexity and diversity in the pose, body shape, and garment shape and texture. Recent geometrically controllable diffusion-based methods mainly rely on prompts to generate appearance. It is hard to balance the realism and the faithfulness of their results to the sketch when the input is coarse. This work presents Sketch2Human, the first system for controllable full-body human image generation guided by a semantic sketch (for geometry control) and a reference image (for appearance control). Our solution is based on the latent space of StyleGAN-Human with inverted geometry and appearance latent codes as input. Specifically, we present a sketch encoder trained with a large synthetic dataset sampled from StyleGAN-Human's latent space and directly supervised by sketches rather than real images. Considering the entangled information of partial geometry and texture in StyleGAN-Human and the absence of disentangled datasets, we design a novel training scheme that creates geometry-preserved and appearance-transferred training data to tune a generator to achieve disentangled geometry and appearance control. Although our method is trained with synthetic data, it can also handle hand-drawn sketches. Qualitative and quantitative evaluations demonstrate the superior performance of our method to state-of-the-art methods.

Abstract:
The process of labeling medical text plays a crucial role in medical research. Nonetheless, creating accurately labeled medical texts of high quality is often a time-consuming task that requires specialized domain knowledge. Traditional methods for generating labeled data typically rely on rigid rule-based approaches, which may not adapt well to new tasks. While recent machine learning (ML) methodologies have mitigated the manual labeling efforts, configuring models to align with specific research requirements can be challenging for labelers without technical expertise. Moreover, automated labeling techniques, such as transfer learning, face difficulties in in directly incorporating expert input, whereas semi-automated methods, like data programming, allow knowledge integration through rules or knowledge bases but may lack continuous result refinement throughout the entire labeling process. In this study, we present a collaborative human-ML teaming workflow that seamlessly integrates visual cluster analysis and active learning to assist domain experts in labeling medical text with high efficiency. Additionally, we introduce an innovative neural network model called the embedding network, which incorporates expert insights to generate task-specific embeddings for medical texts. We integrate the workflow and embedding network into a visual analytics tool named KMTLabeler, equipped with coordinated multi-level views and interactions. Two illustrative case studies, along with a controlled user study, provide substantial evidence of the effectiveness of KMTLabeler in creating an efficient labeling environment for medical text classification.

Abstract:
Point clouds are widely used as a versatile representation of 3D entities and scenes for all scale domains and in a variety of application areas, serving as a fundamental data category to directly convey spatial features. However, due to point sparsity, lack of structure, irregular distribution, and acquisition-related inaccuracies, results of point cloud visualization are often subject to visual complexity and ambiguity. In this regard, non-photorealistic rendering can improve visual communication by reducing the cognitive effort required to understand an image or scene and by directing attention to important features. In the last 20 years, this has been demonstrated by various non-photorealistic rendering approaches that were proposed to target point clouds specifically. However, they do not use a common language or structure for assessment which complicates comparison and selection. Further, recent developments regarding point cloud characteristics and processing, such as massive data size or web-based rendering are rarely considered. To address these issues, we present a survey on non-photorealistic rendering approaches for point cloud visualization, providing an overview of the current state of research. We derive a structure for the assessment of approaches, proposing seven primary dimensions for the categorization regarding intended goals, data requirements, used techniques, and mode of operation. We then systematically assess corresponding approaches and utilize this classification to identify trends and research gaps, motivating future research in the development of effective non-photorealistic point cloud rendering methods.

Abstract:
Recent telepresence systems have shown significant improvements in quality compared to prior systems. However, they struggle to achieve both low cost and high quality at the same time. In this work, we envision a future where telepresence systems become a commodity and can be installed on typical desktops. To this end, we present a high-quality view synthesis method that uses a cost-effective capture system that consists of commodity hardware accessible to the general public. We propose a neural renderer that uses a few RGBD cameras as input to synthesize novel views of a user and their surroundings. At the core of the renderer is Multi-Layer Point Cloud (MPC), a novel 3D representation that improves reconstruction accuracy by removing non-linear biases in depth cameras. Our temporally-aware renderer further improves the stability of synthesized videos by conditioning on past information. Additionally, we propose Spatial Skip Connections (SSC) to improve image upsampling under limited GPU memory. Experimental results show that our renderer outperforms recent methods in terms of view synthesis quality. Our method generalizes to new users and challenging content (e.g. hand gestures and clothing deformation) without costly per-video optimization, object templates, or heavy pre-processing. The code and dataset will be made available.

Abstract:
We propose a new method for computing smooth and integrable cross fields on 2D and 3D surfaces. our approach first computes smooth cross fields by minimizing the Dirichlet energy. Unlike existing optimization-based methods, our technique determines the singularity configuration–i.e., the number, locations, and indices of singularities–by iteratively adjusting them. Singularities can move, merge and split, akin to the behavior of like charges repelling and unlike charges attracting. Once all singularities stop moving, we obtain a cross field with (locally) the lowest Dirichlet energy. In simply connected domains, this cross field is guaranteed to be integrable. However, this property does not hold in multiply connected domains. To make a smooth cross field integrable, we construct a vector field \bf cc that characterizes the deviation of the cross field from a curl-free field. We then optimize the locations of singularities by moving them along the field lines of \bf cc. Our method is fundamentally different from existing integer programming-based approaches, as it avoids combinatorial optimization. It is fully automatic and includes a parameter to control the number of singularities. Our method is well suited for smooth models where exact boundary alignment and sparse hard directional constraints are desired, and can guide seamless conformal parameterization and T-junction-free quadrangulation.

Abstract:
Co-adaptive guidance in visual analytics is a mixed-initiative process in which both the user and the system work together to support each other in solving a given analysis task. While previous studies show the effectiveness of guidance, the impact of guidance design decisions (e.g., the suggestions’ timing, contextualization, or adaptation) and misguidance often remain under-investigated. To investigate these aspects and examine a variety of guidance interaction patterns in a realistic analysis scenario, we present a Wizard of Oz (WOz) study setup in which pairs of participants take the user's and the system's roles, respectively. As users perform their analysis tasks, they are observed by wizards who provide just-in-time guidance as they see fit. Moreover, we designed the study so that wizards would occasionally and unknowingly provide misguidance during the analysis to investigate the users’ confidence in guidance systems. We recruited two groups of participants (12 wizards and 12 users) and paired each participant with two from the other group, obtaining 48 observations. We report insights on interactions between users and wizards. By analyzing these interaction dynamics and the guidance strategies the wizards apply, we derive recommendations for implementing and evaluating future co-adaptive guidance systems.

Abstract:
Whereas contrastive learning eliminates the need for labeled data, existing methods may suffer from inadequate features due to the conventional single shared encoder structure and struggle to fully harness the rich spectrum of 3D augmentations. In this paper, we propose TriCI, a self-supervised method that designs a triple-branch contrastive learning architecture. During contrastive pre-training, we generate three augmented versions of each input point cloud sample and pair each augmented sample with the original one, resulting in three unique positive pairs. We subsequently feed the pairs into three distinct encoders, each of which extracts features from its corresponding input positive pair. We design a novel cross-branch contrastive loss and use it along with the intra-branch contrastive loss to jointly train our network. The proposed cross-branch loss effectively aligns the output features from different perspectives for pre-training and facilitates their integration for downstream tasks, particularly in object-level scenarios. The intra-branch loss helps maximize the feature correspondences within positive pairs. Extensive experiments demonstrate the superiority of our TriCI in self-supervised learning, and show its strong ability in enhancing the performance of downstream object classification and part segmentation tasks. Interestingly, our TriCI achieves a 92.9% accuracy for linear SVM evaluation on ModelNet40, exceeding its closest competitor by 1.7% and even exceeding some supervised methods.

Abstract:
This article proposes a hybrid radiance field representation for unbounded immersive light field reconstruction which supports high-quality rendering and aggressive view extrapolation. The key idea is to first formally separate the foreground and the background and then adaptively balance learning of them during the training process. To fulfill this goal, we represent the foreground and background as two separate radiance fields with two different spatial mapping strategies. We further propose an adaptive sampling strategy and a segmentation regularizer for more clear segmentation and robust convergence. Finally, we contribute a novel immersive light field dataset, named THUImmersive, with the potential to achieve much larger space 6DoF immersive rendering effects compared with existing datasets, by capturing multiple neighboring viewpoints for the same scene, to stimulate the research and AR/VR applications in the immersive light field domain. Extensive experiments demonstrate the strong performance of our method for unbounded immersive light field reconstruction.

Abstract:
Dynamic colored meshes (DCM) are widely used in various applications. However, this kind of meshes may undergo different processes, such as compression or transmission, which can distort them and degrade their quality. To facilitate the development of objective metrics for DCMs and study the influence of typical distortions on their perception, we create the Tencent - Dynamic colored Mesh Database (TDMD) containing eight reference DCM objects with six typical distortions. Using processed video sequences (PVS) derived from the DCM, we conduct a large-scale subjective experiment that resulted in 303 distorted DCM samples with mean opinion scores, making the TDMD the largest available DCM database to our knowledge. This database enables us to study the impact of different types of distortion on human perception and offers recommendations for DCM compression and related tasks. Additionally, we have evaluated three types of state-of-the-art objective metrics on the TDMD, including image-based, point-based, and video-based metrics, on the TDMD. Our experimental results highlight the strengths and weaknesses of each metric, and we provide suggestions about the selection of metrics in practical DCM applications.

Abstract:
Grasping generation holds significant importance in both robotics and AI-generated content. While pure network paradigms based on VAEs or GANs ensure diversity in outcomes, they often fall short of achieving plausibility. Additionally, although those two-step paradigms that first predict contact and then optimize distance yield plausible results, they are always known to be time-consuming. This paper introduces a novel paradigm powered by DDPM, accommodating diverse modalities with varying interaction granularities as its generating conditions, including 3D object, contact affordance, and image content. Our key idea is that the iterative steps inherent to diffusion models can supplant the iterative optimization routines in existing optimization methods, thereby endowing the generated results from our method with both diversity and plausibility. Using the same training data, our paradigm achieves superior generation performance and competitive generation speed compared to optimization-based paradigms. Extensive experiments on both in-domain and out-of-domain objects demonstrate that our method receives significant improvement over the SOTA method. We will release the code for research purposes.

Abstract:
Cybersickness, or sickness induced by virtual reality (VR), negatively impacts the enjoyment and adoption of the technology. One method that has been used to reduce sickness is repeated exposure to VR, herein Cybersickness Abatement from Repeated Exposure (CARE). However, high sickness levels during repeated exposure may discourage some users from returning. Field of view (FOV) restriction reduces cybersickness by minimizing visual motion in the periphery, but also negatively affects the user’s visual experience. This study explored whether CARE that occurs with FOV restriction generalizes to a full FOV experience. Participants played a VR game for up to 20 minutes. Those in the Repeated Exposure Condition played the same VR game on four separate days, experiencing FOV restriction during the first three days and no FOV restriction on the fourth day. Results indicated significant CARE with FOV restriction (Days 1--3). Further, cybersickness on Day 4, without FOV restriction, was significantly lower than that of participants in the Single Exposure Condition, who experienced the game without FOV restriction only on one day. The current findings show that significant CARE can occur while experiencing minimal cybersickness. Results are considered in the context of multiple theoretical explanations for CARE, including sensory rearrangement, adaptation, habituation, and postural control.

Abstract:
Gait disorders are commonly observed in older adults, who frequently experience various issues related to walking. Additionally, researchers and clinicians extensively investigate mobility related to gait in typically and atypically developing children, athletes, and individuals with orthopedic and neurological disorders. Effective gait analysis enables the understanding of the causal mechanisms of mobility and balance control of patients, the development of tailored treatment plans to improve mobility, the reduction of fall risk, and the tracking of rehabilitation progress. However, analyzing gait data is a complex task due to the multivariate nature of the data, the large volume of information to be interpreted, and the technical skills required. Existing tools for gait analysis are often limited to specific patient groups (e.g., cerebral palsy), only handle a specific subset of tasks in the entire workflow, and are not openly accessible. To address these shortcomings, we conducted a requirements assessment with gait practitioners (e.g., researchers, clinicians) via surveys and identified key components of the workflow, including (1) data processing and (2) data analysis and visualization. Based on the findings, we designed VIGMA, an open-access visual analytics framework integrated with computational notebooks and a Python library, to meet the identified requirements. Notably, the framework supports analytical capabilities for assessing disease progression and for comparing multiple patient groups. We validated the framework through usage scenarios with experts specializing in gait and mobility rehabilitation.

Abstract:
Dense sampling of the light field (LF) is essential for various applications, such as virtual reality. However, the collection process is prohibitively expensive due to technological limitations in imaging. Synthesizing novel views from sparse LF data, known as LF Angular Super-Resolution (LFASR), offers an effective solution to this problem. Accurate cross-view interaction is crucial for this task, given the complementary information between LF views. Previous methods, however, suffer from limited reconstruction quality due to inefficient view interaction. To address this, we propose a Multiplane-based Cross-view Interaction Mechanism (MCIM) for robust LFASR. Extensive comparisons with state-of-the-art methods demonstrate that our method achieves superior performance, both visually and quantitatively. Specifically, Drawing inspiration from MultiPlane Images (MPI) in scene modeling, our mechanism incorporates a novel Multiplane Feature Fusion (MPFF) strategy. This strategy facilitates fast and accurate cross-view interaction, enhancing the network’s robustness to scene geometry and suitability for different-baseline LF scenes. Furthermore, to address information redundancy in multiplanes, we leverage the transparency property of MPI and devise a plane selection strategy. Finally, we propose CSTNet, a Cross-Shaped Transformer-based network for LFASR, which employs a cross-shaped self-attention mechanism to enable low-cost training and inference. Experimental results on various angular super-resolution tasks validate that our network achieves state-of-the-art performance on both synthetic and real-world LF scenes.

Abstract:
This paper introduces the concept of Content-Aware Radiance Fields (CoARF), which adaptively aligns the model complexity with the scene intricacy. By examining the intricacies of radiance fields from three perspectives, model complexity is adapted through scalable feature grids, dynamic neural networks, and model quantization. Specifically, we propose a hash collision detection mechanism that removes redundant feature grid by restricting the valid hash collision to reasonable level, making the space complexity scalable. We introduce an uncertainty-aware decoded layer, where simple points are early-exited to prevent them from being processed by deeper network layers, ensuring computational complexity scalable. Furthermore, we propose Learned Bitwidth Quantization (LBQ) and Adversarial Content-Aware Quantization (A-CAQ) paradigms by making the bitwidth of parameters differentiable and trainable, allowing for adjustable quantization schemes. Building on these techniques, the proposed CoARF++ framework enables a scalable pipeline for radiance fields that is tailored to the unique characteristics of scene complexity and quality requirement. Extensive experiments demonstrate a significant and adjustable reduction in model complexity across various NeRF variants, while maintaining the necessary reconstruction and rendering quality, making it advantageous for the practical deployment of radiance field models.

Abstract:
Generative models are becoming a transformative technology for the creation and editing of images. However, it remains challenging to harness these models for precise image manipulation. These challenges often manifest as inconsistency in the editing process, where both the type and amount of semantic change, depend on the image being manipulated. Moreover, there exist many methods for computing image manipulations, whose development is hindered by the matter of inconsistency. This paper aims to address these challenges by improving how we evaluate, compare, and explore the space of manipulations offered by a generative model. We present Concept Lens, a visual interface that is designed to aid users in understanding semantic concepts carried in image manipulations, and how these manipulations vary over generated images. Given the large space of possible images produced by a generative model, Concept Lens is designed to support the exploration of both generated images, and their manipulations, at multiple levels of detail. To this end, the layout of Concept Lens is informed by two hierarchies: a hierarchical organization of (1) original images, grouped by their similarities, and (2) image manipulations, where manipulations that induce similar changes are grouped together. This layout allows one to discover the types of images that consistently respond to a group of manipulations, and vice versa, manipulations that consistently respond to a group of codes. We show the benefits of this design across multiple use cases, specifically, studying the quality of manipulations for a single method, and offering a means of comparing different methods.

Abstract:
Performance models of interaction, such as Fitts’ law, are important tools for predicting and explaining human motor performance and for designing high-performance user interfaces. Extensive prior work has proposed such models for the 3D interaction task of distal pointing, in which the user points their hand or a device at a distant target in order to select it. However, there is no consensus on how to compute the index of difficulty for distal pointing tasks. We present a preliminary study suggesting that existing models may not be sufficient to model distal pointing performance with current virtual reality technologies. Based on these results, we hypothesized that both the form of the model and the standard method for collecting empirical data for pointing tasks might need to change in order to achieve a more accurate and valid distal pointing model. In our main study, we used a new methodology to collect distal pointing data and evaluated traditional models, purely ballistic models, and two-part models. Ultimately, we found that the best model used a simple Fitts’-law-style index of difficulty with angular measures of amplitude and width.

Abstract:
The ability to extract invariant visual features of objects from complex scenes and identify the same objects in different scenes is inborn for humans. To endow AI systems with such capability, we introduce a novel compositional scene understanding method known as Compositional Scene understanding via Global Object-centric representations (CSGOs). CSGO achieves comprehensive scene understanding, including the discovery and identification of objects, by leveraging a set of learnable global object-centric representations in an unsupervised manner. CSGO comprises three components: 1) Local Object-Centric Learning, which is responsible for extracting localized and scene-specific object-centric representations to discover objects; 2) Image Decoding, facilitating the reconstruction of object and scene images using the obtained object-centric representation as input; and 3) Global Object-Centric Learning, identifying the object across diverse scenes according to a set of learnable global object-centric representations, which indicates the scene-free intrinsic attributes (i.e., appearance and shape) of objects. Experimental results on three synthetic datasets and one real-world scene dataset demonstrate that CSGO has excellent object identification and attribute disentanglement abilities. Furthermore, the scene decomposition performance (indicating object discovery performance) of CSGO is superior to comparison methods.

Abstract:
Recent advancements in neural rendering methods, such as Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3D-GS), have significantly revolutionized photo-realistic novel view synthesis of scenes with multiple photos or videos as input. However, existing approaches within the NeRF and 3D-GS frameworks often assume the independence of point sampling and ray casting, which are intrinsic to volume rendering and alpha-blending techniques. These underlying assumptions limit the ability to aggregate context within subspaces, such as densities and colors in the radiance fields and pixels on the image plane, leading to synthesized images that lack fine details and smoothness. To overcome this, we propose a universal framework, MixRF, comprising a Radiance Field Mixer (RF-mixer) and a Color Domain Mixer (CD-mixer), to sufficiently aggregate and fully explore information in neighboring sampled points and casting rays, separately. The RF-mixer treats sampled points as an explicit point cloud, enabling the aggregation of density and color attributes from neighboring points to better capture local geometry and appearance. Meanwhile, the CD-mixer rearranges rendered pixels on the sub-image plane, improving smoothness and recovering fine details and textures. Both mixers employ a kernel-based mixing strategy to facilitate effective and controllable attribute aggregation, ensuring a more comprehensive exploration of radiance values and pixel information. Extensive experiments demonstrate that our MixRF framework is compatible with radiance field-based methods, including NeRF and 3D-GS designs. The proposed framework dramatically enhances performance in both qualitative and quantitative evaluations, with less than a 25%25% increase in computational overhead during inference.

Abstract:
Despite recent advancements in the Large Reconstruction Model (LRM) demonstrating impressive results, when extending its input from single image to multiple images, it exhibits inefficiencies, subpar geometric and texture quality, as well as slower convergence speed than expected. It is attributed to that, LRM formulates 3D reconstruction as a naive images-to-3D translation problem, ignoring the strong 3D coherence among the input images. In this article, we propose a Multi-view Large Reconstruction Model (M-LRM) designed to reconstruct high-quality 3D shapes from multi-views in a 3D-aware manner. Specifically, we introduce a multi-view consistent cross-attention scheme to enable M-LRM to accurately query information from the input images. Moreover, we employ the 3D priors of the input multi-view images to initialize the triplane tokens. Compared to previous methods, the proposed M-LRM can generate 3D shapes of high fidelity. Experimental studies demonstrate that our model achieves a significant performance gain and faster training convergence.

Abstract:
Online group discussions play an important role in education reform by facilitating collaborative learning and knowledge sharing among participants. However, instructors face significant challenges in monitoring discussion progress, tracking student performance and understanding interaction dynamics due to overlapping conversations, time-varying participant behaviors, and hidden interaction patterns. To address these challenges, we propose GroupTrackVis, an interactive visual analytics system that incorporates both advanced algorithms and novel visualization designs, to help instructors analyze group discussions mainly from three perspectives: topic evolution, student performance, and interaction. GroupTrackVis proposes an enhanced topic segmentation algorithm by incorporating word vector weighting and reply relationship analysis, effectively disentangling overlapping discussions. It also extracts six key behavioral attributes from multimodal educational data, offering a comprehensive view of student performance and providing insights into the key factors driving learning outcomes. Additionally, a multi-layer tree network with edge bundling techniques is implemented to clearly visualize the dynamic evolution of student interactions. The integration of algorithms with interactive visualizations enables instructors to explore discussions quickly and dynamically adjust their analysis as the discussion evolves. The effectiveness of GroupTrackVis is demonstrated through two case studies, a user study, and expert interviews, highlighting its ability to support instructors in identifying engaged and disengaged students, and tracking discussion dynamics.

Abstract:
Structured layouts are preferable in many 2D visual contents (e.g., GUIs, webpages) since the structural information allows convenient layout editing. Computational frameworks can help create structured layouts but require heavy labor input. Existing data-driven approaches are effective in automatically generating fixed layouts but fail to produce layout structures. We present StructLayoutFormer, a novel Transformer-based approach for conditional structured layout generation. We use a structure serialization scheme to represent structured layouts as sequences. To better control the structures of generated layouts, we disentangle the structural information from the element placements. Our approach is the first data-driven approach that achieves conditional structured layout generation and produces realistic layout structures explicitly. We compare our approach with existing data-driven layout generation approaches by including post-processing for structure extraction. Extensive experiments have shown that our approach exceeds these baselines in conditional structured layout generation. We also demonstrate that our approach is effective in extracting and transferring layout structures.

Abstract:
Upon completing the design and training phases, deploying a deep learning model to specific hardware becomes necessary prior to its implementation in practical applications. To enhance the performance of the model, the developers must optimize it to decrease inference latency. Auto-scheduling, an automated approach that generates optimization schemes, offers a feasible option for large-scale auto-deployment. Nevertheless, the low-level code generated by auto-scheduling closely resembles hardware coding and may present challenges for human comprehension, thereby hindering future manual optimization efforts. In this study, we introduce ASight, a visual analytics system to assist engineers in identifying performance bottlenecks, comprehending the auto-generated low-level code, and obtaining insights from auto-scheduling optimizations. We develop a subgraph matching algorithm capable of identifying graph isomorphism among Intermediate Representations to track performance bottlenecks from low-level metrics to high-level computational graphs. To address the substantial profiling metrics involved in auto-scheduling and derive optimization design principles by summarizing commonalities among auto-scheduling optimizations, we propose an enhanced visualization for the large search space of auto-scheduling. We validate the effectiveness of ASight through two case studies, one focused on a local machine and the other on a data center, along with a quantitative experiment exploring optimization design principles.

Abstract:
The proliferation of large language models (LLMs) has underscored concerns regarding their security vulnerabilities, notably against jailbreak attacks, where adversaries design jailbreak prompts to circumvent safety mechanisms for potential misuse. Addressing these concerns necessitates a comprehensive analysis of jailbreak prompts to evaluate LLMs’ defensive capabilities and identify potential weaknesses. However, the complexity of evaluating jailbreak performance and understanding prompt characteristics makes this analysis laborious. We collaborate with domain experts to characterize problems and propose an LLM-assisted framework to streamline the analysis process. It provides automatic jailbreak assessment to facilitate performance evaluation and support analysis of components and keywords in prompts. Based on the framework, we design JailbreakLens, a visual analysis system that enables users to explore the jailbreak performance against the target model, conduct multi-level analysis of prompt characteristics, and refine prompt instances to verify findings. Through a case study, technical evaluations, and expert interviews, we demonstrate our system’s effectiveness in helping users evaluate model security and identify model weaknesses.

Abstract:
Dynamic three-dimensional (4D) reconstruction from two-dimensional X-ray coronary angiography (CA) remains a significant clinical problem. Existing CA reconstruction methods often require extensive user interaction or large training datasets. Recently, Neural Radiance Field (NeRF) has successfully reconstructed high-fidelity scenes in natural and medical contexts without these requirements. However, challenges such as sparse-views, intra-scan motion, and complex vessel morphology hinder its direct application to CA data. We introduce NeRF-CA, a first step toward a fully automatic 4D CA reconstruction that achieves reconstructions from sparse coronary angiograms. To the best of our knowledge, we are the first to address the challenges of sparse-views and cardiac motion by decoupling the scene into the moving coronary artery and the static background, effectively translating the problem of motion into a strength. NeRF-CA serves as a first stepping stone for solving the 4D CA reconstruction problem, achieving adequate 4D reconstructions from as few as four angiograms, as required by clinical practice, while significantly outperforming state-of-the-art sparse-view X-ray NeRF. We validate our approach quantitatively and qualitatively using representative 4D phantom datasets and ablation studies.

Abstract:
Object placement in Augmented Reality (AR) is crucial for creating immersive and functional experiences. However, a critical research gap exists in combining user input with efficient automated placement, particularly in understanding spatial relationships and optimal placement. This study addresses this gap by presenting a novel object placement pipeline for AR applications that balances automation with user-directed placement. The pipeline employs entity recognition, object detection, depth estimation along with spawn area allocation to create a placement system. We compared our proposed method against manual placement in a comprehensive evaluation involving 50 participants. The evaluation included user experience questionnaires, a comparative study of task performance, and post-task interviews. Results indicate that our pipeline significantly reduces task completion time while maintaining comparable accuracy to manual placement. The UEQ-S and TENS scores revealed high user satisfaction. While manual placement offered more direct control, our method provided a more streamlined, efficient experience. This study contributes to the field of object placement in AR by demonstrating the potential of automated systems to enhance user experience and task efficiency.

Abstract:
Recent CLIP-guided 3D generation methods have achieved promising results but struggle with generating faithful 3D shapes that conform with input text due to the gap between text and image embeddings. To this end, this paper proposes HOTS3D which makes the first attempt to effectively bridge this gap by aligning text features to the image features with spherical optimal transport (SOT). However, in high-dimensional situations, solving the SOT remains a challenge. To obtain the SOT map for high-dimensional features obtained from CLIP encoding of two modalities, we mathematically formulate and derive the solution based on Villani’s theorem, which can directly align two hyper-sphere distributions without manifold exponential maps. Furthermore, we implement it by leveraging input convex neural networks (ICNNs) for the optimal Kantorovich potential. With the optimally mapped features, a diffusion-based generator is utilized to decode them into 3D shapes. Extensive quantitative and qualitative comparisons with state-of-the-art methods demonstrate the superiority of HOTS3D for text-to-3D generation, especially in the consistency with text semantics.

Abstract:
This paper presents a new algorithm, Weighted Squared Volume Minimization (WSVM), for generating high-quality tetrahedral meshes from closed triangle meshes. Drawing inspiration from the principle of minimal surfaces that minimize squared surface area, WSVM employs a new energy function integrating weighted squared volumes for tetrahedral elements. When minimized with constant weights, this energy promotes uniform volumes among the tetrahedra. Adjusting the weights to account for local geometry further achieves uniform dihedral angles within the mesh. The algorithm begins with an initial tetrahedral mesh generated via Delaunay tetrahedralization and proceeds by sequentially minimizing volume-oriented and then dihedral angle-oriented energies. At each stage, it alternates between optimizing vertex positions and refining mesh connectivity through the iterative process. The algorithm operates fully automatically and requires no parameter tuning. Evaluations on a variety of 3D models demonstrate that WSVM consistently produces tetrahedral meshes of higher quality, with fewer slivers and enhanced uniformity compared to existing methods.

Abstract:
Analyzing the impact of socioeconomic and urban variables on crime is a complex data analysis problem. Exploring synthetic, correlation-based scenarios using changes in a set of variables could alter a region’s definition from unsafe to safe (known counterfactual explanation), which can aid decision-makers in interpreting crime in that region and define public policies to mitigate criminal activity. We propose CounterCrime, a visual analytics tool for crime analysis that uses counterfactual explanations to add insights for this problem. This tool employs various interactive visual metaphors to explore the counterfactual explorations generated in each region. To facilitate exploration, we organize our analysis at three levels: the whole city, the region group, and the regional level. This work proposes a new perspective in crime analysis by creating “what-if” scenarios and allowing decision-makers to anticipate changes that would make a region safer. The tool guides the user in selecting variables with the most significant effect in all city regions. Using a greedy strategy, the system recommends the best variables that may influence crime in unsafe regions as the user explores. Our tool allows for identifying the most appropriate counterfactual explorations at the regional level by grouping them by similarity and determining their feasibility by comparing them with existing examples in other regions. Using crime data from São Paulo, Brazil, we validated our results with case studies. These case studies reveal interesting findings; for example, scenarios that influence crime in a particular unsafe region (or set of regions) might not influence crime in other unsafe regions.

Abstract:
Despite significant advancements in neural simulation techniques for clothing animation, these methods struggle to capture the dynamic details of garments during movement. This limitation restricts their applicability in scenarios where high-quality garment deformation is essential. To address this challenge, we introduce a novel graph learning-based approach to enhance deformation realism through designed mechanisms for mesh information propagation and external optimization strategies during model training. First, we address the issue of over-smoothing common in conventional graph processing techniques by introducing a flexible message-passing method. This approach effectively manages node interactions within the mesh, thereby improving the expressiveness of the model. Furthermore, acknowledging that uniform model supervision typically neglects high-frequency details during optimization, we analyze the spectral properties of clothing meshes. Based on this analysis, we introduce a frequency-division constraint aligned with the characteristics of different frequency bands, which aids in precisely controlling the generation of details. Our model further integrates self-collision and other physics-aware losses, enabling the learning of generalized and fine-grained dynamic deformations. Extensive evaluations and comparisons demonstrate the effectiveness of our approach, showing notable improvements over existing state-of-the-art solutions.

Abstract:
Additive manufacturing (AM) is revolutionizing the production of custom components with complex internal geometries, essential for high-performance applications in diverse fields such as medicine and defense. These AM parts optimize strength while minimizing weight by utilizing internal lattice structures consisting of large quantities of small interconnected struts. However, the complexity of these structures, combined with the challenges of using X-ray Computed Tomography (XCT) data, makes validation of part reliability difficult. This ultimately inhibits the development of novel parts for our collaborating material scientists. We introduce LatticeAnalytics, a novel framework specifically designed for visual inspection of defects in these lattice structures. Our framework offers an end-to-end solution that includes the data management of XCT scans, enables remote access for geographically dispersed teams through a web-based dashboard, and incorporates novel visualizations. Our analysis is facilitated by a coarse alignment between the lattice’s nominal model, a spatial graph, and the XCT data. We employ a simple VR-based approach for fast and rough alignment, followed by an offline registration and identification of the struts. With the nodes and struts aligned and identified in the volume, our framework allows querying of subvolumes containing a single strut at multiple resolutions. This avoids computation over the entire lattice and also allow for easy parallelization of down-stream computations, such as strut-specific metrics. To depict a fast overview of the strut quality, we introduce two innovative visual encodings, crucial for our collaborators’ research in creating novel AM parts: the Contour View and the Roughness Map, which depict critical geometrical and surface features of individual struts in standardized two 2D views. We evaluated the integrated system through expert interviews. The feedback confirms the framework’s practicality and its effectiveness in enhancing current inspection workflows. It solves major bottlenecks for our collaborators, ultimately helping them create novel parts with advanced properties.

Abstract:
Elastic materials often invite the direct touch of users. It is an open question how seeing elastic materials invokes touch desire. The present study proposes a novel visual feature that modulates apparent elasticity and touch desire. The stimulus for our experiment was a clip in which a computer-rendered elastic surface was indented by a needle-like bar. The features of this stimulus that we focused on were spatial deformation range and indentation depth. Observers rated the following three impressions: apparent elasticity, touch desire, and anticipated touch pleasantness. The results showed that both apparent elasticity and touch desire peaked in the middle of the spatial deformation range. The two impressions also depended on indentation depth and were highly correlated with each other. Anticipated touch pleasantness showed a different peak tendency than the other two. An additional block showed that the deformation realism was not related to the above three impressions. The results suggest that apparent elasticity eliciting touch desire can be determined in the parameter space defined by the spatial deformation range and the indentation depth.

Abstract:
Recently, 3D Gaussian Splatting (3DGS) has attracted widespread attention due to its high-quality rendering, and ultra-fast training and rendering speed. However, due to the unstructured and irregular nature of Gaussian point clouds, it is difficult to guarantee geometric reconstruction accuracy and multi-view consistency simply by relying on image reconstruction loss. Although many studies on surface reconstruction based on 3DGS have emerged recently, the quality of their meshes is generally unsatisfactory. To address this problem, we propose a fast planar-based Gaussian splatting reconstruction representation (PGSR) to achieve high-fidelity surface reconstruction while ensuring high-quality rendering. Specifically, we first introduce an unbiased depth rendering method, which directly renders the distance from the camera origin to the Gaussian plane and the corresponding normal map based on the Gaussian distribution of the point cloud, and divides the two to obtain the unbiased depth. We then introduce single-view geometric, multi-view photometric, and geometric regularization to preserve global geometric accuracy. We also propose a camera exposure compensation model to cope with scenes with large illumination variations. Experiments on indoor and outdoor scenes show that the proposed method achieves fast training and rendering while maintaining high-fidelity rendering and geometric reconstruction, outperforming 3DGS-based and NeRF-based methods.

Abstract:
Inserting an object into a background scene has wide applications in image editing and mixed reality. However, existing methods still struggle to seamlessly adapt the object to the background while maintaining its individual characteristics. In this article, we propose to fine-tune a pre-trained diffusion-based insertion model such that it learns to establish a unique correspondence between a few weights and the target object, given as input few-shot images of an object. A novel individualized feature extraction (IFE) module is designed to extract the individual detail features from few-shot object images. Then, the individual features of the target object, together with the semantic features of the target object and the background context features extracted by the pre-trained image encoders are injected into the cross-attention modules of the latent diffusion model, enabling it to learn the correlation information of the target object and the background scene through the attention mechanism. The weights obtained by fine-tuning implicitly serve as an alternative representation of the target object, with which the object can be easily inserted into any background images. Extensive comparative experiments validate the superiority of the proposed method to the state-of-the-art insertion methods in maintaining the individual details of the inserted object and adapting it to background scenes, including allowing the interaction between the inserted object and the background scene, correctly handling their occlusion relationship, maintaining the consistency of their viewpoints and poses.

Abstract:
In dyadic interaction, predicting the listener’s facial reactions is challenging as different reactions could be appropriate in response to the same speaker’s behaviour. Previous approaches predominantly treated this task as an interpolation or fitting problem, emphasizing deterministic outcomes but ignoring the diversity and uncertainty of human facial reactions. Furthermore, these methods often failed to model short-range and long-range dependencies within the interaction context, leading to issues in the synchrony and appropriateness of the generated facial reactions. To address these limitations, this paper reformulates the task as an extrapolation or prediction problem, and proposes an novel framework (called ReactFace) to generate multiple different but appropriate facial reactions from a speaker behaviour rather than merely replicating the corresponding listener facial behaviours. Our ReactFace generates multiple different but appropriate photo-realistic human facial reactions by: (i) learning an appropriate facial reaction distribution representing multiple different but appropriate facial reactions; and (ii) synchronizing the generated facial reactions with the speaker verbal and non-verbal behaviours at each time stamp, resulting in realistic 2D facial reaction sequences. Experimental results demonstrate the effectiveness of our approach in generating multiple diverse, synchronized, and appropriate facial reactions from each speaker’s behaviour. The quality of the generated facial reactions is intimately tied to the speaker’s speech and facial expressions, achieved through our novel speaker-listener interaction modules.

Abstract:
Registering multiple point clouds (MPC) to form a consistent global map is a crucial problem for various 3D mapping applications, which is however highly challenging due to the tightly coupled difficulties of: 1) identifying the registrable point cloud pairs (RPCPs), 2) detecting the point correspondence in each RPCP, 3) estimating the relative transformation for each RPCP, and 4) aligning the RPCPs to generate the global map. This paper proposes an Iterative Pairing and Transformation (IPT) framework to address above difficulties. A Registrability Graph (RG) is in particular proposed, whose vertices represent the nn point clouds and its edges model the registerability between each pair of the point clouds. Registerability is an indicator for whether the relative transformation of the two point clouds can be trustfully estimated. Starting from an NN-clique, the obviously unregistrable pairs will be identified through an adaptive coarse registration stage using RANSAC, leading to a sparser RG. For the remained edges in RG, the relative transformation for each registrable pair is accessed by a learning-based pairwise registration algorithm. These pairwise relative poses among vertices are then input into the global registration step, which optimizes and generates globally consistent pose for each point cloud, as well as a global map. Then, based on the global poses of the point clouds, the edges of the RG will be reevaluated to adjusting the registrable pairs, and this local pairing and global optimization process will repeat until convergence. The experimental results show IPT outperforms current state-of-the-art MPC registration algorithms.

Abstract:
The high performance of tree ensemble classifiers benefits from a large set of rules, which, in turn, makes the models hard to understand. To improve interpretability, existing methods extract a subset of rules for approximation using model reduction techniques. However, by focusing on the reduced rule set, these methods often lose fidelity and ignore anomalous rules that, despite their infrequency, play crucial roles in real-world applications. This paper introduces a scalable visual analysis method to explain tree ensemble classifiers that contain tens of thousands of rules. The key idea is to address the issue of losing fidelity by adaptively organizing the rules as a hierarchy rather than reducing them. To ensure the inclusion of anomalous rules, we develop an anomaly-biased model reduction method to prioritize these rules at each hierarchical level. Synergized with this hierarchical organization of rules, we develop a matrix-based hierarchical visualization to support exploration at different levels of detail. Our quantitative experiments and case studies demonstrate how our method fosters a deeper understanding of both common and anomalous rules, thereby enhancing interpretability without sacrificing comprehensiveness.

Abstract:
In educational Virtual Reality (VR) environments, objects irrelevant to learning can lead to students’ inattention, which adversely affects learning. However, removing these objects from virtual scenes is not feasible, as they are crucial for creating a realistic and immersive experience. Balancing the need to maintain students’ attention while preserving the integrity of scenarios is a challenging task. In this paper, we introduce a gaze-contingent rendering (GCR) technique to address such an issue, which is independent of specific elements or configurations in virtual scenes and adaptable across various contexts. Specifically, we utilize gaze-aware rendering adjustments to adaptively reduce the visibility of objects irrelevant to learning while highlighting relevant ones. We develop three GCR strategies (i.e., blur, pixelation, and underexposure) and investigate how these strategies affect students’ visual attention, academic achievement, and perceptions of the learning activity across different scenarios. Our findings indicate that the proposed rendering strategies effectively achieve the goals of sustaining visual attention and improving academic achievement without significantly impacting immersion or engagement. As an initial exploration of GCR for maintaining attention within educational VR, this study may inspire new directions in future research on GCR and visual attention maintenance in immersive VR.

Abstract:
Visualizing event timelines for collaborative text writing is an important application for navigating and understanding such data, as time passes and the size and complexity of both text and timeline increase. They are often employed by applications such as code repositories and collaborative text editors. In this article, we present a visualization tool to explore historical records of writing of legislative texts, which were discussed and voted on by an assembly of representatives. Our visualization focuses on event timelines from text documents that involve multiple people and different topics, allowing for observation of different proposed versions of said text or tracking data provenance of given text sections, while highlighting the connections between all elements involved. We also describe the process of designing such a tool alongside domain experts, with three steps of evaluation being conducted to verify the effectiveness of our design.

Abstract:
Recently, Large Language Model based Autonomous System (LLMAS) has gained great popularity for its potential to simulate complicated behaviors of human societies. One of its main challenges is to present and analyze the dynamic events evolution of LLMAS. In this work, we present a visualization approach to explore the detailed statuses and agents’ behavior within LLMAS. Our approach outlines a general pipeline that organizes raw execution events from LLMAS into a structured behavior model. We leverage a behavior summarization algorithm to create a hierarchical summary of these behaviors, arranged according to their sequence over time. Additionally, we design a cause trace method to mine the causal relationship between agent behaviors. We then develop AgentLens, a visual analysis system that leverages a hierarchical temporal visualization for illustrating the evolution of LLMAS, and supports users to interactively investigate details and causes of agents’ behaviors. Two usage scenarios and a user study demonstrate the effectiveness and usability of our AgentLens.

Abstract:
We present a novel particle-grid scheme for simulating bubble and foam flow. At the core of our approach lies a particle representation that combines the computational nature of moving least-squares particles and particle level-set methods. Specifically, we assign a dedicated particle system to each individual bubble, enabling accurate tracking of its interface evolution and topological changes in a foaming fluid system. The particles within each bubble's particle system serve dual purposes. First, they function as a surface discretization, allowing for the solution of surfactant flow physics on the bubble's membrane. Additionally, these particles act as interface trackers, facilitating the evolution of the bubble's shape and topology within the multiphase fluid domain. The combination of particle systems from all bubbles contributes to the generation of an unsigned level-set field, further enhancing the simulation of coupled multiphase flow dynamics. By seamlessly integrating our particle representation into a multiphase, volumetric flow solver, our method enables the simulation of a broad range of intricate bubble and foam phenomena. These phenomena exhibit highly dynamic and complex structural evolution, as well as interfacial flow details.

Abstract:
External ventricular drain (EVD) is a common, yet challenging neurosurgical procedure of placing a catheter into the brain ventricular system that requires prolonged training for surgeons to improve the catheter placement accuracy. In this article, we introduce NeuroLens, an Augmented Reality (AR) system that provides neurosurgeons with guidance that aids them in completing an EVD catheter placement. NeuroLens builds on prior work in AR-assisted EVD to present a registered hologram of a patient's ventricles to the surgeons, and uniquely incorporates guidance on the EVD catheter's trajectory, angle of insertion, and distance to the target. The guidance is enabled by tracking the EVD catheter. We evaluate NeuroLens via a study with 33 medical students and 9 neurosurgeons, in which we analyzed participants’ EVD catheter insertion accuracy and completion time, eye gaze patterns, and qualitative responses. Our study, in which NeuroLens was used to aid students and surgeons in inserting an EVD catheter into a realistic phantom model of a human head, demonstrated the potential of NeuroLens as a tool that will aid and educate novice neurosurgeons. On average, the use of NeuroLens improved the EVD placement accuracy of the year 1 students by 39.4%, of the year 2--4 students by 45.7%, and of the neurosurgeons by 16.7%. Furthermore, students who focused more on NeuroLens-provided contextual guidance achieved better results, and novice surgeons improved more than the expert surgeons with NeuroLens's assistance.

Abstract:
To enhance the appeal and informativeness of data news, there is an increasing reliance on data analysis techniques and visualizations, which poses a high demand for journalists’ abilities. While numerous visual analytics systems have been developed for deriving insights, few tools specifically support and disseminate viewpoints for journalism. Thus, this work aims to facilitate the automatic creation of sports news from natural language insights. To achieve this, we conducted an extensive preliminary study on the published sports articles. Based on our findings, we propose a workflow - 1) exploring the data space behind insights, 2) generating narrative structures, 3) progressively generating each episode, and 4) mapping data spaces into communicative visualizations. We have implemented a human-AI interaction system called SNIL, which incorporates user input in conjunction with large language models (LLMs). It supports the modification of textual and graphical content within the episode-based structure by adjusting the description. We conduct user studies to demonstrate the usability of SNIL and the benefit of bridging the gap between analysis tasks and communicative tasks through expert and fan feedback.

Abstract:
In virtual reality applications, in addition to visual feedback, real objects can be used as props for virtual objects to provide passive haptic feedback, which greatly enhances user immersion. Usually, real object props are not one-to-one correspondence with virtual objects. Haptic retargeting technique is proposed to establish the virtual-real correspondence by introducing an offset between the virtual hand and the real hand. Sometimes, the offset is too large to cause user discomfort, and it is necessary to introduce a reset between two haptic retargeting operations to force the virtual hand and the real hand to coincide in order to eliminate the offset. However, too many resets can interfere with this immersion. To address this problem, we propose a haptic retargeting method based on proxy importance calculation using multiple props in virtual reality. The concept of proxy importance for props is introduced first, and then a proxy importance based prop selection and placement method for moving virtual objects are proposed. We also improve the performance of our method by using the props’ weighted proxy importance strategy for multi-user collaboration. Compared to the state-of-the-art methods, our method significantly reduces the number of resets, the task completion time, hand movement distances, and task load without the cost of cybersickness in the single-user task. In the multi-user collaborative task, our method also achieves significant improvement using the strategy that weights the proxy importance of the props.

Abstract:
Treemaps are a powerful tool for representing hierarchical data in a space-efficient manner and are used in various domains, including network security or software development. However, interpreting the topology encoded by nested rectangles can be challenging, particularly compared to tree-structured representations like node-link diagrams or icicle plots. To address this challenge, we introduce TreEducation, a visual education platform designed to improve the visualization literacy skills required for reading treemaps among non-expert users. TreEducation is an online application that combines visualizations, interactions, and gamification elements to facilitate understanding of eight different treemap layout algorithms and enhance students’ learning process. We evaluated TreEducation in a classroom setting and a controlled environment. Our results indicate a significant knowledge gain of students training exclusively with TreEducation and the usefulness of competition as a social gamification element included in our competitive quiz.

Abstract:
Consumer-level omnidirectional video offers an economically viable means to create virtual reality (VR) assets, enabling users to explore and interact within a fully immersive visual environment. However, editing such videos, particularly those with 360^\circ ∘ views and dynamic objects, poses significant challenges. Existing approaches to representing and manipulating omnidirectional content—whether designed for typical 2D perspective imagery or panoramas—often fail to adequately capture the complex spatiotemporal relationships crucial for producing high-quality, editable outputs in dynamic, panoramic settings. To overcome these challenges, we introduce OmniPlane, a novel method that leverages spherical spatiotemporal feature grids to empower the representation and editability of real-world dynamic omnidirectional environments casually captured by commodity omnidirectional cameras. OmniPlane computes spatiotemporal features by fusing vectors or matrices from each learnable spatial and spatiotemporal feature plane within a spherical coordinate system, complemented by a specifically designed weighted sampling strategy respecting the inherent spherical distribution of omnidirectional content. These learned feature planes can be flexibly decomposed into palette-based color bases. This innovative method not only enhances the representation capability of omnidirectional content and dynamics but also enables the recoloring of omnidirectional videos. Extensive experiments and a dedicated user study validate the superior performance of our proposed method in facilitating recolorable representations of dynamic omnidirectional environments.

Abstract:
Topological descriptors such as contour trees are widely utilized in scientific data analysis and visualization, with applications from materials science to climate simulations. It is desirable to preserve topological descriptors when data compression is part of the scientific workflow for these applications. However, classic error-bounded lossy compressors for volumetric data do not guarantee the preservation of topological descriptors, despite imposing strict pointwise error bounds. In this work, we introduce a general framework for augmenting any lossy compressor to preserve the topology of the data during compression. Specifically, our framework quantifies the adjustments (to the decompressed data) needed to preserve the contour tree and then employs a custom variable-precision encoding scheme to store these adjustments. We demonstrate the utility of our framework in augmenting classic compressors (such as SZ3, TTHRESH, and ZFP) and deep learning-based compressors (such as Neurcomp) with topological guarantees.

Abstract:
The growing adoption of social virtual reality (VR) platforms underscores the importance of safeguarding personal VR space to maintain user privacy and security. Teleportation, a prevalent instantaneous locomotion method in VR, facilitates user engagement but can also inadvertently intrude upon personal VR space, thereby raising privacy concerns. This paper introduces three innovative negotiated teleportation techniques designed to secure user-to-user teleportation and protect personal space privacy, all under a unified small-group development framework. We have designed and evaluated three types of negotiated teleportation techniques: Sector technique for directional control, Distance technique for minimum social distance control, and Area technique for defining circular permissible teleportation areas. These techniques foster a collaborative approach to selecting teleportation points that respect personal space. To evaluate the efficacy of these techniques, we conducted a user study with 20 participants who performed social tasks within a virtual campus environment. The findings demonstrate that our techniques significantly enhance privacy protection and alleviate anxiety associated with unwanted proximity in social VR.

Abstract:
We present a novel shape-shifting haptic device, Shiftly, which renders plausible haptic feedback when touching virtual objects in Virtual Reality (VR). By changing its shape, different geometries of virtual objects can be approximated to provide haptic feedback for the user's hand. The device employs only three actuators and three curved origamis that can be programmatically folded and unfolded to create a variety of touch surfaces ranging from flat to curved. In this paper, we present the design of Shiftly, including its kinematic model and integration into VR setups for haptics. We also assessed Shiftly using two user studies. The first study evaluated how well Shiftly can approximate different shapes without visual representation. The second study investigated the realism of the haptic feedback with Shiftly for a user when touching a rendered virtual object. The results showed that our device can provide realistic haptic feedback for flat surfaces, convex shapes of different curvatures, and edge-shaped geometries. Shiftly can less realistically render concave surfaces and objects with small details.

Abstract:
Group selection in virtual reality is an important means of multi-object selection, which allows users to quickly group multiple objects and can significantly improve the operation efficiency of multiple types of objects. In this paper, we propose a group selection method based on multiple rounds of probability permutation, in which the efficiency of group selection is substantially improved by making the object layout of the next round easier to be batch-selected through interactive selection, object grouping probability computation, and position rearrangement in each round of the selection process. We conducted ablation experiments to determine the algorithm coefficients and validate the effectiveness of the algorithm. In addition, an empirical user study was conducted to evaluate the ability of our method to significantly improve the efficiency of the group selection task in an immersive virtual reality environment. The reduced operations also indirectly reduce the user task load and improve usability.

Abstract:
The rising popularity of 360-degree images and virtual reality (VR) has spurred a growing interest among creators in producing visually appealing content through effective color grading processes. Although existing computational approaches have simplified the global color adjustment for entire images with Preferential Bayesian Optimization (PBO), they neglect local colors for points of interest and are not optimized for the immersive nature of VR. In response, we propose a dual-level PBO framework that integrates global and local color adjustments tailored for VR environments. We design and evaluate a novel context-aware preferential Gaussian Process (GP) to learn contextual preferences for local colors, taking into account the dynamic contexts of previously established global colors. Additionally, recognizing the limitations of desktop-based interfaces for comparing 360-degree images, we design three VR interfaces for color comparison. We conduct a controlled user study to investigate the effectiveness of the three VR interface designs and find that users prefer to be enveloped by one 360-degree image at a time and to compare two rather than four color-graded options.

Abstract:
Virtual reality (VR) presents immersive opportunities across many applications, yet the inherent risk of developing cybersickness during interaction can severely reduce enjoyment and platform adoption. Cybersickness is marked by symptoms such as dizziness and nausea, which previous work primarily assessed via subjective post-immersion questionnaires and motion-restricted controlled setups. In this paper, we investigate the dynamic nature of cybersickness while users experience and freely interact in VR. We propose a novel method to continuously identify and quantitatively gauge cybersickness levels from users' passively monitored electroencephalography (EEG) and head motion signals. Our method estimates multitaper spectrums from EEG, integrating specialized EEG processing techniques to counter motion artifacts, and, thus, tracks cybersickness levels in real-time. Unlike previous approaches, our method requires no user-specific calibration or personalization for detecting cybersickness. Our work addresses the considerable challenge of reproducibility and subjectivity in cybersickness research. In addition to our method's implementation, we release our dataset of 16 participants and approximately 2 hours of total recordings to spur future work in this domain. Source code: https://github.com/eth-siplab/EEG_Cybersickness_Estimation_VR-Beyond_Subjectivity.

Abstract:
Navigating multiscale virtual environments necessitates an interaction method to travel across different levels of scale (LoS). Prior research has studied various techniques that enable users to seamlessly adjust their scale to navigate between different LoS based on specific user contexts. We introduce a scroll-based scale control method optimized for exocentric navigation, targeted at scenarios where speed and accuracy in continuous scaling are crucial. We pinpoint the challenges of scale control in settings with multiple LoS and evaluate how distinct designs of scaling techniques influence navigation performance and usability. Through a user study, we investigated two pivotal elements of a scaling technique: the input method and the scaling center. Our findings indicate that our scroll-based input method significantly reduces task completion time and error rate and enhances efficiency compared to the most frequently used bi-manual method. Moreover, we found that the choice of scaling center affects the ease of use of the scaling method, especially when paired with specific input methods.

Abstract:
This paper investigates if and how self-similarity and having motor control impact sense of embodiment, self-identification, and body weight perception in Augmented Reality (AR). We conducted a 2x2 mixed design experiment involving 60 participants who interacted with either synchronously moving virtual humans or independently moving ones, each with self-similar or generic appearances, across two consecutive AR sessions. Participants evaluated their sense of embodiment, self-identification, and body weight perception of the virtual human. Our results show that self-similarity significantly enhanced sense of embodiment, self-identification, and the accuracy of body weight estimates with the virtual human. However, the effects of having motor control over the virtual human movements were notably weaker in these measures than in similar VR studies. Further analysis indicated that not only the virtual human itself but also the participants' body weight, self-esteem, and body shape concerns predict body weight estimates across all conditions. Our work advances the understanding of virtual human body weight perception in AR systems, emphasizing the importance of factors such as coherence with the real-world environment.

Abstract:
This study investigates how both the body part used to control a VR avatar and the avatar's appearance affect redirection detection thresholds. We conducted experiments comparing hand and foot manipulation of two types of avatars: a hand-shaped avatar and an abstract spherical avatar. Our results show that, irrespective of the body part used, the redirection detection threshold increased by 21% when using the hand avatar compared to the abstract avatar. Additionally, when the avatar's position was redirected toward the body midline, the detection threshold increased by 49% compared to redirection away from the midline. No significant differences in detection thresholds were observed between the hand and foot manipulations. These findings suggest that avatar appearance and redirection direction significantly influence user perception in VR environments, offering valuable insights for the design of full-body VR interactions and human augmentation systems.

Abstract:
One of the pleasures of interacting using avatars in VR is being able to play a character very different to yourself. As the scale of characters change relative to a user, there is a need to retarget user motions onto the character, generally maintaining either the user's pose or the position of their wrists and ankles. This retargeting can impact both the functional and social information conveyed by the avatar. Focused on 3rd-person (observed) avatars, this paper presents three studies on these varied aspects of communication. It establishes a baseline for near-field avatar pointing, showing an accuracy of about 5cm. This can be maintained using positional hand constraints, but increases if the user's pose is directly transferred to the character. It is possible to maintain this accuracy with a Semantic Inverse Kinematics formulation that brings the avatar closer to the user's actual pose, but compensates by adjusting the finger pointing direction. Similar results are shown for conveying spatial information, namely object size. The choice of pose or position based retargeting leads to a small change in the perception of avatar personality, indicating an impact on social communication. This effect was not observed in a task where the users' cognitive load was otherwise high, so may be task dependent. It could also become more pronounced for more extreme proportion changes.

Abstract:
In Augmented Reality (AR), virtual content enhances user experience by providing additional information. However, improperly positioned or designed virtual content can be detrimental to task performance, as it can impair users' ability to accurately interpret real-world information. In this paper we examine two types of task-detrimental virtual content: obstruction attacks, in which virtual content prevents users from seeing real-world objects, and information manipulation attacks, in which virtual content interferes with users' ability to accurately interpret real-world information. We provide a mathematical framework to characterize these attacks and create a custom open-source dataset for attack evaluation. To address these attacks, we introduce ViDDAR (Vision language model-based Task-Detrimental content Detector for Augmented Reality), a comprehensive full-reference system that leverages Vision Language Models (VLMs) and advanced deep learning techniques to monitor and evaluate virtual content in AR environments, employing a user-edge-cloud architecture to balance performance with low latency. To the best of our knowledge, ViDDAR is the first system to employ VLMs for detecting task-detrimental content in AR settings. Our evaluation results demonstrate that ViDDAR effectively understands complex scenes and detects task-detrimental content, achieving up to 92.15% obstruction detection accuracy with a detection latency of 533 ms, and an 82.46% information manipulation content detection accuracy with a latency of 9.62 s.

Abstract:
A safety boundary wall in VR is a virtual barrier that defines a safe area, allowing users to navigate and interact without safety concerns. However, existing implementations neglect to utilize the safety boundary wall's large surface for displaying interactive information. In this work, we propose the BoundaryScreen technique based on the “walking outward” metaphor to add interactivity to the safety boundary wall. Specifically, we augment the safety boundary wall by placing the home screen on it. To summon the home screen, the user only needs to walk outward until it appears. Results showed that (i) participants significantly preferred BoundaryScreen in the outermost two-step-wide ring-shaped section of a circular safety area; and (ii) participants exhibited strong “behavioral inertia” for walking, i.e., after completing a routine activity involving constant walking, participants significantly preferred to use the walking-based BoundaryScreen technique to summon the home screen.

Abstract:
Advanced manufacturing creates increasingly complex objects with material compositions that are often difficult to characterize by a single modality. Our collaborating domain scientists are going beyond traditional methods by employing both X-ray and neutron computed tomography to obtain complementary representations expected to better resolve material boundaries. However, the use of two modalities creates its own challenges for visualization, requiring either complex adjustments of bimodal transfer functions or the need for multiple views. Together with experts in nondestructive evaluation, we designed a novel interactive bimodal visualization approach to create a combined view of the co-registered X-ray and neutron acquisitions of industrial objects. Using an automatic topological segmentation of the bivariate histogram of X-ray and neutron values as a starting point, the system provides a simple yet effective interface to easily create, explore, and adjust a bimodal visualization. We propose a widget with simple brushing interactions that enables the user to quickly correct the segmented histogram results. Our semiautomated system enables domain experts to intuitively explore large bimodal datasets without the need for either advanced segmentation algorithms or knowledge of visualization techniques. We demonstrate our approach using synthetic examples, industrial phantom objects created to stress bimodal scanning techniques, and real-world objects, and we discuss expert feedback.

Abstract:
A large number of 3D spectral descriptors have been proposed in the literature, which act as an essential component for 3D deformable shape matching and related applications. An outstanding descriptor should have desirable natures including high-level descriptive capacity, cheap storage, and robustness to a set of nuisances. It is, however, unclear which descriptors are more suitable for a particular application. This paper fills the gap by comprehensively evaluating nine state-of-the-art spectral descriptors on ten popular deformable shape datasets as well as perturbations such as mesh discretization, geometric noise, scale transformation, non-isometric setting, partiality, and topological noise. Our evaluated terms for a spectral descriptor cover four major concerns, i.e., distinctiveness, robustness, compactness, and computational efficiency. In the end, we present a summary of the overall performance and several interesting findings that can serve as guidance for the following researchers to construct a new spectral descriptor and choose an appropriate spectral feature in a particular application.

Abstract:
The use of natural language interfaces (NLIs) to create charts is becoming increasingly popular due to the intuitiveness of natural language interactions. One key challenge in this approach is to accurately capture user intents and transform them to proper chart specifications. This obstructs the wide use of NLI in chart generation, as users’ natural language inputs are generally abstract (i.e., ambiguous or under-specified), without a clear specification of visual encodings. Recently, pre-trained large language models (LLMs) have exhibited superior performance in understanding and generating natural language, demonstrating great potential for downstream tasks. Inspired by this major trend, we propose ChartGPT, generating charts from abstract natural language inputs. However, LLMs are struggling to address complex logic problems. To enable the model to accurately specify the complex parameters and perform operations in chart generation, we decompose the generation process into a step-by-step reasoning pipeline, so that the model only needs to reason a single and specific sub-task during each run. Moreover, LLMs are pre-trained on general datasets, which might be biased for the task of chart generation. To provide adequate visualization knowledge, we create a dataset consisting of abstract utterances and charts and improve model performance through fine-tuning. We further design an interactive interface for ChartGPT that allows users to check and modify the intermediate outputs of each step. The effectiveness of the proposed system is evaluated through quantitative evaluations and a user study.

Abstract:
Hex-dominant mesh generation has received significant attention in recent research due to its superior robustness compared to pure hex-mesh generation techniques. In this work, we introduce the first structure for analyzing hex-dominant meshes. This structure builds on the base complex of pure hex-meshes but incorporates the non-hex elements for a more comprehensive and complete representation. We provide its definition and describe its construction steps. Based on this structure, we present an extraction and categorization of sheets using advanced graph matching techniques to handle the non-hex elements. This enables us to develop an enhanced visual analysis of the structure for any hex-dominant meshes. We apply this structure-based visual analysis to compare hex-dominant meshes generated by different methods to study their advantages and disadvantages. This complements the standard quality metric based on the non-hex element percentage for hex-dominant meshes. Moreover, we propose a strategy to extract a cleaned (optimized) valence-based singularity graph wireframe to analyze the structure for both mesh and sheets. Our results demonstrate that the proposed hybrid base complex provides a coarse representation for mesh element, and the proposed valence singularity graph wireframe provides a better internal visualization of hex-dominant meshes.

Abstract:
In-betweening is a technique for generating transitions given start and target character states. The majority of existing works require multiple (often \geq≥ 10) frames as input, which are not always available. In addition, they produce results that lack diversity, which may not fulfill artists’ requirements. Addressing these gaps, our work deals with a focused yet challenging problem: generating diverse and high-quality transitions given exactly two frames (only the start and target frames). To cope with this challenging scenario, we propose a bi-directional motion generation and stitching scheme which generates forward and backward transitions from the start and target frames with two adversarial autoregressive networks, respectively, and stitches them midway between the start and target frames. In contrast to stitching at the start or target frames, where the ground truth cannot be altered, there is no strict midway ground truth. Thus, our method can capitalize on this flexibility and generate high-quality and diverse transitions simultaneously. Specifically, we employ conditional variational autoencoders (CVAEs) to implement our autoregressive networks and propose a novel stitching loss to stitch the bi-directional generated motions around the midway point. Extensive experiments demonstrate that our method achieves higher motion quality and more diverse results than existing methods on the LaFAN1, Human3.6m and AMASS datasets.

Abstract:
In this article, we present a self-prior-based mesh inpainting framework that requires only an incomplete mesh as input, without the need for any training datasets. Additionally, our method maintains the polygonal mesh format throughout the inpainting process without converting the shape format to an intermediate one, such as a voxel grid, a point cloud, or an implicit function, which are typically considered easier for deep neural networks to process. To achieve this goal, we introduce two graph convolutional networks (GCNs): single-resolution GCN (SGCN) and multi-resolution GCN (MGCN), both trained in a self-supervised manner. Our approach refines a watertight mesh obtained from the initial hole filling to generate a complete output mesh. Specifically, we train the GCNs to deform an oversmoothed version of the input mesh into the expected complete shape. The deformation is described by vertex displacements, and the GCNs are supervised to obtain accurate displacements at vertices in real holes. To this end, we specify several connected regions of the mesh as fake holes, thereby generating meshes with various sets of fake holes. The correct displacements of vertices are known in these fake holes, thus enabling training GCNs with loss functions that assess the accuracy of vertex displacements. We demonstrate that our method outperforms traditional dataset-independent approaches and exhibits greater robustness compared with other deep-learning-based methods for shapes that infrequently appear in shape datasets.

Abstract:
Effective security patrol management is critical for ensuring safety in diverse environments such as art galleries, airports, and factories. The behavior of patrols in these situations can be modeled by patrolling games. They simulate the behavior of the patrol and adversary in the building, which is modeled as a graph of interconnected nodes representing rooms. The designers of algorithms solving the game face the problem of analyzing complex graph layouts with temporal dependencies. Therefore, appropriate visual support is crucial for them to work effectively. In this paper, we present a novel tool that helps the designers of patrolling games explore the outcomes of the proposed algorithms and approaches, evaluate their success rate, and propose modifications that can improve their solutions. Our tool offers an intuitive and interactive interface, featuring a detailed exploration of patrol routes and probabilities of taking them, simulation of patrols, and other requested features. In close collaboration with experts in designing patrolling games, we conducted three case studies demonstrating the usage and usefulness of our tool. The prototype of the tool, along with exemplary datasets, is available at https://gitlab.fi.muni.cz/formela/strategy-vizualizer.

Abstract:
Scalar field comparison is a fundamental task in scientific visualization. In topological data analysis, we compare topological descriptors of scalar fields—such as persistence diagrams and merge trees—because they provide succinct and robust abstract representations. Several similarity measures for topological descriptors seem to be both asymptotically and practically efficient with polynomial time algorithms, but they do not scale well when handling large-scale, time-varying scientific data and ensembles. In this paper, we propose a new framework to facilitate the comparative analysis of merge trees, inspired by tools from locality sensitive hashing (LSH). LSH hashes similar objects into the same hash buckets with high probability. We propose two new similarity measures for merge trees that can be computed via LSH, using new extensions to Recursive MinHash and subpath signature, respectively. Our similarity measures are extremely efficient to compute and closely resemble the results of existing measures such as merge tree edit distance or geometric interleaving distance. Our experiments demonstrate the utility of our LSH framework in applications such as shape matching, clustering, key event detection, and ensemble summarization.

Abstract:
Building Information Modeling (BIM) describes a central data pool covering the entire life cycle of a construction project. Similarly, Building Energy Modeling (BEM) describes the process of using a 3D representation of a building as a basis for thermal simulations to assess the building's energy performance. This paper explores the intersection of BIM and BEM, focusing on the challenges and methodologies in converting BIM data into BEM representations for energy performance analysis. BEMTrace integrates 3D data wrangling techniques with visualization methodologies to enhance the accuracy and traceability of the BIM-to-BEM conversion process. Through parsing, error detection, and algorithmic correction of BIM data, our methods generate valid BEM models suitable for energy simulation. Visualization techniques provide transparent insights into the conversion process, aiding error identification, validation, and user comprehension. We introduce context-adaptive selections to facilitate user interaction and to show that the BEMTrace workflow helps users understand complex 3D data wrangling processes.

Abstract:
Projecting high-dimensional vectors into two dimensions for visualization, known as embedding visualization, facilitates perceptual reasoning and interpretation. Comparing multiple embedding visualizations drives decision-making in many domains, but traditional comparison methods are limited by a reliance on direct point correspondences. This requirement precludes comparisons without point correspondences, such as two different datasets of annotated images, and fails to capture meaningful higher-level relationships among point groups. To address these shortcomings, we propose a general framework for comparing embedding visualizations based on shared class labels rather than individual points. Our approach partitions points into regions corresponding to three key class concepts-confusion, neighborhood, and relative size-to characterize intra- and inter-class relationships. Informed by a preliminary user study, we implemented our framework using perceptual neighborhood graphs to define these regions and introduced metrics to quantify each concept. We demonstrate the generality of our framework with usage scenarios from machine learning and single-cell biology, highlighting our metrics' ability to draw insightful comparisons across label hierarchies. To assess the effectiveness of our approach, we conducted an evaluation study with five machine learning researchers and six single-cell biologists using an interactive and scalable prototype built with Python, JavaScript, and Rust. Our metrics enable more structured comparisons through visual guidance and increased participants' confidence in their findings.

Abstract:
Citations allow quickly identifying related research. If multiple publications are selected as seeds, specific suggestions for related literature can be made based on the number of incoming and outgoing citation links to this selection. Interactively adding recommended publications to the selection refines the next suggestion and incrementally builds a relevant collection of publications. Following this approach, the paper presents a search and foraging approach, PUREsuggest, which combines citation-based suggestions with augmented visualizations of the citation network. The focus and novelty of the approach is, first, the transparency of how the rankings are explained visually and, second, that the process can be steered through user-defined keywords, which reflect topics of interests. The system can be used to build new literature collections, to update and assess existing ones, as well as to use the collected literature for identifying relevant experts in the field. We evaluated the recommendation approach through simulated sessions and performed a user study investigating search strategies and usage patterns supported by the interface.

Abstract:
Classical bibliography, by researching preserved catalogs from both official archives and personal collections of accumulated books, examines the books throughout history, thereby revealing cultural development across historical periods. In this work, we collaborate with domain experts to accomplish the task of data annotation concerning Chinese ancient catalogs. We introduce the CataAnno system that facilitates users in completing annotations more efficiently through cross-linked views, recommendation methods and convenient annotation interactions. The recommendation method can learn the background knowledge and annotation patterns that experts subconsciously integrate into the data during prior annotation processes. CataAnno searches for the most relevant examples previously annotated and recommends to the user. Meanwhile, the cross-linked views assist users in comprehending the correlations between entries and offer explanations for these recommendations. Evaluation and expert feedback confirm that the CataAnno system, by offering high-quality recommendations and visualizing the relationships between entries, can mitigate the necessity for specialized knowledge during the annotation process. This results in enhanced accuracy and consistency in annotations, thereby enhancing the overall efficiency.

Abstract:
Participatory budgeting (PB) is a democratic approach to allocating municipal spending that has been adopted in many places in recent years, including in Chicago. Current PB voting resembles a ballot where residents are asked which municipal projects, such as school improvements and road repairs, to fund with a limited budget. In this work, we ask how interactive visualization can benefit PB by conducting a design probe-based interview study (N=13) with policy workers and academics with expertise in PB, urban planning, and civic HCI. Our probe explores how graphical elicitation of voter preferences and a dashboard of voting statistics can be incorporated into a realistic PB tool. Through qualitative analysis, we find that visualization creates opportunities for city government to set expectations about budget constraints while also granting their constituents greater freedom to articulate a wider range of preferences. However, using visualization to provide transparency about PB requires efforts to mitigate potential access barriers and mistrust. We call for more visualization professionals to help build civic capacity by working in and studying political systems.

Abstract:
The increasing reliance on Large Language Models (LLMs) for health information seeking can pose severe risks due to the potential for misinformation and the complexity of these topics. This paper introduces KnowNet a visualization system that integrates LLMs with Knowledge Graphs (KG) to provide enhanced accuracy and structured exploration. Specifically, for enhanced accuracy, KnowNet extracts triples (e.g., entities and their relations) from LLM outputs and maps them into the validated information and supported evidence in external KGs. For structured exploration, KnowNet provides next-step recommendations based on the neighborhood of the currently explored entities in KGs, aiming to guide a comprehensive understanding without overlooking critical aspects. To enable reasoning with both the structured data in KGs and the unstructured outputs from LLMs, KnowNet conceptualizes the understanding of a subject as the gradual construction of graph visualization. A progressive graph visualization is introduced to monitor past inquiries, and bridge the current query with the exploration history and next-step recommendations. We demonstrate the effectiveness of our system via use cases and expert interviews.

Abstract:
Composite visualization represents a widely embraced design that combines multiple visual representations to create an integrated view. However, the traditional approach of creating composite visualizations in immersive environments typically occurs asynchronously outside of the immersive space and is carried out by experienced experts. In this work, we aim to empower users to participate in the creation of composite visualization within immersive environments through embodied interactions. This could provide a flexible and fluid experience with immersive visualization and has the potential to facilitate understanding of the relationship between visualization views. We begin with developing a design space of embodied interactions to create various types of composite visualizations with the consideration of data relationships. Drawing inspiration from people's natural experience of manipulating physical objects, we design interactions based on the combination of 3D manipulations in immersive environments. Building upon the design space, we present a series of case studies showcasing the interaction to create different kinds of composite visualizations in virtual reality. Subsequently, we conduct a user study to evaluate the usability of the derived interaction techniques and user experience of creating composite visualizations through embodied interactions. We find that empowering users to participate in composite visualizations through embodied interactions enables them to flexibly leverage different visualization views for understanding and communicating the relationships between different views, which underscores the potential of several future application scenarios.

Abstract:
When using exploratory visual analysis to examine multivariate hierarchical data, users often need to query data to narrow down the scope of analysis. However, formulating effective query expressions remains a challenge for multivariate hierarchical data, particularly when datasets become very large. To address this issue, we develop a declarative grammar, HiRegEx (Hierarchical data Regular Expression), for querying and exploring multivariate hierarchical data. Rooted in the extended multi-level task topology framework for tree visualizations (e-MLTT), HiRegEx delineates three query targets (node, path, and subtree) and two aspects for querying these targets (features and positions), and uses operators developed based on classical regular expressions for query construction. Based on the HiRegEx grammar, we develop an exploratory framework for querying and exploring multivariate hierarchical data and integrate it into the TreeQueryER prototype system. The exploratory framework includes three major components: top-down pattern specification, bottom-up data-driven inquiry, and context-creation data overview. We validate the expressiveness of HiRegEx with the tasks from the e-MLTT framework and showcase the utility and effectiveness of TreeQueryER system through a case study involving expert users in the analysis of a citation tree dataset.

Abstract:
Various data visualization applications such as reverse engineering and interactive authoring require a vocabulary that describes the structure of visualization scenes and the procedure to manipulate them. A few scene abstractions have been proposed, but they are restricted to specific applications for a limited set of visualization types. A unified and expressive model of data visualization scenes for different applications has been missing. To fill this gap, we present Manipulable Semantic Components (MSC), a computational representation of data visualization scenes, to support applications in scene understanding and augmentation. MSC consists of two parts: a unified object model describing the structure of a visualization scene in terms of semantic components, and a set of operations to generate and modify the scene components. We demonstrate the benefits of MSC in three applications: visualization authoring, visualization deconstruction and reuse, and animation specification.

Abstract:
The importance of data charts is self-evident, given their ability to express complex data in a simple format that facilitates quick and easy comparisons, analysis, and consumption. However, the inherent visual nature of the charts creates barriers for people with visual impairments to reap the associated benefits to the same extent as their sighted peers. While extant research has predominantly focused on understanding and addressing these barriers for blind screen reader users, the needs of low-vision screen magnifier users have been largely overlooked. In an interview study, almost all low-vision participants stated that it was challenging to interact with data charts on small screen devices such as smartphones and tablets, even though they could technically “see” the chart content. They ascribed these challenges mainly to the magnification-induced loss of visual context that connected data points with each other and also with chart annotations, e.g., axis values. In this paper, we present a method that addresses this problem by automatically transforming charts that are typically non-interactive images into personalizable interactive charts which allow selective viewing of desired data points and preserve visual context as much as possible under screen enlargement. We evaluated our method in a usability study with 26 low-vision participants, who all performed a set of representative chart-related tasks under different study conditions. In the study, we observed that our method significantly improved the usability of charts over both the status quo screen magnifier and a state-of-the-art space compaction-based solution.

Abstract:
Haptic feedback provides an essential sensory stimulus crucial for interaction and analyzing three-dimensional spatio-temporal phenomena on surface visualizations. Given its ability to provide enhanced spatial perception and scene maneuverability, virtual reality (VR) catalyzes haptic interactions on surface visualizations. Various interaction modes, encompassing both mid-air and on-surface interactions—with or without the application of assisting force stimuli—have been explored using haptic force feedback devices. In this paper, we evaluate the use of on-surface and assisted on-surface haptic modes of interaction compared to a no-haptic interaction mode. A force-based haptic stylus is used for all three modalities; the on-surface mode uses collision based forces, whereas the assisted on-surface mode is accompanied by an additional snapping force. We conducted a within-subjects user study involving fundamental interaction tasks performed on surface visualizations. Keeping a consistent visual design across all three modes, our study incorporates tasks that require the localization of the highest, lowest, and random points on surfaces; and tasks that focus on brushing curves on surfaces with varying complexity and occlusion levels. Our findings show that participants took almost the same time to brush curves using all the interaction modes. They could draw smoother curves using the on-surface interaction modes compared to the no-haptic mode. However, the assisted on-surface mode provided better accuracy than the on-surface mode. The on-surface mode was slower in point localization, but the accuracy depended on the visual cues and occlusions associated with the tasks. Finally, we discuss participant feedback on using haptic force feedback as a tangible input modality and share takeaways to aid the design of haptics-based tangible interactions for surface visualizations.

Abstract:
Probability density function (PDF) curves are among the few charts on a Cartesian coordinate system that are commonly presented without y-axes. This design decision may be due to the lack of relevance of vertical scaling in normal PDFs. In fact, as long as two normal PDFs have the same means and standard deviations (SDs), they can be scaled to occupy different amounts of vertical space while still remaining statistically identical. Because unfixed PDF height increases as SD decreases, visualization designers may find themselves tempted to vertically shrink low-SD PDFs to avoid occlusion or save white space in their figures. Although irregular vertical scaling has been explored in bar and line charts, the visualization community has yet to investigate how this visual manipulation may affect reader comparisons of PDFs. In this paper, we present two preregistered experiments (n = 600, n = 401) that systematically demonstrate that vertical scaling can lead to misinterpretations of PDFs. We also test visual interventions to mitigate misinterpretation. In some contexts, we find including a y-axis can help reduce this effect. Overall, we find that keeping vertical scaling consistent, and therefore maintaining equal pixel areas under PDF curves, results in the highest likelihood of accurate comparisons. Our findings provide insights into the impact of vertical scaling on PDFs, and reveal the complicated nature of proportional area comparisons.

Abstract:
Data tables are one of the most common ways in which people encounter data. Although mostly built with text and numbers, data tables have a spatial layout and often exhibit visual elements meant to facilitate their reading. Surprisingly, there is an empirical knowledge gap on how people read tables and how different visual aids affect people's reading of tables. In this work, we seek to address this vacuum through a controlled study. We asked participants to repeatedly perform four different tasks with four table representation conditions (plain tables, tables with zebra striping, tables with cell background color encoding cell value, and tables with in-cell bars with lengths encoding cell value). We analyzed completion time, error rate, gaze-tracking data, mouse movement and participant preferences. We found that color and bar encodings help for finding maximum values. For a more complex task (comparison of proportional differences) color and bar helped less than zebra striping. We also characterize typical human behavior for the four tasks. These findings inform the design of tables and research directions for improving presentation of data in tabular form.

Abstract:
We propose the notion of attention-aware visualizations (AAVs) that track the user's perception of a visual representation over time and feed this information back to the visualization. Such context awareness is particularly useful for ubiquitous and immersive analytics where knowing which embedded visualizations the user is looking at can be used to make visualizations react appropriately to the user's attention: for example, by highlighting data the user has not yet seen. We can separate the approach into three components: (1) measuring the user's gaze on a visualization and its parts; (2) tracking the user's attention over time; and (3) reactively modifying the visual representation based on the current attention metric. In this paper, we present two separate implementations of AAV: a 2D data-agnostic method for web-based visualizations that can use an embodied eyetracker to capture the user's gaze, and a 3D data-aware one that uses the stencil buffer to track the visibility of each individual mark in a visualization. Both methods provide similar mechanisms for accumulating attention over time and changing the appearance of marks in response. We also present results from a qualitative evaluation studying visual feedback and triggering mechanisms for capturing and revisualizing attention.

Abstract:
Graphs are often used to model relationships between entities. The identification and visualization of clusters in graphs enable insight discovery in many application areas, such as life sciences and social sciences. Force-directed graph layouts promote the visual saliency of clusters, as they bring adjacent nodes closer together, and push non-adjacent nodes apart. At the same time, matrices can effectively show clusters when a suitable row/column ordering is applied, but are less appealing to untrained users not providing an intuitive node-link metaphor. It is thus worth exploring layouts combining the strengths of the node-link metaphor and node ordering. In this work, we study the impact of node ordering on the visual saliency of clusters in orderable node-link diagrams, namely radial diagrams, arc diagrams and symmetric arc diagrams. Through a crowdsourced controlled experiment, we show that users can count clusters consistently more accurately, and to a large extent faster, with orderable node-link diagrams than with three state-of-the art force-directed layout algorithms, i.e., ‘Linlog’, ‘Backbone’ and ‘sfdp’. The measured advantage is greater in case of low cluster separability and/or low compactness. A free copy of this paper and all supplemental materials are available at https://osf.io/kc3dg/.

Abstract:
In this study, we address the growing issue of misleading charts, a prevalent problem that undermines the integrity of information dissemination. Misleading charts can distort the viewer's perception of data, leading to misinterpretations and decisions based on false information. The development of effective automatic detection methods for misleading charts is an urgent field of research. The recent advancement of multimodal Large Language Models (LLMs) has introduced a promising direction for addressing this challenge. We explored the capabilities of these models in analyzing complex charts and assessing the impact of different prompting strategies on the models' analyses. We utilized a dataset of misleading charts collected from the internet by prior research and crafted nine distinct prompts, ranging from simple to complex, to test the ability of four different multimodal LLMs in detecting over 21 different chart issues. Through three experiments–from initial exploration to detailed analysis–we progressively gained insights into how to effectively prompt LLMs to identify misleading charts and developed strategies to address the scalability challenges encountered as we expanded our detection range from the initial five issues to 21 issues in the final experiment. Our findings reveal that multimodal LLMs possess a strong capability for chart comprehension and critical thinking in data interpretation. There is significant potential in employing multimodal LLMs to counter misleading information by supporting critical thinking and enhancing visualization literacy. This study demonstrates the applicability of LLMs in addressing the pressing concern of misleading charts.

Abstract:
Merge trees are a valuable tool in the scientific visualization of scalar fields; however, current methods for merge tree comparisons are computationally expensive, primarily due to the exhaustive matching between tree nodes. To address this challenge, we introduce the Merge Tree Neural Network (MTNN), a learned neural network model designed for merge tree comparison. The MTNN enables rapid and high-quality similarity computation. We first demonstrate how to train graph neural networks, which emerged as effective encoders for graphs, in order to produce embeddings of merge trees in vector spaces for efficient similarity comparison. Next, we formulate the novel MTNN model that further improves the similarity comparisons by integrating the tree and node embeddings with a new topological attention mechanism. We demonstrate the effectiveness of our model on real-world data in different domains and examine our model's generalizability across various datasets. Our experimental analysis demonstrates our approach's superiority in accuracy and efficiency. In particular, we speed up the prior state-of-the-art by more than 100× on the benchmark datasets while maintaining an error rate below 0.1%.

Abstract:
Data visualization aids in making data analysis more intuitive and in-depth, with widespread applications in fields such as biology, finance, and medicine. For massive and continuously growing streaming time series data, these data are typically visualized in the form of line charts, but the data transmission puts significant pressure on the network, leading to visualization lag or even failure to render completely. This paper proposes a universal sampling algorithm FPCS, which retains feature points from continuously received streaming time series data, compensates for the frequent fluctuating feature points, and aims to achieve efficient visualization. This algorithm bridges the gap in sampling for streaming time series data. The algorithm has several advantages: (1) It optimizes the sampling results by compensating for fewer feature points, retaining the visualization features of the original data very well, ensuring high-quality sampled data; (2) The execution time is the shortest compared to similar existing algorithms; (3) It has an almost negligible space overhead; (4) The data sampling process does not depend on the overall data; (5) This algorithm can be applied to infinite streaming data and finite static data.

Abstract:
Mixed Reality (MR) is gaining prominence in manual task skill learning due to its in-situ, embodied, and immersive experience. To teach manual tasks, current methodologies break the task into hierarchies (tasks into subtasks) and visualize not only the current subtasks but also the future ones that are causally related. We investigate the impact of visualizing causality within an MR framework on manual task skill learning. We conducted a user study with 48 participants, experimenting with how presenting tasks in hierarchical causality levels (no causality, event-level, interaction-level, and gesture-level causality) affects user comprehension and performance in a complex assembly task. The research finds that displaying all causality levels enhances user understanding and task execution, with a compromise of learning time. Based on the results, we further provide design recommendations and in-depth discussions for future manual task learning systems.

Abstract:
At the beginning of the COVID-19 pandemic, HealthLink BC (HLBC) rapidly integrated physicians into the triage process of their virtual healthcare service to improve patient outcomes and satisfaction with this service and preserve health care system capacity. We present the design and implementation of a visual analytics tool, VIVA (Virtual healthcare Interactions using Visual Analytics), to support HLBC in analysing various forms of usage data from the service. We abstract HLBC’s data and data analysis tasks, which we use to inform our design of VIVA. We also present the interactive workflow abstraction of Scan, Act, Adapt. We validate VIVA’s design through three case studies with stakeholder domain experts. We also propose the Controllability Through Configuration model to conduct and analyze design studies, and discuss architectural evolution of VIVA through that lens. It articulates configuration, both that specified by a developer or technical power user and that constructed automatically through log data from previous interactive sessions, as a bridge between the rigidity of hardwired programming and the time-consuming implementation of full end-user interactivity.

Abstract:
Causal networks are widely used in many fields to model the complex relationships between variables. A recent approach has sought to construct causal networks by leveraging the wisdom of crowds through the collective participation of humans. While this can yield detailed causal networks that model the underlying phenomena quite well, it requires a large number of individuals with domain understanding. We adopt a different approach: leveraging the causal knowledge that large language models, such as OpenAI’s GPT-4, have learned by ingesting massive amounts of literature. Within a dedicated visual analytics interface, called CausalChat, users explore single variables or variable pairs recursively to identify causal relations, latent variables, confounders, and mediators, constructing detailed causal networks through conversation. Each probing interaction is translated into a tailored GPT-4 prompt and the response is conveyed through visual representations which are linked to the generated text for explanations. We demonstrate the functionality of CausalChat across diverse data contexts and conduct user studies involving both domain experts and laypersons.

Abstract:
Holography is a promising approach to recreate lifelike 3D scenes. However, due to the current Spatial Light Modulators (SLMs) lacking sufficient pixels, the defocused planes of holograms always exhibit obvious interference phenomena. The methods based on random phase can alleviate this problem, but they always affect the imaging quality of the focal plane. Meanwhile, direct current (DC) noise of nondiffracted light in SLMs, coupled with ubiquitous dynamic noise, has long been a fundamental issue affecting holographic display quality. In this study, we proposed a method based on static high-pass filtering and time-multiplexing that overcomes the traditional tradeoff between divergence capability of holograms and display quality in focal planes. Simultaneously, the proposed method can eliminate DC and dynamic noise with a simple and robust structure. Moreover, we further extended artificial intelligence-driven algorithms to achieve higher-quality on-axis amplitude-only holograms. The Simulations and experiments demonstrated that the developed method is a promising and easily generalized solution for time-multiplexing holography.

Abstract:
In educators' ongoing pursuit of enriching remote education, Virtual Reality (VR)-based telepresence has shown significant promise due to its immersive and interactive nature. Existing approaches often rely on point cloud or NeRF-based techniques to deliver realistic representations of teachers and classrooms to remote students. However, achieving low latency is non-trivial, and maintaining high-fidelity rendering under such constraints poses an even greater challenge. This paper introduces LiteAT, a data-lightweight and user-adaptive VR telepresence system, to enable real-time, immersive learning experiences. LiteAT employs a Gaussian Splatting-based reconstruction pipeline that integrates an SMPL-X-driven dynamic human model with a static classroom, supporting lightweight data transmission and high-quality rendering. To enable efficient and personalized exploration in the virtual classroom, we propose a user-adaptive viewpoint recommendation framework that dynamically suggests high-quality viewpoints tailored to user preferences. Candidate viewpoints are evaluated based on multiple visual quality factors and are continuously optimized based on recent user behavior and scene dynamics. Quantitative experiments and user studies validate the effectiveness of LiteAT across multiple evaluation metrics. LiteAT establishes a versatile and scalable foundation for immersive telepresence, potentially supporting real-time scenarios such as procedural teaching, multimodal instruction, and collaborative learning.

Abstract:
Head-worn augmented reality (AR) continues to evolve through critical advancements in power optimizations, AI capabilities, and naturalistic user interactions. Eye-tracking sensors play a key role in these advancements. At the same time, eye-tracking data is not well understood by users and can reveal sensitive information. Our work contributes visualizations based on visceral notice to increase privacy awareness of eye-tracking data in AR. We also evaluated user perceptions towards privacy noise mechanisms applied to gaze data visualized through these visceral interfaces. While privacy mechanisms have been evaluated against privacy attacks, we are the first to evaluate them subjectively and understand their influence on data-sharing attitudes. Despite our participants being highly concerned with eye-tracking privacy risks, we found 47% of our participants still felt comfortable sharing raw data. When applying privacy noise, 70% to 76% felt comfortable sharing their gaze data for the Weighted Smoothing and Gaussian Noise privacy mechanisms, respectively. This implies that participants are still willing to share raw gaze data even though overall data-sharing sentiments decreased after experiencing the visceral interfaces and privacy mechanisms. Our work implies that increased access and understanding of privacy mechanisms are critical for gaze-based AR applications; further research is needed to develop visualizations and experiences that relay additional information about how raw gaze data can be used for sensitive inferences, such as age, gender, and ethnicity. We intend to open-source our codebase to provide AR developers and platforms with the ability to better inform users about privacy concerns and provide access to privacy mechanisms. A pre-print of this paper and all supplemental materials are available at https://bmdj-vt.github.io/project_pages/privacy_notice.

Abstract:
We proposed viewpoint-tolerant shared depth perception without individual tracking by leveraging human cognitive compensation in universally 3D rendered images on a wall-sized display. While traditional 3D-perception-enabled display systems have primarily focused on single-user scenarios-adapting rendering based on head and eye tracking-the use of wall-sized displays to extend spatial experiences and support perceptually coherent multi-user interactions remains underexplored. We investigated the effects of virtual depths (dv) and absolute viewing distance (da) on human cognitive compensation factors (perceived distance difference, viewing angle threshold, and perceived presence) to construct the wall display-based eXtended Reality (XR) space. Results show that participants experienced a compelling depth perception even from off-center angles of 23°-37°, and largely increasing virtual depth worsens depth perception and presence factors, highlighting the importance of balancing extended depth of virtual space and viewing distance from the wall-sized display. Drawing on these findings, wall-sized displays in venues such as museums, galleries, and classrooms can evolve beyond 2D information sharing to offer immersive, spatially extended group experiences without individualized tracking or wearables.

Abstract:
Cybersickness significantly impairs user comfort and immersion in virtual reality (VR). Effective identification of cybersickness leveraging physiological, visual, and motion data is a critical prerequisite for its mitigation. However, current methods primarily employ direct feature fusion across modalities, which often leads to limited accuracy due to inadequate modeling of inter-modal relationships. In this paper, we propose a multimodal contrastive learning method for cybersickness recognition. First, we introduce Brain Connectivity Graph Representation (BCGR), an innovative graph-based representation that captures cybersickness-related connectivity patterns across modalities. We further develop three BCGR instances: E-BCGR, constructed based on EEG signals; MV-BCGR, constructed based on video and motion data; and S-BCGR, obtained through our proposed standardized decomposition algorithm. Then, we propose a connectivity-constrained contrastive fusion module, which aligns E-BCGR and MV-BCGR into a shared latent space via graph contrastive learning while utilizing S-BCGR as a connectivity constraint to enhance representation quality. Moreover, we construct a multimodal cybersickness dataset comprising synchronized EEG, video, and motion data collected in VR environments to promote further research in this domain. Experimental results demonstrate that our method outperforms existing state-of-the-art methods across four critical evaluation metrics: accuracy, sensitivity, specificity, and the area under the curve. Source code: https://github.com/PEKEW/cybersickness-bcgr.

Abstract:
We propose a visuo-tactile feedback method that combines virtual hand visualization and fingertip vibrations to modulate affective roughness perception in VR. While prior work has focused on object-based textures and vibrotactile feedback, the role of visual feedback on virtual hands remains underexplored. Our approach introduces affective visual cues including line shape, motion, and color applied to hand outlines, and examines their influence on both affective responses (arousal, valence) and perceived roughness. Results show that sharp contours enhanced perceived roughness, increased arousal, and reduced valence, intensifying the emotional impact of haptic feedback. In contrast, color affected valence only, with red consistently lowering emotional positivity. These effects were especially noticeable at lower haptic intensities, where visual cues extended affective modulation into mid-level perceptual ranges. Overall, the findings highlight how integrating expressive visual cues with tactile feedback can enrich affective rendering and offer flexible emotional tuning in immersive VR interactions.

Abstract:
User interfaces for inspecting spatio-temporal events often allow their users to filter the events by specifying a time window with a time slider. We consider the case that filtered events are visualized on a map using textual or iconic labels. However, to ensure a clear visualization, not all filtered events are annotated with a label. We present algorithms for setting up a data structure that encodes for every possible time window the set of displayed labels. Our algorithms ensure that the displayed labels never overlap and guarantee the stability of the labeling during certain basic interactions with the time slider. Assuming that the labels have different priorities (weights), we aim to maximize the weight of the displayed labels integrated over all possible time windows. As basic interactions, we consider moving the entire time window, symmetrically scaling it, and dragging one of its endpoints. We consider two stability requirements: (1) during a basic interaction, a label should appear and disappear at most once; (2) if a label is displayed for a time window QQ, then it is also displayed for all the time windows contained in QQ and that contain its timestamp. We prove that finding an optimal solution is NP-hard and propose efficient constant-factor approximation algorithms for unit-square and unit-disk labels, as well as a fast greedy heuristic for arbitrarily shaped labels. In experiments on real-world data, we compare the non-exact algorithms with an exact approach through integer linear programming.

Abstract:
Dashboards serve as effective visualization tools for conveying complex information. However, there exists a knowledge gap regarding how dashboard designs impact user engagement, necessitating designers to rely on their design expertise. Saliency has been used to comprehend viewing behaviors and assess visualizations, yet existing saliency models are primarily designed for single-view visualizations. To address this, we conduct an eye-tracking study to quantify participants’ viewing patterns on dashboards. We collect eye-movement data from 60 participants, each viewing 36 dashboards (16 representative dashboards shared across all and 20 unique to each participant), totaling 1,216 dashboards and 2,160 eye-movement data instances. Analysis of the data from 16 dashboards viewed by all participants provides insights into how dashboard objects and layout designs influence viewing behaviors. Our analysis confirms known viewing patterns and reveals new patterns related to dashboard layout designs. Using the eye-movement data and identified patterns, we develop a saliency model to predict viewing behaviors with dashboards. Compared to state-of-the-art models for single-view visualizations, our model demonstrates overall improvement in prediction performance for dashboards. Finally, we propose potential dashboard design guidelines, illustrate an application case, and discuss general scanning strategies along with limitations and future work.

Abstract:
The advances in multi-modal foundation models (FMs) (e.g., CLIP and LLaVA) have facilitated the auto-labeling of large-scale datasets, enhancing model performance in challenging downstream tasks such as open-vocabulary object detection and segmentation. However, the quality of FM-generated labels is less studied as existing approaches focus more on data quantity over quality. This is because validating large volumes of data without ground truth presents a considerable challenge in practice. Existing methods typically rely on limited metrics to identify problematic data, lacking a comprehensive perspective, or apply human validation to only a small data fraction, failing to address the full spectrum of potential issues. To overcome these challenges, we introduce VISTA, a visual analytics framework that improves data quality to enhance the performance of multi-modal models. Targeting the complex and demanding domain of open-vocabulary image segmentation, VISTA integrates multi-phased data validation strategies with human expertise, enabling humans to identify, understand, and correct hidden issues within FM-generated labels. Through detailed use cases on two benchmark datasets and expert reviews, we demonstrate VISTA’s effectiveness from both quantitative and qualitative perspectives.

Abstract:
Visual Analytics (VA) researchers frequently collaborate closely with domain experts to derive requirements and select appropriate solutions to fulfill these requirements. Despite strides made in exploring requirement and solution spaces, challenges persist due to the absence of guidance in the initial consideration space and the lack of shared problem-solving knowledge, often resulting in suboptimal solutions. To address these issues, we conducted an empirical study of VA research, with a focus on mapping the relations between requirement and solution spaces. Analyzing 220 VA papers, we formulate refined topologies for data, requirements, and solutions. We propose conceptualizing the connections between requirements, data, and solutions through knowledge graphs and utilizing solution paths to encapsulate fundamental problem-solving knowledge in visual analytics research. Through the integration of solution paths into a graph and analyzing their interconnections, we identified a subset of problem-driven design patterns that demonstrated the efficacy of our approach. By externalizing problem-solving knowledge and formulating problem-driven design patterns, our aim is to streamline the exploration of consideration space, facilitating the inclusion of “good” solutions, and establish a benchmark for shared design decisions among researchers and readers.

Abstract:
As a pivotal application of Augmented Reality (AR) technology, AR games empower players to bridge reality with virtuality, offering a distinct and immersive experience set apart from traditional games. However, when creating AR games, one of the most formidable challenges faced by designers pertains to the unpredictability of intricate real-world environments, which hinders crafting naturally integrated scenes where virtual objects harmoniously blend with the players’ surroundings. In this paper, we introduce EverywhereAR, a system that is capable of flexibly realizing the designer’s idea in various real-world scenes. It provides a designer-friendly Game Scene Template development interface, for designers to quickly graphify their inspirations. To achieve the best AR game scene, this work proposes a highly customizable integration method. According to the integrated AR scene graph, the system will arrange each virtual object in a reasonable position to make the generated game scene look natural. We conducted an experiment to evaluate our system’s performance across various game scene templates and real-world environments. Results from the experiment indicated that our system was able to generate AR game scenes matching the quality of scenes manually created by professional designers. In addition, we conducted another experiment to assess the effectiveness and usability of the proposed interface. The experiment results showed that the interface was intuitive and efficient, allowing users to create a simple game scene within one minute.

Abstract:
Human point cloud registration is a critical problem in the fields of computer vision and computer graphics applications. Currently, due to the presence of joint hinges and limb occlusions in human point clouds, point cloud alignment is challenging. To address these two limits, this paper proposes an unsupervised non-rigid human point cloud registration method based on deformation field fusion. The method mainly consists of the deep dynamic link deformation field estimation module and the probabilistic alignment deformation field estimation module. The deep dynamic link deformation field estimation module uses a time series network to convert non-rigid deformation into multiple rigid deformations. Then, feature extraction is performed to estimate the deformation field based on the rigid deformations. The probabilistic alignment deformation field estimation module builds on a Gaussian mixture model and adds local and global constraint conditions for deformation field estimation. Finally, the two deformation fields are fused into the total deformed field by aligning them, which enhances the sensitivity to both global and local feature information. The experimental results on public datasets and real private datasets demonstrate that the proposed method has higher accuracy and better robustness under joint hinges and limb adhesion conditions.

Abstract:
Projection mapping (PM) is useful in the product design process, since it seamlessly bridges a physical mockup and its digital twin by allowing designers to interactively explore new textures, colors, and shapes without the need to create new physical mockups. While PM has proven effective for car interior design, previous research focused solely on supporting the design of dashboards and instrument panels, neglecting evaluation in realistic driving scenarios. This paper introduces a self-contained car interior design support system that extends beyond the dashboard to include the A-pillars. Additionally, to enable designers to evaluate their designs in authentic driving conditions, we integrate a driving simulator, complete with a motion platform, into the PM system. Through the construction of a prototype, we demonstrate the feasibility of our proposed system. Finally, through user studies, we derive guidelines for PM-based car interior design to optimize the user experience.

Abstract:
Illumination estimation from a single indoor image is a promising yet challenging task. Existing indoor illumination estimation methods mainly regress lighting parameters or infer a panorama from a limited field-of-view image. Nevertheless, these methods fail to recover a panorama with both well-distributed illumination and detailed environment textures, leading to a lack of realism in rendering the embedded 3D objects with complex materials. This paper presents a novel multi-stage illumination estimation framework named IllumiDiff. Specifically, in Stage I, we first estimate illumination conditions from the input image, including the illumination distribution as well as the environmental texture of the scene. In Stage II, guided by the estimated illumination conditions, we design a conditional panoramic texture diffusion model to generate a high-quality LDR panorama. In Stage III, we leverage the illumination conditions to further reconstruct the LDR panorama to an HDR panorama. Extensive experiments demonstrate that our IllumiDiff can generate an HDR panorama with realistic illumination distribution and rich texture details from a single limited field-of-view indoor image. The generated panorama can produce impressive rendering results for the embedded 3D objects with various materials.

Abstract:
Particle advection is one of the foundational algorithms for visualization and analysis and is central to understanding vector fields common to scientific simulations. Achieving efficient performance with large data in a distributed memory setting is notoriously difficult. Because of its simplicity and minimized movement of large vector field data, the Parallelize over Data (POD) algorithm has become a de facto standard. Despite its simplicity and ubiquitous usage, the scaling issues with the POD algorithm are known and have been described throughout the literature. In this paper, we describe a set of in-depth analyses of the POD algorithm that shed new light on the underlying causes for the poor performance of this algorithm. We designed a series of representative workloads to study the performance of the POD algorithm and executed them on a supercomputer while collecting timing and statistical data for analysis. we then performed two different types of analysis. In the first analysis, we introduce two novel metrics for measuring algorithmic efficiency over the course of a workload run. The second analysis was from the perspective of the particles being advected. Using particle-centric analysis, we identify that the overheads associated with particle movement between processes (not the communication itself) have a dramatic impact on the overall execution time. These overheads become particularly costly when flow features span multiple blocks, resulting in repeated particle circulation (which we term “ping pong particles”) between blocks. Our findings shed important light on the underlying causes of poor performance and offer directions for future research to address these limitations.

Abstract:
In recent years, significant progress has been witnessed in the field of neural 3D avatar reconstruction. Among all related tasks, building an animatable avatar from monocular videos is one of the most challenging ones, yet it also has a wide range of applications. The “animatable” means that we need to transfer any arbitrary and unseen poses onto the avatar and generate new 3D videos. Thanks to the rise of the powerful representation of NeRF, generating a high-fidelity animatable avatar from videos has become easier and more accessible. Despite their impressive visual results, the substantial training and rendering overhead dramatically hamper their applications. 3D Gaussian Splatting, as a timely new representation, has demonstrated its high-quality and high-efficiency rendering. This has led to many concurrent works to introduce 3D-GS to animatable avatar building. Although they demonstrate very high-fidelity renderings for poses similar to the training video frames, poor results are produced when the poses are far from training. We argue that this is primarily because the Gaussian points lack structures. Thus, we suggest involving DMTet to represent the coarse geometry of the avatar. In our representation, the majority of Gaussian points are bound to the mesh vertices, while some free Gaussian is allowed to expand to better fit the given video. Furthermore, we develop a dual-space optimization framework to jointly optimize the DMTet, Gaussian points, and skinning weights under two spaces. In this sense, Gaussian points are transformed in a constrained way, which dramatically improves the generalization ability for unseen poses. This is well demonstrated via extensive experiments.

Abstract:
This paper introduces the Simultaneous Presence (SP) Continuum, a novel concept designed to enhance the use of portals in virtual reality (VR) by understanding users’ experiences across multiple environments. Portals traditionally provide access to secondary worlds, but their limited field of view (FoV) can hinder full engagement with these environments. To overcome this, we introduce portal overlays, which expand the portal's FoV by superimposing the secondary world over the user's view of the primary world. We explore several overlay techniques—Contours, Blended (Opacity and Stencil), and Absolute—to adjust user experience across the SP Continuum. The Contours overlay, renders only the edges of objects, offering a minimalistic view and immersion of the secondary world. The Blended overlay allows for adjustable immersion between worlds through opacity or the distribution of pixels. The Absolute overlay virtually immerses users completely in the secondary world. Importantly, these overlays are context-activated instead of always on, to suit situational needs. Our study revealed high SP was possible in both worlds, and overlays significantly impacted users’ experiences on the SP Continuum. The Blended overlay provided balanced SP, while Contours had the highest primary presence but lower secondary presence. In contrast, the Absolute overlay, alternating full immersion between worlds, had the highest secondary presence but resulted in reduced primary presence, slower navigation and selection times, higher effort and frustration, and lower usability.

Abstract:
We present FLINT (learning-based FLow estimation and temporal INTerpolation), a novel deep learning-based approach to estimate flow fields for 2D+time and 3D+time scientific ensemble data. FLINT can flexibly handle different types of scenarios with (1) a flow field being partially available for some members (e.g., omitted due to space constraints) or (2) no flow field being available at all (e.g., because it could not be acquired during an experiment). The design of our architecture allows to flexibly cater to both cases simply by adapting our modular loss functions, effectively treating the different scenarios as flow-supervised and flow-unsupervised problems, respectively (with respect to the presence or absence of ground-truth flow). To the best of our knowledge, FLINT is the first approach to perform flow estimation from scientific ensembles, generating a corresponding flow field for each discrete timestep, even in the absence of original flow information. Additionally, FLINT produces high-quality temporal interpolants between scalar fields. FLINT employs several neural blocks, each featuring several convolutional and deconvolutional layers. We demonstrate performance and accuracy for different usage scenarios with scientific ensembles from both simulations and experiments.

Abstract:
Guidance-enhanced approaches are used to support users in making sense of their data and overcoming challenging analytical scenarios. While recent literature underscores the value of guidance, a lack of clear explanations to motivate system interventions may still negatively impact guidance effectiveness. Hence, guidance-enhanced VA approaches require meticulous design, demanding contextual adjustments for developing appropriate explanations. Our article discusses the concept of explainable guidance and how it impacts the user–system relationship—specifically, a user's trust in guidance within the VA process. We subsequently propose a model that supports the design of explainability strategies for guidance in VA. The model builds upon flourishing literature in explainable AI, available guidelines for developing effective guidance in VA systems, and accrued knowledge on user–system trust dynamics. Our model responds to challenges concerning guidance adoption and context-effectiveness by fostering trust through appropriately designed explanations. To demonstrate the model's value, we employ it in designing explanations within two existing VA scenarios. We also describe a design walk-through with a guidance expert to showcase how our model supports designers in clarifying the rationale behind system interventions and designing explainable guidance.

Abstract:
This article introduces a novel latent 3D diffusion model for generating neural voxel fields with precise part-aware structures and high-quality textures. In comparison to existing methods, this approach incorporates two key designs to guarantee high-quality and accurate part-aware generation. On one hand, we introduce a latent 3D diffusion process for neural voxel fields, incorporating part-aware information into the diffusion process and allowing generation at significantly higher resolutions to capture rich textural and geometric details accurately. On the other hand, a part-aware shape decoder is introduced to integrate the part codes into the neural voxel fields, guiding accurate part decomposition and producing high-quality rendering results. Importantly, part-aware learning establishes structural relationships to generate texture information for similar regions, thereby facilitating high-quality rendering results. We evaluate our approach across eight different data classes through extensive experimentation and comparisons with state-of-the-art methods. The results demonstrate that our proposed method has superior generative capabilities in part-aware shape generation, outperforming existing state-of-the-art methods. Moreover, we have conducted image- and text-guided shape generation via the conditioned diffusion process, showcasing the advanced potential in multi-modal guided shape generation.

Abstract:
Redirected walking (RDW) allows users to explore vast virtual spaces by walking in confined real spaces, yet suffers from frequent boundary collisions due to physical constraints. The major solution is to use the reset strategy to steer users away from boundaries. However, most reset methods guide users to fixed spots or follow constant patterns, neglecting spatial features and users’ movement trends. In this article, we propose an innovative predictive reset method based on spatial probability density distribution to jointly involve impacts of spatial feature and walking intention for forecasting the user’s possible positional distribution, and thereby determines the optimal reset direction by maximizing walking expectation. Given a space, we calculate the stationary layout energy to indicate traveling difficulties of all positions. Meanwhile, we exploit a novel intention inference model to anticipate the probability distribution of the user’s presence across adjacent positions. Furthermore, we incorporate the obstacle energy attenuation to predict the obstacle avoidance behaviors. All aforementioned factors are amalgamated into a potential region energy map, and then we integrate energy maps of virtual and real spaces into a fusion energy map to enable the prediction considering both spaces simultaneously. Thus, the optimal reset direction is derived by maximizing the fusion energy. Simulation and user studies are conducted on a broad dataset containing plentiful virtual and real spaces. The results demonstrate that our method effectively reduces the physical collisions and increase the continuous walking distance compared to prevalent reset methods, while exhibiting superior applicability when combined with various RDW controllers.

Abstract:
We present the first real-time method for inserting a rigid virtual object into a neural radiance field (NeRF), which produces realistic lighting and shadowing effects, as well as allows interactive manipulation of the object. By exploiting the rich information about lighting and geometry in a NeRF, our method overcomes several challenges of object insertion in augmented reality. For lighting estimation, we produce accurate and robust incident lighting that combines the 3D spatially-varying lighting from NeRF and an environment lighting to account for sources not covered by the NeRF. For occlusion, we blend the rendered virtual object with the background scene using an opacity map integrated from the NeRF. For shadows, with a precomputed field of spherical signed distance fields, we query the visibility term for any point around the virtual object, and cast soft, detailed shadows onto 3D surfaces. Compared with state-of-the-art techniques, our approach can insert virtual objects into scenes with superior fidelity, and has great potential to be further applied to augmented reality systems.

Abstract:
Digital garments are set to revolutionize the apparel industry in the way we design, produce, market, sell and try-on real garments. But for digital garments to play a central role, from designer to consumer, they must be a faithful digital replica of their real counterpart: a digital twin. Yet, most industry-grade tools used in the apparel industry do not focus on accuracy, but rather on producing fast and plausible drapes for interactive editing and quick feedback, thus limiting the value and the potential of digital garments. The key to accuracy lies in using the proper underlying simulation technology, well documented in the academic literature but historically sidelined in the apparel industry in favor of simulation speed. In this paper, we describe our industry-grade cloth simulation engine, built with a strong focus on accuracy rather than sheer speed. Using a global integration scheme and adopting state of the art simulation practices from the Computer Graphics field, we evaluate a wide range of algorithms to improve its convergence and overall performance. We provide qualitative and quantitative insights on the cost and capabilities of each of these features, with the aim of giving valuable feedback and useful guidelines to practitioners seeking to implement an accurate and robust draping simulator.

Abstract:
Existing facial editing methods have achieved remarkable results, yet they often fall short in supporting multimodal conditional local facial editing. One of the significant evidences is that their output image quality degrades dramatically after several iterations of incremental editing, as they do not support local editing. In this paper, we present a novel multimodal generative and fusion framework for globally-consistent local facial editing (FACEMUG) that can handle a wide range of input modalities and enable fine-grained and semantic manipulation while remaining unedited parts unchanged. Different modalities, including sketches, semantic maps, color maps, exemplar images, text, and attribute labels, are adept at conveying diverse conditioning details, and their combined synergy can provide more explicit guidance for the editing process. We thus integrate all modalities into a unified generative latent space to enable multimodal local facial edits. Specifically, a novel multimodal feature fusion mechanism is proposed by utilizing multimodal aggregation and style fusion blocks to fuse facial priors and multimodalities in both latent and feature spaces. We further introduce a novel self-supervised latent warping algorithm to rectify misaligned facial features, efficiently transferring the pose of the edited image to the given latent codes. We evaluate our FACEMUG through extensive experiments and comparisons to state-of-the-art (SOTA) methods. The results demonstrate the superiority of FACEMUG in terms of editing quality, flexibility, and semantic control, making it a promising solution for a wide range of local facial editing tasks.

Abstract:
No-reference point cloud quality assessment (PCQA) based on bitstreams uses information extracted from the bitstream for quality monitoring at network nodes. We develop a no-reference PCQA model based on bitstreams for the perceived quality assessment of Octree-Lifting coded point clouds. At first, our research explores the essential correlation between subjective visual quality degradation and the texture quantization parameter (TQP) when using lossless geometric coding. Then, we enhance the proposed model by incorporating texture complexity (TC) while taking into account the dependence of perceptual coding distortion on the texture characteristics of a point cloud. We estimate TC by utilizing TQP and calculating the average standard deviation of the Y-component of the attribute value (Y\_ stdY_std), both of which are extracted from the bitstream. Then, a texture distortion assessment model is constructed based on TQP and Y\_ stdY_std. The integration of the texture distortion model with the position quantization scale (PQS) results in the derivation of an overall no-reference bitstream-based PCQA model, named streamPCQ-OL. The findings from the conducted experiments highlight a significant superiority of the proposed model over existing approaches in terms of performance.

Abstract:
Direct Volume Rendering (DVR) plays an important role in scientific data visualization. To generate photo-realistic DVR results, the physical light transport throughout the volume is simulated by applying the Monte Carlo-based volumetric path tracing (VPT) approach. For real-time applications, due to the time constraint for rendering each frame, only a limited number of samples shall be taken for the computation per pixel. This can result in a significant amount of noise in the rendering results. This paper describes our optimized VPT sampling algorithm and a novel denoising technique to generate consistently high-quality realistic DVR results in real time. We develop a new shading model that can reduce estimation variance to enhance the quality of DVR results. Additionally, a hybrid acceleration structure is created by integrating both octree and macrocell to improve sampling efficiency. This allows the acquisition of sufficiently more shading samples while maintaining the desired interactive frame rate. To further eliminate remaining noise and improve temporal stability of DVR results, we develop a novel spatiotemporal denoising framework. Our denoiser decouples the estimated radiance into high-detail low-noise and low-detail high-noise components. Different denoising algorithms are separately applied to these components to reduce noise without introducing blurring artifacts. Our DVR system can consistently offer high rendering quality and good temporal stability across DVR result frames in real time. During fast user interactions and with rapid alterations of the illumination condition, our rendering method can still provide good visual comfort and representation accuracy without visible latency.

Abstract:
This article presents a review and analysis of evaluation practices within the visualization and visual analytics (VIS) domain, with a focus on domain application work accepted at the IEEE VIS conference from 2018 to 2022. Through the analysis of 140 pertinent article, we establish a detailed classification principle for evaluation practices, using the Who, When, What, and How indicators. This principle covers facets such as analysis methods, targets, scenarios, participant expertise, and stages of occurrence. By systematically categorizing the application domains presented in these works, we apply our established classification principle to discern and categorize the evaluation practices within them, identifying the prevailing characteristics and trends. The article explores the variety of evaluation methods employed across different application domains and observes the distinctions in their usage. In conclusion, we provide insights and highlight concerns for conducting evaluations in upcoming domain application research. Our findings are intended to inform and guide subsequent studies in a similar context.

Abstract:
Multiple tools are available to reduce cybersickness (sickness caused by virtual reality), but past research has not investigated the combined effects of multiple mitigation tools. Field of view (FOV) restriction limits peripheral vision during self-motion, and ample evidence supports its effectiveness for reducing cybersickness. Snap turning involves discrete rotations of the user's perspective without presenting intermediate views, although reports on its effectiveness at reducing cybersickness are limited and equivocal. Both mitigation tools reduce the visual motion that can cause cybersickness. The current study (N = 201) investigated the individual and combined effects of FOV restriction and snap turning on cybersickness when playing a consumer virtual reality game. FOV restriction and snap turning in isolation reduced cybersickness compared to a control condition without mitigation tools. Yet, the combination of FOV restriction and snap turning did not further reduce cybersickness beyond the individual tools in isolation, and in some cases the combination of tools led to cybersickness similar to that in the no mitigation control. These results indicate that caution is warranted when combining multiple cybersickness mitigation tools, which can interact in unexpected ways.

Abstract:
Nowadays, orthodontics has become an important part of modern personal life to assist one in improving mastication and raising self-esteem. However, the quality of orthodontic treatment still heavily relies on the empirical evaluation of experienced doctors, which lacks quantitative assessment and requires patients to visit clinics frequently for in-person examination. To resolve the aforementioned problem, we propose a novel and practical mobile device-based framework for precisely measuring tooth movement in treatment, so as to simplify and strengthen the traditional tooth monitoring process. To this end, we formulate the tooth movement monitoring task as a multi-view multi-object pose estimation problem via different views that capture multiple texture-less and severely occluded objects (i.e. teeth). Specifically, we exploit a pre-scanned 3D tooth model and a sparse set of multi-view tooth images as inputs for our proposed tooth monitoring framework. After extracting tooth contours and localizing the initial camera pose of each view from the initial configuration, we propose a joint pose estimation scheme to precisely estimate the 3D pose of each individual tooth, so as to infer their relative offsets during treatment. Furthermore, we introduce the metric of Relative Pose Bias to evaluate the individual tooth pose accuracy in a small scale. We demonstrate that our approach is capable of reaching high accuracy and efficiency as practical orthodontic treatment monitoring requires.

Abstract:
When analyzing heterogeneous data comprising numerical and categorical attributes, it is common to treat the different data types separately or transform the categorical attributes to numerical ones. The transformation has the advantage of facilitating an integrated multi-variate analysis of all attributes. We propose a novel technique for transforming categorical data into interpretable numerical feature vectors using Large Language Models (LLMs). The LLMs are used to identify the categorical attributes’ main characteristics and assign numerical values to these characteristics, thus generating a multi-dimensional feature vector. The transformation can be computed fully automatically, but due to the interpretability of the characteristics, it can also be adjusted intuitively by an end user. We provide a respective interactive tool that aims to validate and possibly improve the AI-generated outputs. Having transformed a categorical attribute, we propose novel methods for ordering and color-coding the categories based on the similarities of the feature vectors.

Abstract:
Mastering the correct use of laboratory equipment is a fundamental skill for undergraduate science students involved in laboratory-based training. However, hands-on laboratory time is often limited, and remote students may struggle as their absence from the physical lab limits their skill development. An air displacement micropipette was selected for this investigation, as accuracy and correct technique are essential in generating reliable assay data. Handling small liquid volumes demands hand dexterity and practice to achieve proficiency. This research assesses the importance of tactile authenticity during training by faithfully replicating the micropipette's key physical and operational characteristics. We developed a custom haptic training approach called ‘Smart Pipette’ which promotes accurate operation and enhances laboratory dexterity training. A comparative user study with 34 participants evaluated the effectiveness of the Smart Pipette custom haptic device against training with off-the-shelf hardware, specifically the Quest VR hand controller, which was chosen because it is held mid-air similar to a laboratory micropipette. Both training conditions are integrated with the same self-paced virtual simulation displayed on a computer screen, offering audiovisual instructions and real-time guidance. Results demonstrated that participants trained with the Smart Pipette custom haptic exhibited increased accuracy and precision while making fewer errors than those trained with off-the-shelf hardware. The Smart Pipette and the Quest VR controller had no significant differences in cognitive load or system usability scores. Tactile authentic interaction devices address challenges faced by online learners, while their applicability extends to traditional classrooms, where real-time feedback significantly enhances overall training performance outcomes.

Abstract:
The article presents a novel approach to visualizing adversarial robustness (called robustness below) of deep neural networks (DNNs). Traditional tests only return a value reflecting a DNN's overall robustness across a fixed number of test samples. Unlike them, we use test samples to train a generative model (GM) and render a DNN's robustness distribution over infinite generated samples within the GM's latent space. The approach extends test samples, enabling users to obtain new test samples to improve feature coverage constantly. Moreover, the distribution provides more information about a DNN's robustness, enabling users to understand a DNN's robustness comprehensively. We propose three methods to resolve the challenges of realizing the approach. Specifically, we (1) map a GM's high-dimensional latent space onto a plane with less information loss for visualization, (2) design a network to predict a DNN's robustness on massive samples to speed up the distribution rendering, and (3) develop a system to supports users to explore the distribution from multiple perspectives. Subjective and objective experiment results prove the usability and effectiveness of the approach.

Abstract:
Full-body 3D scanning simplifies the acquisition of digital body models. However, current systems are bulky, intricate, and costly, with strict clothing constraints. We propose a pipeline that combines inner body shape inference and parametric model registration for reconstructing the corresponding body model from a single front scan of a clothed body. Three networks modules (Scan2Front-Net, Front2Back-Net, and Inner2Corr-Net) with relatively independent functions are proposed for predicting front inner, back inner, and parametric model reference point clouds, respectively. We consider the back inner point cloud as an axial offset of the front inner point cloud and divide the body into 14 parts. This offset relationship is then learned within the same body parts to reduce the ambiguity of the inference. The predicted front and back inner point clouds are concatenated as inner body point cloud, and then reconstruction is achieved by registering the parametric body model through a point-to-point correspondence between the reference point cloud and the inner body point cloud. Qualitative and quantitative analysis show that the proposed method has significant advantages in terms of body shape completion and reconstruction body model accuracy.

Affiliations: Department of Computer Science, University of Hong Kong, Hong Kong, China; School of Information Science and Technology, Shanghai Tech University and Shanghai Engineering Research Center of Intelligent Vision and Imaging, Shanghai, China; Faculty of Dentistry, University of Hong Kong, Hong Kong, China; Department of Mechanical and Aerospace Engineering, Hong Kong University of Science and Technology, Hong Kong, China; Shandong University, Jinan, China; Texas A&M University, College Station, TX, USA

Abstract:
Invisible orthodontics, commonly known as clear alignment treatment, offers a more comfortable and aesthetically pleasing alternative in orthodontic care, attracting considerable attention in the dental community in recent years. It replaces conventional metal braces with a series of removable, and transparent aligners. Each aligner is crafted to facilitate a gradual adjustment of the teeth, ensuring progressive stages of dental correction. This necessitates the design for teeth motion. Here we present an automatic method and a system for generating collision-free teeth motion planning while avoiding gaps between adjacent teeth, which is unacceptable in clinical practice. To tackle this task, we formulate it as a constrained optimization problem and utilize the interior point method for its solution. We also developed an interactive system that enables dentists to easily visualize and edit the paths. Our method significantly speeds up the clear aligner planning process, creating the desired motion paths for a full set of teeth in under five minutes—a task that typically requires several hours of manual work. Our experiments and user studies confirm the effectiveness of this method in planning teeth movement, showcasing its potential to streamline orthodontic procedures.

Abstract:
We present a comprehensive pipeline, integrated with a visual analytics system called GapMiner, capable of exploring and exploiting untapped opportunities within the empty regions of high-dimensional datasets. Our approach utilizes a novel Empty-Space Search Algorithm (ESA) to identify the center points of these uncharted voids, which represent reservoirs for potentially valuable new configurations. Initially, this process is guided by user interactions through GapMiner, which visualizes Empty-Space Configurations (ESCs) within the context of the dataset and allows domain experts to explore and refine ESCs for subsequent validation in domain experiments or simulations. These activities iteratively enhance the dataset and contribute to training a connected deep neural network (DNN). As training progresses, the DNN gradually assumes the role of identifying and validating high-potential ESCs, reducing the need for direct user involvement. Once the DNN achieves sufficient accuracy, it autonomously guides the exploration of optimal configurations by predicting performance and refining configurations through a combination of gradient ascent and improved empty-space searches. Domain experts were actively involved throughout the system’s development. Our findings demonstrate that this methodology consistently generates superior novel configurations compared to conventional randomization-based approaches. We illustrate its effectiveness in multiple case studies with diverse objectives.

Abstract:
Controllable head avatar reconstruction via the inversion of few-shot images using 3D generative models has demonstrated significant potential for efficient avatar creation. However, under limited input conditions, existing one-shot inversion methods often fail to produce high-fidelity results, frequently leading to shape distortions, expression deviations, and identity inconsistencies. To address these limitations, we propose EGAvatar, a novel and efficient 3DGAN inversion framework designed to generate high-fidelity, generalizable head avatars from few-shot images. The core principle of EGAvatar is a decoupling-by-inverting strategy, built upon an animatable 3DGAN prior. Specifically, we introduce an effective animatable 3DGAN model that synthesizes high-quality 3D avatars by integrating a coarse 3D triplane representation (derived from a latent 3DGAN) with an offset 3D triplane (learned via a triplane 3DGAN). Leveraging this architecture, we design a 3DGAN-based inversion approach to reconstruct 3D avatars efficiently. Additionally, we incorporate an expression-view disentanglement mechanism to maintain consistent appearance across varying expressions and viewpoints, thereby enhancing the generalizability of avatar reconstruction from limited input images. Extensive experiments conducted on two publicly available benchmarks and a private dataset demonstrate that EGAvatar outperforms existing state-of-the-art methods in both qualitative and quantitative evaluations. Notably, EGAvatar achieves superior performance while requiring significantly fewer input images and offering more efficient training and inference.

Abstract:
Most consumer cameras use rolling shutter (RS) exposure, the captured videos often suffer from distortions (e.g., skew and jelly effect). Also, these videos are impeded by the limited bandwidth and frame rate, which inevitably affect the video streaming experience. In this paper, we excavate the potential of event cameras as they enjoy high temporal resolution. Accordingly, we propose a framework to recover the global shutter (GS) high frame rate (i.e., slow motion) video without RS distortion from an RS camera and event camera. One challenge is the lack of real-world datasets for supervised training. Therefore, we explore self-supervised learning with the key idea of estimating the displacement field—a non-linear and dense 3D spatiotemporal representation of all pixels during the exposure time. This allows for a mutual reconstruction between RS and GS frames and facilitates slow-motion video recovery. We then combine the input RS frames with the DF to map them to the GS frames (RS-to-GS). Given the under-constrained nature of this mapping, we integrate it with the inverse mapping (GS-to-RS) and RS frame warping (RS-to-RS) for self-supervision. We evaluate our framework via objective analysis (i.e., quantitative and qualitative comparisons on four datasets) and subjective studies (i.e., user study). The results show that our framework can recover slow-motion videos without distortion, with much lower bandwidth (94% drop) and higher inference speed ( 16\; \rm ms/\rm frame16 ms / frame ) under 32 ×32× frame interpolation.

Abstract:
Despite the growing popularity of data videos, guidance on designing narrative climaxes that maximise viewers’ emotional engagement remains scarce. To address this gap, our work leverages emotional theory to derive patterns for crafting emotionally resonant climaxes of data videos. We first analyzed the climaxes of 96 data videos, categorizing them into eight emotional dimensions based on Plutchik’s basic emotion model. Based on data analysis, we then formulated 40 patterns for creating narrative climaxes. To evaluate the patterns when applied as design hints, we conducted a user study with 48 participants, where Group A created data video climaxes using our patterns, Group B created them without our patterns, and Group C used other patterns as the baseline. Evaluations by two experts and 20 general audiences revealed that the climaxes created with the patterns were more emotionally engaging. The participants also praised the clarity and practicality of the patterns.

Abstract:
Non-local similarity (NLS) has been successfully applied to point cloud denoising. However, existing non-local methods either involve high algorithmic complexity in capturing NLS or suffer from diminished accuracy in estimating low-rank matrices. To address these problems, we propose a Point Cloud Denoising framework using \gammaγ-norm minimization based on Curvature Entropy (PCD-\gammaγCE) for efficiently removing noise. First, we develop a structure descriptor, which exploits Curvature Entropy (CE) to accurately capture shape variation details of Non-Local Similar Structure (NLSS), and employs Angle Subdivision (AS) of NLSS to control the complexity of initial normal matrix construction. Second, we introduce \gammaγ-norm to construct a low-rank denoising model for initial normal matrix, thereby providing a nearly unbiased estimation of rank function with better robustness to noise. Extensive experiments on synthetic and raw scanned point clouds show that our approach outperforms the popular denoising methods, with a 99.90% time reduction and gains in Mean Square Error (MSE) and Chamfer Distance (CD) compared with the Weighted Nuclear Norm Minimization (WNNM) method.

Abstract:
Interactive visualization editors empower users to author visualizations without writing code, but do not provide guidance on the art and craft of effective visual communication. In this article, we explore the potential of using an off-the-shelf large language models (LLMs) to provide actionable and customized feedback to visualization designers. Our implementation, Visualizationary, demonstrates how ChatGPT can be used for this purpose through two key components: a preamble of visualization design guidelines and a suite of perceptual filters that extract salient metrics from a visualization image. We present findings from a longitudinal user study involving 13 visualization designers—6 novices, 4 intermediates, and 3 experts—who authored a new visualization from scratch over several days. Our results indicate that providing guidance in natural language via an LLM can aid even seasoned designers in refining their visualizations.

Abstract:
The demand for full-reference point cloud quality assessment (PCQA) has extended across various point cloud services. Unlike image quality assessment, where the reference and the distorted images are naturally aligned in coordinates and thus allow point-to-point (P2P) color assessment, the coordinates and attributes of a 3D point cloud may both suffer from distortion, making the P2P evaluation unsuitable. To address this, PCQA methods usually define a set of key points and construct a neighborhood around each key point for neighbor-to-neighbor (N2N) computation on geometry and attribute. However, state-of-the-art PCQA methods often exhibit limitations in certain scenarios due to insufficient consideration of key points and neighborhoods. To overcome these challenges, this paper proposes PQI, a simple yet efficient metric to index point cloud quality. PQI suggests using scale-wise key points to uniformly perceive distortions within a point cloud, along with a mild neighborhood size associated with each key point for compromised N2N computation. To achieve this, PQI employs a multiscale framework to obtain key points, ensuring comprehensive feature representation and distortion detection throughout the entire point cloud. Such a multiscale method merges every eight points into one in the downsampling processing, implicitly embedding neighborhood information into a single point and thereby eliminating the need for an explicitly large neighborhood. Further, within each neighborhood, simple features, such as geometry Euclidean distance difference and attribute value difference, are extracted. Feature similarity is then calculated between the reference and the distorted samples at each scale and linearly weighted to generate the final PQI score. Extensive experiments demonstrate the superiority of PQI, consistently achieving high performance across several widely recognized PCQA datasets. Moreover, PQI is highly appealing for practical applications due to its low complexity and flexible scale options.

Abstract:
3D representations of large-scale and urban scenes are crucial across various industries, including autonomous driving, urban planning, natural resource supervision and many more. Large-scale industrial reconstructions are inherently complex and multifaceted. Many existing surveys primarily focus on academic progressions and often neglect the intricate and diverse needs of industry. This survey aims to bridge this gap by providing a comprehensive analysis of 3D reconstruction methods, with a focus on industrial requirements such as scalability and integration of human interaction. Our approach involves utilizing Affinity Diagramming to systematically analyze qualitative data gathered from industrial partners. This methodology enables us to gain deep insights into how recent literature addresses these specific industrial needs. The survey encompasses various aspects, including input and reconstruction modalities, architectural models, datasets, evaluation metrics, and the incorporation of prior knowledge. We further discuss practical implications derived from our analysis, highlighting key considerations for future advancements in 3D reconstruction methods tailored for large-scale applications.

Abstract:
Incorporating automatic style extraction and transfer from existing well-designed graph visualizations can significantly alleviate the designer’s workload. There are many types of graph visualizations. In this paper, our work focuses on node-link diagrams. We present a novel approach to streamline the design process of graph visualizations by automatically extracting visual styles from well-designed examples and applying them to other graphs. Our formative study identifies the key styles that designers consider when crafting visualizations, categorizing them into global and local styles. Leveraging deep learning techniques such as saliency detection models and multi-label classification models, we develop end-to-end pipelines for extracting both global and local styles. Global styles focus on aspects such as color scheme and layout, while local styles are concerned with the finer details of node and edge representations. Through a user study and evaluation experiment, we demonstrate the efficacy and time-saving benefits of our method, highlighting its potential to enhance the graph visualization design process.

Abstract:
Nowadays, 3D scenes are not merely static arrangements of objects. With the development of transformable modules, furniture objects can be translated, rotated, and even reshaped to achieve scenes with different functions (e.g., from a bedroom to a living room). Transformable domestic space, therefore, studies how a layout can change its function by reshaping and rearranging transformable modules, resulting in various transformable layouts. In practice, a rearrangement is dynamically conducted by reshaping/translating/rotating furniture objects with proper schedules, which can consume more time for designers than static scene design. Due to changes in objects’ functions, potential transformable layouts may also be extensive, making it hard to explore desired layouts. We present a system for exploring transformable layouts. Given a single input scene consisting of transformable modules, our system first attempts to derive more layouts by reshaping and rearranging the modules. The derived scenes are organized into a graph-like hierarchy according to their functions, where edges represent functional evolutions (e.g., a living room can be reshaped to a bedroom), and nodes represent layouts that are dynamically transformable through translating/rotating/reshaping modules. The resulting hierarchy lets scene designers interactively explore possible scene variants and preview the animated rearrangement process. Experiments show that our system is efficient for generating transformable layouts, sensible for organizing functional hierarchies, and inspiring for providing ideas during interactions.

Abstract:
In this paper, we propose a novel rendering framework based on neural radiance fields (NeRF) named HH-NeRF that can generate high-resolution audio-driven talking portrait videos with high fidelity and fast rendering. Specifically, our framework includes a detail-aware NeRF module and an efficient conditional super-resolution module. First, a detail-aware NeRF is proposed to efficiently generate a high-fidelity low-resolution talking head, by using the encoded volume density estimation and audio-eye-aware color calculation. This module can capture natural eye blinks and high-frequency details, and maintain a similar rendering time as previous fast methods. Secondly, we present an efficient conditional super-resolution module on the dynamic scene to directly generate the high-resolution portrait with our low-resolution head. Incorporated with the prior information, such as depth map and audio features, our new proposed efficient conditional super resolution module can adopt a lightweight network to efficiently generate realistic and distinct high-resolution videos. Extensive experiments demonstrate that our method can generate more distinct and fidelity talking portraits on high resolution (900 × 900) videos compared to state-of-the-art methods.

Abstract:
Dashboards remain ubiquitous tools for analyzing data and disseminating the findings. Understanding the range of dashboard designs, from simple to complex, can support development of authoring tools that enable end-users to meet their analysis and communication goals. Yet, there has been little work that provides a quantifiable, systematic, and descriptive overview of dashboard design patterns. Instead, existing approaches only consider a handful of designs, which limits the breadth of patterns that can be surfaced. More quantifiable approaches, inspired by machine learning (ML), are presently limited to single visualizations or capture narrow features of dashboard designs. To address this gap, we present an approach for modeling the content and composition of dashboards using a graph representation. The graph decomposes dashboard designs into nodes featuring content “blocks’; and uses edges to model “relationships”, such as layout proximity and interaction, between nodes. To demonstrate the utility of this approach, and its extension over prior work, we apply this representation to derive a census of 25,620 dashboards from Tableau Public, providing a descriptive overview of the core building blocks of dashboards in the wild and summarizing prevalent dashboard design patterns. We discuss concrete applications of both a graph representation for dashboard designs and the resulting census to guide the development of dashboard authoring tools, making dashboards accessible, and for leveraging AI/ML techniques. Our findings underscore the importance of meeting users where they are by broadly cataloging dashboard designs, both common and exotic.

Abstract:
Visual analytics (VA) requires analysts to iteratively propose analysis tasks based on observations and execute tasks by creating visualizations and interactive exploration to gain insights. This process demands skills in programming, data processing, and visualization tools, highlighting the need for a more intelligent, streamlined VA approach. Large language models (LLMs) have recently been developed as agents to handle various tasks with dynamic planning and tool-using capabilities, offering the potential to enhance the efficiency and versatility of VA. We propose LightVA, a lightweight VA framework that supports task decomposition, data analysis, and interactive exploration through human-agent collaboration. Our method is designed to help users progressively translate high-level analytical goals into low-level tasks, producing visualizations and deriving insights. Specifically, we introduce an LLM agent-based task planning and execution strategy, employing a recursive process involving a planner, executor, and controller. The planner is responsible for recommending and decomposing tasks, the executor handles task execution, including data analysis, visualization generation and multi-view composition, and the controller coordinates the interaction between the planner and executor. Building on the framework, we develop a system with a hybrid user interface that includes a task flow diagram for monitoring and managing the task planning process, a visualization panel for interactive data exploration, and a chat view for guiding the model through natural language instructions. We examine the effectiveness of our method through a usage scenario and an expert study.

Abstract:
The neural radiance fields (NeRF) have emerged as a prominent methodology for synthesizing realistic images of novel views. While neural radiance representations based on voxels or mesh individually offer distinct advantages, excelling in either rendering quality or speed, each has limitations in the other aspect. In response, we propose a hybrid representation named Vosh, seamlessly combining both voxel and mesh components in hybrid rendering for view synthesis. Vosh is meticulously crafted by optimizing the voxel grid based on neural rendering, strategically meshing a portion of the volumetric density field to surface. Therefore, it excels in fast rendering scenes with simple geometry and textures through its mesh component, while simultaneously enabling high-quality rendering in intricate regions by leveraging voxel component. The flexibility of Vosh is showcased through the ability to adjust hybrid ratios, providing users the ability to control the balance between rendering quality and speed based on flexible usage. Experimental results demonstrate that our method achieves commendable trade-off between rendering quality and speed, and notably has real-time performance on mobile devices.

Abstract:
Large-scale text-to-image diffusion models have achieved great success in synthesizing high-quality and diverse images given target text prompts. Despite the revolutionary image generation ability, current state-of-the-art models still struggle to deal with multi-concept generation accurately in many cases. This phenomenon is known as “concept bleeding” and displays as the unexpected overlapping or merging of various concepts. This paper presents a general approach for text-to-image diffusion models to address the mutual interference between different subjects and their attachments in complex scenes, pursuing better text-image consistency. The core idea is to isolate the synthesizing processes of different concepts. We propose to bind each attachment to corresponding subjects separately with split text prompts. Besides, we introduce a revision method to fix the concept bleeding problem in multi-subject synthesis. We first depend on pre-trained object detection and segmentation models to obtain the layouts of subjects. Then we isolate and resynthesize each subject individually with corresponding text prompts to avoid mutual interference. Overall, we achieve a training-free strategy, named Isolated Diffusion, to optimize multi-concept text-to-image synthesis. It is compatible with the latest Stable Diffusion XL (SDXL) and prior Stable Diffusion (SD) models. We compare our approach with alternative methods using a variety of multi-concept text prompts and demonstrate its effectiveness with clear advantages in text-image consistency and user study.

Abstract:
Recently, learning multi-view neural surface reconstruction with the supervision of point clouds or depth maps has been a promising way. However, due to weak perception and underutilization of prior information, current methods still struggle with the challenges of limited accuracy and excessive time complexity. In addition, prior data perturbation is also an important yet rarely considered issue, often resulting in distorted geometry. To address these challenges, we propose a novel point-guided method named PG-NeuS, which achieves accurate and efficient reconstruction while robustly coping with point noise. Specifically, the aleatoric uncertainty of the point cloud is modeled to capture the noise distribution, estimating the reliability of each point and enhancing robustness against noise. Moreover, a Neural Projection module is proposed to connect points and images, adding geometric constraints to the implicit surface and achieving more precise point guidance. To better compensate for geometric bias between volume rendering and point modeling, we additionally design a Bias network that leverages the geometric information in high-fidelity points to enhance detail representation. Benefiting from the effective point guidance, the proposed PG-NeuS achieves an 11x speed increase and a 33.3% accuracy improvement compared to NeuS on DTU, even with a lightweight network. Extensive experiments show that our method yields high-quality surfaces with high efficiency, especially for fine-grained details and smooth regions, outperforming the state-of-the-art methods. Moreover, it exhibits strong robustness to noisy data and sparse data.

Abstract:
Eye tracking technology, essential for enhancing user experience in virtual reality (VR) and augmented reality (AR) devices, has been widely incorporated into advanced head-mounted devices like the Apple Vision Pro and PICO 4 Pro, becoming a standard feature. However, dedicated eye tracking datasets for such devices are severely lacking, with existing datasets commonly facing issues like camera skew and low resolution, particularly failing to adequately consider the diversity in wearing postures. To address this gap, we have developed the Posture-Variant Eye Tracking Dataset (PVEye), which includes 11,044,800 high-resolution near-eye images from 104 participants, showcasing a rich variety of wearing postures. This dataset aims to advance the development and application of appearance-based eye tracking methods. Utilizing this dataset, our evaluations demonstrate that the appearance-based method, particularly the NVGaze model, provides improved accuracy and robustness compared to the traditional feature-based method. Crucially, our experiments indicate that variations in wearing posture can significantly impact eye tracking performance, with posture-related errors contributing approximately 45% to the overall error variance. Moreover, the study delves into the specific impact of calibration and other critical factors on eye tracking performance, offering insights for further optimization of tracking effectiveness.

Abstract:
Transparency is commonly utilized in visualizations to overlay color-coded histograms or sets, thereby facilitating the visual comparison of categorical data. However, these charts often suffer from significant overlap between objects, resulting in substantial color interactions. Existing color blending models struggle in these scenarios, frequently leading to ambiguous color mappings and the introduction of false colors. To address these challenges, we propose an automated approach for generating optimal color encodings to enhance the perception of translucent charts. Our method harnesses color nameability to maximize the association between composite colors and their respective class labels. We introduce a color-name aware (CNA) optimization framework that generates maximally coherent color assignments and transparency settings while ensuring perceptual discriminability for all segments in the visualization. We demonstrate the effectiveness of our technique through crowdsourced experiments with composite histograms, showing how our technique can significantly outperform both standard and visualization-specific color blending models. Furthermore, we illustrate how our approach can be generalized to other visualizations, including parallel coordinates and Venn diagrams. We provide an open-source implementation of our technique as a web-based tool.

Abstract:
Registering 3D point clouds with low overlap is challenging in 3D computer vision, primarily due to difficulties in identifying small overlap regions and removing correspondence outliers. We observe that the neighborhood similarity can be utilized to detect point correspondence, and the consistent neighborhood correspondence can be used as a criterion to detect robust overlapping regions. So that a Double-layer Multi-scale Star-graph (DMS) structure is proposed to detect robust correspondences using two different types of multi-scale star-graphs. The first-layer Multi-scale Neighbor Feature Star-graphs (MNFS) takes each point as the center and its multi-scale nearest neighbors as the leaves. The MNFS enables to establish the initial correspondence candidate set between the two point clouds based on multi-scale neighborhood topology and feature similarity. Subsequently, each pair of corresponding points find their nearest neighbors within the correspondence sets to construct a Multi-scale Matching Star-graphs (MMS) on each side, so the mutual correspondence relationships between the MMS vertices are identified. These identified mutual correspondences are treated as vertices to construct the Multi-scale Correspondence Star-graphs (MCS), that indicate the relationships among the correspondences. We design edge weight and vertex weight criterion in MCS to detect only the robust correspondence set that has strong neighborhood consistency, so as to reject the outliers. Finally, the point cloud registration is conducted based on the detected robust correspondence. The experimental results demonstrate clearly that the proposed DMS method exhibits superior robustness when compared to existing state-of-the-art registration algorithms.

Abstract:
In this paper, we present a novel heuristic algorithm for the stable but NP-complete deformation-based edit distance on merge trees. Our key contribution is the introduction of a user-controlled look-ahead parameter that allows to trade off accuracy and computational cost. We achieve a fixed parameter tractable running time that is polynomial in the size of the input but exponential in the look-ahead value. This extension unlocks the potential of the deformation-based edit distance in handling saddle swaps, while maintaining feasible computation times. Experimental results demonstrate the computational efficiency and effectiveness of this approach in handling specific perturbations.

Abstract:
The proliferation of large language models (LLMs) has revolutionized the capabilities of natural language interfaces (NLIs) for data analysis. LLMs can perform multi-step and complex reasoning to generate data insights based on users’ analytic intents. However, these insights often entangle with an abundance of contexts in analytic conversations such as code, visualizations, and natural language explanations. This hinders efficient recording, organization, and navigation of insights within the current chat-based LLM interfaces. In this paper, we first conduct a formative study with eight data analysts to understand their general workflow and pain points of insight management during LLM-powered data analysis. Accordingly, we introduce InsightLens, an interactive system to overcome such challenges. Built upon an LLM-agent-based framework that automates insight recording and organization along with the analysis process, InsightLens visualizes the complex conversational contexts from multiple aspects to facilitate insight navigation. A user study with twelve data analysts demonstrates the effectiveness of InsightLens, showing that it significantly reduces users’ manual and cognitive effort without disrupting their conversational data analysis workflow, leading to a more efficient analysis experience.

Abstract:
Drawing on social influence and behavioral intention theories, coworkers' risk-taking serves as an “extra motive” —an exogenous factor—for risk-taking behaviors among workers in the workplace. Social influence theories have shown that social factors, such as social pressure and coworker risk-taking, may predict risk-taking behaviors and significantly affect decision-making. While immersive technologies have been widely used to create close-to-real simulations for construction safety-related studies, there is a paucity of research considering the impact of social presence in evaluating workers' risk decision-making within immersive environments. To bridge this gap, this study developed a state-of-the-art Augmented Virtuality (AV) environment to investigate roofers' risk-taking behaviors when exposed to social stressors (working alongside a safe/unsafe peer). In this augmented virtuality environment, a virtual peer with safe and unsafe behaviors was simulated in order to impose peer pressure and increase participants' sense of social presence. Participants were asked to install asphalt shingles on a physical section of a roof (passive haptics) while the rest of the environment was projected virtually. During shingle installation, participants' cognitive and behavioral responses were captured using psychophysiological wearable technologies and self-report measures. The results demonstrated that the developed AV model could successfully enhance participants' sense of presence and social presence while serving as an appropriate platform for assessing individuals' decision-making orientations and behavioral changes in the presence of social stressors. Such information shows the value of immersive technologies to examine the naturalistic responses of individuals without exposing them to actual risks.

Abstract:
We propose LAPIG, a language guided projector image generation method with surface adaptation and stylization. LAPIG consists of a projector-camera system and a target textured projection surface. LAPIG takes the user text prompt as input and aims to transform the surface style using the projector. LAPIG's key challenge is that due to the projector's physical brightness limitation and the surface texture, the viewer's perceived projection may suffer from color saturation and artifacts in both dark and bright regions, such that even with the state-of-the-art projector compensation techniques, the viewer may see clear surface texture-related artifacts. Therefore, how to generate a projector image that follows the user's instruction while also displaying minimum surface artifacts is an open problem. To address this issue, we propose projection surface adaptation (PSA) that can generate compensable surface stylization. We first train two networks to simulate the projector compensation and project-and-capture processes, this allows us to find a satisfactory projector image without real project-and-capture and utilize gradient descent for fast convergence. Then, we design content and saturation losses to guide the projector image generation, such that the generated image shows no clearly perceivable artifacts when projected. Finally, the generated image is projected for visually pleasing surface style morphing effects. The source code and more results are available on the project page: https://Yu-chen-Deng.github.io/LAPIG/.

Abstract:
The Point-and-Teleport locomotion in VR allows for instantaneous movement and helps minimize motion sickness. However, precise targeting can be difficult when the intended destination is obscured by an obstruction. This work presents Preview Teleport, an enhanced point-and-teleport technique that incorporates a preview window, enabling users to visualize the landing site behind obstructions in a seamless, single-step process. In a study of 28 participants, we investigated how window position, arc type, camera pose — defined as the three critical design aspects of the preview window — affect user performance. Additionally, we explored the impact of environment familiarity, simulated through a see-through effect. Our findings suggest Preview Teleport boosts efficiency, enhances spatial awareness and reduces fatigue, though it slightly compromises accuracy relative to methods without previews. This drop in accuracy is due to Preview Teleport's use of a preview window for directly targeting distant, obscured areas. In contrast, no-preview methods typically involve users first teleporting to a nearby visible location, then making a final adjustment to the obscured destination, allowing for more precise targeting. Within preview teleport designs, the window attachment to the controller, the camera pose at user-height, and the use of the standard parabolic-arc pointer are recommended. Notably, the rotated-align arc pointer demonstrates superior accuracy in targeting within familiar environments.

Abstract:
Virtual Reality (VR) technologies offer compelling experiences by allowing users to immerse themselves in simulated environments interacting through avatars. However, despite its ability to evoke emotional responses, and seeing ‘through the eyes’ of the displayed other, it remains unclear to what extent VR actually fosters perspective-taking (PT) or thinking about others' thoughts and feelings. It might be that the common belief that one can “become someone else” through VR is misleading, and that engaging situations through a different viewpoint does not produce a different cognitive standpoint. To test this, we conducted a 2 (perspective, first-person or third-person) by 2 (perspective-taking task or no task) to examine effects on perspective taking, measured via audio-recordings afforded by the think-aloud protocol. Our data demonstrate that while first-person perspective (1PP) facilitates perceived embodiment, it has no appreciable influence on perspective-taking. Regardless of 1PP or third-person perspective (3PP), perspective-taking was substantially and significantly increased when users were given a specific task prompting them to actively consider a character's perspective. Without such tasks, it seems that participants default to their own viewpoints. These data highlight the need for intentional design in VR experiences to consider content rather than simply viewpoint as key to authentic perspective-taking. To truly harness VR's potential as an “empathy machine,” developers must integrate targeted perspective-taking tasks or story prompts, ensuring that cognitive engagement is an active component of the experience.

Abstract:
In Virtual Reality (VR), existing hand-scale passive interaction techniques are unsuitable for continuous large-scale renders: room-scale proxies lack portability, and wearable robotic arms are energy-intensive and induce friction. This paper presents a technique for providing wall haptics in VR which supports portable, passive, and large-scale user interactions. We propose a potential solution, PantographHaptics, a technique which uses the scaling properties of a pantograph to passively render two-degree-of-freedom body-scale surfaces to overcome the limitations present in existing methods. We demonstrate PantographHaptics through two prototypes: HapticLever, a grounded system, and Feedbackpack, a wearable device. We evaluate these prototypes with technical and user evaluations. Our 9-participant first study compares HapticLever against traditional haptic modalities, while our 7-participant second study verifies Feedbackpack's usability and interaction fidelity.

Abstract:
Hand redirection, which subtly adjusts a user's hand movements in a virtual environment, can modify perception and movement by providing real-time corrections to motor feedback. In the context of motor learning and rehabilitation, observing replays of movements has been shown to enhance motor function. The application of hand redirection to these replays by making movements appear larger or smaller than they actually are has the potential to improve motor function. However, the detection threshold for hand redirection, specifically in the context of motion replays, remains unclear, as it has primarily been studied in real-time feedback settings. This study aims to determine the threshold at which hand redirection during post-exercise replay sessions becomes detectable. We conducted two psychophysical experiments to evaluate how much discrepancy between replayed and actual movements can go unnoticed by users, both with hand redirection (N=20) and without (N=18). Our findings reveal a tendency for the amount of movement during replay to be underestimated. Furthermore, compared to conventional real-time hand redirection without replay, replay manipulations involving redirection applied during the preceding reaching task resulted in a significantly larger JND. These insights are crucial for leveraging hand redirection techniques in replay-based motor learning applications.

Abstract:
Real camera footage is subject to noise, motion blur (MB) and depth of field (DoF). In some applications these might be considered distortions to be removed, but in others it is important to model them because it would be ineffective, or interfere with an aesthetic choice, to simply remove them. In augmented reality applications where virtual content is composed into a live video feed, we can model noise, MB and DoF to make the virtual content visually consistent with the video. Existing methods for this typically suffer two main limitations. First, they require a camera calibration step to relate a known calibration target to the specific cameras response. Second, existing work require methods that can be (differentiably) tuned to the calibration, such as slow and specialized neural networks. We propose a method which estimates parameters for noise, MB and DoF instantly, which allows using off-the-shelf real-time simulation methods from e.g., a game engine in compositing augmented content. Our main idea is to unlock both features by showing how to use modern computer vision methods that can remove noise, MB and DoF from the video stream, essentially providing self-calibration. This allows to auto-tune any black-box real-time nose+MB-DoF method to deliver fast and high-fidelity augmentation consistency.

Abstract:
This paper presents a Task-Free eye-tracking dataset for Dynamic Point Clouds (TF-DPC) aimed at investigating visual attention. The dataset is composed of eye gaze and head movements collected from 24 participants observing 19 scanned dynamic point clouds in a Virtual Reality (VR) environment with 6 degrees of freedom. We compare the visual saliency maps generated from this dataset with those from a prior task-dependent experiment (focused on quality assessment) to explore how high-level tasks influence human visual attention. To measure the similarity between these visual saliency maps, we apply the well-known Pearson correlation coefficient and an adapted version of the Earth Mover's Distance metric, which takes into account both spatial information and the degrees of saliency. Our experimental results provide both qualitative and quantitative insights, revealing significant differences in visual attention due to task influence. This work enhances our understanding of the visual attention for dynamic point cloud (specifically human figures) in VR from gaze and human movement trajectories, and highlights the impact of task-dependent factors, offering valuable guidance for advancing visual saliency models and improving VR perception.

Abstract:
Change detection (CD) is critical in everyday tasks. While current algorithmic approaches for CD are improving, they remain imprecise, often requiring human intervention. Cognitive science research focuses on understanding CD mechanisms, especially through change blindness studies. However, these do not address the primary requirement in real-life CD - detecting changes as effectively as possible. Such a requirement is directly relevant to the visual comparison field - studying visualisation techniques to compare data and identify differences or changes effectively. Recent studies have used Virtual Reality (VR) to improve visual comparison by providing an immersive platform where users can interact with 3D data at a real-life scale, enhancing spatial reasoning. We believe VR could also improve CD performance accordingly. Particularly, VR offers stereoscopic depth perception over traditional displays, potentially enhancing the detection of spatial change. In this paper, we develop and analyse three 3D visual comparison techniques for CD in VR: Sliding Window, 3D Slider, and Switch Back. These techniques are evaluated under synthetic but realistic environments and frequently occurring Perceptual Challenges, including different Changed Object Size, Lighting Variation, and Scene Drift conditions. Experimental results reveal significant differences between the techniques in detection time measures and subjective user experience.

Abstract:
The Proteus effect describes how users of virtual environments adjust their attitudes to match stereotypes associated with their avatar's appearance. While numerous studies have demonstrated this phenomenon's reliability, its underlying processes remain poorly understood. This work investigates deindividuation's hypothesized but unproven role within the Proteus effect. Deindividuated individuals tend to follow situational norms rather than personal ones. Therefore, together with high embodiment and identification processes, deindividuation may lead to a stronger Proteus effect. We present two experimental studies. First, we demonstrated the emergence of the Proteus effect in a real-world academic context: engineering students got better scores in a statistical task when embodying Albert Einstein's avatar compared to a control one. In the second study, we tested the role of deindividuation by manipulating participants' exposure to different identity cues during the task. While we could not find a significant effect of deindividuation on the participants' performance, our results highlight an unexpected pattern, with embodiment as a negative predictor and identification as a positive predictor of performance. These results open avenues for further research on the processes involved in the Proteus effect, particularly those focused on the relation between the avatar and the nature of the task to be performed. All supplemental materials are available at https://osf.io/au3wk/.

Abstract:
Virtual Reality (VR) offers a unique collaborative experience, with parallel views playing a pivotal role in Collaborative Virtual Environments by supporting the transfer and delivery of items. Sharing and manipulating partners' views provides users with a broader perspective that helps them identify the targets and partner actions. We proposed TeamPortal accordingly and conducted two user studies with 72 participants (36 pairs) to investigate the potential benefits of interactive, shared perspectives in VR collaboration. Our first study compared ShaView and TeamPortal against a baseline in a collaborative task that encompassed a series of searching and manipulation tasks. The results show that TeamPortal significantly reduced movement and increased collaborative efficiency and social presence in complex tasks. Following the results, the second study evaluated three variants: TeamPortal+, SnapTeamPortal+, and DropTeamPortal+. The results show that both SnapTeamPortal+ and DropTeamPortal+ improved task efficiency and willingness to further adopt these technologies, though SnapTeamPortal+ reduced co-presence. Based on the findings, we proposed three design implications to inform the development of future VR collaboration systems.

Abstract:
The growing availability of affordable Virtual Reality (VR) hardware and the increasing interest in the Metaverse are driving the expansion of Social VR (SVR) platforms. These platforms allow users to embody avatars in immersive social virtual environments, enabling real-time interactions using consumer devices. Beyond merely replicating real-life social dynamics, SVR platforms offer opportunities to surpass real-world constraints by augmenting these interactions. One example of such augmentation is Artificial Facial Mimicry (AFM), which holds significant potential to enhance social experiences. Mimicry, the unconscious imitation of verbal and non-verbal behaviors, has been shown to positively affect human-agent interactions, yet its role in avatar-mediated human-to-human communication remains under-explored. AFM presents various possibilities, such as amplifying emotional expressions, or substituting one emotion for another to better align with the context. Furthermore, AFM can address the limitations of current facial tracking technologies in fully capturing users' emotions. To investigate the potential benefits of AFM in SVR, an automated AM system was developed. This system provides AFM, along with other kinds of head mimicry (nodding and eye contact), and it is compatible with consumer VR devices equipped with facial tracking. This system was deployed within a test-bench immersive SVR application. A between-dyads user study was conducted to assess the potential benefits of AFM for interpersonal communication while maintaining avatar behavioral naturalness, comparing the experiences of pairs of participants communicating with AFM enabled against a baseline condition. Subjective measures revealed that AFM improved interpersonal closeness, aspects of social attraction, interpersonal trust, social presence, and naturalness compared to the baseline condition. These findings demonstrate AFM's positive impact on key aspects of social interaction and highlight its potential applications across various SVR domains.

Abstract:
The real-time generation of editable mesh animations in XR applications has been a focal point of research in the XR field. However, easily controlling the generated editable meshes remains a significant challenge. Existing methods often suffer from slow generation speeds and suboptimal results, failing to accurately simulate target objects' complex details and shapes, which does not meet user expectations. Additionally, the final generated meshes typically require manual user adjustments, and it is difficult to generate multiple target models simultaneously. To overcome these limitations, a universal control scheme for particles based on the sampling features of the target is proposed. It introduces a spatially adaptive control algorithm for particle coupling by adjusting the magnitude of control forces based on the spatial features of model sampling, thereby eliminating the need for parameter dependency and enabling the control of multiple types of models within the same scene. We further introduce boundary correction techniques to improve the precision in generating target shapes while reducing particle splashing. Moreover, a distance-adaptive particle fragmentation mechanism prevents unnecessary particle accumulation. Experimental results demonstrate that the method has better performance in controlling complex structures and generating multiple targets at the same time compared to existing methods. It enhances control accuracy for complex structures and targets under the condition of sparse model sampling. It also consistently delivers outstanding results while maintaining high stability and efficiency. Ultimately, we were able to create a set of smooth editable meshes and developed a solution for integrating this algorithm into VR and AR animation applications.

Abstract:
Navigating topographic charts involves nuanced skills and real-world correspondence, influenced by individual learning styles. Traditionally, 2D and 3D representations bridge map-reality gaps, but sandboxes often introduce manual errors and reliance on personal interpretation. This paper examines VR's potential in military planning, focusing on terrain interpretation, data visualization, and scale transitions. A study with 36 army cadets investigates VR's effectiveness in enhancing spatial perception and real-world task performance. Findings suggest that VR improves position choices and grades and reduces result disparities, particularly benefiting users with lower spatial skills. The research also evaluates the impact of different scales on VR planning, offering insights into potential advantages and challenges. Future studies should explore control issues and completion times in real-world scenarios.

Abstract:
Visual guidance (VG) plays an essential role in directing user attention in virtual reality (VR) and augmented reality (AR) environments. However, traditional approaches rely on explicit visual annotations, which often compromise visual clarity and increase user cognitive load. To address this issue, we propose an unobtrusive VG technique based on color vibration, a phenomenon in which rapidly alternating colors at frequencies above 25 Hz are perceived as a single intermediate color. Our work explores a perceptual state that exists between complete color fusion and visible flicker, where color differences remain detectable without conscious awareness of vibration. Through two experimental studies, we first identified the thresholds separating complete fusion, this intermediate perceptual state, and visible flicker by systematically varying color vibration parameters. Subsequently, we applied color vibrations with derived thresholds to natural image regions and validated their attention-guiding capabilities using eye-tracking measurements. The results demonstrate that controlled color vibration successfully directs user attention while maintaining low cognitive demand, providing an effective method for implementing unobtrusive VG in VR and AR systems.

Abstract:
Hydropower turbines are large-scale equipment essential to sustainable energy supply chains, and engineers have few opportunities to examine their internal structure. Our Immersive Analytics (IA) application is part of a research project that combines and compares simulated water turbine flows and sensor-measured data, looking for data-driven predictions of the lifetime of the mechanical parts of hydroelectric power plants. Our prototype combines spatial and abstract data in an immersive environment in which the user can navigate through a full-scale model of a water turbine, view simulated water flows of three different energy supply conditions, and visualize and interact with sensor-collected data situated at the reference position of the sensors in the actual turbine. In this paper, we detail our design process, which resulted from consultations with domain experts and a literature review, give an overview of our prototype, and present its evaluation, resulting from semi-structured interviews with experts and qualitative thematic analysis. Our findings confirm the current literature that IA applications add value to the presentation and analysis of situated data, as they show that we advance in the design directions for IA applications for domain experts that combine abstract and spatial data, with conclusions on how to avoid skepticism from such professionals.

Abstract:
With the continuous advancement of artificial intelligence technology, data-driven methods for reconstructing and animating virtual agents have achieved increasing levels of realism. However, there is limited research on how these novel data-driven methods, combined with voice cues, affect user perceptions. We use advanced data-driven methods to reconstruct stylized agents and combine them with synthesized voices to study their effects on users' trust and other perceptions (e.g. social presence and empathy). Through an experiment with 27 participants, our findings reveal that stylized virtual agents enhance user trust to a degree comparable to real style, while voice has a negligible effect on trust. Additionally, elder agents are more likely to be trusted. The style of the agents also plays a key role in participants' perceived realism, and audio-visual matching significantly enhances perceived empathy. These results provide new insights into designing trustworthy virtual agents and further support and validate the audio-visual integration theory.

Abstract:
With the rise of short-form video platforms and the increasing availability of data, we see the potential for people to share short-form videos embedded with data in situ (e.g., daily steps when running) to increase the credibility and expressiveness of their stories. However, creating and sharing such videos in situ is challenging since it involves multiple steps and skills (e.g., data visualization creation and video editing), especially for amateurs. By conducting a formative study (N=10) using three design probes, we collected the motivations and design requirements. We then built VisTellAR, a mobile AR authoring tool, to help amateur video creators embed data visualizations in short-form videos in situ. A two-day user study shows that participants (N=12) successfully created various videos with data visualizations in situ and they confirmed the ease of use and learning. AR pre-stage authoring was useful to assist people in setting up data visualizations in reality with more designs in camera movements and interaction with gestures and physical objects to storytelling.

Abstract:
In various scientific and industrial domains, analyzing multivariate spatial data, i.e., vectors associated with spatial locations, is common practice. To analyze those datasets, analysts may turn to methods such as Spatial Blind Source Separation (SBSS). Designed explicitly for spatial data analysis, SBSS finds latent components in the dataset and is superior to popular non-spatial methods, like PCA. However, when analysts try different tuning parameter settings, the amount of latent components complicates analytical tasks. Based on our years-long collaboration with SBSS researchers, we propose a visualization approach to tackle this challenge. The main component is UnDRground Tubes (UT), a general-purpose idiom combining ideas from set visualization and multidimensional projections. We describe the UT visualization pipeline and integrate UT into an interactive multiple-view system. We demonstrate its effectiveness through interviews with SBSS experts, a qualitative evaluation with visualization experts, and computational experiments. SBSS experts were excited about our approach. They saw many benefits for their work and potential applications for geostatistical data analysis more generally. UT was also well received by visualization experts. Our benchmarks show that UT projections and its heuristics are appropriate.

Abstract:
Shape is commonly used to distinguish between categories in multi-class scatterplots. However, existing guidelines for choosing effective shape palettes rely largely on intuition and do not consider how these needs may change as the number of categories increases. Unlike color, shapes can not be represented by a numerical space, making it difficult to propose general guidelines or design heuristics for using shape effectively. This paper presents a series of four experiments evaluating the efficiency of 39 shapes across three tasks: relative mean judgment tasks, expert preference, and correlation estimation. Our results show that conventional means for reasoning about shapes, such as filled versus unfilled, are insufficient to inform effective palette design. Further, even expert palettes vary significantly in their use of shape and corresponding effectiveness. To support effective shape palette design, we developed a model based on pairwise relations between shapes in our experiments and the number of shapes required for a given design. We embed this model in a palette design tool to give designers agency over shape selection while incorporating empirical elements of perceptual performance captured in our study. Our model advances understanding of shape perception in visualization contexts and provides practical design guidelines that can help improve categorical data encodings.

Abstract:
Evaluating large language models (LLMs) presents unique challenges. While automatic side-by-side evaluation, also known as LLM-as-a-judge, has become a promising solution, model developers and researchers face difficulties with scalability and interpretability when analyzing these evaluation outcomes. To address these challenges, we introduce LLM Comparator, a new visual analytics tool designed for side-by-side evaluations of LLMs. This tool provides analytical workflows that help users understand when and why one LLM outperforms or underperforms another, and how their responses differ. Through close collaboration with practitioners developing LLMs at Google, we have iteratively designed, developed, and refined the tool. Qualitative feedback from these users highlights that the tool facilitates in-depth analysis of individual examples while enabling users to visually overview and flexibly slice data. This empowers users to identify undesirable patterns, formulate hypotheses about model behavior, and gain insights for model improvement. LLM Comparator has been integrated into Google's LLM evaluation platforms and open-sourced.

Abstract:
We introduce DiffFit, a differentiable algorithm for fitting protein atomistic structures into an experimental reconstructed Cryo-Electron Microscopy (cryo-EM) volume map. In structural biology, this process is necessary to semi-automatically composite large mesoscale models of complex protein assemblies and complete cellular structures that are based on measured cryo-EM data. The current approaches require manual fitting in three dimensions to start, resulting in approximately aligned structures followed by an automated fine-tuning of the alignment. The DiffFit approach enables domain scientists to fit new structures automatically and visualize the results for inspection and interactive revision. The fitting begins with differentiable three-dimensional (3D) rigid transformations of the protein atom coordinates followed by sampling the density values at the atom coordinates from the target cryo-EM volume. To ensure a meaningful correlation between the sampled densities and the protein structure, we proposed a novel loss function based on a multi-resolution volume-array approach and the exploitation of the negative space. This loss function serves as a critical metric for assessing the fitting quality, ensuring the fitting accuracy and an improved visualization of the results. We assessed the placement quality of DiffFit with several large, realistic datasets and found it to be superior to that of previous methods. We further evaluated our method in two use cases: automating the integration of known composite structures into larger protein complexes and facilitating the fitting of predicted protein domains into volume densities to aid researchers in identifying unknown proteins. We implemented our algorithm as an open-source plugin (github.com/nanovis/DiffFit) in ChimeraX, a leading visualization software in the field. All supplemental materials are available at osf. io/5tx4q.

Abstract:
In volume visualization, visualization synthesis has attracted much attention due to its ability to generate novel visualizations without following the conventional rendering pipeline. However, existing solutions based on generative adversarial networks often require many training images and take significant training time. Still, issues such as low quality, consistency, and flexibility persist. This paper introduces StyleRF-VolVis, an innovative style transfer framework for expressive volume visualization (VolVis) via neural radiance field (NeRF). The expressiveness of StyleRF-VolVis is upheld by its ability to accurately separate the underlying scene geometry (i.e., content) and color appearance (i.e., style), conveniently modify color, opacity, and lighting of the original rendering while maintaining visual content consistency across the views, and effectively transfer arbitrary styles from reference images to the reconstructed 3D scene. To achieve these, we design a base NeRF model for scene geometry extraction, a palette color network to classify regions of the radiance field for photorealistic editing, and an unrestricted color network to lift the color palette constraint via knowledge distillation for non-photorealistic editing. We demonstrate the superior quality, consistency, and flexibility of StyleRF-VolVis by experimenting with various volume rendering scenes and reference images and comparing StyleRF-VolVis against other image-based (AdaIN), video-based (ReReVST), and NeRF-based (ARF and SNeRF) style rendering solutions.

Abstract:
Onboarding a user to a visualization dashboard entails explaining its various components, including the chart types used, the data loaded, and the interactions available. Authoring such an onboarding experience is time-consuming and requires significant knowledge and little guidance on how best to complete this task. Depending on their levels of expertise, end users being onboarded to a new dashboard can be either confused and overwhelmed or disinterested and disengaged. We propose interactive dashboard tours (D-Tours) as semi-automated onboarding experiences that preserve the agency of users with various levels of expertise to keep them interested and engaged. Our interactive tours concept draws from open-world game design to give the user freedom in choosing their path through onboarding. We have implemented the concept in a tool called D-TOUR Prototype, which allows authors to craft custom interactive dashboard tours from scratch or using automatic templates. Automatically generated tours can still be customized to use different media (e.g., video, audio, and highlighting) or new narratives to produce an onboarding experience tailored to an individual user. We demonstrate the usefulness of interactive dashboard tours through use cases and expert interviews. Our evaluation shows that authors found the automation in the D-Tour Prototype helpful and time-saving, and users found the created tours engaging and intuitive. This paper and all supplemental materials are available at https://osf.io/6fbjp/.

Abstract:
People commonly utilize visualizations not only to examine a given dataset, but also to draw generalizable conclusions about the underlying models or phenomena. Prior research has compared human visual inference to that of an optimal Bayesian agent, with deviations from rational analysis viewed as problematic. However, human reliance on non-normative heuristics may prove advantageous in certain circumstances. We investigate scenarios where human intuition might surpass idealized statistical rationality. In two experiments, we examine individuals' accuracy in characterizing the parameters of known data-generating models from bivariate visualizations. Our findings indicate that, although participants generally exhibited lower accuracy compared to statistical models, they frequently outperformed Bayesian agents, particularly when faced with extreme samples. Participants appeared to rely on their internal models to filter out noisy visualizations, thus improving their resilience against spurious data. However, participants displayed overconfidence and struggled with uncertainty estimation. They also exhibited higher variance than statistical machines. Our findings suggest that analyst gut reactions to visualizations may provide an advantage, even when departing from rationality. These results carry implications for designing visual analytics tools, offering new perspectives on how to integrate statistical models and analyst intuition for improved inference and decision-making. The data and materials for this paper are available at https://osf.io/qmfv6

Abstract:
Providing effective guidance for users has long been an important and challenging task for efficient exploratory visual analytics, especially when selecting variables for visualization in high-dimensional datasets. Correlation is the most widely applied metric for guidance in statistical and analytical tools, however a reliance on correlation may lead users towards false positives when interpreting causal relations in the data. In this work, inspired by prior insights on the benefits of counterfactual visualization in supporting visual causal inference, we propose a novel, simple, and efficient counterfactual guidance method to enhance causal inference performance in guided exploratory analytics based on insights and concerns gathered from expert interviews. Our technique aims to capitalize on the benefits of counterfactual approaches while reducing their complexity for users. We integrated counterfactual guidance into an exploratory visual analytics system, and using a synthetically generated ground-truth causal dataset, conducted a comparative user study and evaluated to what extent counterfactual guidance can help lead users to more precise visual causal inferences. The results suggest that counterfactual guidance improved visual causal inference performance, and also led to different exploratory behaviors compared to correlation-based guidance. Based on these findings, we offer future directions and challenges for incorporating counterfactual guidance to better support exploratory visual analytics.

Abstract:
Multi-objective evolutionary algorithms (MOEAs) have emerged as powerful tools for solving complex optimization problems characterized by multiple, often conflicting, objectives. While advancements have been made in computational efficiency as well as diversity and convergence of solutions, a critical challenge persists: the internal evolutionary mechanisms are opaque to human users. Drawing upon the successes of explainable AI in explaining complex algorithms and models, we argue that the need to understand the underlying evolutionary operators and population dynamics within MOEAs aligns well with a visual analytics paradigm. This paper introduces ParetoTracker, a visual analytics framework designed to support the comprehension and inspection of population dynamics in the evolutionary processes of MOEAs. Informed by preliminary literature review and expert interviews, the framework establishes a multi-level analysis scheme, which caters to user engagement and exploration ranging from examining overall trends in performance metrics to conducting fine-grained inspections of evolutionary operations. In contrast to conventional practices that require manual plotting of solutions for each generation, ParetoTracker facilitates the examination of temporal trends and dynamics across consecutive generations in an integrated visual interface. The effectiveness of the framework is demonstrated through case studies and expert interviews focused on widely adopted benchmark optimization problems.

Abstract:
Referential gestures, or as termed in linguistics, deixis, are an essential part of communication around data visualizations. Despite their importance, such gestures are often overlooked when documenting data analysis meetings. Transcripts, for instance, fail to capture gestures, and video recordings may not adequately capture or emphasize them. We introduce a novel method for documenting collaborative data meetings that treats deixis as a first-class citizen. Our proposed framework captures cursor-based gestural data along with audio and converts them into interactive documents. The framework leverages a large language model to identify word correspondences with gestures. These identified references are used to create context-based annotations in the resulting interactive document. We assess the effectiveness of our proposed method through a user study, finding that participants preferred our automated interactive documentation over recordings, transcripts, and manual note-taking. Furthermore, we derive a preliminary taxonomy of cursor-based deictic gestures from participant actions during the study. This taxonomy offers further opportunities for better utilizing cursor-based deixis in collaborative data analysis scenarios.

Abstract:
Visualizing relational data is crucial for understanding complex connections between entities in social networks, political affiliations, or biological interactions. Well-known representations like node-link diagrams and adjacency matrices offer valuable insights, but their effectiveness relies on the ability to identify patterns in the underlying topological structure. Reordering strategies and layout algorithms play a vital role in the visualization process since the arrangement of nodes, edges, or cells influences the visibility of these patterns. The BioFabric visualization combines elements of node-link diagrams and adjacency matrices, leveraging the strengths of both, the visual clarity of node-link diagrams and the tabular organization of adjacency matrices. A unique characteristic of BioFabric is the possibility to reorder nodes and edges separately. This raises the question of which combination of layout algorithms best reveals certain patterns. In this paper, we discuss patterns and anti-patterns in BioFabric, such as staircases or escalators, relate them to already established patterns, and propose metrics to evaluate their quality. Based on these quality metrics, we compared combinations of well-established reordering techniques applied to BioFabric with a well-known benchmark data set. Our experiments indicate that the edge order has a stronger influence on revealing patterns than the node layout. The results show that the best combination for revealing staircases is a barycentric node layout, together with an edge order based on node indices and length. Our research contributes a first building block for many promising future research directions, which we also share and discuss. A free copy of this paper and all supplemental materials are available at https://osf.io/9mt8r/?view_only=b7t0dfbe550e3404f83059afdc60184c6.

Abstract:
Visualization items—factual questions about visualizations that ask viewers to accomplish visualization tasks-are regularly used in the field of information visualization as educational and evaluative materials. For example, researchers of visualization literacy require large, diverse banks of items to conduct studies where the same skill is measured repeatedly on the same participants. Yet, generating a large number of high-quality, diverse items requires significant time and expertise. To address the critical need for a large number of diverse visualization items in education and research, this paper investigates the potential for large language models (LLMS) to automate the generation of multiple-choice visualization items. Through an iterative design process, we develop the VILA (Visualization Items Generated by Large LAnguage Models) pipeline, for efficiently generating visualization items that measure people's ability to accomplish visualization tasks. We use the VILA pipeline to generate 1,404 candidate items across 12 chart types and 13 visualization tasks. In collaboration with 11 visualization experts, we develop an evaluation rulebook which we then use to rate the quality of all candidate items. The result is the VILA bank of ~1, 100 items. From this evaluation, we also identify and classify current limitations of the VILA pipeline, and discuss the role of human oversight in ensuring quality. In addition, we demonstrate an application of our work by creating a visualization literacy test, VILA-VLAT, which measures people's ability to complete a diverse set of tasks on various types of visualizations; comparing it to the existing VLAT, VILA-VLAT shows moderate to high convergent validity (R = 0.70). Lastly, we discuss the application areas of the VILA pipeline and the VILA bank and provide practical recommendations for their use. All supplemental materials are available at https://osf.io/ysrhq/.

Abstract:
Understanding the input and output of data wrangling scripts is crucial for various tasks like debugging code and onboarding new data. However, existing research on script understanding primarily focuses on revealing the process of data transformations, lacking the ability to analyze the potential scope, i.e., the space of script inputs and outputs. Meanwhile, constructing input/output space during script analysis is challenging, as the wrangling scripts could be semantically complex and diverse, and the association between different data objects is intricate. To facilitate data workers in understanding the input and output space of wrangling scripts, we summarize ten types of constraints to express table space and build a mapping between data transformations and these constraints to guide the construction of the input/output for individual transformations. Then, we propose a constraint generation model for integrating table constraints across multiple transformations. Based on the model, we develop Ferry, an interactive system that extracts and visualizes the data constraints describing the input and output space of data wrangling scripts, thereby enabling users to grasp the high-level semantics of complex scripts and locate the origins of faulty data transformations. Besides, Ferry provides example input and output data to assist users in interpreting the extracted constraints and checking and resolving the conflicts between these constraints and any uploaded dataset. Ferry's effectiveness and usability are evaluated through two usage scenarios and two case studies, including understanding, debugging, and checking both single and multiple scripts, with and without executable data. Furthermore, an illustrative application is presented to demonstrate Ferry's flexibility.

Abstract:
In the biomedical domain, visualizing the document embeddings of an extensive corpus has been widely used in information-seeking tasks. However, three key challenges with existing visualizations make it difficult for clinicians to find information efficiently. First, the document embeddings used in these visualizations are generated statically by pretrained language models, which cannot adapt to the user's evolving interest. Second, existing document visualization techniques cannot effectively display how the documents are relevant to users' interest, making it difficult for users to identify the most pertinent information. Third, existing embedding generation and visualization processes suffer from a lack of interpretability, making it difficult to understand, trust and use the result for decision-making. In this paper, we present a novel visual analytics pipeline for user-driven document representation and iterative information seeking (VADIS). VADIS introduces a prompt-based attention model (PAM) that generates dynamic document embedding and document relevance adjusted to the user's query. To effectively visualize these two pieces of information, we design a new document map that leverages a circular grid layout to display documents based on both their relevance to the query and the semantic similarity. Additionally, to improve the interpretability, we introduce a corpus-level attention visualization method to improve the user's understanding of the model focus and to enable the users to identify potential oversight. This visualization, in turn, empowers users to refine, update and introduce new queries, thereby facilitating a dynamic and iterative information-seeking experience. We evaluated VADIS quantitatively and qualitatively on a real-world dataset of biomedical research papers to demonstrate its effectiveness.

Abstract:
The widespread use of Deep Neural Networks (DNNs) has recently resulted in their application to challenging scientific visualization tasks. While advanced DNNs demonstrate impressive generalization abilities, understanding factors like prediction quality, confidence, robustness, and uncertainty is crucial. These insights aid application scientists in making informed decisions. However, DNNs lack inherent mechanisms to measure prediction uncertainty, prompting the creation of distinct frameworks for constructing robust uncertainty-aware models tailored to various visualization tasks. In this work, we develop uncertainty-aware implicit neural representations to model steady-state vector fields effectively. We comprehensively evaluate the efficacy of two principled deep uncertainty estimation techniques: (1) Deep Ensemble and (2) Monte Carlo Dropout, aimed at enabling uncertainty-informed visual analysis of features within steady vector field data. Our detailed exploration using several vector data sets indicate that uncertainty-aware models generate informative visualization results of vector field features. Furthermore, incorporating prediction uncertainty improves the resilience and interpretability of our DNN model, rendering it applicable for the analysis of non-trivial vector field data sets.

Abstract:
Annotations are widely used in information visualization to guide attention, clarify patterns, and support interpretation. We present a comprehensive survey of 191 research articles describing empirical studies, tools, techniques, and systems that incorporate annotations across various visualization contexts. Based on a structured analysis, we characterize annotations by their types, generation methods, and targets, and examine their use across four primary application domains: user engagement, storytelling, collaboration, and exploratory data analysis. We also discuss key trends, practical challenges, and open research directions. These findings offer a foundation for designing more effective annotation systems and advancing future research on annotation in visualization.

Abstract:
In this paper, we propose p-Blend, an efficient and effective blendshape perturbation mechanism designed to defend against both intra- and cross-app re-identification attacks in virtual reality. p-Blend provides privacy protection when streaming blendshape data to third-party applications on VR devices. In its design, we consider both privacy and utility. p-Blend not only perturbs blendshape values to resist re-identification attacks but also preserves the smoothness of facial animations and the naturalness of facial expressions, ensuring the continued usability of the data. We validate the effectiveness of p-Blend through extensive empirical evaluations and user studies. Quantitative experiments on a large-scale dataset collected from 45 participants demonstrate that p-Blend significantly reduces re-identification accuracy across a range of machine learning models. While pure-random perturbation fails to prevent attacks that exploit statistical features, p-Blend effectively mitigates these risks in both raw and statistical blendshape data. Additionally, user study results show that facial animations generated from p-Blend-perturbed blendshapes maintain greater smoothness and naturalness compared to those using purely random perturbation. The codes and dataset are available at https://github.com/jingwei1016/p-Blend.

Abstract:
Image-based scene understanding allows Augmented Reality (AR) systems to provide contextual visual guidance in unprepared, real-world environments. While effective on video see-through (VST) head-mounted displays (HMDs), such methods suffer on optical see-through (OST) HMDs due to misregistration between the world-facing camera and the user's eye perspective. To approximate the user's true eye view, we implement and evaluate three software-based eye-perspective rendering (EPR) techniques on a commercially available, untethered OST HMD (Microsoft HoloLens 2): (1) Plane-Proxy EPR, projecting onto a fixed-distance plane; (2) Mesh-Proxy EPR, using SLAM-based reconstruction for projection; and (3) Gaze-Proxy EPR, a novel eye-tracking-based method that aligns the projection with the user's gaze depth. A user study on real-world tasks underscores the importance of accurate EPR and demonstrates gaze-proxy as a lightweight alternative to geometry-based methods. We release our EPR framework as open source.

Abstract:
As virtual reality (VR) continues to expand, particularly in social VR platforms and immersive gaming environments, understanding the factors that shape user experience is becoming increasingly important. Avatars and locomotion methods play central roles in influencing how users identify with their virtual representations and navigate virtual spaces. Despite extensive research on these elements individually, their relationship remains underexplored. In particular, little is known about how congruence between avatar appearance and locomotion method affects user perceptions. This study investigates the impact of avatar-locomotion congruence on user experience and avatar identification in VR. We conducted a within-subjects experiment with 30 participants, employing two visually distinct avatar types (human and gorilla) and two locomotion methods (human-like arm-swinging and gorilla-like arm-rolling), to assess their individual and combined effects. Our results indicate that congruence between avatar appearance and locomotion method enhances both avatar identification and user experience. These findings contribute to the understanding of the relationship between avatars and locomotion in VR, with potential applications in enhancing user experience in immersive gaming, social VR, and gamified remote physical therapy.

Abstract:
Flood risk communication in disaster-prone communities often relies on traditional tools (e.g., paper and browser-based hazard/flood maps) that struggle to engage community stakeholders and reflect intuitive flood situations. In this paper, we applied the transitional mixed reality (MR) interface concept from pioneering work and extended it for flood risk communication scenarios through co-design with community stakeholders to help vulnerable residents understand flood risk and facilitate preparedness. Starting with an initial transitional MR prototype, we conducted three iterative workshops - each dedicated to device usability, visualization techniques, and interaction methods. We collaborated with diverse community stakeholders in flood-prone areas, collecting feedback to refine the system according to community needs. Our preliminary evaluation indicates that this co-designed system significantly improves user understanding and engagement compared to traditional tools, though some older residents faced usability challenges. We detailed this iterative co-design process, critical insights and design implications, offering our work as a practical case of mixed reality application in strengthening flood risk communication. We also discuss the system's potential to support community-driven collaboration in flood preparedness.

Abstract:
Virtual reality (VR) is increasingly recognized as a powerful tool for science education, offering interactive environments to explore intangible concepts. Traditional teaching methods often struggle to convey abstract concepts in science, where many phenomena are not directly observable. VR can address this issue by modeling and visualizing complex and unobservable entities and processes, allowing learners to dynamically interact with what would otherwise not be directly perceptible. However, relatively few controlled studies have compared immersive VR learning with equivalent hands-on laboratory learning in physics education, particularly for more abstract topics. In this work, we designed a VR-based physics lab that is capable of visualizing electrons and electromagnetic fields to teach fundamental concepts of electronics and magnetism, closely replicating a traditional electronics learning kit used as a baseline for comparison. We evaluated the impact of the two conditions (VR versus traditional) on students' learning outcomes, motivation, engagement, and cognitive load. Our results show significantly higher knowledge retention in the VR group compared to the traditional group. Also, while there were no significant differences in immediate comprehension between the two groups, participants in the VR group spent substantially more time engaged with the learning content. These findings highlight the potential of visually enriched virtual environments to enhance the learning experience and improve knowledge retention of intangible scientific concepts.

Abstract:
Embodied conversational agents (ECAs) capable of non-verbal behaviors have been developed to address the limitations of voice-only assistants, with research exploring their use in mixed and augmented reality (AR), suggesting they may soon interact with us more naturally in physical spaces. Traditionally, AI voice assistants are activated through wake-up keywords, and since they are invisible, their method of appearance has not been a concern. However, for ECAs in AR, the question of how they should enter the user's space when summoned remains underexplored. In this paper, we focused on the plausibility of ECAs' entering action into the user's field of view in AR. We analyzed its impact on user experience, concentrating on perceived social presence and co-presence of the agent. Three entrance styles were chosen for comparison: an obviously impossible one, a possible one, and an intermediate one, alongside a voice-only condition. We designed and conducted a within-subjects study with 38 participants. Our results indicated that while the plausibility of the action had less impact on functionality compared to the embodiment itself, it significantly affected social/co-presence. These findings highlight the importance of entrance design for future AR agent experiences.

Abstract:
Extended Reality (XR) has the potential to improve efficiency and safety in the user-centered development of human-robot interaction. However, the validity of using XR prototyping for user studies for contact-intensive robotic tasks remains underexplored. These in-contact tasks are particularly relevant due to challenges arising from indirect force perception in robot control. Therefore, in this work, we investigate a representative example of such a task: robotic ultrasound. A user study was conducted to assess the transferability of results from a simulated user study to real-world conditions, comparing two force-assistance approaches. The XR simulation replicates the physical study set-up employing a virtual robotic arm, its control interface, ultrasound imaging, and two force-assistance methods: automation and force visualization. Our results indicate that while differences in force deviation, perceived workload, and trust emerge between real and simulated setups, the overall findings remain consistent. Specifically, partial automation of robot control improves performance and trust while reducing workload, and visual feedback decreases force deviation in both real and simulated conditions. These findings highlight the potential of XR for comparative studies, even in complex robotic tasks.

Abstract:
The increasing interest surrounding 3D human heads for digital avatars and simulations has highlighted the need for accurate internal modeling rather than solely focusing on external approximations. Existing approaches rely on traditional optimization techniques applied to explicit 3D representations like point clouds and meshes, leading to computational inefficiencies and challenges in capturing local geometric features. To tackle these problems, we propose a novel modeling method called DDF-ISM. It leverages a probabilistic Directed Distance Field for Internal Structure Modeling, facilitating efficient and anatomically accurate deformation of different parts of the human head. DDF-ISM comprises two key components: 1) a probabilistic DDF network for implicit representation of the target model to provide crucial local geometric information, and 2) a conditioned deformation network guided by the local geometry. Additionally, we introduce a large-scale dataset of human heads with internal structures derived from high-quality Computed Tomography (CT) scans, along with well-designed template models encompassing skull, mandible, brain, and head surface. Evaluation on this dataset showcases the superiority of our approach over existing methods, exhibiting superior performance in both modeling quality and efficiency.

Abstract:
Large language models (LLMs) have gained widespread popularity due to their ability to perform ad-hoc natural language processing (NLP) tasks with simple natural language prompts. Part of the appeal for LLMs is their approachability to the general public, including individuals with little technical expertise in NLP. However, prompts can vary significantly in terms of their linguistic structure, context, and other semantics, and modifying one or more of these aspects can result in significant differences in task performance. Non-expert users may find it challenging to identify the changes needed to improve a prompt, especially when they lack domain-specific knowledge and appropriate feedback. To address this challenge, we present PromptAid, a visual analytics system designed to interactively create, refine, and test prompts through exploration, perturbation, testing, and iteration. PromptAid uses coordinated visualizations which allow users to improve prompts via three strategies: keyword perturbations, paraphrasing perturbations, and obtaining the best set of in-context few-shot examples. PromptAid was designed through a pre-study involving NLP experts, and evaluated via a robust mixed-methods user study. Our findings indicate that PromptAid helps users to iterate over prompts with less cognitive overhead, generate diverse prompts with the help of recommendations, and analyze the performance of the generated prompts while surpassing existing state-of-the-art prompting interfaces in performance.

Abstract:
With the widespread use of virtual reality (VR), VR sickness is becoming a key barrier for users to have prolonged VR experience. To alleviate VR sickness, researchers focused on reducing sensory mismatch by aligning the visual information with other sensory cues during the VR experience. We present a wearable haptic interface enabling neck muscle vibration (NMV) with a multi-stimulus configuration. NMV’s vibration on muscle spindles causes a haptic proprioceptive illusion of muscle stretch. Through NMV, we provide a simulated sensation of neck rotation to users without physically rotating the neck. For a left and right rotation on the yaw axis, we vibrated the sternocleidomastoid (SCM) muscles and splenius capitis (SC) muscles. Our lightweight interface vibrates different combinations of actuators on the left and right SCM and SC muscles to deliver multi-stimulus NMV in a desired illusory direction. We found that NMV sensation differs among individuals and is less effective during neck rotation. Based on these results, we developed a calibration and rendering process for NMV using real-time VR rotation information with varying viewpoint control. Our evaluation, which used a VR scene mimicking a common VR experience, showed that NMV effectively reduces rotation-induced VR sickness and improves the overall VR experience, such as presence.

Abstract:
Data slice finding is an emerging technique for validating machine learning (ML) models by identifying and analyzing subgroups in a dataset that exhibit poor performance, often characterized by distinct feature sets or descriptive metadata. However, in the context of validating vision models involving unstructured image data, this approach faces significant challenges, including the laborious and costly requirement for additional metadata and the complex task of interpreting the root causes of underperformance. To address these challenges, we introduce AttributionScanner, an innovative human-in-the-loop Visual Analytics (VA) system, designed for metadata-free data slice finding. Our system identifies interpretable data slices that involve common model behaviors and visualizes these patterns through an Attribution Mosaic design. Our interactive interface provides straightforward guidance for users to detect, interpret, and annotate predominant model issues, such as spurious correlations (model biases) and mislabeled data, with minimal effort. Additionally, it employs a cutting-edge model regularization technique to mitigate the detected issues and enhance the model’s performance. The efficacy of AttributionScanner is demonstrated through use cases involving two benchmark datasets, with qualitative and quantitative evaluations showcasing its substantial effectiveness in vision model validation, ultimately leading to more reliable and accurate models.

Abstract:
In this paper, we introduce a novel method of Filtering and Serializing Spatial Information to tackle uncalibrated photometric stereo tasks, termed FSSI-PS. Photometric stereo aims to recover surface normals from images with varying lighting and is crucial for tasks like 3D reconstruction and defect detection. Current methods in complex surface reconstruction are costly and inaccurate due to redundant feature representations from GCN or Transformer modules, caused by the weak global information extraction capability of GCNs or the large computational cost of Transformers. Furthermore, the trainset’s lack of richness in texture complexity makes reconstruction more difficult. We address these issues by optimizing feature maps and dataset richness through serializing and filtering. First, we use Mamba-RNN to optimize feature representation by directly fusing feature maps, which reduces redundancy and uses minimal computational resources. Specifically, we treat input spatial information as a sequence and serialize it by sorting. Furthermore, we introduce the Mean Angular Variation metric to assess reconstruction difficulty by measuring texture complexity. It classifies PS-Sculpture and PS-Blobby into three categories: Difficult, Normal, and Simple. We use this to construct DNS-S+B, a photometric stereo training set with rich complexity levels. Our method is compared with state-of-the-art methods on the DiLiGenT and LUCES benchmarks to highlight effectiveness.

Abstract:
Ubiquitous applications of Deep neural networks (DNNs) in different artificial intelligence systems have led to their adoption in solving challenging visualization problems in recent years. While sophisticated DNNs offer an impressive generalization, it is imperative to comprehend the quality, confidence, robustness, and uncertainty associated with their prediction. A thorough understanding of these quantities produces actionable insights that help application scientists make informed decisions. Unfortunately, the intrinsic design principles of the DNNs cannot beget prediction uncertainty, necessitating separate formulations for robust uncertainty-aware models for diverse visualization applications. To that end, this contribution demonstrates how the prediction uncertainty and sensitivity of DNNs can be estimated efficiently using various methods and then interactively compared and contrasted for deep image synthesis tasks. Our inspection suggests that uncertainty-aware deep visualization models generate illustrations of informative and superior quality and diversity. Furthermore, prediction uncertainty improves the robustness and interpretability of deep visualization models, making them practical and convenient for various scientific domains that thrive on visual analyses.

Abstract:
Generative text-to-image models, which allow users to create appealing images through a text prompt, have seen a dramatic increase in popularity in recent years. However, most users have a limited understanding of how such models work and often rely on trial and error strategies to achieve satisfactory results. The prompt history contains a wealth of information that could provide users with insights into what has been explored and how the prompt changes impact the output image, yet little research attention has been paid to the visual analysis of such process to support users. We propose the Image Variant Graph, a novel visual representation designed to support comparing prompt-image pairs and exploring the editing history. The Image Variant Graph models prompt differences as edges between corresponding images and presents the distances between images through projection. Based on the graph, we developed the PrompTHis system through co-design with artists. Based on the review and analysis of the prompting history, users can better understand the impact of prompt changes and have a more effective control of image generation. A quantitative user study and qualitative interviews demonstrate that PrompTHis can help users review the prompt history, make sense of the model, and plan their creative process.

Abstract:
Human eye gaze plays a significant role in many virtual and augmented reality (VR/AR) applications, such as gaze-contingent rendering, gaze-based interaction, or eye-based activity recognition. However, prior works on gaze analysis and prediction have only explored eye-head coordination and were limited to human-object interactions. We first report a comprehensive analysis of eye-body coordination in various human-object and human-human interaction activities based on four public datasets collected in real-world (MoGaze), VR (ADT), as well as AR (GIMO and EgoBody) environments. We show that in human-object interactions, e.g., pick and place, eye gaze exhibits strong correlations with full-body motion while in human-human interactions, e.g., chat and teach, a person's gaze direction is correlated with the body orientation towards the interaction partner. Informed by these analyses we then present Pose2Gaze – a novel eye-body coordination model that uses a convolutional neural network and a spatio-temporal graph convolutional neural network to extract features from head direction and full-body poses, respectively, and then uses a convolutional neural network to predict eye gaze. We compare our method with state-of-the-art methods that predict eye gaze only from head movements and show that Pose2Gaze outperforms these baselines with an average improvement of 24.0% on MoGaze, 10.1% on ADT, 21.3% on GIMO, and 28.6% on EgoBody in mean angular error, respectively. We also show that our method significantly outperforms prior methods in the sample downstream task of eye-based activity recognition. These results underline the significant information content available in eye-body coordination during daily activities and open up a new direction for gaze prediction.

Abstract:
With the growing complexity and volume of data, visualizations have become more intricate, often requiring advanced techniques to convey insights. These complex charts are prevalent in everyday life, and individuals who lack knowledge in data visualization may find them challenging to understand. This paper investigates using Large Language Models (LLMs) to help users with low data literacy understand complex visualizations. While previous studies focus on text interactions with users, we noticed that visual cues are also critical for interpreting charts. We introduce an LLM application that supports both text and visual interaction for guiding chart interpretation. Our study with 26 participants revealed that the in-situ support effectively assisted users in interpreting charts and enhanced learning by addressing specific chart-related questions and encouraging further exploration. Visual communication allowed participants to convey their interests straightforwardly, eliminating the need for textual descriptions. However, the LLM assistance led users to engage less with the system, resulting in fewer insights from the visualizations. This suggests that users, particularly those with lower data literacy and motivation, may have over-relied on the LLM agent. We discuss opportunities for deploying LLMs to enhance visualization literacy while emphasizing the need for a balanced approach.

Abstract:
Multivariate networks are commonly found in real-world data-driven applications. Uncovering and understanding the relations of interest in multivariate networks is not a trivial task. This paper presents a visual analytics workflow for studying multivariate networks to extract associations between different structural and semantic characteristics of the networks (e.g., what are the combinations of attributes largely relating to the density of a social network?). The workflow consists of a neural-network-based learning phase to classify the data based on the chosen input and output attributes, a dimensionality reduction and optimization phase to produce a simplified set of results for examination, and finally an interpreting phase conducted by the user through an interactive visualization interface. A key part of our design is a composite variable construction step that remodels nonlinear features obtained by neural networks into linear features that are intuitive to interpret. We demonstrate the capabilities of this workflow with multiple case studies on networks derived from social media usage and also evaluate the workflow with qualitative feedback from experts.

Abstract:
Dense SLAM based on monocular cameras does indeed have immense application value in the field of AR/VR, especially when it is performed on a mobile device. In this article, we propose a novel method that integrates a light-weight depth completion network into a sparse SLAM system using a multi-basis depth representation, so that dense mapping can be performed online even on a mobile phone. Specifically, we present a specifically optimized multi-basis depth completion network, called BBC-Net, tailored to the characteristics of traditional sparse SLAM systems. BBC-Net can predict multiple balanced bases and a confidence map from a monocular image with sparse points generated by off-the-shelf keypoint-based SLAM systems. The final depth is a linear combination of predicted depth bases that can be easily optimized by tuning the corresponding weights. To seamlessly incorporate the weights into traditional SLAM optimization and ensure efficiency and robustness, we design a set of depth weight factors, which makes our network a versatile plug-in module, facilitating easy integration into various existing sparse SLAM systems and significantly enhancing global depth consistency through bundle adjustment. To verify the portability of our method, we integrate BBC-Net into two representative SLAM systems. The experimental results on various datasets show that the proposed method achieves better performance in monocular dense mapping than the state-of-the-art methods. We provide an online demo running on a mobile phone, which verifies the efficiency and mapping quality of the proposed method in real-world scenarios.

Abstract:
Neural Radiance Fields (NeRFs) have shown impressive capabilities in synthesizing photorealistic novel views. However, their application to room-size scenes is limited by the requirement of several hundred views with accurate poses for training. To address this challenge, we propose SN^22eRF, a framework which can reconstruct the neural radiance field with significantly fewer views and noisy poses by exploiting multiple priors. Our key insight is to leverage both multi-view and monocular priors to constrain the optimization of NeRF in the setting of sparse and noisy pose inputs. Specifically, we extract and match key points to constrain pose optimization and use Ray Transformer with a monocular depth estimator to provide dense depth prior for geometry optimization. Benefiting from these priors, our approach achieves state-of-the-art accuracy in novel view synthesis for indoor room scenarios.

Abstract:
Most 360 virtual reality (VR) contents have been developed without considering that users could be affected by VR sickness. Accordingly, users’ viewing safety has been steadily highlighted as a critical problem in the VR market. In this study, we investigate a novel VR sickness mitigation framework based on human visual characteristics for the rendered VR content. First, we build a large-scale 360 VR content database termed VRSP360 (VR Sickness and Presence 360) dedicated to the analysis of VR sickness and thoroughly conduct eye-tracking experiments to measure human perception. In the experiment, we observe that the users’ gaze distribution is highly center-biased when they experience excessive VR sickness. From this observation, we design a foveated filtering framework that limits high-frequency textures in the peripheral view to mitigate VR sickness. Particularly, given the human visual system's (HVS) non-uniform resolution with respect to the fovea, we also adopt the foveation-based filtering method using the trade-off between sickness mitigation and presence conservation, which reduces any loss in perceptual quality despite the filtering. We further demonstrate that our framework can effectively compress visual information by applying foveated compression. In addition, we develop two metrics (visual texture index and perceptual information index) to measure the effective preservation of user-perceived information despite the filtration of peripheral vision textures by our proposed mitigation method. Through rigorous subjective evaluation on both original content and its VR-sickness-mitigated version, we demonstrate that the proposed framework successfully mitigates VR sickness with a reduction rate of ～∼19% on the proposed dataset.

Abstract:
Although existing speech-driven talking face generation methods achieve significant progress, they are far from real-world application due to the avatar-specific training demand and unstable lip movements. To address the above issues, we propose the GSmoothFace, a novel two-stage generalized talking face generation model guided by a fine-grained 3D face model, which can synthesize smooth lip dynamics while preserving the speaker’s identity. Our proposed GSmoothFace model mainly consists of the Audio to Expression Prediction (A2EP) module and the Target Adaptive Face Translation (TAFT) module. Specifically, we first develop the A2EP module to predict expression parameters synchronized with the driven speech. It uses a transformer to capture the long-term audio context and learns the parameters from the fine-grained 3D facial vertices, resulting in accurate and smooth lip-synchronization performance. Afterward, the well-designed TAFT module, empowered by Morphology Augmented Face Blending (MAFB), takes the predicted expression parameters and target video as inputs to modify the facial region of the target video without distorting the background content. The TAFT effectively exploits the identity appearance and background context in the target video, which makes it possible to generalize to different speakers without retraining. Both quantitative and qualitative experiments confirm the superiority of our method in terms of realism, lip-synchronization, and visual quality.

Abstract:
Reconstructing 3D models of dynamic, real-world objects with high-fidelity textures from monocular frame sequences has been a challenging problem in recent years. This difficulty stems from factors such as shadows, indirect illumination, and inaccurate object-pose estimations due to occluding hand-object interactions. To address these challenges, we propose a novel approach that predicts the hand’s impact on environmental visibility and indirect illumination on the object’s surface albedo. Our method first learns the geometry and low-fidelity texture of the object, hand, and background through composite rendering of radiance fields. Simultaneously, we optimize the hand and object poses to achieve accurate object-pose estimations. We then refine physics-based rendering parameters—including roughness, specularity, albedo, hand visibility, skin color reflections, and environmental illumination—to produce precise albedo, and accurate hand illumination and shadow regions. Our approach surpasses state-of-the-art methods in texture reconstruction and, to the best of our knowledge, is the first to account for hand-object interactions in object texture reconstruction.

Abstract:
In mixed reality (MR) avatar-mediated telepresence, avatar movement must be adjusted to convey the user’s intent in a dissimilar space. This paper presents a novel neural network-based framework designed for translating upper-body gestures, which adjusts virtual avatar movements in dissimilar environments to accurately reflect the user’s intended gestures in real-time. Our framework translates a wide range of upper-body gestures, including eye gaze, deictic gestures, free-form gestures, and the transitions between them. A key feature of our framework is its ability to generate natural upper-body gestures for users of different sizes, irrespective of handedness and eye dominance, even though the training is based on data from a single person. Unlike previous methods that require paired motion between users and avatars for training, our framework uses an unpaired approach, significantly reducing training time and allowing for generating a wider variety of motion types. These advantages were made possible by designing two separate networks: the Motion Progression Network, which interprets sparse tracking signals from the user to determine motion progression, and the Upper-body Gesture Network, which autoregressively generates the avatar’s pose based on these progressions. We demonstrate the effectiveness of our framework through quantitative comparisons with state-of-the-art methods, qualitative animation results, and a user evaluation in MR telepresence scenarios.

Abstract:
To address the need for high-complexity low-cost interaction methods for mobile VR, we present a Cardboard Controller, supporting 6-degree-of-freedom target selection while being made of low-cost, highly accessible materials. We present two studies, one evaluating selection activation methods, and the other comparing performance and user experience of the Cardboard Controller using ray-casting and the virtual hand. Our Cardboard Controller has comparable throughput and task completion time to similar 3D input devices and can effectively support pointing and grabbing interactions, particularly when objects are within reach. We propose guidelines for designing low-cost interaction methods and input devices for mobile VR to encourage future research towards the democratization of VR.

Abstract:
We introduce MOST, a novel MOtion diffuSion model via Temporal clip Banzhaf interaction, aimed at addressing the persistent challenge of generating human motion from rare language prompts. While previous approaches struggle with coarse-grained matching and overlook important semantic cues due to motion redundancy, our key insight lies in leveraging fine-grained clip relationships to mitigate these issues. MOST’s retrieval stage presents the first formulation of its kind - temporal clip Banzhaf interaction - which precisely quantifies textual-motion coherence at the clip level. This facilitates direct, fine-grained text-to-motion clip matching and eliminates prevalent redundancy. In the generation stage, a motion prompt module effectively utilizes retrieved motion clips to produce semantically consistent movements. Extensive evaluations confirm that MOST achieves state-of-the-art text-to-motion retrieval and generation performance by comprehensively addressing previous challenges, as demonstrated through quantitative and qualitative results highlighting its effectiveness, especially for rare prompts.

Abstract:
Redirected walking (RDW) expands the virtually reachable areas within confined physical spaces by real-walking locomotion. However, existing RDW controllers struggle with extracting spatial features, hindering the improvement for physical obstacle avoidance. To overcome this, we propose a novel spatial walkability-aware redirection controller utilizing deep reinforcement learning (DRL), which learns to enhance obstacle avoidance capability by leveraging comprehensive spatial features. Based on information entropy, we innovatively introduce the spatial walkability entropy (SWE) metric to characterize the walkability and safety of each physical position by assessing the difficulty of reaching its surroundings. Guided by this, we design a novel joint reward that considers both the SWE distribution and the user’s virtual-physical alignment, providing ample guidance for learning. Moreover, unlike existing controllers employing traditional reset strategies, we propose a novel reset method that maximizes regional entropy to guide users towards more open areas, reducing the re-collision risk. Extensive simulation experiments compare our controller with state-of-the-art (SOTA) redirection controllers. The results demonstrate that our controller significantly reduces physical collisions across various virtual-physical scenarios. Moreover, live user experiments confirm that our controller offers a superior roaming experience in practical settings.

Abstract:
Deep learning has the potential to revolutionize sports performance, with applications ranging from perception and comprehension to decision. This article presents a comprehensive survey of deep learning in sports performance, focusing on three main aspects: algorithms, datasets and virtual environments, and challenges. First, we discuss the hierarchical structure of deep learning algorithms in sports performance which includes perception, comprehension and decision while comparing their strengths and weaknesses. Second, we list widely used existing datasets in sports and highlight their characteristics and limitations. Finally, we summarize current challenges and point out future trends of deep learning in sports. Our survey provides valuable reference material for researchers interested in deep learning in sports applications.

Abstract:
Dimensionality reduction is used as an important tool for unraveling the complexities of high-dimensional datasets in many fields of science, such as cell biology, chemical informatics, and physics. Visualizations of the dimensionally-reduced data enable scientists to delve into the intrinsic structures of their datasets and align them with established hypotheses. Visualization researchers have thus proposed many dimensionality reduction methods and interactive systems designed to uncover latent structures. At the same time, different scientific domains have formulated guidelines or common workflows for using dimensionality reduction techniques and visualizations for their respective fields. In this work, we present a critical analysis of the usage of dimensionality reduction in scientific domains outside of computer science. First, we conduct a bibliometric analysis of 21,249 academic publications that use dimensionality reduction to observe differences in the frequency of techniques across fields. Next, we conduct a survey of a 71-paper sample from four fields: biology, chemistry, physics, and business. Through this survey, we uncover common workflows, processes, and usage patterns, including the mixed use of confirmatory data analysis to validate a dataset and projection method and exploratory data analysis to then generate more hypotheses. We also find that misinterpretations and inappropriate usage is common, particularly in the visual interpretation of the resulting dimensionally reduced view. Lastly, we compare our observations with recent works in the visualization community in order to match work within our community to potential areas of impact outside our community. By comparing the usage found within scientific fields to the recent research output of the visualization community, we offer both validation of the progress of visualization research into dimensionality reduction and a call for action to produce techniques that meet the needs of scientific users.

Affiliations: School of Mathematics and Computer Sciences, Nanchang University, Nanchang, China; School of Architecture & Urban Planning, Shenzhen University, Shenzhen, China; School of Geodesy and Geomatics, Wuhan University, Wuhan, China; State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, Wuhan, China; School of Safety Science and Emergency Management, Wuhan University of Technology, Wuhan, China; School of Remote Sensing & Geomatics Engineering, Nanjing University of Information Science & Technology, Nanjing, China; College of Civil Engineering, Nanjing Forestry University, Nanjing, China; Guangzhou Urban Planning & Design Survey Research Institute, Guangzhou, China

Abstract:
Point cloud registration serves as a fundamental problem across multiple fields including computer vision, computer graphics, and remote sensing. While local feature descriptors (LFDs) have long been established as a cornerstone for point cloud registration and the LFD-based approach has been extensively studied, the field has witnessed significant advancements in recent years. Despite these developments, the research community lacks a systematic review to consolidate these contributions, leaving many researchers unaware of recent progress in LFD-based registration. To address this gap, we present a comprehensive review that critically examines both state-of-the-art and widely referenced methods across all subtasks of LFD-based registration. Our work provides: (1) an extensive survey of existing methodologies, (2) in-depth analysis of their respective strengths and limitations, (3) insightful observations and practical recommendations, and (4) a thorough summary of relevant applications and publicly available datasets. This systematic overview offers valuable guidance for researchers pursuing future investigations in this domain.

Abstract:
The transfer function (TF) design is crucial for enhancing the visualization quality and understanding of volume data in volume rendering. Recent research has proposed various multidimensional TFs to utilize diverse attributes extracted from volume data for controlling individual voxel rendering. Although multidimensional TFs enhance the ability to segregate data, manipulating various attributes for the rendering is cumbersome. In contrast, low-dimensional TFs are more beneficial as they are easier to manage, but separating volume data during rendering is problematic. This paper proposes a novel approach, a two-level transfer function, for rendering volume data by reducing TF dimensions. The proposed technique involves extracting multidimensional TF attributes from volume data and applying t-Stochastic Neighbor Embedding (t-SNE) to the TF attributes for dimensionality reduction. The two-level transfer function combines the classical 2D TF and t-SNE TF in the conventional direct volume rendering pipeline. The proposed approach is evaluated by comparing segments in t-SNE TF and rendering images using various volume datasets. The results of this study demonstrate that the proposed approach can effectively allow us to manipulate multidimensional attributes easily while maintaining high visualization quality in volume rendering.

Abstract:
Rendering photo-realistic novel-view images of complex scenes has been a long-standing challenge in computer graphics. In recent years, great research progress has been made in enhancing rendering quality and accelerating rendering speed in the realm of view synthesis. However, when rendering complex dynamic scenes with sparse views, the rendering quality remains limited due to occlusion problems. Besides, for rendering high-resolution images on dynamic scenes, the rendering speed is still far from real-time. In this work, we propose a generalizable view synthesis method that can render high-resolution novel-view images of complex static and dynamic scenes in real-time from sparse views. To address the occlusion problems arising from the sparsity of input views and the complexity of captured scenes, we introduce an explicit 3D visibility reasoning approach that can efficiently estimate the visibility of sampled 3D points to the input views. The proposed visibility reasoning approach is fully differentiable and can gracefully fit inside the volume rendering pipeline, allowing us to train our networks with only multi-view images as supervision while refining geometry and texture simultaneously. Besides, each module in our pipeline is carefully designed to bypass the time-consuming MLP querying process and enhance the rendering quality of high-resolution images, enabling us to render high-resolution novel-view images in real-time. Experimental results show that our method outperforms previous view synthesis methods in both rendering quality and speed, particularly when dealing with complex dynamic scenes with sparse views.

Abstract:
Graphical overlays that layer visual elements onto charts, are effective to convey insights and context in financial narrative visualizations. However, automating graphical overlays is challenging due to complex narrative structures and limited understanding of effective overlays. To address the challenge, we first summarize the commonly used graphical overlays and narrative structures, and the proper correspondence between them in financial narrative visualizations, elected by a survey of 1752 layered charts with corresponding narratives. We then design FinFlier, a two-stage innovative system leveraging a knowledge-grounding large language model to automate graphical overlays for financial visualizations. The text-data binding module enhances the connection between financial vocabulary and tabular data through advanced prompt engineering, and the graphics overlaying module generates effective overlays with narrative sequencing. We demonstrate the feasibility and expressiveness of FinFlier through a gallery of graphical overlays covering diverse financial narrative visualizations. Performance evaluations and user studies further confirm system’s effectiveness and the quality of generated layered charts.

Abstract:
Neural implicit fields, such as the neural signed distance field (SDF) of a shape, have emerged as a powerful representation for many applications, e.g., encoding a 3D shape and performing collision detection. Typically, implicit fields are encoded by Multi-layer Perceptrons (MLP) with positional encoding (PE) to capture high-frequency geometric details. However, a notable side effect of such PE-equipped MLPs is the noisy artifacts present in the learned implicit fields. While increasing the sampling rate could in general mitigate these artifacts, in this paper we aim to explain this adverse phenomenon through the lens of Fourier analysis. We devise a tool to determine the appropriate sampling rate for learning an accurate neural implicit field without undesirable side effects. Specifically, we propose a simple yet effective method to estimate the intrinsic frequency of a given network with randomized weights based on the Fourier analysis of the network's responses. It is observed that a PE-equipped MLP has an intrinsic frequency much higher than the highest frequency component in the PE layer. Sampling against this intrinsic frequency following the Nyquist-Sannon sampling theorem allows us to determine an appropriate training sampling rate. We empirically show in the setting of SDF fitting that this recommended sampling rate is sufficient to secure accurate fitting results, while further increasing the sampling rate would not further noticeably reduce the fitting error. Training PE-equipped MLPs simply with our sampling strategy leads to performances superior to the existing methods.

Abstract:
Point scene instance mesh reconstruction is a challenging task since it requires both scene-level instance segmentation and instance-level mesh reconstruction from partial observations simultaneously. Previous works either adopt a detection backbone or a segmentation one, and then directly employ a mesh reconstruction network to produce complete meshes from incomplete instance point clouds. To further boost the mesh reconstruction quality with both local details and global smoothness, in this work, we propose JIMR, a joint framework with two cascaded stages for semantic and geometry understanding. In the first stage, we propose to perform both instance segmentation and object detection simultaneously. By making both tasks promote each other, this design facilitates subsequent mesh reconstruction by providing more precisely-segmented instance points and better alignment benefiting from predicted complete bounding boxes. In the second stage, we propose a complete-then-reconstruct procedure, where the completion module explicitly disentangles completion from reconstruction, and enables the usage of pre-trained weights of existing powerful completion and reconstruction networks. Moreover, we propose a comprehensive confidence score to filter proposals considering the quality of instance segmentation, bounding box detection, semantic classification, and mesh reconstruction at the same time. Experiments show that our proposed JIMR outperforms state-of-the-art methods regarding instance reconstruction qualitatively and quantitatively.

Abstract:
To understand the surface evolution and potential habitability of other planets we must analyse their geology – the 3D structure and chemistry of the rocks that are exposed at the surface. Although rovers capture this 3D structure using stereo camera systems and other instruments, when we present this information to mission scientists for analysis it is generally confined to the 2D plane of a computer screen, and the spatial information is lost at the point when it is needed most. To address this problem, we design, develop, and evaluate a prototype Virtual Environment to present geological data in the 3D form in which it was originally captured, and users are supplied with a toolkit for measurement and annotation of data. We observed that users were inspired by the environment and felt more connected to it because they could move within the data; they valued the tools but did not trust the scale and therefore did not always trust the results. We conclude with recommendations for others working in this application area, and pose a series of questions for future research.

Abstract:
Impact dynamics are crucial for estimating the growth patterns of NFT projects by tracking the diffusion and decay of their relative appeal among stakeholders. Machine learning methods for impact dynamics analysis are incomprehensible and rigid in terms of their interpretability and transparency, whilst stakeholders require interactive tools for informed decision-making. Nevertheless, developing such a tool is challenging due to the substantial, heterogeneous NFT transaction data and the requirements for flexible, customized interactions. To this end, we integrate intuitive visualizations to unveil the impact dynamics of NFT projects. We first conduct a formative study and summarize analysis criteria, including substitution mechanisms, impact attributes, and design requirements from stakeholders. Next, we propose the Minimal Substitution Model to simulate substitutive systems of NFT projects that can be feasibly represented as node-link graphs. Particularly, we utilize attribute-aware techniques to embed the project status and stakeholder behaviors in the layout design. Accordingly, we develop a multi-view visual analytics system, namely NFTracer, allowing interactive analysis of impact dynamics in NFT transactions. We demonstrate the informativeness, effectiveness, and usability of NFTracer by performing two case studies with domain experts and one user study with stakeholders. The studies suggest that NFT projects featuring a higher degree of similarity are more likely to substitute each other. The impact of NFT projects within substitutive systems is contingent upon the degree of stakeholders’ influx and projects’ freshness.

Abstract:
The issue of traffic congestion poses a significant obstacle to the development of global cities. One promising solution to tackle this problem is intelligent traffic signal control (TSC). Recently, TSC strategies leveraging reinforcement learning (RL) have garnered attention among researchers. However, the evaluation of these models has primarily relied on fixed metrics like reward and queue length. This limited evaluation approach provides only a narrow view of the model's decision-making process, impeding its practical implementation. Moreover, effective TSC necessitates coordinated actions across multiple intersections. Existing visual analysis solutions fall short when applied in multi-agent settings. In this study, we delve into the challenge of interpretability in multi-agent reinforcement learning (MARL), particularly within the context of TSC. We propose MARLens, a visual analytics system tailored to understand MARL-based TSC. Our system serves as a versatile platform for both RL and TSC researchers. It empowers them to explore the model's features from various perspectives, revealing its decision-making processes and shedding light on interactions among different agents. To facilitate quick identification of critical states, we have devised multiple visualization views, complemented by a traffic simulation module that allows users to replay specific training scenarios. To validate the utility of our proposed system, we present three comprehensive case studies, incorporate insights from domain experts through interviews, and conduct a user study. These collective efforts underscore the feasibility and effectiveness of MARLens in enhancing our understanding of MARL-based TSC systems and pave the way for more informed and efficient traffic management strategies.

Abstract:
In volume visualization, users can interactively explore the three-dimensional data by specifying color and opacity mappings in the transfer function (TF) or adjusting lighting parameters, facilitating meaningful interpretation of the underlying structure. However, rendering large-scale volumes demands powerful GPUs and high-speed memory access for real-time performance. While existing novel view synthesis (NVS) methods offer faster rendering speeds with lower hardware requirements, the visible parts of a reconstructed scene are fixed and constrained by preset TF settings, significantly limiting user exploration. This article introduces inverse volume rendering via Gaussian splatting (iVR-GS), an innovative NVS method that reduces the rendering cost while enabling scene editing for interactive volume exploration. Specifically, we compose multiple iVR-GS models associated with basic TFs covering disjoint visible parts to make the entire volumetric scene visible. Each basic model contains a collection of 3D editable Gaussians, where each Gaussian is a 3D spatial point that supports real-time scene rendering and editing. We demonstrate the superior reconstruction quality and composability of iVR-GS against other NVS solutions (Plenoxels, CCNeRF, and base 3DGS) on various volume datasets. The code is available at https://github.com/TouKaienn/iVR-GS.

Abstract:
Audio-driven 3D facial animation is crucial for enhancing the metaverse's realism, immersion, and interactivity. While most existing methods focus on generating highly realistic and lively 2D talking head videos by leveraging extensive 2D video datasets these approaches work in pixel space and are not easily adaptable to 3D environments. We present VASA-Rig, which has achieved a significant advancement in the realism of lip-audio synchronization, facial dynamics, and head movements. In particular, we introduce a novel rig parameter-based emotional talking face dataset and propose the Latents2Rig model, which facilitates the transformation of 2D facial animations into 3D. Unlike mesh-based models, VASA-Rig outputs rig parameters, instantiated in this paper as 174 Metahuman rig parameters, making it more suitable for integration into industry-standard pipelines. Extensive experimental results demonstrate that our approach significantly outperforms existing state-of-the-art methods in terms of both realism and accuracy.

Abstract:
Spatial audio and directional tactile cues are crucial for target localization and collision avoidance, especially when visual information is limited or cognitive load is high. As multisensory cues, spatially aligned auditory and tactile stimuli can perform better than misaligned ones, but achieving precise alignment is challenging. This study aims to (1) identify just noticeable differences (JNDs) of direction between auditory and tactile stimuli in the horizontal plane and (2) evaluate hazard avoidance performance with misaligned cues within the JNDs. The estimated JNDs increase from 26° to 84° as the stimuli shifted from the center to the side of the body. We also demonstrate that perceptual alignment of auditory and tactile cues within the JND ranges yielded comparable hazard avoidance performance and usability to exact physical alignment. Our study offers useful insights for more efficient and accurate spatial audio-tactile rendering systems for extended reality (XR).

Abstract:
We present Explainable XR, an end-to-end framework for analyzing user behavior in diverse eXtended Reality (XR) environments by leveraging Large Language Models (LLMs) for data interpretation assistance. Existing XR user analytics frameworks face challenges in handling cross-virtuality – AR, VR, MR – transitions, multi-user collaborative application scenarios, and the complexity of multimodal data. Explainable XR addresses these challenges by providing a virtuality-agnostic solution for the collection, analysis, and visualization of immersive sessions. We propose three main components in our framework: (1) A novel user data recording schema, called User Action Descriptor (UAD), that can capture the users' multimodal actions, along with their intents and the contexts; (2) a platform-agnostic XR session recorder, and (3) a visual analytics interface that offers LLM-assisted insights tailored to the analysts' perspectives, facilitating the exploration and analysis of the recorded XR session data. We demonstrate the versatility of Explainable XR by demonstrating five use-case scenarios, in both individual and collaborative XR applications across virtualities. Our technical evaluation and user studies show that Explainable XR provides a highly usable analytics solution for understanding user actions and delivering multifaceted, actionable insights into user behaviors in immersive environments.

Abstract:
Airflow is recognized as an effective method for inducing the illusion of self-motion (vection) and reducing motion sickness in virtual reality. However, the quantitative relationship between virtual motion and the airflow perceived as consistent with it has not been fully explored. To address this gap, this study conducted three experiments. In Experiment 1, we carried out a series of cross-modal matching tasks to establish the relationship between the speed of virtual motion and the airflow speed perceived as consistent with it, revealing a strong linear correlation. In Experiment 2, we introduced the concept of an “Airflow Gradient“ to simulate the bodily sensation of curvilinear motion and examined the relationship between the radius and angular velocity of the motion and the difference in airflow speed between the left and right sides. The results indicated a linear relationship between the radius and the left-right airflow speed difference, while the angular velocity showed a near-quadratic pattern, similar to the centripetal acceleration formula. Based on these findings, Experiment 3 developed a dynamic airflow scheme and compared it with constant airflow and no-airflow conditions during locomotion tasks in a complex urban environment. The results demonstrated that dynamic airflow, which ensures consistency between visual and bodily vection, further reduces motion sickness, enhances presence, and provides a more natural and consistent virtual motion experience.

Abstract:
Visual impairments significantly impact individuals' ability to perceive their surroundings, affecting everyday tasks and spatial navigation. This study explores SEEK VR,s a multi-modal virtual reality game designed to foster empathy and raise awareness about the challenges faced by visually impaired individuals. By integrating visual feedback, 3D spatial audio, and haptic feedback, the game provides an immersive experience that helps participants understand the physical and emotional struggles of visual impairment. The paper includes a review of related work on empathy-driven VR games, a detailed description of the design and implementation of SEEK VR, and the technical aspects of its multimodal interactions. A user study with 24 participants demonstrated significant increases in empathy, particularly in empathy and willingness to help visually impaired individuals in real-world scenarios. These findings highlight the potential of VR serious games to promote social awareness and empathy through immersive, multi-modal interactions.

Abstract:
We investigate the perception of visual variables on wall-sized tiled displays within an immersive environment. We designed and conducted two formal user studies focusing on elementary visualization reading tasks in VR. The first study compared three different virtual display arrangements (Flat, Cylinder, and Cockpit). It showed that participants made smaller errors on virtual curved walls (Cylinder and Cockpit) compared to Flat. Following that, we compared the results with those from a previous study conducted in a real-world setting. The comparative analysis showed that virtual curved walls resulted in smaller errors than the real-world flat wall display, but with longer task completion time. The second study evaluated the impact of four 3D user interaction techniques (Selection, Walking, Steering, and Teleportation) on performing the elementary task on the virtual Flat wall display. The results confirmed that interaction techniques further improved task performance. Finally, we discuss the limitations and future work.

Abstract:
Virtual environments (VEs) are pivotal for virtual, augmented, and mixed reality systems. Despite advances in 3D generation and reconstruction, the direct creation of 3D objects within an established 3D scene (represented as NeRF) for novel VE creation remains a relatively unexplored domain. This process is complex, requiring not only the generation of high-quality 3D objects but also their seamless integration into the existing scene. To this end, we propose a novel pipeline featuring an intuitive interface, dubbed GO-NeRF. Our approach takes text prompts and user-specified regions as inputs and leverages the scene context to generate 3D objects within the scene. We employ a compositional rendering formulation that effectively integrates the generated 3D objects into the scene, utilizing optimized 3D-aware opacity maps to avoid unintended modifications to the original scene. Furthermore, we develop tailored optimization objectives and training strategies to enhance the model's ability to capture scene context and mitigate artifacts, such as floaters, that may occur while optimizing 3D objects within the scene. Extensive experiments conducted on both forward-facing and 360°scenes demonstrate the superior performance of our proposed method in generating objects that harmonize with surrounding scenes and synthesizing high-quality novel view images. The code will be at https://daipengwa.github.io/G0-NeRF/.

Abstract:
Distractions in mixed reality (MR) environments can significantly influence user experience, affecting key factors such as presence, reaction time, cognitive load, and Break in Presence (BIP). Presence measures immersion, reaction time captures user responsiveness, cognitive load reflects mental effort, and BIP represents moments when attention shifts from the virtual to the real world, breaking immersion. While prior work has established that distractions impact these factors individually, the relationship between these constructs remains underexplored, particularly in MR environments where users engage with both real and virtual stimuli. To address this gap, we have presented a theoretical model to understand how congruent and incongruent distractions affect all these constructs. We conducted a within-subject study (N = 54) where participants performed image-sorting tasks under different distraction conditions. Our findings show that incongruent distractions significantly increase cognitive load, slow reaction times, and elevate BIP frequency, with presence mediating these effects.

Abstract:
Private conversations in social Virtual Reality (VR) environments lack the nuanced cues of physical interactions, potentially diminishing the sense of privacy and social presence. This paper introduces Whisper, a novel multisensory interaction technique designed to enhance private conversations for social VR applications. We first conducted a formative study (N=20) to understand private conversation demands, limitations of existing methods, and user expectations in social VR. Informed by these insights, Whisper incorporates visual (avatar proximity, gestures and illumination), auditory (voice conversation), and tactile (simulated airflow) elements to simulate the act of whispering, providing users with an intuitive and immersive method of private communication. The technique also features a contextual record to maintain conversation continuity. We evaluated Whisper through a comparative user study (N=24) in party and classroom scenarios. Results demonstrate that Whisper significantly outperforms existing methods in sense of privacy, mode distinguishability, intimacy, perceptual realism, and social presence.

Abstract:
Photos taken in unconstrained tourist environments often present challenges for accurate 3D scene reconstruction due to variable appearances and transient occlusions, which can introduce artifacts in novel view synthesis. Recently, in-the-wild 3D scene reconstruction has been achieved realistic rendering with Neural Radiance Fields (NeRFs). With the advancement of 3D Gaussian Splatting (3DGS), some methods also attempt to reconstruct 3D scenes from unconstrained photo collections and achieve real-time rendering. However, the rapid convergence of 3DGS is misaligned with the slower convergence of neural network-based appearance encoder and transient mask predictor, hindering the reconstruction efficiency. To address this, we propose a novel sky-aware framework for scene reconstruction from unconstrained photo collection using 3DGS. Firstly, we observe that the learnable per-image transient mask predictor in previous work is unnecessary. By introducing a simple yet efficient greedy supervision strategy, we directly utilize the pseudo mask generated by a pretrained semantic segmentation network as the transient mask, thereby achieving more efficient and higher quality in-the-wild 3D scene reconstruction. Secondly, we find that separately estimating appearance embeddings for the sky and building significantly improves reconstruction efficiency and accuracy. We analyze the underlying reasons and introduce a neural sky module to generate diverse skies from latent sky embeddings extract from unconstrained images. Finally, we propose a mutual distillation learning strategy to constrain sky and building appearance embeddings within the same latent space, further enhancing reconstruction efficiency and quality. Extensive experiments on multiple datasets demonstrate that the proposed framework outperforms existing methods in novel view and appearance synthesis, offering superior rendering quality with faster convergence and rendering speed.

Abstract:
Photorealistic stylization of 3D scenes aims to generate photorealistic images from arbitrary novel views according to a given style image, while ensuring consistency when rendering video from different viewpoints. Some existing stylization methods using neural radiance fields can effectively predict stylized scenes by combining the features of the style image with multi-view images to train 3D scenes. However, these methods generate novel view images that contain undesirable artifacts. In addition, they cannot achieve universal photorealistic stylization for a 3D scene. Therefore, a stylization image needs to retrain a 3D scene representation network based on a neural radiation field. We propose a novel photorealistic 3D scene stylization transfer framework to address these issues. It can realize photorealistic 3D scene style transfer with a 2D style image for novel view video rendering. We first pre-trained a 2D photorealistic style transfer network, which can satisfy the photorealistic style transfer between any content image and style image. Then, we use voxel features to optimize a 3D scene and obtain the geometric representation of the scene. Finally, we jointly optimize a hypernetwork to realize the photorealistic style transfer of arbitrary style images. In the transfer stage, we use a pre-trained 2D photorealistic network to constrain the photorealistic style of different views and different style images in the 3D scene. The experimental results show that our method not only realizes the 3D photorealistic style transfer of arbitrary style images, but also outperforms the existing methods in terms of visual quality and consistency.

Abstract:
We propose a new barrier-based box-constrained convex QP solver based on a primal-dual interior point method to efficiently solve large-scale pressure Poisson problems with non-negative pressure constraints, which commonly arise in liquid animation. The performance of prior active-set-based approaches is limited by the need to repeatedly update the active set. Our solver eliminates this issue by entirely avoiding the use of an active set, which in turn makes the inner problems of our Newton iteration process fully unconstrained. For efficiency, exploiting the solution uniqueness of convex QPs and the fact that the pressure constraints are simple box constraints, we aggressively update solution candidates without performing any step selection procedure (such as line search) and instead directly clamp candidates back to the bounds wherever constraint violations occur. Additionally, to accelerate the inner linear solves, we present a topology-aware geometry-inspired aggregation algebraic multigrid preconditioner and describe in detail several key performance optimizations that we incorporate. We demonstrate the efficacy of our solver in various practical scenarios and show that it often surpasses various alternatives in terms of speed and memory usage.

Abstract:
View-based methods have demonstrated promising performance in 3D shape understanding. However, they tend to make strong assumptions about the relations between views or learn the multi-view correlations indirectly, which limits the flexibility of exploring inter-view correlations and the effectiveness of target tasks. To overcome the above problems, this article investigates flexible organization and explicit correlation learning for multiple views. In particular, we propose to incorporate different views of a 3D shape into a permutation-invariant set, referred to as View Set, which removes rigid relation assumptions and facilitates adequate information exchange and fusion among views. Based on that, we devise a nimble Transformer model, named VSFormer, to explicitly capture pairwise and higher-order correlations of all elements in the set. Meanwhile, we theoretically reveal a natural correspondence between the Cartesian product of a view set and the correlation matrix in the attention mechanism, which supports our model design. Comprehensive experiments suggest that VSFormer has better flexibility, efficient inference efficiency and superior performance. Notably, VSFormer reaches state-of-the-art results on various 3 d recognition datasets, including ModelNet40, ScanObjectNN and RGBD. It also establishes new records on the SHREC’17 retrieval benchmark.

Affiliations: School of Computer and Data Engineering, NingboTech University, Ningbo, Zhejiang, China; School of Software Technology, Zhejiang University, Hangzhou, Zhejiang, China; School of Big Data and Artificial Intelligence Management, Xi’an Jiaotong University, Xi’an, Shaanxi, China; School of Computer Science and Technology, ShanDong University, Jinan, Shandong, China; Graphics & Geometric Computing Laboratory, School of Mathematical Sciences, University of Science and Technology of China, Hefei, Anhui, China

Abstract:
Identifying points of interest (POIs) on the surface of 3D shapes is a significant challenge in geometric processing research. The complex connection between POIs and their geometric descriptors, combined with the small percentage of POIs on the shape, makes detecting POIs on any given 3D shape a highly challenging task. Existing methods directly detect POIs from the entire 3D shape, resulting in low efficiency and accuracy. Therefore, we propose a novel multi-modal POI detection method using a coarse-to-fine approach, with the key idea of reducing data complexity and enabling more efficient and accurate subsequent POI detection by first identifying and processing important regions on the 3D shape. It first obtains important areas on the 3D shape through 2D projected images, then processes points within these regions using attention mechanisms. Extensive experiments demonstrate that our method outperforms existing POI detection techniques.

Abstract:
Small multiples are a popular visualization method, displaying different views of a dataset using multiple frames, often with the same scale and axes. However, there is a need to address their potential constraints, especially in the context of human cognitive capacity limits. These limits dictate the maximum information our mind can process at once. We explore the issue of capacity limitation by testing competing theories that describe how the number of frames shown in a display, the scale of the frames, and time constraints impact user performance with small multiples of line charts in an energy grid scenario. In two online studies (Experiment 1 n = 141 and Experiment 2 n = 360) and a follow-up eye-tracking analysis (n = 5), we found a linear decline in accuracy with increasing frames across seven tasks, which was not fully explained by differences in frame size, suggesting visual search challenges. Moreover, the studies demonstrate that highlighting specific frames can mitigate some visual search difficulties but, surprisingly, not eliminate them. This research offers insights into optimizing the utility of small multiples by aligning them with human limitations.

Abstract:
Implicit Neural representations (INRs) are widely used for scientific data reduction and visualization by modeling the function that maps a spatial location to a data value. Without any prior knowledge about the spatial distribution of values, we are forced to sample densely from INRs to perform visualization tasks like iso-surface extraction which can be very computationally expensive. Recently, range analysis has shown promising results in improving the efficiency of geometric queries, such as ray casting and hierarchical mesh extraction, on INRs for 3D geometries by using arithmetic rules to bound the output range of the network within a spatial region. However, the analysis bounds are often too conservative for complex scientific data. In this article, we present an improved technique for range analysis by revisiting the arithmetic rules and analyzing the probability distribution of the network output within a spatial region. We model this distribution efficiently as a Gaussian distribution by applying the central limit theorem. Excluding low probability values, we are able to tighten the output bounds, resulting in a more accurate estimation of the value range, and hence more accurate identification of iso-surface cells and more efficient iso-surface extraction on INRs. Our approach demonstrates superior performance in terms of the iso-surface extraction time on four datasets compared to the original range analysis method and can also be generalized to other geometric query tasks.

Abstract:
Head-mounted displays (HMDs) in room-scale virtual reality are usually tracked using inside-out visual SLAM algorithms. Alternatively, to track the motion of the HMD with respect to a fixed real-world reference frame, an outside-in instrumentation like a motion capture system can be adopted. However, outside-in tracking systems may temporarily lose tracking as they suffer by occlusion and blind spots. A possible solution is to adopt a hybrid approach where the inside-out tracker of the HMD is augmented with an outside-in sensing system. On the other hand, when the tracking signal of the outside-in system is recovered after a loss of tracking the transition from inside-out tracking to hybrid tracking may generate a discontinuity, i.e a sudden change of the virtual viewpoint, that can be uncomfortable for the user. Therefore, hybrid tracking solutions for HMDs require advanced sensor fusion algorithms to obtain a smooth transition. This work proposes a method for hybrid tracking of a HMD with smooth transitions based on an adaptive complementary filter. The proposed approach can be configured with several parameters that determine a trade-off between user experience and tracking error. A user study was carried out in a room-scale virtual reality environment, where users carried out two different tasks while multiple signal tracking losses of the outside-in sensor system occurred. The results show that the proposed approach improves user experience compared to a standard Extended Kalman Filter, and that tracking error is lower compared to a state-of-the-art complementary filter when configured for the same quality of user experience.

Abstract:
As basketball's popularity surges, fans often find themselves confused and overwhelmed by the rapid game pace and complexity. Basketball tactics, involving a complex series of actions, require substantial knowledge to be fully understood. This complexity leads to a need for additional information and explanation, which can distract fans from the game. To tackle these challenges, we present Sportify, a Visual Question Answering system that integrates narratives and embedded visualization for demystifying basketball tactical questions, aiding fans in understanding various game aspects. We propose three novel action visualizations (i.e., Pass, Cut, and Screen) to demonstrate critical action sequences. To explain the reasoning and logic behind players' actions, we leverage a large-language model (LLM) to generate narratives. We adopt a storytelling approach for complex scenarios from both first and third-person perspectives, integrating action visualizations. We evaluated Sportify with basketball fans to investigate its impact on understanding of tactics, and how different personal perspectives of narratives impact the understanding of complex tactic with action visualizations. Our evaluation with basketball fans demonstrates Sportify's capability to deepen tactical insights and amplify the viewing experience. Furthermore, third-person narration assists people in getting in-depth game explanations while first-person narration enhances fans' game engagement.

Abstract:
Contour trees describe the topology of level sets in scalar fields and are widely used in topological data analysis and visualization. A main challenge of utilizing contour trees for large-scale scientific data is their computation at scale using high-performance computing. To address this challenge, recent work has introduced distributed hierarchical contour trees for distributed computation and storage of contour trees. However, effective use of these distributed structures in analysis and visualization requires subsequent computation of geometric properties and branch decomposition to support contour extraction and exploration. In this work, we introduce distributed algorithms for augmentation, hypersweeps, and branch decomposition that enable parallel computation of geometric properties, and support the use of distributed contour trees as query structures for scientific exploration. We evaluate the parallel performance of these algorithms and apply them to identify and extract important contours for scientific visualization.

Abstract:
We present a systematic review, an empirical study, and a first set of considerations for designing visualizations in motion, derived from a concrete scenario in which these visualizations were used to support a primary task. In practice, when viewers are confronted with embedded visualizations, they often have to focus on a primary task and can only quickly glance at a visualization showing rich, often dynamically updated, information. As such, the visualizations must be designed so as not to distract from the primary task, while at the same time being readable and useful for aiding the primary task. For example, in games, players who are engaged in a battle have to look at their enemies but also read the remaining health of their own game character from the health bar over their character's head. Many trade-ofts are possible in the design of embedded visualizations in such dynamic scenarios, which we explore in-depth in this paper with a focus on user experience. We use video games as an example of an application context with a rich existing set of visualizations in motion. We begin our work with a systematic review of in-game visualizations in motion. Next, we conduct an empirical user study to investigate how different embedded visualizations in motion designs impact user experience. We conclude with a set of considerations and trade-offs for designing visualizations in motion more broadly as derived from what we learned about video games. All supplemental materials of this paper are available at osf.io/3v8wm/.

Abstract:
Propagation analysis refers to studying how information spreads on social media, a pivotal endeavor for understanding social sentiment and public opinions. Numerous studies contribute to visualizing information spread, but few have considered the implicit and complex diffusion patterns among multiple platforms. To bridge the gap, we summarize cross-platform diffusion patterns with experts and identify significant factors that dissect the mechanisms of cross-platform information spread. Based on that, we propose an information diffusion model that estimates the likelihood of a topic/post spreading among different social media platforms. Moreover, we propose a novel visual metaphor that encapsulates cross-platform propagation in a manner analogous to the spread of seeds across gardens. Specifically, we visualize platforms, posts, implicit cross-platform routes, and salient instances as elements of a virtual ecosystem — gardens, flowers, winds, and seeds, respectively. We further develop a visual analytic system, namely BloomWind, that enables users to quickly identify the cross-platform diffusion patterns and investigate the relevant social media posts. Ultimately, we demonstrate the usage of BloomWind through two case studies and validate its effectiveness using expert interviews.

Abstract:
Dimensionality reduction techniques are widely used for visualizing high-dimensional data. However, support for interpreting patterns of dimension reduction results in the context of the original data space is often insufficient. Consequently, users may struggle to extract insights from the projections. In this paper, we introduce DimBridge, a visual analytics tool that allows users to interact with visual patterns in a projection and retrieve corresponding data patterns. DimBridge supports several interactions, allowing users to perform various analyses, from contrasting multiple clusters to explaining complex latent structures. Leveraging first-order predicate logic, DimBridge identifies subspaces in the original dimensions relevant to a queried pattern and provides an interface for users to visualize and interact with them. We demonstrate how DimBridge can help users overcome the challenges associated with interpreting visual patterns in projections.

Abstract:
Multi-modal embeddings form the foundation for vision-language models, such as CLIP embeddings, the most widely used text-image embeddings. However, these embeddings are vulnerable to subtle misalignment of cross-modal features, resulting in decreased model performance and diminished generalization. To address this problem, we design ModalChorus, an interactive system for visual probing and alignment of multi-modal embeddings. ModalChorus primarily offers a two-stage process: 1) embedding probing with Modal Fusion Map (MFM), a novel parametric dimensionality reduction method that integrates both metric and nonmetric objectives to enhance modality fusion; and 2) embedding alignment that allows users to interactively articulate intentions for both point-set and set-set alignments. Quantitative and qualitative comparisons for CLIP embeddings with existing dimensionality reduction (e.g., t-SNE and MDS) and data fusion (e.g., data context map) methods demonstrate the advantages of MFM in showcasing cross-modal features over common vision-language datasets. Case studies reveal that ModalChorus can facilitate intuitive discovery of misalignment and efficient re-alignment in scenarios ranging from zero-shot classification to cross-modal retrieval and generation.

Abstract:
We apply an approach from cognitive linguistics by mapping Conceptual Metaphor Theory (CMT) to the visualization domain to address patterns of visual conceptual metaphors that are often used in science infographics. Metaphors play an essential part in visual communication and are frequently employed to explain complex concepts. However, their use is often based on intuition, rather than following a formal process. At present, we lack tools and language for understanding and describing metaphor use in visualization to the extent where taxonomy and grammar could guide the creation of visual components, e.g., infographics. Our classification of the visual conceptual mappings within scientific representations is based on the breakdown of visual components in existing scientific infographics. We demonstrate the development of this mapping through a detailed analysis of data collected from four domains (biomedicine, climate, space, and anthropology) that represent a diverse range of visual conceptual metaphors used in the visual communication of science. This work allows us to identify patterns of visual conceptual metaphor use within the domains, resolve ambiguities about why specific conceptual metaphors are used, and develop a better overall understanding of visual metaphor use in scientific infographics. Our analysis shows that ontological and orientational conceptual metaphors are the most widely applied to translate complex scientific concepts. To support our findings we developed a visual exploratory tool based on the collected database that places the individual infographics on a spatio-temporal scale and illustrates the breakdown of visual conceptual metaphors.

Abstract:
This paper defines, analyzes, and discusses the emerging genre of visualization atlases. We currently witness an increase in web-based, data-driven initiatives that call themselves “atlases” while explaining complex, contemporary issues through data and visualizations: climate change, sustainability, AI, or cultural discoveries. To understand this emerging genre and inform their design, study, and authoring support, we conducted a systematic analysis of 33 visualization atlases and semi-structured interviews with eight visualization atlas creators. Based on our results, we contribute (1) a definition of a visualization atlas as a compendium of (web) pages aimed at explaining and supporting exploration of data about a dedicated topic through data, visualizations and narration. (2) a set of design patterns of 8 design dimensions, (3) insights into the atlas creation from interviews and (4) the definition of 5 visualization atlas genres. We found that visualization atlases are unique in the way they combine i) exploratory visualization, ii) narrative elements from data-driven storytelling and iii) structured navigation mechanisms. They target a wide range of audiences with different levels of domain knowledge, acting as tools for study, communication, and discovery. We conclude with a discussion of current design practices and emerging questions around the ethics and potential real-world impact of visualization atlases, aimed to inform the design and study of visualization atlases.

Abstract:
Large Language Models (LLMs) are powerful but also raise significant security concerns, particularly regarding the harm they can cause, such as generating fake news that manipulates public opinion on social media and providing responses to unethical activities. Traditional red teaming approaches for identifying AI vulnerabilities rely on manual prompt construction and expertise. This paper introduces AdversaFlow, a novel visual analytics system designed to enhance LLM security against adversarial attacks through human-AI collaboration. AdversaFlow involves adversarial training between a target model and a red model, featuring unique multi-level adversarial flow and fluctuation path visualizations. These features provide insights into adversarial dynamics and LLM robustness, enabling experts to identify and mitigate vulnerabilities effectively. We present quantitative evaluations and case studies validating our system's utility and offering insights for future AI security solutions. Our method can enhance LLM security, supporting downstream scenarios like social media regulation by enabling more effective detection, monitoring, and mitigation of harmful content and behaviors.

Abstract:
Room-scale immersive data visualisations provide viewers a wide-scale overview of a large dataset, but to interact precisely with individual data points they typically have to navigate to change their point of view. In traditional screen-based visualisations, focus-and-context techniques allow visualisation users to keep a full dataset in view while making detailed selections. Such techniques have been studied extensively on desktop to allow precise selection within large data sets, but they have not been explored in immersive 3D modalities. In this paper we develop a novel immersive focus-and-context technique based on a “magic portal” metaphor adapted specifically for data visualisation scenarios. An extendable-hand interaction technique is used to place a portal close to the region of interest. The other end of the portal then opens comfortably within the user's physical reach such that they can reach through to precisely select individual data points. Through a controlled study with 12 participants, we find strong evidence that portals reduce overshoots in selection and overall hand trajectory length, reducing arm and shoulder fatigue compared to ranged interaction without the portal. The portals also enable us to use a robot arm to provide haptic feedback for data within the limited volume of the portal region. In a second study with another 12 participants we found that haptics provided a positive experience (qualitative feedback) but did not significantly reduce fatigue. We demonstrate applications for portal-based selection through two use-case scenarios.

Abstract:
Existing deep learning-based surrogate models facilitate efficient data generation, but fall short in uncertainty quantification, efficient parameter space exploration, and reverse prediction. In our work, we introduce SurroFlow, a novel normalizing flow-based surrogate model, to learn the invertible transformation between simulation parameters and simulation outputs. The model not only allows accurate predictions of simulation outcomes for a given simulation parameter but also supports uncertainty quantification in the data generation process. Additionally, it enables efficient simulation parameter recommendation and exploration. We integrate SurroFlow and a genetic algorithm as the backend of a visual interface to support effective user-guided ensemble simulation exploration and visualization. Our framework significantly reduces the computational costs while enhancing the reliability and exploration capabilities of scientific surrogate models.

Abstract:
This paper presents an interactive technique to explain visual patterns in network visualizations to analysts who do not understand these visualizations and who are learning to read them. Learning a visualization requires mastering its visual grammar and decoding information presented through visual marks, graphical encodings, and spatial configurations. To help people learn network visualization designs and extract meaningful information, we introduce the concept of interactive pattern explanation that allows viewers to select an arbitrary area in a visualization, then automatically mines the underlying data patterns, and explains both visual and data patterns present in the viewer's selection. In a qualitative and a quantitative user study with a total of 32 participants, we compare interactive pattern explanations to textual-only and visual-only (cheatsheets) explanations. Our results show that interactive explanations increase learning of i) unfamiliar visualizations, ii) patterns in network science, and iii) the respective network terminology.

Abstract:
“Correlation does not imply causation” is a famous mantra in statistical and visual analysis. However, consumers of visualizations often draw causal conclusions when only correlations between variables are shown. In this paper, we investigate factors that contribute to causal relationships users perceive in visualizations. We collected a corpus of concept pairs from variables in widely used datasets and created visualizations that depict varying correlative associations using three typical statistical chart types. We conducted two MTurk studies on (1) preconceived notions on causal relations without charts, and (2) perceived causal relations with charts, for each concept pair. Our results indicate that people make assumptions about causal relationships between pairs of concepts even without seeing any visualized data. Moreover, our results suggest that these assumptions constitute causal priors that, in combination with visualized association, impact how data visualizations are interpreted. The results also suggest that causal priors may lead to over- or under-estimation in perceived causal relations in different circumstances, and that those priors can also impact users' confidence in their causal assessments. In addition, our results align with prior work, indicating that chart type may also affect causal inference. Using data from the studies, we develop a model to capture the interaction between causal priors and visualized associations as they combine to impact a user's perceived causal relations. In addition to reporting the study results and analyses, we provide an open dataset of causal priors for 56 specific concept pairs that can serve as a potential benchmark for future studies. We also suggest remaining challenges and heuristic-based guidelines to help designers improve visualization design choices to better support visual causal inference.

Abstract:
Visual validation of regression models in scatterplots is a common practice for assessing model quality, yet its efficacy remains unquantified. We conducted two empirical experiments to investigate individuals' ability to visually validate linear regression models (linear trends) and to examine the impact of common visualization designs on validation quality. The first experiment showed that the level of accuracy for visual estimation of slope (i.e., fitting a line to data) is higher than for visual validation of s lope (i.e., accepting a shown line). Notably, we found bias toward slopes that are “too steep” in both cases. This lead to novel insights that participants naturally assessed regression with orthogonal distances between the points and the line (i.e., ODR regression) rather than the common vertical distances (OLS regression). In the second experiment, we investigated whether incorporating common designs for regression visualization (error lines, bounding boxes, and confidence intervals) would improve visual validation. Even though error lines reduced validation bias, results failed to show the desired improvements in accuracy for any design. Overall, our findings suggest caution in using visual model validation for linear trends in scatterplots.

Abstract:
While previous work has found success in deploying visualizations as museum exhibits, it has not investigated whether museum context impacts visitor behaviour with these exhibits. We present an interactive Deep-time Literacy Visualization Exhibit (DeLVE) to help museum visitors understand deep time (lengths of extremely long geological processes) by improving proportional reasoning skills through comparison of different time periods. DeLVE uses a new visualization idiom, Connected Multi-Tier Ranges, to visualize curated datasets of past events across multiple scales of time, relating extreme scales with concrete scales that have more familiar magnitudes and units. Museum staff at three separate museums approved the deployment of DeLVE as a digital kiosk, and devoted time to curating a unique dataset in each of them. We collect data from two sources, an observational study and system trace logs. We discuss the importance of context: similar museum exhibits in different contexts were received very differently by visitors. We additionally discuss differences in our process from Sedlmair et al.'s design study methodology which is focused on design studies triggered by connection with collaborators rather than the discovery of a concept to communicate. Supplemental materials are available at: https://osf.io/z53dq/

Abstract:
Exploratory data science is an iterative process of obtaining, cleaning, profiling, analyzing, and interpreting data. This cyclical way of working creates challenges within the linear structure of computational notebooks, leading to issues with code quality, recall, and reproducibility. To remedy this, we present Loops, a set of visual support techniques for iterative and exploratory data analysis in computational notebooks. Loops leverages provenance information to visualize the impact of changes made within a notebook. In visualizations of the notebook provenance, we trace the evolution of the notebook over time and highlight differences between versions. Loops visualizes the provenance of code, markdown, tables, visualizations, and images and their respective differences. Analysts can explore these differences in detail in a separate view. Loops not only makes the analysis process transparent but also supports analysts in their data science work by showing the effects of changes and facilitating comparison of multiple versions. We demonstrate our approach's utility and potential impact in two use cases and feedback from notebook users from various backgrounds. This paper and all supplemental materials are available at https://osf.io/79eyn.

Abstract:
In the era of immersive consumer electronics, such as AR/VR headsets and smart devices, people increasingly seek ways to express their identity through virtual fashion. However, existing 3D garment design tools remain inaccessible to everyday users due to steep technical barriers and limited data. In this work, we introduce a 3D sketch-driven 3D garment generation framework that empowers ordinary users — even those without design experience — to create high-quality digital clothing through simple 3D sketches in AR/VR environments. By combining a conditional diffusion model, a sketch encoder trained in a shared latent space, and an adaptive curriculum learning strategy, our system interprets imprecise, free-hand input and produces realistic, personalized garments. To address the scarcity of training data, we also introduce KO3DClothes, a new dataset of paired 3D garments and user-created sketches. Extensive experiments and user studies confirm that our method significantly outperforms existing baselines in both fidelity and usability, demonstrating its promise for democratized fashion design on next-generation consumer platforms.

Abstract:
Combining sparse IMUs and a monocular camera is a new promising setting to perform real-time human motion capture. This paper proposes a diffusion-based solution to learn human motion priors and fuse the two modalities of signals together seamlessly in a unified framework. By delicately considering the characteristics of the two signals, the sequential visual information is considered as a whole and transformed into a condition embedding, while the inertial measurement is concatenated with the noisy body pose frame by frame to construct a sequential input for the diffusion model. Firstly, we observe that the visual information may be unavailable in some frames due to occlusions or subjects moving out of the camera view. Thus incorporating the sequential visual features as a whole to get a single feature embedding is robust to the occasional degenerations of visual information in those frames. On the other hand, the IMU measurements are robust to occlusions and always stable when signal transmission has no problem. So incorporating them frame-wisely could better explore the temporal information for the system. Experiments have demonstrated the effectiveness of the system design and its state-of-the-art performance in pose estimation compared with the previous works. The code will be released.

Abstract:
Virtual Reality (VR) technologies in fields such as telehealth, teleconferencing, and virtual education are significantly affected by end-to-end latency, which notably impacts users’ interactive experience and performance. Previous research suggests that a perceptual threshold may exist—once latency is reduced below a certain level, users no longer perceive it, and their interactive performance remains largely unaffected. However, there is no consensus on the exact value of this absolute latency perception threshold. In this study, we employed an experimental design based on Fitts’ law to investigate whether interaction strategies and task difficulty can alter the latency perception threshold (LPT), and how variations in this threshold influence users’ interactive performance. The results show that the LPT is approximately 130-170 ms, and that when interaction strategies prioritize speed or when tasks become more challenging, users exhibit heightened sensitivity to latency. Due to the presence of the LPT, the effect of latency on interactive performance follows a nonlinear pattern, and building on this finding, we refined a Fitts’ law model to incorporate the influence of latency. Notably, electroencephalogram (EEG) signals can still capture users’ perception of latency when they are unaware of minor latency, demonstrating a level of sensitivity that exceeds conscious awareness. Our findings provide insights into latency effects on performance and perception, guiding the design of more responsive VR interaction systems.

Abstract:
Due to the intrinsic complexity of high-dimensional (HD) data, dimensionality reduction (DR) techniques cannot preserve all the structural characteristics of the original data. Therefore, DR techniques focus on preserving either local neighborhood structures (local techniques) or global structures such as pairwise distances between points (global techniques). However, both approaches can mislead analysts to erroneous conclusions about the overall arrangement of manifolds in HD data. For example, local techniques may exaggerate the compactness of individual manifolds, while global techniques may fail to separate clusters that are well-separated in the original space. In this research, we provide a deeper insight into Uniform Manifold Approximation with Two-phase Optimization (UMATO), a DR technique that addresses this problem by effectively capturing local and global structures. UMATO achieves this by dividing the optimization process of UMAP into two phases. In the first phase, it constructs a skeletal layout using representative points, and in the second phase, it projects the remaining points while preserving the regional characteristics. Quantitative experiments validate that UMATO outperforms widely used DR techniques, including UMAP, in terms of global structure preservation, with a slight loss in local structure. We also confirm that UMATO outperforms baseline techniques in terms of scalability and stability against initialization and subsampling, making it more effective for reliable HD data analysis. Finally, we present a case study and a qualitative demonstration that highlight UMATO’s effectiveness in generating faithful projections, enhancing the overall reliability of visual analytics using DR.

Abstract:
We present a method for modeling and rendering irregular and heterogeneous glass objects, with a specific emphasis on stained glass windows and window works often encountered in architecture from middle age to 18th century. The artisanal production of sheet glass results in glass panels displaying a vast variety of surface and volume irregularities like bubbles, irregular surface or smoothly varying refractive index, all of which contribute to the specific visual aspect of old glass. We propose to account for all the aforementioned effects in a unified framework based on signed distance functions and an analytic solution of the ray tracing equations on tetrahedral volume elements. We demonstrate how to construct an unbiased estimator for the transmitted lighting produced by such panels by using Fermat’s principle and results from seismic ray theory. We use texture coordinates to map arbitrary sections of a complex glass panel onto the individual faces of a mesh, allowing the modeling and rendering of complex 3-dimensional objects composed of colored glass facets such as stained glass windows.

Affiliations: Department of Electrical Engineering, Graduate Institute of Communication Engineering, National Taiwan University, Taipei, Taiwan; Department of Psychology, Graduate Institute of Brain and Mind Sciences, Neurobiology and Cognitive Science Center, Center for Artificial Intelligence and Advanced Robotics, National Taiwan University, Taipei, Taiwan; Department of Electrical Engineering, Graduate Institute of Communication Engineering, and Graduate Institute of Networking and Multimedia, National Taiwan University, Taipei, Taiwan

Abstract:
Light field AR glasses can provide better visual comfort than conventional AR glasses; however, studies on user performance comparison between them are notably scarce. In this article, we present a systematic method employing a serial visual search task without confounding factors to quantify and compare the user performance and experience between these two types of AR glasses at two different viewing distances, 30 cm and 60 cm, and in two modes, purely virtual VR mode and virtual-real integration AR mode. The results show that the light field AR glasses led to a significantly faster reaction speed and higher accuracy than the conventional AR glasses at 30 cm in the AR mode. The participant feedback also shows that the former led to better virtual-real integration. User performance and experience of the light field AR glasses remained consistent across different viewing distances. Although the conventional AR glasses had a better search efficiency than the light field AR glasses at 60 cm in both AR and VR modes, it had more negative feedback from the participants. Overall, the design of this experiment successfully allows us to quantify the effect of VAC and underscores the strength of the evaluation method.

Abstract:
Designing interactions between humans and virtual characters requires careful consideration of various human perceptions and user experiences. While numerous studies have explored the effects of several virtual characters' properties, the impacts of the virtual character's personality and age on human perceptions and experiences have yet to be thoroughly investigated. To address this gap, we conducted a within-group study (N = 28) following a 2 (personality: egoism vs. altruism) × 2 (age: child vs. adult) design to explore how the personality and age factors influence human perception and experience during interactions with virtual characters. In each condition of our study, our participants co-solved a jigsaw puzzle with a virtual character that embodied combinations of personality and age. After each condition, participants completed a survey. We also asked them to provide written feedback at the end of the study. Our statistical analyses revealed that the virtual character's personality and age significantly influenced participants' perceptions and experiences. The personality factor affected perceptions of altruism, anthropomorphism, likability, safety, and all aspects of user experience, including perceived collaboration, rapport, emotional reactivity, and the desire for future interaction. Additionally, the virtual character's age affected our participants' ratings of the uncanny valley and likability. We also identified an interaction effect between personality and age factors on the virtual character's anthropomorphism. Based on our findings, we offered guidelines and insights for researchers aiming to design collaborative experiences with virtual characters of different personalities and ages.

Abstract:
The virtual content in augmented reality (AR) can introduce misleading or harmful information, leading to semantic misunderstandings or user errors. In this work, we focus on visual information manipulation (VIM) attacks in AR, where virtual content changes the meaning of real-world scenes in subtle but impactful ways. We introduce a taxonomy that categorizes these attacks into three formats: character, phrase, and pattern manipulation, and three purposes: information replacement, information obfuscation, and extra wrong information. Based on the taxonomy, we construct a dataset, AR-VIM, which consists of 452 raw-AR video pairs spanning 202 different scenes, each simulating a real-world AR scenario. To detect the attacks in the dataset, we propose a multimodal semantic reasoning framework, VIM-Sense. It combines the language and visual understanding capabilities of vision-language models (VLMs) with optical character recognition (OCR)-based textual analysis. VIM-Sense achieves an attack detection accuracy of 88.94% on AR-VIM, consistently outperforming vision-only and text-only baselines. The system achieves an average attack detection latency of 7.07 seconds in a simulated video processing framework and 7.17 seconds in a real-world evaluation conducted on a mobile Android AR application.

Abstract:
Immersive 3D mid-air sketching systems liberate users from the confines of traditional 2D sketching canvases. However, complications from perceptual challenges in Virtual Reality (VR), combined with the ergonomic and cognitive challenges of sketching in all three dimensions in mid-air lower the accuracy and aesthetic quality of 3D sketches. This paper explores how color-to-depth and opacity cues support users to create and perceive freehand, 3D strokes in room-scale sketching, unlocking a full 360° of freedom for creation. We implemented three graphic depth shader cues modifying the (1) alpha, (2) hue, and (3) value levels of a single color to dynamically adjust the color and transparency of meshes relative to their depth from the user. We investigated how these depth cues influence sketch efficiency, sketch quality, and total sketch experience with 24 participants in a comparative, counterbalanced, 4 x 1 within-subjects user study. First, with our graphic depth shader cues we could successfully transfer results of prior research in seated sketching tasks to room-scale scenarios. Our color-to-depth cues improved the similarity of sketches to target models. This highlights the usefulness of the color-to-depth approach even for the increased range of motion and depth in room-scale sketching. Second, our shaders assisted participants to complete tasks faster, spend a greater percentage of task time sketching, reduced the feeling of mental tiredness and improved the feeling of sketch efficiency in room-scale sketching. We discuss these findings and share our insights and conclusions to advance the research on improving spatial cognition in immersive sketching systems.

Abstract:
Simultaneous localization and mapping (SLAM) plays a fundamental role in extended reality (XR) applications. As the standards for immersion in XR continue to increase, the demands for SLAM benchmarking have become more stringent. Trajectory accuracy is the key metric, and marker-based optical motion capture (MoCap) systems are widely used to generate ground truth (GT) because of their drift-free and relatively accurate measurements. However, the precision of MoCap-based GT is limited by two factors: the spatiotemporal calibration with the device under test (DUT) and the inherent jitter in the MoCap measurements. These limitations hinder accurate SLAM benchmarking, particularly for key metrics like rotation error and inter-frame jitter, which are critical for immersive XR experiences. This paper presents a novel continuous-time maximum likelihood estimator to address these challenges. The proposed method integrates auxiliary inertial measurement unit (IMU) data to compensate for MoCap jitter. Additionally, a variable time synchronization method and a pose residual based on screw congruence constraints are proposed, enabling precise spatiotemporal calibration across multiple sensors and the DUT. Experimental results demonstrate that our approach outperforms existing methods, achieving the precision necessary for comprehensive benchmarking of state-of-the-art SLAM algorithms in XR applications. Furthermore, we thoroughly validate the practicality of our method by benchmarking several leading XR devices and open-source SLAM algorithms. The code is publicly available at https://github.com/ylab-xrpg/xr-hpgt.

Abstract:
Measuring visitor engagement in informal learning environments presents critical challenges for optimizing educational experiences and spatial design. While existing research predominantly focuses on formal settings, systematic analysis of multidimensional engagement in complex environments like museums remains underdeveloped. This study introduces the first integrated head-mounted display (HMD)-based framework combining meso-scale spatial analysis through behavioral engagement heatmaps with micro-level temporal engagement modeling via headset pose and eye-tracking data. Our edge-optimized long short-term memory (LSTM) model achieves real-time engagement measurement with a mean squared error (MSE) of 0.145 using four physiologically grounded features from a two-stage user study (N=20 for feature analysis, N=15 for modeling). The framework synthesizes planar trajectory heatmaps and panoramic fixation distributions to enable both real-time adaptive support and post-hoc exhibition design insights. Results demonstrate HMDs' potential as precision measurement tools, establishing methodological foundations for intelligent heritage environments that dynamically respond to engagement states through integrated meso-micro analytics.

Abstract:
Virtual reality (VR) is increasingly adopted for collaborative teaching and learning, enabling immersive and interactive experiences. As Artificial Intelligence (AI) tutors begin to take on roles alongside human instructors, it becomes crucial to understand how their integration influences interaction dynamics and role perception in these settings. This study investigates the role of Embodied Virtual Agents (EVAs) substituting human instructors in virtual training, specifically addressing the previously underexplored issue of EVA appearance consistency with instructors in Human-Agent Teaming (HAT). We recruited 21 participants to compare three conditions: No Agent, Shared-Appearance (SA) EVA, and Unique-Appearance (UA) EVA, where an EVA substitutes for the instructor during temporary absences. We evaluated collaboration efficiency, user perception/preference, and HAT dynamics. Our findings confirm that EVAs significantly enhance task efficiency compared to no support and reveal a key trade-off regarding appearance: SA fosters perceived continuity and trust but risks ambiguity and uncanny effects, while UA provides transparency and role clarity but may disrupt experiential coherence. These results have implications for designing dynamic HAT systems where control may shift. We discuss the benefits and limitations of each approach and offer design recommendations for future mixed-agency interfaces.

Abstract:
Thanks to the development of virtual reality (VR) technology, there is growing research on VR avatar body deformation effects. However, previous research mainly focused on passive body deformation expression, leaving users with limited methods to actively control their virtual bodies. To address this gap, we explored user-controlled forearm deformation by investigating how hand and forearm gestures can be mapped to various degrees of avatar forearm deformation. We conducted a gesture design workshop with six designers to generate gesture sets for different forearm deformations and deformation degrees, resulting in 15 gesture sets. Then, we selected the three highest-rated gesture sets and conducted a comparative study to evaluate the sense of embodiment and user performance across the three gesture sets. Our findings provide design suggestions for gesture-controlled forearm deformation in VR.

Abstract:
Large language models (LLMs) have evolved into LLM agents that can use the user conversation context and respond according to the roles of the LLM agents. Recent studies have suggested that LLM-based agents can be used as human-like partners in social interactions. However, the role of the environmental context, particularly spatial information of user space, in the interaction between humans and LLM agents has not been explored. In this study, participants engaged in counselling conversations under three different conditions based on their spatial awareness levels. The dependent measures included copresence, trust, therapist alliances, and self-disclosure. The results suggested that participants in the condition where the LLM actively reflected spatial information generally reported higher levels of user experience. Interestingly, when the LLM actively reflected the spatial context of the user, the participants tended to describe themselves and express their emotions more. These findings suggest that spatially aware LLM agents can contribute to better social interactions between humans and LLM agents. Our findings can be used to design future augmented reality applications in the counselling, education, and healthcare industries.

Abstract:
Zoning out, a form of cognitive disengagement, seriously challenges the effectiveness of virtual reality (VR) based upper limb rehabilitation. As therapy often involves repetitive tasks requiring sustained attention, undetected lapses in focus can reduce motor learning, engagement, and overall recovery outcomes. This research addresses this gap by proposing ZonAware, a novel strategy integrating real-time zoning out detection with adaptive intervention to enhance user engagement during VR rehabilitation. ZonAware identifies zoning out using five eye-tracking metrics: blink frequency, blink duration, pupil size, eye openness, and gaze duration. These signals are analysed through lightweight statistical models (Z-Score, Boxplot, and Modified Z-Score), with a hard voting mechanism producing binary classifications in real-time. Upon detection, a pattern changing intervention subtly modulates task difficulty by temporarily increasing, then decreasing it, to regain user focus without breaking immersion. Three user studies involving 70 healthy participants and 22 patients demonstrated the strategy's effectiveness. ZonAware achieved 98.24% detection accuracy with low latency (82–150 ms), reducing zoning out frequency by 53.57% and shortening disengagement duration from 18.1 to 4.8 seconds. The approach also improved user engagement, performance, and emotional motivation. ZonAware delivers one of the first real-time zoning out solutions for VR rehabilitation, offering an interpretable, theory-driven approach that enhances attention, engagement, and adaptability in human-computer interaction.

Abstract:
Swarm fragmentation, the breakdown of communication and coordination among robots, can critically compromise a swarm's mission. Integrating Augmented Reality support into swarm monitoring-especially through co-located visualisations anchored directly on the robots- may enable human operators to detect early signs of fragmentation and intervene effectively. In this work, we propose three localised visual cues-targeting robot connectivity, dominant decision influences, and movement direction-to make explicit the underlying Perception-Decision-Action (PDA) loop of each robot. Through an immersive Virtual Reality user study, 51 participants were tasked with both anticipating potential fragmentation and selecting the appropriate control to prevent it, while observing swarms exhibiting expansion, densification, flocking, and swarming behaviours. Our results reveal that a visualisation emphasising inter-robot connectivity significantly improves anticipation of fragmentation, though none of the cues consistently enhance control selection over a baseline condition. These findings underscore the potential of co-located AR-enhanced visual feedback to support human-swarm interaction and inform the design of future AR-based supervisory systems for robot swarms. A free copy of this paper and all supplemental materials are available at https://osf.io/49gny.

Abstract:
Dynamic Facial Projection Mapping (DFPM) overlays computer-generated images onto human faces to create immersive experiences that have been used in the makeup and entertainment industries. In this study, we propose two concepts to reduce the misalignment artifacts between projected images and target faces, which is a persistent challenge for DFPM. Our first concept is a high-speed face-tracking method that exploits temporal information. We first introduce a cropped-area-limited inter/extrapolation-based face detection framework, which allows parallel execution with facial landmark detection. We then propose a novel hybrid facial landmark detection method that combines fast Ensemble of Regression Trees (ERT)-based detections and an auxiliary detection. ERT-based detections rapidly produce results in 0.107 ms using temporal information with the support of auxiliary detection to recover from detection errors. To train the facial landmark detection method, we propose an innovative method for simulating high-frame-rate video annotations to address the lack of publicly available high-frame-rate annotated datasets. Our second concept is a lens-shift co-axial projector-camera setup that maintains a high optical alignment with only a 1.274-pixel error between 1 m and 2 m depth. This setup reduces misalignment by applying the same optical designs to the projector and camera without causing large misalignment as in conventional methods. Based on these concepts, we developed a novel high-speed DFPM system that achieves nearly perfect alignment with human visual perception.

Affiliations: Department of Computer Science and Guangxi Key Laboratory of Multimedia Communications and Network Technology, Guangxi University, Nanning, China; Department of Computer Science, University of Rochester, Rochester, NY, USA; School of Creative Media and the Department of Computer Science, City University of Hong Kong, Kowloon Tong, Hong Kong; Department of Computer Science and Engineering, Hong Kong University of Science and Technology, Hong Kong; Department of Computer Science and Technology, Tsinghua University, Beijing, China

Abstract:
Heads-up computing aims to provide synergistic digital assistance that minimally interferes with users’ on-the-go daily activities. Currently, the input modalities of heads-up computing are mainly voice and finger gestures. In this work, we propose and evaluate the AmplitudeArrow (AA) technique designed for on-the-go AR menu selection to demonstrate that consecutive simple head gestures can also be an effective input modality for heads-up computing. Specifically, AA arranges menu icons into one/two row(s). To select a target icon, the user first makes their head yaw to pre-select the target icon or the column containing it and then makes their head pitch to make the arrow in the target icon expand until the arrow covers the target icon completely, i.e., the pitch amplitude surpasses the selection confirmation threshold. User studies indicated that AA demonstrated robust resistance to walking-caused head perturbation and external factors such as other people/obstacles, delivering high accuracy (error rate < < 5%%) and fast speed (< < 1.5s per selection) when there were no more than six icon columns (twelve icons) distributed horizontally and evenly in a menu area with a horizontal visual angle of 43^\circ 43∘.

Abstract:
Network visualizations are understudied in graphical perception. As a result, most network visualization designs still largely rely on designer intuition and algorithm optimizations rather than being guided by knowledge of human perception. The lack of perceptual understanding of network visualizations also limits the generalizability of past empirical evaluations, given their focus on performance over causal interpretation. To bridge this gap between perception and network visualization, we introduce a framework highlighting five key perceptual mechanisms used in node-link diagrams and adjacency matrices: attention, visual search, perceptual organization, ensemble coding, and object recognition. Our framework describes the role these perceptual mechanisms play in common network analytical tasks. We use the framework to revisit four past empirical investigations and outline future design experiments that can help produce more perceptually effective network visualizations. We anticipate this connection will afford translational understanding to guide more effective network visualization design and offer hypotheses for perception-aware network visualizations.

Abstract:
Celtic knots, an ancient art form often linked to Celtic heritage, have been used historically in the decoration of monuments and manuscripts, often symbolizing the notions of eternity and interconnectedness. This paper introduces the framework CelticGraph designed for illustrating graphs in the style of Celtic knots and links. The process of creating these drawings raises interesting combinatorial concepts in the theory of circuits in planar graphs. Further, CelticGraph uses a novel algorithm to represent edges as Bézier curves, aiming to show each link as a smooth curve with limited curvature. We also show that with our production mechanisms we can compute any 4-regular plane graph and thereby any celtic knot or link. The CelticGraph framework for drawing graphs as celtic knots and links is implemented as an add-on of Vanted, a network visualization and analysis tool.

Abstract:
Visual Analytics (VA) has become a paramount discipline in supporting data analysis in many scientific domains, empowering the human user with automatic capabilities while keeping the lead in the analysis. At the same time, designing an effective VA solution is not a simple task, requiring its adaptation to the problem at hand and the intended user of the system. In this scenario, the User-Centered Design (UCD) methodology provides the framework to incorporate user needs into the design of a VA solution. On the other hand, its implementation mainly relies on qualitative feedback, with the designer missing tools supporting her in quantitatively reporting the user feedback and using it to hypothesize and test the successive changes to the VA solution. To overcome this limitation, we propose TraVIS, a Visual Analytics solution allowing the loading of a web-based VA system, collecting user traces, and analyzing them with respect to the system at hand. In this process, the designer can leverage the collected traces and relate them to the tasks the VA solution supports and how those can be achieved. Using TraVIS, the designer can identify ineffective interaction paths, analyze the user traces support to task completion, hypothesize corrections to the design, and evaluate the effect of changes. We evaluated TraVIS through experimentation with 11 VA systems from literature, a use case, and user evaluation with five experts. Results show the benefits that TraVIS provides in terms of identifying design problems and efficient support for UCD.

Abstract:
Data visualization recommendation aims to assist the user in creating visualizations from a given dataset. The process of creating appropriate visualizations requires expert knowledge of the available data model as well as the dashboard application that is used. To relieve the user from requiring this knowledge and from the manual process of creating numerous visualizations or dashboards, we present a context-aware visualization recommender system (VisCARS) for monitoring applications that automatically recommends a personalized dashboard to the user, based on the system they are monitoring and the task they are trying to achieve. Through a knowledge graph-based approach, expert knowledge about the data and the application is included as contextual features to improve the recommendation process. A dashboard ontology is presented that describes key components in a dashboard ecosystem in order to semantically annotate all the knowledge in the graph. The recommender system leverages knowledge graph embedding and comparison techniques in combination with a context-aware collaborative filtering approach to derive recommendations based on the context, i.e., the state of the monitored system, and the end-user preferences. The proposed methodology is implemented and integrated in a dynamic dashboard solution. The resulting recommender system is evaluated on a smart healthcare use-case through a quantitative performance and scalability analysis as well as a qualitative user study. The results highlight the performance of the proposed solution compared to the state-of-the-art and its potential for time-critical monitoring applications.

Abstract:
We contribute an analysis of the prevalence and relative performance of archetypal VR menu techniques. An initial survey of 108 menu interfaces in 84 popular commercial VR applications establishes common design characteristics. These characteristics motivate the design of raycast, direct, and marking menu archetypes, and a two-experiment comparison of their relative performance with one and two levels of hierarchy using 8 or 24 items. With a single-level menu, direct input is the fastest interaction technique in general, and is unaffected by number of items. With a two-level hierarchical menu, marking is fastest regardless of item number. Menus using raycasting, the most common menu interaction technique, were among the slowest of the tested menus but were rated most consistently usable. Using the combined results, we provide design and implementation recommendations with applications to general VR menu design.

Abstract:
The reconstruction of indoor scenes from multi-view RGB images is challenging due to the coexistence of flat and texture-less regions alongside delicate and fine-grained regions. Recent methods leverage neural radiance fields aided by predicted surface normal priors to recover the scene geometry. These methods excel in producing complete and smooth results for floor and wall areas. However, they struggle to capture complex surfaces with high-frequency structures due to the inadequate neural representation and the inaccurately predicted normal priors. This work aims to reconstruct high-fidelity surfaces with fine-grained details by addressing the above limitations. To improve the capacity of the implicit representation, we propose a hybrid architecture to represent low-frequency and high-frequency regions separately. To enhance the normal priors, we introduce a simple yet effective image sharpening and denoising technique, coupled with a network that estimates the pixel-wise uncertainty of the predicted surface normal vectors. Identifying such uncertainty can prevent our model from being misled by unreliable surface normal supervisions that hinder the accurate reconstruction of intricate geometries. Experiments on the benchmark datasets show that our method outperforms existing methods in terms of reconstruction quality. Furthermore, the proposed method also generalizes well to real-world indoor scenarios captured by our hand-held mobile phones.

Abstract:
Due to sensor limitations, the light field (LF) images captured by the LF camera suffer from low dynamic range and are prone to poor exposure. To solve this problem, combining multi-exposure technology with LF camera imaging can achieve high dynamic range (HDR) LF imaging. However, for dynamic scenes, this approach tends to produce disturbing ghosting artifacts and destroy the parallax structure of the generated results. To this end, this paper proposes a novel ghost-free HDR LF imaging method using multi-attention learning and exposure guidance. Specifically, the proposed method first designs a multi-scale cross-attention module to achieve efficient multi-exposure LF feature alignment. After that, a dual self-attention-driven Transformer block is constructed to excavate the geometric information of LF and fuse the aligned LF features. In particular, exposure masks derived from middle-exposure are introduced in the feature fusion to guide the network to focus on information recovery in low- and high-brightness regions. Besides, a local compensation module is integrated to cope with local alignment errors and refine details. Finally, a multi-objective reconstruction strategy combined with exposure masks is employed to restore high-quality HDR LF images. Extensive experimental results on the benchmark dataset show that the proposed method generates HDR LF results with high spatial-angular quality consistency and outperforms the state-of-the-art methods in quantitative and qualitative comparisons. Furthermore, the proposed method can enhance the performance of existing LF applications, such as depth estimation.

Abstract:
The walking sensation is a result of the synthesis of multisensory inputs from various systems. The vestibular system, typically used for detecting acceleration, is a crucial component of the walking sensation. This study investigated the use of galvanic vestibular stimulation(GVS) to enhance the sensation of walking in virtual reality (VR) environments, particularly when users are seated and not engaged in active movements. GVS is a transcutaneous electric stimulation technique to evoke vestibular sensory responses and involves the application of a penetrating current to vestibular afferents. This study revealed that the pseudo-walking sensation can be intensified by applying lateral GVS. However, no difference was observed when it was synchronized with the walking rhythm represented by foot-sole vibration patterns. Furthermore, the study compares the effectiveness of lateral versus anterior-posterior GVS in enhancing walking sensations in VR. The findings provide novel perspectives on enhancing the VR walking experience through vestibular stimulation, even in scenarios in which the user is seated.

Abstract:
Grounded theory (GT) is a research methodology that entails a systematic workflow for theory generation grounded on emergent data. In this article, we juxtapose GT workflows with typical workflows in visualization and visual analytics (VIS), unveiling the characteristics shared by these workflows. We explore the research landscape of VIS to study where GT is applied to generate VIS theories, explicitly as well as implicitly. We discuss “why” GT can potentially play a significant role in VIS. We outline a “how” methodology for conducting GT research in VIS, which addresses the need for theoretical advancement in VIS while benefiting from other methods and techniques in VIS. We illustrate this “how” methodology with a use case of adopting GT approaches in studying visualization guidelines.

Abstract:
Extracting geometric features from 3D point clouds is widely applied in many tasks, including registration and recognition. We propose a simple yet effective method, termed height-azimuth image based transformation-invariant net (HA-TiNet), to learn a distinctive, general and rotation-invariant 3D local descriptor. HA-TiNet is composed of a height-azimuth image generator and a feature extraction net. Based on a local reference axis (LRA), the height-azimuth image generator first partitions local region along the plane-radial direction, and then implements a statistic of height and azimuth information in each divided space to generate a set of height-azimuth images. The generated height-azimuth images are invariant in the rotation around x- and y-axes and have high accuracy due to the high repeatability of an LRA. Besides, they can be easily embedded in 2D convolutional neural networks (CNNs). Our feature extraction net learns the information on the height-azimuth images using a ResNet-based backbone and a rotation-invariant layer. The ResNet-based backbone is lightweight while very effective. The rotation-invariant layer removes the rotation-variance around z-axis, making our descriptor have full rotation-invariance. Extensive experiments on indoor and outdoor datasets show that our method presents superior overall performance, and exhibits strong descriptiveness and generalization ability compared to the state-of-the-art descriptors.

Abstract:
Sensemaking on a large collection of documents (corpus) is a challenging task often found in fields such as market research, legal studies, intelligence analysis, political science, or computational linguistics. Previous works approach this problem from topic- and entity-based perspectives, but the capability of the underlying NLP model limits their effectiveness. Recent advances in prompting with LLMs present opportunities to enhance such approaches with higher accuracy and customizability. However, poorly designed prompts and visualizations could mislead users into falsely interpreting the visualizations and hinder the system's trustworthiness. In this paper, we address this issue by taking into account the user analysis tasks and visualization goals in the prompt-based data extraction stage, thereby extending the concept of Model Alignment. We present HINTs, a VA system for supporting sensemaking on large collections of documents, combining previous entity-based and topic-based approaches. The visualization pipeline of HINTs consists of three stages. First, entities and topics are extracted from the corpus with prompts. Then, the result is modeled as a hypergraph and hierarchically clustered. Finally, an enhanced space-filling curve layout is applied to visualize the hypergraph for interactive exploration. The system further integrates an LLM-based intelligent chatbot agent in the interface to facilitate the sensemaking of interested documents. To demonstrate the generalizability and effectiveness of the HINTs system, we present two case studies on different domains and a comparative user study. We report our insights on the behavior patterns and challenges when intelligent agents are used to facilitate sensemaking. We find that while intelligent agents can address many challenges in sensemaking, the visual hints that visualizations provide are still necessary. We discuss limitations and future work for combining interactive visualization and LLMs more profoundly to better support corpus analysis.

Abstract:
Augmented Reality (AR) teleconferencing allows spatially distributed users to interact with each other in 3D through agents in their own physical environments. Existing methods leveraging volumetric capturing and reconstruction can provide a high-fidelity experience but are often too complex and expensive for everyday use. Other solutions target mobile and effortless-to-setup teleconferencing on AR Head Mounted Displays (HMD). They directly transplant the conventional video conferencing onto an AR-HMD platform or use avatars to represent remote participants. However, they can only support either a high fidelity or a high level of co-presence. Moreover, the limited Field of View (FoV) of HMDs could further degrade users’ immersive experience. To achieve a balance between fidelity and co-presence, we explore using life-size 2D video-based avatars (video avatars for short) in AR teleconferencing. Specifically, with the potential effect of FoV on users’ perception of proximity, we first conducted a pilot study to explore the local-user-centered optimal placement of video avatars in small-group AR conversations. With the placement results, we then implement a proof-of-concept prototype of video-avatar-based teleconferencing. We conduct user evaluations with our prototype to verify its effectiveness in balancing fidelity and co-presence. Following the indication in the pilot study, we further quantitatively explore the effect of FoV size on the video avatar’s optimal placement through a user study involving more FoV conditions in a VR-simulated environment. We regress placement models to serve as references for computationally determining video avatar placements in such teleconferencing applications on various existing AR HMDs and future ones with bigger FoVs.

Abstract:
Fast detection of exact point-to-point geodesic paths on meshes is still challenging with existing methods. For this, we present a method to reduce the region to be investigated on the mesh for efficiency. It is by our observation that a mesh and its simplified one are very alike so that the geodesic path between two defined points on the mesh and the geodesic path between their corresponding two points on the simplified mesh are very near to each other in the 3D Euclidean space. Thus, with the geodesic path on the simplified mesh, we can generate a region on the original mesh that contains the geodesic path on the mesh, called the search region, by which existing methods can reduce the search scope in detecting geodesic paths, and so obtaining acceleration. We demonstrate the rationale behind our proposed method. Experimental results show that we can promote existing methods well, e.g., the global exact method VTP (vertex-oriented triangle propagation) can be sped up by even over 200 times when handling large meshes. Our search region can also speed up path initialization using the Dijkstra algorithm to promote local methods, e.g., obtaining an acceleration of at least two times in our tests.

Abstract:
Implicit neural representation (INR) has been a powerful paradigm for effectively compressing time-varying volumetric data. However, the optimization process can span days or even weeks due to its reliance on coordinate-based inputs and outputs for modeling volumetric data. To address this issue, we introduce a divide-and-conquer INR (DCINR), significantly accelerating the compressing process of time-varying volumetric data in hours. Our approach starts by dividing the data set into a set of non-overlapping blocks. Then, we apply a block selection strategy to weed out redundant blocks to reduce the computation cost without sacrificing performance. In parallel, each selected block is modeled by a tiny INR, with the size of the INR being adapted to match the information richness in the block. The block size is determined by maximizing the average network capacity. After optimization, the optimized INRs are utilized to decompress the data set. By evaluating our approach across various time-varying volumetric data sets, DCINR surpasses learning-based and lossy compression approaches in compression ratio, visual fidelity, and various performance metrics. Additionally, this method operates within a comparable compression time to that of lossy compressors, achieves extreme compression ratios ranging from thousands to tens of thousands, and preserves features with high quality.

Abstract:
Augmented Reality (AR) has long been expected to help users improve their working efficiency. However, due to the absence of intelligent systems, existing AR applications are greatly affected by the virtual content interference with real-world activities. Unlike existing work, which focuses more on hiding virtual content to reduce interference, in this work, we propose an innovative AR Task Support System where virtual contents actively guide users with task completion. During task execution, our system proactively searches for and tracks key objects in the scene, and uses this context information to automatically select appropriate virtual content and display positions. Through introducing open-world prompt-based visual models, our system can effectively retrieve few-shot or even zero-shot objects that are uncommon in the dataset. This approach extends the use of AR Task Support System beyond controlled industrial settings to more uncontrolled daily scenarios, overcoming the limitations of existing systems. It also significantly reduces development costs for developers. We demonstrate the advantages of our system over traditional virtual content management systems through a series of experiments that are closer to users’ real usage situations.

Abstract:
Extended reality (XR)—which includes virtual reality (VR) and augmented reality (AR)—is becoming increasingly popular for sharing scientific knowledge. This research evaluates the state-of-the-art in XR for scientific communication. Our two-phase methodology began with a Systematic Literature Review, identifying 94 relevant articles and conference papers from the last decade (2013-2023) sourced from the Web of Science and SCOPUS databases. These publications show scholars and practitioners using XR to convey scientific findings, foster awareness, ignite interest, shape opinions, and enhance understanding. In the second phase, we applied data clustering and analysis. Our findings highlight a significant increase in XR studies over the last decade, with the XR technologies used for communication (N = 24), dissemination (N = 23), educational/training (N = 21), and decision-making (N = 10). Our results indicate the need to establish clearer guidelines for aligning science communication and to create more possibilities to publish peer-reviewed research in.

Abstract:
As virtual objects contain increasingly rich attribute information, small wearable fingertip devices need to have higher degrees of freedom (DoFs) to convey the haptic sensation of virtual objects. In order to effectively display the shape features of virtual objects to users through curvature, we designed a 6-DoF wearable fingertip device (WFD). This WFD combines a 6-DoF Stewart parallel mechanism, consisting of a static platform and a mobile platform connected by six revolute-spherical-spherical kinematic chains. The translation and rotation of the mobile platform are driven by six miniature servo motors, which can simulate haptic sensations such as making and breaking contact, sliding, and skin stretch when the fingertip interacts with a virtual surface. The WFD is fixed at the user’s dominant index finger using hook-and-loop fasteners, with a size of 68 × 59 × 56 mm^33 and a mass of 45.5 g. We analyzed and validated the kinematic model of the WFD and tested its force output capability. Finally, we invited 15 adults to conduct three subjective perception experiments to evaluate the performance of the WFD in curvature perception and shape display. The experimental results show that: (1) The just noticeable difference (JND) for curvature identification using the WFD is 3.02\pm±0.23 m^-1-1; (2) The 6-DoF haptic feedback provided by the WFD improves the accuracy of curved surface recognition from 53.4\pm±7.1% in 3-DoF to 72.0\pm±5.9%; (3) Even without visual feedback, the shape recognition accuracy of the WFD when combined with the Touch device reaches 82.3\pm±8.2% . Experimental results show that the WFD has good performance and potential in curvature perception and shape display.

Abstract:
Geospatial data are often uncertain due to measurement, spatial, or temporal limitations. A knowledge gap exists about how geospatial uncertainty visualization techniques influence human factors measures. This comprehensive review synthesized the current literature on visual representations of uncertainty in geospatial data applications, identifying the breadth of techniques and the relationships between strategies and human performance and perception outcomes. Eligible articles described and evaluated at least one method for representing uncertainty in geographical data with participants, including land, ocean, weather, climate, and positioning data. Forty articles were included. Uncertainty was visualized using multivariate and univariate maps through colours, shapes, boundary regions, textures, symbols, grid noise, and text. There were varying effects, and no definitive superior method was identified. The predominant user focus was on novices. Trends were observed in supporting users understand uncertainty, user preferences, confidence, decision-making performance, and response times for different techniques and application contexts. The findings highlight the impacts of different categorizations within colour and shape techniques, heterogeneity in perception and performance evaluation, performance and perception mismatch, and differences and similarities between novices and experts. Contextual factors and user characteristics, including understanding the decision-maker's tasks, user type, and desired outcomes for decision-support appear to be important factors influencing the design of effective uncertainty visualizations. Future research on geospatial applications of uncertainty visualizations can expand on the observed trends with consistent and standardized measurement and reporting, further explore human performance and perception impacts with 3-dimensional and interactive uncertainty visualizations, and perform real-world evaluations within various contexts.

Abstract:
Measuring the 3D shape of semi-transparent surfaces with projector-camera 3D scanners is a difficult task because these surfaces weakly reflect light in a diffuse manner, and transmit a large part of the incident light. The task is even harder in the presence of participating background surfaces. The two methods proposed in this paper use sinusoidal patterns, each with a frequency chosen in the frequency range allowed by the projection optics of the projector-camera system. They differ in the way in which the camera-projector correspondence map is established, as well as in the number of patterns and the processing time required. The first method utilizes the discrete Fourier transform, performed on the intensity signal measured at a camera pixel, to inventory projector columns illuminating directly and indirectly the scene point imaged by that pixel. The second method goes beyond discrete Fourier transform and achieves the same goal by fitting a proposed analytical model to the measured intensity signal. Once the one (camera pixel) to many (projector columns) correspondence is established, a surface continuity constraint is applied to extract the one to one correspondence map linked to the semi-transparent surface. This map is used to determine the 3D point cloud of the surface by triangulation. Experimental results demonstrate the performance (accuracy, reliability) achieved by the proposed methods.

Abstract:
Accurate segmentation of 3D point clouds in indoor scenes remains a challenging task, often hindered by the labor-intensive nature of data annotation. While weakly supervised learning approaches have shown promise in leveraging partial annotations, they frequently struggle with imbalanced performance between foreground and background elements due to the complex structures and proximity of objects in indoor environments. To address this issue, we propose a novel foreground-aware label enhancement method utilizing visual boundary priors. Our approach projects 3D point clouds onto 2D planes and applies 2D image segmentation to generate pseudo-labels for foreground objects. These labels are subsequently back-projected into 3D space and used to train an initial segmentation model. We further refine this process by incorporating prior knowledge from projected images to filter the predicted labels, followed by model retraining. We introduce this technique as the Foreground Boundary Prior (FBP), a versatile, plug-and-play module designed to enhance various weakly supervised point cloud segmentation methods. We demonstrate the efficacy of our approach on the widely-used 2D-3D-Semantic dataset, employing both random-sample and bounding-box based weak labeling strategies. Our experimental results show significant improvements in segmentation performance across different architectural backbones, highlighting the method's effectiveness and portability.

Abstract:
The manipulation of virtual 3D objects is essential for a variety of handheld AR scenarios. However, the mapping of commonly supported 2D touch gestures to manipulations in 3D space is not trivial. As an alternative, our work explores the use of haptic props that facilitate direct manipulation of virtual 3D objects with 6 degrees of freedom. In an experiment, we instructed 20 participants to solve 2D and 3D docking tasks in AR, to compare traditional 2D touch gestures with prop-based interactions using three prop shapes (cube, rhombicuboctahedron, sphere). Our findings highlight benefits of haptic props for 3D manipulation tasks with respect to task performance, user experience, preference, and workload. For 2D tasks, the benefits of haptic props are less pronounced. Finally, while we found no significant impact of prop shape on task performance, this appears to be subject to personal preference.

Abstract:
Deep learning approaches have provided state-of-the-art performance in many applications by relying on large and overparameterized neural networks. However, such networks are very brittle and are difficult to deploy on resource-limited platforms. Model pruning, i.e., reducing the size of the network, is a widely adopted strategy that can lead to a more robust and compact model. Many heuristics exist for model pruning, but our understanding of the pruning process remains limited due to the black-box nature of a neural network model. Empirical studies show that some heuristics improve performance whereas others can make models more brittle. This work aims to shed light on how different pruning methods alter the network’s internal feature representation and the corresponding impact on model performance. To facilitate a comprehensive comparison and characterization of the high-dimensional model feature space, we introduce a visual geometric analysis of feature representations. We evaluated a set of critical geometric concepts decomposed from the commonly adopted classification loss and used them to design a visualization system to compare and highlight the impact of pruning on model performance and feature representation. The proposed tool provides an environment for an in-depth comparison of pruning methods and a comprehensive understanding of how the model responds to common data corruption. By leveraging the proposed visualization, machine learning researchers can reveal the similarities between pruning methods and redundancy in robustness evaluation benchmarks, obtain geometric insights about the differences between pruned models that achieve superior robustness performance, and identify samples that are robust or fragile to model pruning and common data corruption.

Abstract:
Real-time rendering for light field displays (LFDs) enables human-computer three-dimensional (3D) interaction with smooth and realistic experience, which is potentially useful for 3D gaming, virtual reality, or surgical navigation. However, existing rendering methods realize a real-time rendering normally by sacrificing the 3D image resolution. We propose a culling-based real-time rendering (CBR) method with an accurate ray sampling, aimed at rendering the optimal synthetic images in real-time, which reproduce the high resolution 3D images. The synthetic image generation and display processes are first analyzed, identifying the condition of accurate ray sampling for generating the optimal synthetic images. Then the CBR method is proposed, significantly improving the generation speed of the optimal synthetic images under the accurate ray sampling condition by employing triangle culling in geometry shading stage and pixel culling using pre-computed viewpoint mask texture. Experiments show that accurate ray sampling generates the highest 3D image resolution, and the CBR method achieves superior frame rate, typically rendering 39.7 frames per second (fps) for a LFD with 4,225 viewpoints and a model of 16,301 triangle faces.

Affiliations: School of Information Science and Technology, Shanghai Engineering Research Center of Intelligent Vision and Imaging, Key Laboratory of Intelligent Perception and Human-Machine Collaboration, Ministry of Education, ShanghaiTech University, Shanghai, China; CFETS Information Technology, Shanghai Company, Ltd., Shanghai, China; Baiont Technology, Sinovation Ventures AI Institute, Beijing, China; Department of Computer Science and Engineering, The Hong Kong University of Science and Technology, Hong Kong; School of Entrepreneurship and Management, ShanghaiTech University, Shanghai, China

Abstract:
The fund investment industry heavily relies on the expertise of fund managers, who bear the responsibility of managing portfolios on behalf of clients. With their investment knowledge and professional skills, fund managers gain a competitive advantage over the average investor in the market. Consequently, investors prefer entrusting their investments to fund managers rather than directly investing in funds. For these investors, the primary concern is selecting a suitable fund manager. While previous studies have employed quantitative or qualitative methods to analyze various aspects of fund managers, such as performance metrics, personal characteristics, and performance persistence, they often face challenges when dealing with a large candidate space. Moreover, distinguishing whether a fund manager's performance stems from skill or luck poses a challenge, making it difficult to align with investors’ preferences in the selection process. To address these challenges, this study characterizes the requirements of investors in selecting suitable fund managers and proposes an interactive visual analytics system called FMLens. This system streamlines the fund manager selection process, allowing investors to efficiently assess and deconstruct fund managers’ investment styles and abilities across multiple dimensions. Additionally, the system empowers investors to scrutinize and compare fund managers’ performances. The effectiveness of the approach is demonstrated through two case studies and a qualitative user study. Feedback from domain experts indicates that the system excels in analyzing fund managers from diverse perspectives, enhancing the efficiency of fund manager evaluation and selection.

Abstract:
In Virtual Reality (VR), navigating small physical spaces often relies on handheld controllers, such as teleportation and joystick movements, due to the limited space for natural walking. However, walking-based techniques can enhance immersion by enabling more natural movement. This paper presents three room-connection techniques - portals, corridors, and central hubs - that can be used in virtual environments (VEs) to create “impossible spaces”. These spaces use overlapping areas to maximize available physical space, promising for walking even in constrained spaces. We conducted a user study with 33 participants to assess the effectiveness of these techniques within a small physical area (2.5 x 2.5 m). The results show that all three techniques are viable for connecting rooms in VR, each offering distinct characteristics. Each method positively impacts presence, cybersickness, spatial awareness, orientation, and overall user experience. Specifically, portals offer a flexible and straightforward solution, corridors provide a seamless and natural transition between spaces, and central hubs simplify navigation. The primary contribution of this work is demonstrating how these room-connection techniques can be applied to dynamically adapt VEs to fit small, uncluttered physical spaces, such as those commonly available to VR users at home. Applications such as virtual museum tours, training simulations, and emergency preparedness exercises can greatly benefit from these methods, providing users with a more natural and engaging experience, even within the limited space typical in home settings.

Abstract:
Augmented Reality (AR) offers new opportunities for interacting with our surroundings. However, creating AR services for specific environments such as homes, factories, or buildings remains challenging for users without development skills, as it involves complex 3D editors and advanced coding workflows. This paper presents a comparative study of three distinct tools, dedicated to such novice users, for prototyping augmented reality AR services. These tools include desktop, Virtual Reality (VR), and AR editors, focusing on the positioning of AR assets. In a between-subjects design experiment, where each user tested only one editor, we used two scenarios (smart-home and smart-building) to assess performance, usability, induced workload, and global user experience for each tool. Additionally, the two scenarios allowed us to examine the impact of the target environment size on the results, with a fivefold difference between the sizes of the two environments tested. Our observations indicate that the AR and VR tools outperformed the desktop editor in several criteria, such as task completion duration, usability and enjoyment, suggesting they not only provide a viable alternative to desktop editors for novice users but could also be prioritized. The differences induced by the scenario and environment size were minimal, suggesting their low impact. Future studies should explore this further with larger differences in environment size.

Abstract:
Robotic ultrasound systems have the potential to improve medical diagnostics, but patient acceptance remains a key challenge. To address this, we propose a novel system that combines an AI-based virtual agent, powered by a large language model (LLM), with three mixed reality visualizations aimed at enhancing patient comfort and trust. The LLM enables the virtual assistant to engage in natural, conversational dialogue with patients, answering questions in any format and offering real-time reassurance, creating a more intelligent and reliable interaction. The virtual assistant is animated as controlling the ultrasound probe, giving the impression that the robot is guided by the assistant. The first visualization employs augmented reality (AR), allowing patients to see the real world and the robot with the virtual avatar superimposed. The second visualization is an augmented virtuality (AV) environment, where the real-world body part being scanned is visible, while a 3D Gaussian Splatting reconstruction of the room, excluding the robot, forms the virtual environment. The third is a fully immersive virtual reality (VR) experience, featuring the same 3D reconstruction but entirely virtual, where the patient sees a virtual representation of their body being scanned in a robot-free environment. In this case, the virtual ultrasound probe, mirrors the movement of the probe controlled by the robot, creating a synchronized experience as it touches and moves over the patient's virtual body. We conducted a comprehensive agent-guided robotic ultrasound study with all participants, comparing these visualizations against a standard robotic ultrasound procedure. Results showed significant improvements in patient trust, acceptance, and comfort. Based on these findings, we offer insights into designing future mixed reality visualizations and virtual agents to further enhance patient comfort and acceptance in autonomous medical procedures.

Abstract:
The perception of one's own body is subject to systematic distortions and can be influenced by exposure to visual stimuli showing distorted bodies. In Virtual Reality (VR), echoing such body judgment inaccuracies, avatars with strong appearance dissimilarities with respect to users' bodies can be successfully embodied. The present experimental work investigates, in the healthy population, the perception of the own body in immersive and embodied VR, as well as the impact of being co-present with virtual humans on such self-perception. Participants were successively presented with different avatars, corresponding to various upper- and lower-body proportions, and were asked to compare them with their perceived own body morphology. To investigate the influence of co-present virtual humans on this judgment, the task was performed in co-presence with virtual agents corresponding to various body appearances. Results show an overall overestimation of one's leg length and no influence of the co-present agent's appearance. Importantly, the embodiment scores reflect such body morphology judgment inaccuracy, with participants reporting lower levels of embodiment for avatars with very short legs than for avatars with very long legs. Our findings suggest specifics of embodied body judgment methods, likely resulting from the experience of embodying the avatar as compared to visual appreciation only.

Abstract:
As Augmented and Virtual Reality (AR/VR) advances, secure and user-friendly authentication becomes vital. We evaluated 17 authentication schemes across gaze, gesture, PIN, spatial, and recognition-based categories using a systematic framework focused on effectiveness, security, and usability. Our analysis revealed varied performance and significant gaps requiring standardized methods. For example, Beat-PIN demonstrated strong security with 140-bit entropy, while RubikAuth achieved high usability with authentication times of 1.69 seconds. Gaze-based methods, though innovative, faced accuracy issues. We also observed a preference for schemes like In-Air Handwriting and Things, which balanced security and ease of use. By extending Bonneau et al.'s framework [5] to develop an AR/VR-specific evaluation model, we identified schemes like RubikAuth and Things as particularly promising for AR/VR. This study highlights the strengths and limitations of current methods and emphasizes the need for cross-modal and context-aware techniques to advance AR/VR authentication.

Abstract:
Leveraging real-time eye tracking, foveated rendering optimizes hardware efficiency and enhances visual quality virtual reality (VR). This approach leverages eye-tracking techniques to determine where the user is looking, allowing the system to render high-resolution graphics only in the foveal region—the small area of the retina where visual acuity is highest, while the peripheral view is rendered at lower resolution. However, modern deep learning-based gaze-tracking solutions often exhibit a long-tail distribution of tracking errors, which can degrade user experience and reduce the benefits of foveated rendering by causing misalignment and decreased visual quality. This paper introduces FovealNet, an advanced AI-driven gaze tracking framework designed to optimize system performance by strategically enhancing gaze tracking accuracy. To further reduce the implementation cost of the gaze tracking algorithm, FovealNet employs an event-based cropping method that eliminates over 64.8% of irrelevant pixels from the input image. Additionally, it incorporates a simple yet effective token-pruning strategy that dynamically removes tokens on the fly without compromising tracking accuracy. Finally, to support different runtime rendering configurations, we propose a system performance-aware multi-resolution training strategy, allowing the gaze tracking DNN to adapt and optimize overall system performance more effectively. Evaluation results demonstrate that FovealNet achieves at least 1.42× speed up compared to previous methods and 13% increase in perceptual quality for foveated output. The code is available at https://github.com/wl3181/FovealNet.

Abstract:
The lack of haptic feedback in touchless human-robot interaction is critical in applications such as robotic ultrasound, where force perception is crucial to ensure image quality. Augmented reality (AR) is a promising tool to address this limitation by providing sensory substitution through visual or vibrotactile feedback. The implementation of visual force feedback requires consideration not only of feedback design but also of positioning. Therefore, we implemented two different visualization types at three different positions and investigated the effects of vibrotactile feedback on these approaches. Furthermore, we examined the effects of multimodal feedback compared to visual or vibrotactile output alone. Our results indicate that sensory substitution eases the interaction in contrast to a feedback-less baseline condition, with the presence of visual support reducing average force errors and being subjectively preferred by the participants. However, the more feedback was provided, the longer users needed to complete their tasks. Regarding visualization design, a 2D bar visualization reduced force errors compared to a 3D arrow concept. Additionally, the visualizations being displayed directly on the ultrasound screen were subjectively preferred. With findings regarding feedback modality and visualization design our work represents an important step toward sensory substitution for touchless human-robot interaction.

Abstract:
In recent years, various head-worn virtual reality (VR) techniques have emerged to enhance object selection for occluded or distant targets. However, many approaches focus solely on ray-casting inputs, restricting their use with other input methods, such as bare hands. Additionally, some techniques speed up selection by changing the user's perspective or modifying the scene context, which may complicate interactions when users plan to resume or manipulate the scene afterward. To address these challenges, we present FocalSelect, a heuristic selection technique that builds 3D disambiguation through head-hand coordination and scoring-based functions. Our interaction design adheres to the principle that the intended selection range is a small sector of the headset's viewing frustum, allowing optimal targets to be identified within this scope. We also introduce a density-aware adjustable occlusion plane for effective depth culling of rendered objects. Two experiments are conducted to assess the adaptability of FocalSelect across different input modalities and its performance against five selection techniques. The results indicate that FocalSelect enhances selection experiences in occluded and remote scenarios while preserving the spatial context among objects. This preservation helps maintain users' understanding of the original scene and facilitates further manipulation. We also explore potential applications and enhancements to demonstrate more practical implementations of FocalSelect.

Abstract:
Avoiding obstacles while navigating is a challenge for people with low vision, who have impaired yet functional vision, which impacts their mobility, safety, and independence. This study investigates the impact of using Augmented Reality (AR) to enhance the visibility of obstacles for people with low vision. Twenty-five participants (14 with low vision and 11 typically sighted) wore smart glasses and completed a real-world obstacle course under two conditions: with obstacles enhanced using 3D AR markings and without any enhancement (i.e., passthrough only - control condition). Our results reveal that AR enhancements significantly decreased walking time, with the low vision group demonstrating a notable reduction in time. Additionally, the path length was significantly shorter with AR enhancements. The decrease in time and path length did not lead to more collisions, suggesting improved obstacle avoidance. Participants also reported a positive user experience with the AR system, highlighting its potential to enhance mobility for low vision users. These results suggest that AR technology can play a critical role in supporting the independence and confidence of low vision individuals in mobility tasks within complex environments. We discuss design guidelines for future AR systems to assist low vision people.

Abstract:
Collaborative work in large virtual environments often requires transitions from loosely-coupled collaboration at different locations to tightly-coupled collaboration at a common meeting point. Inspired by prior work on the continuum between these extremes, we present two novel interaction techniques designed to share spatial context while collaborating over large virtual distances. The first method replicates the familiar setup of a video conference by providing users with a virtual tablet to share video feeds with their peers. The second method called PASCAL (Parallel Avatars in a Shared Collaborative Aura Link) enables users to share their immediate spatial surroundings with others by creating synchronized copies of it at the remote locations of their collaborators. We evaluated both techniques in a within-subject user study, in which 24 participants were tasked with solving a puzzle in groups of two. Our results indicate that the additional contextual information provided by PASCAL had significantly positive effects on task completion time, ease of communication, mutual understanding, and co-presence. As a result, our insights contribute to the repertoire of successful interaction techniques to mediate between loosely- and tightly-coupled work in collaborative virtual environments.

Abstract:
Previous works have shown the potential of immersive technologies to make physical activities a more engaging experience. With encountered-type haptic feedback, users can perceive a more realistic sensation for exertion interaction in substitutions reality. Although substitutional reality has utilized physical environments, props, and devices to provide encountered-type haptic feedback, these cannot withstand the fierce force of humans and do not give feedback when users move around simultaneously, such as in combat sports. In this work, we present Hit Around, a substitutional moving robot for immersive and exertion interaction, in which the user can move and punch the virtual opponent and perceive encountered-type haptic feedback anywhere. We gathered insight into immersive exertion interaction from three exhibitions with iterative prototypes, then designed and implemented the hardware system and application. To understand the ability of mobility and weight loading, we conducted two technical evaluations and a laboratory experiment to validate the feasibility. Finally, a field deployment study explored the limitations and challenges of developing immersive exertion interaction with encountered-type haptics.

Abstract:
Facial expressions are crucial for many eXtended Reality (XR) use cases, from mirrored self exposures to social XR, where users interact via their avatars as digital alter egos. However, current XR devices differ in sensor coverage of the face region. Hence, a faithful reconstruction of facial expressions either has to exclude these areas or synthesize missing animation data with model-based approaches, potentially leading to perceivable mismatches between executed and perceived expression. This paper investigates potential effects of the coverage of facial animations (none, partial, or whole) on important factors of self-perception. We exposed 83 participants to their mirrored personalized avatar. They were shown their mirrored avatar face with upper and lower face animation, upper face animation only, lower face animation only, or no face animation. Whole animations were rated higher in virtual embodiment and slightly lower in uncanniness. Missing animations did not differ from partial ones in terms of virtual embodiment. Contrasts showed significantly lower humanness, lower eeriness, and lower attractiveness for the partial conditions. For questions related to self-identification, effects were mixed. We discuss participants' shift in body part attention across conditions. Qualitative results show participants perceived their virtual representation as fascinating yet uncanny.

Abstract:
Genomics is at the core of precision medicine, and there are high expectations on genomics-enabled improvement of patient outcomes in the years to come. Around the world, initiatives to increase the use of DNA sequencing in clinical routine are being deployed, such as the use of broad panels in the standard care for oncology patients. Such a development comes at the cost of increased demands on throughput in genomic data analysis. In this paper, we use the task of copy number variant (CNV) analysis as a context for exploring visualization concepts for clinical genomics. CNV calls are generated algorithmically, but time-consuming manual intervention is needed to separate relevant findings from irrelevant ones in the resulting large call candidate lists. We present a visualization environment, named Copycat, to support this review task in a clinical scenario. Key components are a scatter-glyph plot replacing the traditional list visualization, and a glyph representation designed for at-a-glance relevance assessments. Moreover, we present results from a formative evaluation of the prototype by domain specialists, from which we elicit insights to guide both prototype improvements and visualization for clinical genomics in general.

Abstract:
Obtaining high-quality labeled training data poses a significant bottleneck in the domain of machine learning. Data programming has emerged as a new paradigm to address this issue by converting human knowledge into labeling functions (LFs) to quickly produce low-cost probabilistic labels. To ensure the quality of labeled data, data programmers commonly iterate LFs for many rounds until satisfactory performance is achieved. However, the challenge in understanding the labeling iterations stems from interpreting the intricate relationships between data programming elements, exacerbated by their many-to-many and directed characteristics, inconsistent formats, and the large scale of data typically involved in labeling tasks. These complexities may impede the evaluation of label quality, identification of areas for improvement, and the effective optimization of LFs for acquiring high-quality labeled data. In this article, we introduce EvoVis, a visual analytics method for multi-class text labeling tasks. It seamlessly integrates relationship analysis and temporal overview to display contextual and historical information on a single screen, aiding in explaining the labeling iterations in data programming. We assessed its utility and effectiveness through case studies and user studies. The results indicate that EvoVis can effectively assist data programmers in understanding labeling iterations and improving the quality of labeled data, as evidenced by an increase of 0.16 in the average F1 score when compared to the default analysis tool.

Abstract:
The rapid, accurate, and robust computation of virtual human figures’ “in-between” pose transitions from available and sometimes sparse inputs is of fundamental significance to 3D interactive graphics and computer animation. Various methods have been proposed to produce natural lifelike transitions of human pose automatically in recent decades. Nevertheless, conventional pure model-driven methods require heuristic knowledge (e.g., least motion guided by physics laws) and ad-hoc clues (e.g., splines with non-uniform time warp) that are difficult to obtain, learn, and infer. With the fast emergence of large-scale datasets readily available to animators in the most recent years, deep models afford a powerful alternative to tackle the aforementioned challenges. However, pure data-driven methods still suffer from the remaining challenges such as unseen data in practice and less generative power in model/domain/data transfer, and the measurement of the generative power has always been omitted in these works. In essence, data-driven methods solely rely on the qualities and quantities of training datasets. In this paper, we propose a hybrid approach built upon the seamless integration of data-driven and model-driven methods, called Dynamic Motion Transition (DMT), with the following salient modeling advantages: (1) The data augmentation capability based on the limited human locomotion data capture and the concept of force-derived directly from physical laws; (2) Force learning by which skeleton joints are driven to move, and the Conditional Temporal Transformer (CTT) being trained to learn the force change in the local range, both at the fine level; and (3) At the coarse level, the effective and flexible creation of the subsequent step motion using Dynamic Movement Primitives (DMP) until the target is reached. Our extensive experiments have confirmed that our model can outperform the state-of-the-art methods under the newly devised metric by virtue of the least action loss function. In addition, our novel method and system are of immediate benefit to many other animation tasks such as motion synthesis and control, and motion tracking and prediction in this bigdata graphics era.

Abstract:
Recovering a user-special and controllable human model from a single RGB image is a nontrivial challenge. Existing methods usually generate static results with an image consistent subject's pose. Our work aspires to achieve pose-controllable human reconstruction from a single image by learning a dynamic (multi-pose) implicit field. We first construct a feature-embedded human model (FEHM) as a bridge to propagate image features to different pose spaces. Based on FEHM, we then encode three pose-decoupled features. Global image features represent user-specific shapes in images and replace widely used pixel-aligned ways to avoid unwanted shape-pose entanglement. Spatial color features propagate FEHM-embedded image cues into 3D pose space to provide spatial high-frequency guidance. Spatial geometry features improve reconstruction robustness by using the surface shape of the FEHM as the prior. Finally, new implicit functions are designed to predict the dynamic human implicit fields. For effective supervision, a realistic human avatar dataset, SimuSCAN, with 1000+ models is constructed using a low-cost hierarchical mesh registration method. Extensive experiments demonstrate that our method achieves the state-of-the-art reconstruction level.

Abstract:
New web technologies have enabled the deployment of powerful GPU-based computational pipelines that run entirely in the web browser, opening a new frontier for accessible scientific visualization applications. However, these new capabilities do not address the memory constraints of lightweight end-user devices encountered when attempting to visualize the massive data sets produced by today's simulations and data acquisition systems. We propose a novel implicit isosurface rendering algorithm for interactive visualization of massive volumes within a small memory footprint. We achieve this by progressively traversing a wavefront of rays through the volume and decompressing blocks of the data on-demand to perform implicit ray-isosurface intersections, displaying intermediate results each pass. We improve the quality of these intermediate results using a pretrained deep neural network that reconstructs the output of early passes, allowing for interactivity with better approximates of the final image. To accelerate rendering and increase GPU utilization, we introduce speculative ray-block intersection into our algorithm, where additional blocks are traversed and intersected speculatively along rays to exploit additional parallelism in the workload. Our algorithm is able to trade-off image quality to greatly decrease rendering time for interactive rendering even on lightweight devices. Our entire pipeline is run in parallel on the GPU to leverage the parallel computing power that is available even on lightweight end-user devices. We compare our algorithm to the state of the art in low-overhead isosurface extraction and demonstrate that it achieves 1.7×1.7×–5.7×5.7× reductions in memory overhead and up to 8.4×8.4× reductions in data decompressed.

Abstract:
This research explores a novel paradigm for preserving topological segmentations in existing error-bounded lossy compressors. Today's lossy compressors rarely consider preserving topologies such as Morse-Smale complexes, and the discrepancies in topology between original and decompressed datasets could potentially result in erroneous interpretations or even incorrect scientific conclusions. In this paper, we focus on preserving Morse-Smale segmentations in 2D/3D piecewise linear scalar fields, targeting the precise reconstruction of minimum/maximum labels induced by the integral line of each vertex. The key is to derive a series of edits during compression time. These edits are applied to the decompressed data, leading to an accurate reconstruction of segmentations while keeping the error within the prescribed error bound. To this end, we develop a workflow to fi x ex trema an d in tegral lines alternatively until convergence within finite iterations. We accelerate each workflow component with shared-memory/GPU parallelism to make the performance practical for coupling with compressors. We demonstrate use cases with fluid dynamics, ocean, and cosmology application datasets with a significant acceleration with an NVIDIA A100 GPU.

Abstract:
Dynamic data visualizations can convey large amounts of information over time, such as using motion to depict changes in data values for multiple entities. Such dynamic displays put a demand on our visual processing capacities, yet our perception of motion is limited. Several techniques have been shown to improve the processing of dynamic displays. Staging the animation to sequentially show steps in a transition and tracing object movement by displaying trajectory histories can improve processing by reducing the cognitive load. In this paper, We examine the effectiveness of staging and tracing in dynamic displays. We showed participants animated line charts depicting the movements of lines and asked them to identify the line with the highest mean and variance. We manipulated the animation to display the lines with or without staging, tracing and history, and compared the results to a static chart as a control. Results showed that tracing and staging are preferred by participants, and improve their performance in mean and variance tasks respectively. They also preferred display time 3 times shorter when staging is used. Also, encoding animation speed with mean and variance in congruent tasks is associated with higher accuracy. These findings help inform real-world best practices for building dynamic displays. The supplementary materials can be found at https://osf.io/8c95v/

Abstract:
The semantic similarity between documents of a text corpus can be visualized using map-like metaphors based on two-dimensional scatterplot layouts. These layouts result from a dimensionality reduction on the document-term matrix or a representation within a latent embedding, including topic models. Thereby, the resulting layout depends on the input data and hyperparameters of the dimensionality reduction and is therefore affected by changes in them. Furthermore, the resulting layout is affected by changes in the input data and hyperparameters of the dimensionality reduction. However, such changes to the layout require additional cognitive efforts from the user. In this work, we present a sensitivity study that analyzes the stability of these layouts concerning (1) changes in the text corpora, (2) changes in the hyperparameter, and (3) randomness in the initialization. Our approach has two stages: data measurement and data analysis. First, we derived layouts for the combination of three text corpora and six text embeddings and a grid-search-inspired hyperparameter selection of the dimensionality reductions. Afterward, we quantified the similarity of the layouts through ten metrics, concerning local and global structures and class separation. Second, we analyzed the resulting 42 817 tabular data points in a descriptive statistical analysis. From this, we derived guidelines for informed decisions on the layout algorithm and highlight specific hyperparameter settings. We provide our implementation as a Git repository at hpicgs/Topic-Models-and-Dimensionality-Reduction-Sensitivity-Study and results as Zenodo archive at DOI:10.5281/zenodo.12772898.

Abstract:
Annotations play a vital role in highlighting critical aspects of visualizations, aiding in data externalization and exploration, collaborative sensemaking, and visual storytelling. However, despite their widespread use, we identified a lack of a design space for common practices for annotations. In this paper, we evaluated over 1,800 static annotated charts to understand how people annotate visualizations in practice. Through qualitative coding of these diverse real-world annotated charts, we explored three primary aspects of annotation usage patterns: analytic purposes for chart annotations (e.g., present, identify, summarize, or compare data features), mechanisms for chart annotations (e.g., types and combinations of annotations used, frequency of different annotation types across chart types, etc.), and the data source used to generate the annotations. We then synthesized our findings into a design space of annotations, highlighting key design choices for chart annotations. We presented three case studies illustrating our design space as a practical framework for chart annotations to enhance the communication of visualization insights. All supplemental materials are available at https://shorturl.at/bAGM1.

Abstract:
A year ago, we submitted an IEEE VIS paper entitled “Swaying the Public? Impacts of Election Forecast Visualizations on Emotion, Trust, and Intention in the 2022 U.S. Midterms” [50], which was later bestowed with the honor of a best paper award. Yet, studying such a complex phenomenon required us to explore many more design paths than we could count, and certainly more than we could document in a single paper. This paper, then, is the unwritten prequel—the backstory. It chronicles our journey from a simple idea—to study visualizations for election forecasts—through obstacles such as developing meaningfully different, easy-to-understand forecast visualizations, crafting professional-looking forecasts, and grappling with how to study perceptions of the forecasts before, during, and after the 2022 U.S. midterm elections. This journey yielded a rich set of original knowledge. We formalized a design space for two-party election forecasts, navigating through dimensions like data transformations, visual channels, and types of animated narratives. Through qualitative evaluation of ten representative prototypes with 13 participants, we then identified six core insights into the interpretation of uncertainty visualizations in a U.S. election context. These insights informed our revisions to remove ambiguity in our visual encodings and to prepare a professional-looking forecasting website. As part of this story, we also distilled challenges faced and design lessons learned to inform both designers and practitioners. Ultimately, we hope our methodical approach could inspire others in the community to tackle the hard problems inherent to designing and evaluating visualizations for the general public.

Abstract:
Recent advancements in Large Language Models (LLMs) and Prompt Engineering have made chatbot customization more accessible, significantly reducing barriers to tasks that previously required programming skills. However, prompt evaluation, especially at the dataset scale, remains complex due to the need to assess prompts across thousands of test instances within a dataset. Our study, based on a comprehensive literature review and pilot study, summarized five critical challenges in prompt evaluation. In response, we introduce a feature-oriented workflow for systematic prompt evaluation. In the context of text summarization, our workflow advocates evaluation with summary characteristics (feature metrics) such as complexity, formality, or naturalness, instead of using traditional quality metrics like ROUGE. This design choice enables a more user-friendly evaluation of prompts, as it guides users in sorting through the ambiguity inherent in natural language. To support this workflow, we introduce Awesum, a visual analytics system that facilitates identifying optimal prompt refinements for text summarization through interactive visualizations, featuring a novel Prompt Comparator design that employs a BubbleSet-inspired design enhanced by dimensionality reduction techniques. We evaluate the effectiveness and general applicability of the system with practitioners from various domains and found that (1) our design helps overcome the learning curve for non-technical people to conduct a systematic evaluation of summarization prompts, and (2) our feature-oriented workflow has the potential to generalize to other NLG and image-generation tasks. For future works, we advocate moving towards feature-oriented evaluation of LLM prompts and discuss unsolved challenges in terms of human-agent interaction.

Abstract:
Emerging multimodal large language models (MLLMs) exhibit great potential for chart question answering (CQA). Recent efforts primarily focus on scaling up training datasets (i.e., charts, data tables, and question-answer (QA) pairs) through data collection and synthesis. However, our empirical study on existing MLLMs and CQA datasets reveals notable gaps. First, current data collection and synthesis focus on data volume and lack consideration of fine-grained visual encodings and QA tasks, resulting in unbalanced data distribution divergent from practical CQA scenarios. Second, existing work follows the training recipe of the base MLLMs initially designed for natural images, under-exploring the adaptation to unique chart characteristics, such as rich text elements. To fill the gap, we propose a visualization-referenced instruction tuning approach to guide the training dataset enhancement and model development. Specifically, we propose a novel data engine to effectively filter diverse and high-quality data from existing datasets and subsequently refine and augment the data using LLM-based generation techniques to better align with practical QA tasks and visual encodings. Then, to facilitate the adaptation to chart characteristics, we utilize the enriched data to train an MLLM by unfreezing the vision encoder and incorporating a mixture-of-resolution adaptation strategy for enhanced fine-grained recognition. Experimental results validate the effectiveness of our approach. Even with fewer training examples, our model consistently outperforms state-of-the-art CQA models on established benchmarks. We also contribute a dataset split as a benchmark for future research. Source codes and datasets of this paper are available at https://github.com/zengxingchen/ChartQA-MLLM.

Abstract:
To deploy machine learning models on-device, practitioners use compression algorithms to shrink and speed up models while maintaining their high-quality output. A critical aspect of compression in practice is model comparison, including tracking many compression experiments, identifying subtle changes in model behavior, and negotiating complex accuracy-efficiency trade-offs. However, existing compression tools poorly support comparison, leading to tedious and, sometimes, incomplete analyses spread across disjoint tools. To support real-world comparative workflows, we develop an interactive visual system called COMPRESS AND COMPARE. Within a single interface, COMPRESS AND COMPARE surfaces promising compression strategies by visualizing provenance relationships between compressed models and reveals compression-induced behavior changes by comparing models' predictions, weights, and activations. We demonstrate how COMPRESS AND COMPARE supports common compression analysis tasks through two case studies, debugging failed compression on generative language models and identifying compression artifacts in image classification models. We further evaluate COMPRESS AND COMPARE in a user study with eight compression experts, illustrating its potential to provide structure to compression workflows, help practitioners build intuition about compression, and encourage thorough analysis of compression's effect on model behavior. Through these evaluations, we identify compression-specific challenges that future visual analytics tools should consider and COMPRESS AND COMPARE visualizations that may generalize to broader model comparison tasks.

Abstract:
Synthetic Lethal (SL) relationships, though rare among the vast array of gene combinations, hold substantial promise for targeted cancer therapy. Despite advancements in AI model accuracy, there is still a significant need among domain experts for interpretive paths and mechanism explorations that align better with domain-specific knowledge, particularly due to the high costs of experimentation. To address this gap, we propose an iterative Human-AI collaborative framework with two key components: 1) Human-Engaged Knowledge Graph Refinement based on Metapath Strategies, which leverages insights from interpretive paths and domain expertise to refine the knowledge graph through metapath strategies with appropriate granularity. 2) Cross-Granularity SL Interpretation Enhancement and Mechanism Analysis, which aids experts in organizing and comparing predictions and interpretive paths across different granularities, uncovering new SL relationships, enhancing result interpretation, and elucidating potential mechanisms inferred by Graph Neural Network (GNN) models. These components cyclically optimize model predictions and mechanism explorations, enhancing expert involvement and intervention to build trust. Facilitated by SLInterpreter, this framework ensures that newly generated interpretive paths increasingly align with domain knowledge and adhere more closely to real-world biological principles through iterative Human-AI collaboration. We evaluate the framework's efficacy through a case study and expert interviews.

Abstract:
We present the results of an exploratory study on how pairs interact with speech commands and touch gestures on a wall-sized display during a collaborative sensemaking task. Previous work has shown that speech commands, alone or in combination with other input modalities, can support visual data exploration by individuals. However, it is still unknown whether and how speech commands can be used in collaboration, and for what tasks. To answer these questions, we developed a functioning prototype that we used as a technology probe. We conducted an in-depth exploratory study with 10 participant pairs to analyze their interaction choices, the interplay between the input modalities, and their collaboration. While touch was the most used modality, we found that participants preferred speech commands for global operations, used them for distant interaction, and that speech interaction contributed to the awareness of the partner's actions. Furthermore, the likelihood of using speech commands during collaboration was related to the personality trait of agreeableness. Regarding collaboration styles, participants interacted with speech equally often whether they were in loosely or closely coupled collaboration. While the partners stood closer to each other during close collaboration, they did not distance themselves to use speech commands. From our findings, we derive and contribute a set of design considerations for collaborative and multimodal interactive data analysis systems. All supplemental materials are available at https://osf.io/8gpv2.

Abstract:
Data videos increasingly becoming a popular data storytelling form represented by visual and audio integration. In recent years, more and more researchers have explored many narrative structures for effective and attractive data storytelling. Meanwhile, the Hero's Journey provides a classic narrative framework specific to the Hero's story that has been adopted by various mediums. There are continuous discussions about applying Hero's Journey to data stories. However, so far, little systematic and practical guidance on how to create a data video for a specific story type like the Hero's Journey, as well as how to manipulate its sound and visual designs simultaneously. To fulfill this gap, we first identified 48 data videos aligned with the Hero's Journey as the common storytelling from 109 high-quality data videos. Then, we examined how existing practices apply Hero's Journey for creating data videos. We coded the 48 data videos in terms of the narrative stages, sound design, and visual design according to the Hero's Journey structure. Based on our findings, we proposed a design space to provide practical guidance on the narrative, visual, and sound custom design for different narrative segments of the hero's journey (i.e., Departure, Initiation, Return) through data video creation. To validate our proposed design space, we conducted a user study where 20 participants were invited to design data videos with and without our design space guidance, which was evaluated by two experts. Results show that our design space provides useful and practical guidance for data storytellers effectively creating data videos with the Hero's Journey.

Abstract:
Emotion is an important factor to consider when designing visualizations as it can impact the amount of trust viewers place in a visualization, how well they can retrieve information and understand the underlying data, and how much they engage with or connect to a visualization. We conducted five crowdsourced experiments to quantify the effects of color, chart type, data trend, data variability and data density on emotion (measured through self-reported arousal and valence). Results from our experiments show that there are multiple design elements which influence the emotion induced by a visualization and, more surprisingly, that certain data characteristics influence the emotion of viewers even when the data has no meaning. In light of these findings, we offer guidelines on how to use color, scale, and chart type to counterbalance and emphasize the emotional impact of immutable data characteristics.

Abstract:
A layered graph is an important category of graph in which every node is assigned to a layer, and layers are drawn as parallel or radial lines. They are commonly used to display temporal data or hierarchical graphs. Previous research has demonstrated that minimizing edge crossings is the most important criterion to consider when looking to improve the readability of such graphs. While heuristic approaches exist for crossing minimization, we are interested in optimal approaches to the problem that prioritize human readability over computational scalability. We aim to improve the usefulness and applicability of such optimal methods by understanding and improving their scalability to larger graphs. This paper categorizes and evaluates the state-of-the-art linear programming formulations for exact crossing minimization and describes nine new and existing techniques that could plausibly accelerate the optimization algorithm. Through a computational evaluation, we explore each technique's effect on calculation time and how the techniques assist or inhibit one another, allowing researchers and practitioners to adapt them to the characteristics of their graphs. Our best-performing techniques yielded a median improvement of 2.5-17 × depending on the solver used, giving us the capability to create optimal layouts faster and for larger graphs. We provide an open-source implementation of our methodology in Python, where users can pick which combination of techniques to enable according to their use case. A free copy of this paper and all supplemental materials, datasets used, and source code are available at https://osf.io/5vq79.

Abstract:
Large Language Models (LLMs) like GPT-4 which support multimodal input (i.e., prompts containing images in addition to text) have immense potential to advance visualization research. However, many questions exist about the visual capabilities of such models, including how well they can read and interpret visually represented data. In our work, we address this question by evaluating the GPT-4 multimodal LLM using a suite of task sets meant to assess the model's visualization literacy. The task sets are based on existing work in the visualization community addressing both automated chart question answering and human visualization literacy across multiple settings. Our assessment finds that GPT-4 can perform tasks such as recognizing trends and extreme values, and also demonstrates some understanding of visualization design best-practices. By contrast, GPT-4 struggles with simple value retrieval when not provided with the original dataset, lacks the ability to reliably distinguish between colors in charts, and occasionally suffers from hallucination and inconsistency. We conclude by reflecting on the model's strengths and weaknesses as well as the potential utility of models like GPT-4 for future visualization research. We also release all code, stimuli, and results for the task sets at the following link: https://doi.org/10.17605/OSF.IO/F39J6

Abstract:
Interactive visualizations are powerful tools for Exploratory Data Analysis (EDA), but how do they affect the observations analysts make about their data? We conducted a qualitative experiment with 13 professional data scientists analyzing two datasets with Jupyter notebooks, collecting a rich dataset of interaction traces and think-aloud utterances. By qualitatively coding participant utterances, we introduce a formalism that describes EDA as a sequence of analysis states, where each state is comprised of either a representation an analyst constructs (e.g., the output of a data frame, an interactive visualization, etc.) or an observation the analyst makes (e.g., about missing data, the relationship between variables, etc.). By applying our formalism to our dataset, we identify that interactive visualizations, on average, lead to earlier and more complex insights about relationships between dataset attributes compared to static visualizations. Moreover, by calculating metrics such as revisit count and representational diversity, we uncover that some representations serve more as “planning aids” during EDA rather than tools strictly for hypothesis-answering. We show how these measures help identify other patterns of analysis behavior, such as the “80-20 rule”, where a small subset of representations drove the majority of observations. Based on these findings, we offer design guidelines for interactive exploratory analysis tooling and reflect on future directions for studying the role that visualizations play in EDA.

Abstract:
Translating natural language to visualization (NL2VIS) has shown great promise for visual data analysis, but it remains a challenging task that requires multiple low-level implementations, such as natural language processing and visualization design. Recent advancements in pre-trained large language models (LLMs) are opening new avenues for generating visualizations from natural language. However, the lack of a comprehensive and reliable benchmark hinders our understanding of LLMs' capabilities in visualization generation. In this paper, we address this gap by proposing a new NL2VIS benchmark called VisEval. Firstly, we introduce a high-quality and large-scale dataset. This dataset includes 2,524 representative queries covering 146 databases, paired with accurately labeled ground truths. Secondly, we advocate for a comprehensive automated evaluation methodology covering multiple dimensions, including validity, legality, and readability. By systematically scanning for potential issues with a number of heterogeneous checkers, VisEval provides reliable and trustworthy evaluation outcomes. We run VisEval on a series of state-of-the-art LLMs. Our evaluation reveals prevalent challenges and delivers essential insights for future advancements.

Abstract:
Stigma towards individuals with schizophrenia reduces quality of life, creating a barrier to accessing education and employment opportunities. Schizophrenia is one of the most stigmatized mental health conditions, and stigma is prevalent particularly among healthcare professionals. In this study, we investigated whether Virtual Reality (VR) can be incorporated into interventions to reduce stigma. In particular, we compared the effectiveness of three VR conditions based on intergroup contact theory in reducing stigma in form of implicit and explicit attitudes, and behavioral intentions. Through an immersive virtual consultation in a clinical setting, participants (N=60) experienced one of three different conditions: the Doctor's perspective (embodiment in a majority group member during contact), the Patient's perspective (embodiment in a minority group member) and a Third-person perspective (vicarious contact). Results demonstrated an increase of stigma on certain explicit measures (perceived recovery and social restriction) but also an increase of empathy (perspective-taking, empathic concern) across all conditions regardless of perspective. More importantly, participants' viewpoint influenced the desire for social distance differently depending on the perspective: the Third-person observation significantly increased the desire for social distance, Doctor embodiment marginally decreased it, while Patient embodiment showed no significant change. No change was found in the Implicit Association Test. These findings suggest that VR intergroup contact can effectively reduce certain dimensions of stigma toward schizophrenia, but the type of perspective experienced significantly impacts outcomes.

Abstract:
All-day smart glasses are likely to emerge as platforms capable of continuous contextual sensing, uniquely positioning them for unprecedented assistance in our daily lives. Integrating the multi-modal AI agents required for human memory enhancement while performing continuous sensing, however, presents a major energy efficiency challenge for all-day usage. Achieving this balance requires intelligent, context-aware sensor management. Our approach, EgoTrigger, leverages audio cues from the microphone to selectively activate power-intensive cameras, enabling efficient sensing while preserving substantial utility for human memory enhancement. EgoTrigger uses a lightweight audio model (YAMNet) and a custom classification head to trigger image capture from hand-object interaction (HOI) audio cues, such as the sound of a drawer opening or a medication bottle being opened. In addition to evaluating on the QA-Ego4D dataset, we introduce and evaluate on the Human Memory Enhancement Question-Answer (HME-QA) dataset. Our dataset contains 340 human-annotated first-person QA pairs from full-length Ego4D videos that were curated to ensure that they contained audio, focusing on HOI moments critical for contextual understanding and memory. Our results show EgoTrigger can use 54% fewer frames on average, significantly saving energy in both power-hungry sensing components (e.g., cameras) and downstream operations (e.g., wireless transmission), while achieving comparable performance on datasets for an episodic memory task. We believe this context-aware triggering strategy represents a promising direction for enabling energy-efficient, functional smart glasses capable of all-day use - supporting applications like helping users recall where they placed their keys or information about their routine activities (e.g., taking medications).

Abstract:
This paper presents a scoping review of immersive virtual nature experiences delivered via head-mounted displays (HMDs) and their role in promoting well-being, psychological restoration, cognitive performance, and nature connectedness. As access to natural environments becomes increasingly constrained by urbanization, technological lifestyles, and environmental change, immersive technologies offer a scalable and accessible alternative for engaging with nature. Guided by three core research questions, this review explores how HMD-mediated immersive technologies have been used to promote nature connectedness and well-being, what trends and outcomes have been observed across applications, and what methodological gaps or limitations exist in this growing body of work. Fifty-five peer-reviewed studies were analyzed and categorized into six key implication areas: emotional well-being, stress reduction, cognitive performance, attention recovery, restorative benefits, and nature connectedness. The review identifies immersive virtual nature as a promising application of extended reality (XR) technologies, with potential across healthcare, education, and daily life, while also emphasizing the need for more consistent methodologies and long-term research.

Abstract:
When communicating with embodied conversational agents (ECAs) in virtual reality, there might be delays in the responses of the agents lasting several seconds, for example, due to more extensive computations of the answers when large language models are used. Such delays might lead to unnatural or frustrating interactions. In this paper, we investigate filler types to mitigate these effects and lead to a more positive experience and perception of the agent. In a within-subject study, we asked 24 participants to communicate with ECAs in virtual reality, comparing four strategies displayed during the delays: a multimodal behavioral filler consisting of conversational and gestural fillers, a base condition with only idle motions, and two symbolic indicators with progress bars, one embedded as a badge on the agent, the other one external and visualized as a thinking bubble. Our results indicate that the behavioral filler improved perceived response time, three subscales of presence, humanlikeness, and naturalness. Participants looked away from the face more often when symbolic indicators were displayed, but the visualizations did not lead to a more positive impression of the agent or to increased presence. The majority of participants preferred the behavioral fillers, only 12.5% and 4.2% favored the symbolic embedded and external conditions, respectively.

Abstract:
Immersive embodiment holds great promise for motor skill acquisition, however the design and effect of real-time visual guidance across perspectives and body representations remain underexplored. This study introduces a puppet-inspired visual feedback framework that uses continuous visual linkages - line, color, and thickness cues - to externalize spatial deviation and scaffold embodied learning. To evaluate its effectiveness, we conducted a controlled virtual reality experiment (N = 40) involving gesture imitation tasks with fine (sign language) and gross (aviation marshalling) motor components, under first- and third-person viewpoints. Results showed that color-based guidance significantly improved imitation accuracy, short-term learning, and perceived embodiment, especially in finger-based and first-person settings. Subjective assessments (NASA-TLX, Motivation, IPQ, Embodiment) confirmed improvements in presence, agency, and task engagement.

Abstract:
Understanding 3D object shapes necessitates shape representation by object parts abstracted from results of instance and semantic segmentation. Promising shape representations enable computers to interpret a shape with meaningful parts and identify their repeatability. However, supervised shape representations depend on costly annotation efforts, while current unsupervised methods work under strong semantic priors and involve multi-stage training, thereby limiting their generalization and deployment in shape reasoning and understanding. Driven by the tendency of high-dimensional semantically similar features to lie in or near low-dimensional subspaces, we introduce a one-stage, fully unsupervised framework towards semantic-aware shape representation. This framework produces joint instance segmentation, semantic segmentation, and shape abstraction through sparse representation and feature alignment of object parts in a high-dimensional space. For sparse representation, we devise a sparse latent membership pursuit method that models each object part feature as a sparse convex combination of point features at either the semantic or instance level, promoting part features in the same subspace to exhibit similar semantics. For feature alignment, we customize an attention-based strategy in the feature space to align instance- and semantic-level object part features and reconstruct the input shape using both of them, ensuring geometric reusability and semantic consistency of object parts. To firm up semantic disambiguation, we construct cascade unfrozen learning on geometric parameters of object parts. Experiments conducted on benchmark datasets confirm that our approach results in instance- and semantic-level joint segmentation and shape abstraction with repeatable primitives, providing coherent semantic interpretations of 3D object shapes across categories in a one-stage, fully unsupervised manner, without relying on annotations or heuristic semantic priors.

Abstract:
Photoinduced electronic transitions are complex quantum-mechanical processes where electrons move between energy levels due to the absorption of light. This induces dynamics i.e., coupled changes in the electronic structure and nuclear geometry, that drive physical and chemical processes of importance in diverse fields ranging from photobiology and materials design to medicine. The evolving electronic structure can be characterized by two electron density fields: hole and particle natural transition orbitals (NTOs). A study of the two density fields helps understand the movement of electronic charge from one part of the molecule to another, specifically the donor and acceptor regions. Previous works in this area rely on side-by-side visual comparisons of isosurfaces, statistical approaches, or visual analysis of bivariate fields restricted to limited time instances. We propose a new method to analyze time-varying bivariate fields with a large number of instances, as pertinent to understand electronic structure changes during light-induced dynamics. Since the NTO fields depend on the nuclear geometry, the nuclear motion leads to a large number of bivariate field instances. Structures like tracking graphs have been used to analyze time-varying univariate fields. This article presents a structured and practical approach to feature-directed visual exploration of time-varying bivariate fields using continuous scatterplots (CSPs) and image moment-based descriptors, tailored for studying the evolving electronic structure following photoexcitation. The CSP of the bivariate field at every time step is represented using an image moment vector of length 4. The collection of all image moment vector descriptors is considered as a point cloud in \mathbb R^4R4 and visualized using principal component analysis. Choosing an appropriate pair of principal components results in a representation of the point cloud as a curve on the plane. This representation supports tasks such as identifying interesting time steps, identifying patterns within the bivariate field, and tracking their evolution over time. We present two case studies on excited-state dynamics in molecular systems that demonstrate how the time-varying bivariate field analysis helps provide application-specific insights.

Abstract:
The lack of visually guided data exploration tools limits the scope of research questions communication scientists are able to study. The Comment Analyzer steps in where traditional statistical tools fail when it comes to researching the commenting behavior of news article readers. The basis of such an analysis are comment-thread corpora in which comments are tagged with various deliberative quality indicators as well as political stance. Our analysis tool provides a visual querying system for the exploration and analysis of such corpora and allows social scientists to gain insights into the distributions and relations between comment attributes, the homogeneity of thread sets, frequent thread structures and changes in comment qualities over the course of a single but in particular of multiple threads at once. We developed the tool in close collaboration with communication scientists in a user-centered approach. The system has proven its utility in thorough reviews with the communication scientists, by corroborating existing findings in the literature but particularly by provoking and answering new research questions. Final reviews with five independent experts confirmed these observations and revealed the potential of the Comment Analyzer for other datasets currently being created and analyzed in the communication sciences.

Abstract:
We investigate the concept of Augmented Dynamic Data Physicalization, the combination of shape-changing physical data representations with high-resolution virtual content. Tangible data sculptures, for example using mid-air shape-changing interfaces, are aesthetically appealing and persistent, but also technically and spatially limited. Blending them with Augmented Reality overlays such as scales, labels, or other contextual information opens up new possibilities. We explore the potential of this promising combination and propose a set of essential visualization components and interaction principles. They facilitate sophisticated hybrid data visualizations, for example Overview & Detail techniques or 3D view aggregations. We discuss three implemented applications that demonstrate how our approach can be used for personal information hubs, interactive exhibitions, and immersive data analytics. Based on these use cases, we conducted hands-on sessions with external experts, resulting in valuable feedback and insights. They highlight the potential of combining dynamic physicalizations with dynamic AR overlays to create rich and engaging data experiences.

Abstract:
Research on smooth vector graphics is separated into two independent research threads: one on interpolation-based gradient meshes and the other on diffusion-based curve formulations. With this paper, we propose a mathematical formulation that unifies gradient meshes and curve-based approaches as solution to a Poisson problem. To combine these two well-known representations, we first generate a non-overlapping intermediate patch representation that specifies for each patch a target Laplacian and boundary conditions. Unifying the treatment of boundary conditions adds further artistic degrees of freedoms to the existing formulations, such as Neumann conditions on diffusion curves. To synthesize a raster image for a given output resolution, we then rasterize boundary conditions and Laplacians for the respective patches and compute the final image as solution to a Poisson problem. We evaluate the method on various test scenes containing gradient meshes and curve-based primitives. Since our mathematical formulation works with established smooth vector graphics primitives on the front-end, it is compatible with existing content creation pipelines and with established editing tools. Rather than continuing two separate research paths, we hope that a unification of the formulations will lead to new rasterization and vectorization tools in the future that utilize the strengths of both approaches.

Abstract:
Large Language Models (LLMs) have gained significant attention but also raised concerns due to the risk of misuse. Jailbreak prompts, a popular type of adversarial attack towards LLMs, have appeared and constantly evolved to breach the safety protocols of LLMs. To address this issue, LLMs are regularly updated with safety patches based on reported jailbreak prompts. However, malicious users often keep their successful jailbreak prompts private to exploit LLMs. To uncover these private jailbreak prompts, extensive analysis of large-scale conversational datasets is necessary to identify prompts that still manage to bypass the system's defenses. This task is highly challenging due to the immense volume of conversation data, diverse characteristics of jailbreak prompts, and their presence in complex multi-turn conversations. To tackle these challenges, we introduce JailbreakHunter, a visual analytics approach for identifying jailbreak prompts in large-scale human-LLM conversational datasets. We have designed a workflow with three analysis levels: group-level, conversation-level, and turn-level. Group-level analysis enables users to grasp the distribution of conversations and identify suspicious conversations using multiple criteria, such as similarity with reported jailbreak prompts in previous research and attack success rates. Conversation-level analysis facilitates the understanding of the progress of conversations and helps discover jailbreak prompts within their conversation contexts. Turn-level analysis allows users to explore the semantic similarity and token overlap between a single-turn prompt and the reported jailbreak prompts, aiding in the identification of new jailbreak strategies. The effectiveness and usability of the system were verified through multiple case studies and expert interviews.

Abstract:
Interactively visualizing large finite element simulation data on High-Performance Computing (HPC) systems poses several difficulties. Some of these relate to unstructured data, which, even on a single node, is much more expensive to render compared to structured volume data. Worse yet, in the data parallel rendering context, such data with highly non-convex spatial domain boundaries will cause rays along its silhouette to enter and leave a given rank's domains at different distances. This straddling, in turn, poses challenges for both ray marching, which usually assumes successive elements to share a face, and compositing, which usually assumes a single fragment per pixel per rank. We holistically address these issues using a combination of three inter-operating techniques: first, we use a highly optimized GPU ray marching technique that, given an entry point, can march a ray to its exit point with high-performance by exploiting an exclusive-or (XOR) based compaction scheme. Second, we use hardware-accelerated ray tracing to efficiently find the proper entry points for these marching operations. Third, we use a “deep” compositing scheme to properly handle cases where different ranks’ ray segments interleave in depth. We use GPU-to-GPU remote direct memory access (RDMA) to achieve interactive frame rates of 10–15 frames per second and higher for our motivating use case, the Fun3D NASA Mars Lander.

Abstract:
Neural implicit function based on signed distance field (SDF) has achieved impressive progress in reconstructing 3D models with high fidelity. However, such approaches can only represent closed surfaces. Recent works based on unsigned distance function (UDF) are proposed to handle both watertight and single-layered open surfaces. Nonetheless, as UDF is signless, its direct output is limited to the point cloud, which imposes an additional challenge on extracting high-quality meshes from discrete points. To address this challenge, we present a novel neural implicit representation coded HSDF, which is a hybrid of signed and unsigned distance fields. In particular, HSDF is able to represent arbitrary topologies containing both closed and open surfaces while being compatible with existing iso-surface extraction techniques for easy field-to-mesh conversion. In addition to predicting a UDF, we propose to learn an additional sign field. Unlike traditional SDF, HSDF is able to locate the surface of interest before level surface extraction by generating surface points following NDF (Chibane et al. 2020). We are then able to obtain open surfaces via an adaptive meshing approach that only instantiates regions containing surfaces into a polygon mesh. HSDF benefits downstream tasks like neural rendering, as it enables the rendering of back-faces of open surfaces. We also propose HSDF-Net, a dedicated learning framework that factorizes the learning of HSDF into two easier sub-problems. Experiments and evaluations show that HSDF outperforms the state-of-the-art techniques both qualitatively and quantitatively on some of the used datasets.

Affiliations: Computational Media and Arts Thrust in the Information Hub, The Hong Kong University of Science and Technology (Guangzhou), Hong Kong; Computer Science & Software Engineering Department, Concordia University, Quebec, Canada; Department of Computing in the School of Advanced Technology, Xi’an Jiaotong-Liverpool University, Suzhou, China; Department of Industrial Design in the Design School, Xi’an Jiaotong-Liverpool University, Suzhou, China; Department of Intelligent Science in the School of Advanced Technology, Xi’an Jiaotong-Liverpool University, Suzhou, China

Abstract:
Prior work has shown that frame rate significantly influences user behavior in fast-response tasks in 2D and 3D contexts. However, its impact on a steering task, which involves navigating an object along a path from the start to the end, remains relatively unexplored, especially in the context of virtual reality (VR). This task is considered a typical non-fast-response activity, as it does not demand rapid reactions within a limited time frame. Our work aims to understand and model users’ steering behavior and predict movement time with different task complexities and frame rates in VR environments. We first conducted a user study to collect user behavior in a steering task with four factors: frame rate, path length, width, and radius of curvature. Based on the results, we then quantified the effects of frame rate and built two predictive models. Our models exhibited the best fit (r^2> 0.957r2>0.957) and over 17% improvement in prediction accuracy for movement time compared to existing models. Our models’ robustness was further validated by applying them to predict steering performance with different VR tasks and frame rates. The two models keep the best predictability for both movement time and speed.

Abstract:
Recent years have witnessed growing interest in understanding the sensitivity of machine learning to training data characteristics. While researchers have claimed the benefits of activities such as a human-in-the-loop approach of interactive label correction for improving model performance, there have been limited studies to quantitatively probe the relationship between the cost of label correction and the associated benefit in model performance. We employ a simulation-based approach to explore the efficacy of label correction under diverse task conditions, namely different datasets, noise properties, and machine learning algorithms. We measure the impact of label correction on model performance under the best-case scenario assumption: perfect correction (perfect human and visual systems), serving as an upper-bound estimation of the benefits derived from visual interactive label correction. The simulation results reveal a trade-off between the label correction effort expended and model performance improvement. Notably, task conditions play a crucial role in shaping the trade-off. Based on the simulation results, we develop a set of recommendations to help practitioners determine conditions under which interactive label correction is an effective mechanism for improving model performance.

Abstract:
Analytic provenance can be visually encoded to help users track their ongoing analysis trajectories, recall past interactions, and inform new analytic directions. Despite its significance, provenance is often hardwired into analytics systems, affording limited user control and opportunities for self-reflection. We thus propose modeling provenance as an attribute that is available to users during analysis. We demonstrate this concept by modeling two provenance attributes that track the recency and frequency of user interactions with data. We integrate these attributes into a visual data analysis system prototype, ProvenanceLens, wherein users can visualize their interaction recency and frequency by mapping them to encoding channels (e.g., color, size) or applying data transformations (e.g., filter, sort). Using ProvenanceLens as a design probe, we conduct an exploratory study with sixteen users to investigate how these provenance-tracking affordances are utilized for both decision-making and self-reflection. We find that users can accurately and confidently answer questions about their analysis, and we show that mismatches between the user's mental model and the provenance encodings can be surprising, thereby prompting useful self-reflection. We also report on the user strategies surrounding these affordances, and reflect on their intuitiveness and effectiveness in representing provenance.

Abstract:
Point cloud completion is essential for restoring 3D geometric data lost due to occlusions or sensor limitations. Existing methods often rely on k-nearest neighbor(KNN)-based local feature extraction, which focuses on neighborhoods around central points while neglecting critical global structural information. Additionally, Transformer-based approaches, while effective at modeling global context, typically use central point feature sequences to reduce computational complexity. This windowed attention strategy compromises the preservation of global context, leading to incomplete modeling of the point cloud’s overall structure. To address these challenges, we propose SDA-Net, a dual-attention point cloud completion network utilizing multiple serialization strategies. These strategies transform unordered point clouds into structured sequences, enabling comprehensive modeling of inter-point relationships. Additionally, the dual-attention mechanism enhances global feature extraction through complementary spatial and channel-wise self-attention, effectively compensating for the loss of global context. Extensive experiments demonstrate that SDA-Net achieves state-of-the-art performance, including an average Chamfer Distance (CD) of 6.48 on the PCN dataset. Furthermore, it excels in real-world applications, accurately reconstructing fine-grained details in LiDAR-scanned point clouds.

Abstract:
Human hand and head movements are the most pervasive input modalities in extended reality (XR) and are significant for a wide range of applications. However, prior works on hand and head modelling in XR only explored a single modality or focused on specific applications. We present HaHeAE – a novel self-supervised method for learning generalisable joint representations of hand and head movements in XR. At the core of our method is an autoencoder (AE) that uses a graph convolutional network-based semantic encoder and a diffusion-based stochastic encoder to learn the joint semantic and stochastic representations of hand-head movements. It also features a diffusion-based decoder to reconstruct the original signals. Through extensive evaluations on three public XR datasets, we show that our method 1) significantly outperforms commonly used self-supervised methods by up to 74.1% in terms of reconstruction quality and is generalisable across users, activities, and XR environments, 2) enables new applications, including interpretable hand-head cluster identification and variable hand-head movement generation, and 3) can serve as an effective feature extractor for downstream tasks. Together, these results demonstrate the effectiveness of our method and underline the potential of self-supervised methods for jointly modelling hand-head behaviours in extended reality.

Abstract:
Redirected walking (RDW) is a method for exploring virtual spaces larger than physical spaces while preserving a natural walking sensation. Expanding the range of visual manipulation gains that can be applied without causing discomfort is necessary to apply RDW in practice. Ankle tendon electrical stimulation (TES) can expand the range by inducing body tilt sensation and sway. Therefore, in this study, we proposed a locomotion method that applies ankle TES to RDW. In Experiment 1, we evaluated the effect of TES on the detection threshold (DT), which is the maximal gain at which visual manipulation remains unnoticed. The results indicated that the DT was expanded when TES was applied to induce the body tilt sensation in the same direction as the RDW’s visual manipulation. Specifically, the pooled mean of the DT was expanded by more than 18%. In Experiment 2, we evaluated the applicability, a supplementary index for assessing locomotion techniques. The results demonstrated that ankle TES mitigates the reduction of the applicability, especially under a curvature gain of \pm 0.3 [m^-1]±0.3[m-1].

Abstract:
Unsigned distance fields (UDFs) provide a flexible representation for models with complex topologies, but accurately extracting their zero level sets remains challenging, particularly in preserving topological correctness and fine geometric details. We present DCUDF2, an enhanced method that builds upon DCUDF to address these limitations. Our approach introduces an accuracy-aware loss function with self-adaptive weights, enabling precise geometric fitting while avoiding over-smoothing. To improve robustness, we propose a topology correction strategy that reduces the sensitivity to hyper-parameter settings. Furthermore, we develop new operations leveraging self-adaptive weights to accelerate convergence and improve runtime efficiency. Extensive experiments on diverse datasets demonstrate that DCUDF2 consistently outperforms DCUDF and existing methods in both geometric fidelity and topological accuracy.

Abstract:
3D Face shape stylization refers to transforming a realistic 3D face shape into a different style, such as a cartoon face style. To solve this problem, this paper proposes modeling this task as a deformation transfer problem. This approach significantly reduces labor costs, as the artists would only need to create a single template for each face style. Realistic facial features of the original 3D face e.g. the nose or chin shape, would thus be automatically transferred to those in the style template. Deformation transfer methods, however, have two drawbacks. They are slow and they require re-optimization for every new input face. To address these weaknesses, we propose a neural network-based 3D face shape stylization method. This method is trained through weakly supervised learning, and its template's structure is preserved using our novel template-guided mesh smoothing regularization. Our method is the first learning-based deformation transfer method for 3D face shape stylization. Its employment offers the useful and practical benefit of not requiring paired training data. The experiments show that the quality of the stylized faces obtained by our method is comparable to that of the traditional deformation transfer method, achieving an average Chamfer Distance of approximately 0.01 mm. However, our approach significantly boosts the processing speed, achieving a rate approximately 3,000 times faster than the traditional deformation transfer.

Abstract:
The transportation network is an important element in an urban system that supports daily activities, enabling people to travel from one place to another. One of the key challenges is the network complexity, which is composed of many node pairs distributed over the area. This spatial characteristic results in the high dimensional network problem in understanding the ‘cause’ of problems such as traffic congestion. Recent studies have proposed visual analytics systems aimed at understanding these underlying causes. Despite these efforts, the analysis of such causes is limited to identified patterns. However, given the intricate distribution of roads and their mutual influence, new patterns continuously emerge across all roads within urban transportation. At this stage, a well-defined visual analytics system can be a good solution for transportation practitioners. In this paper, we propose a system, CATOM (Causal Topology Map), for the cause-effect analysis of traffic patterns based on Granger causality for extracting causal topology maps. CATOM discovers causal relationships between roads through the Granger causality test and quantifies these relationships through the causal density. During the design process, the system was developed to fully utilize spatial information with visualization techniques to overcome the previous problems in the literature. We also evaluate the usability of our approach by conducting a SUS(System Usability Scale) test and traffic cause analysis with the real-world data from two study sites in collaboration with domain experts.

Abstract:
This article introduces the Least Increase aversion (LIA) protocol to investigate the relative impact of factors that may trigger cybersickness. The protocol is inspired by the Subjective Matching methodology (SMT) from which it borrows the incremental construction of a richer VR experience, except that the full-blown target experience may cause undesired discomfort. In the first session, the participant briefly encounter all factors at the maximum level. Then in the second session they start with the minimum level of all factors as a Baseline. Subsequently, we expect the participant to minimize their exposure to the most adverse factors. This a pproach ranks the factors from mildest to worst and helps detect individual susceptibility to cybersickness triggers. To validate the applicability of LIA protocol, we further evaluate it with an experiment to identify individual susceptibility to three rotational axes (Yaw, Pitch, and Roll). The findings not only confirm the protocol's capability to accurately discern individual rankings of various factors to cybersickness but also indicate that individual susceptibility is more intricate and multifaceted than initially anticipated.

Abstract:
Surgical navigation systems involve various technologies of segmentation, calibration, registration, tracking, and visualization. These systems aim to superimpose multisource information in the surgical field and provide surgeons with a composite overlay (augmented-reality) view, improving the operative precision and experience. Surgical 3-D tracking is the key to build these systems. Unfortunately, surgical 3-D tracking is still a challenge to endoscopic and robotic navigation systems and easily gets trapped in image artifacts, tissue deformation, and inaccurate positional (e.g., electromagnetic) sensor measurements. This work explores a new monocular endoscope hybrid 3-D tracking method called spatially constrained adaptive differential evolution that combines two spatial constraints with observation-recall adaptive propagation and observation-based fitness computing for stochastic optimization. Specifically, we spatially constraint inaccurate electromagnetic sensor measurements to the centerline of anatomical tubular structures to keep them physically locating inside the tubes, as well as interpolate these measurements to reduce jitter errors for smooth 3-D tracking. We then propose observation-recall adaptive propagation with fitness computing to precisely fuse the constrained sensor measurements, preoperative images, and endoscopic video sequences for accurate hybrid 3-D tracking. Additionally, we also propose a new marker-free hybrid registration strategy to precisely align positional sensor measurements to preoperative images. Our new framework was evaluated on a large amount of clinical data acquired from various surgical endoscopic procedures, with the experimental results showing that it certainly outperforms current surgical 3-D approaches. In particular, the position and rotation errors were significantly reduced from (6.55, 11.4) to (3.02 mm, 8.54^\circ∘).

Abstract:
We propose a self-supervised approach for 3D dynamic reconstruction of articulated motions based on Generative Adversarial Networks and Neural Radiance Fields. Our method reconstructs articulated objects and recover their continuous motions and attributes from an unordered, discontinuous image set. Notably, we treat motion states as time-independent, recognizing that articulated objects can exhibit identical motions at different times. The key insight of our approach utilizes generative adversarial networks to create a continuous implicit motion state space. Initially, we employ a motion network extracts discrete motion states from images as anchors. These anchors are then expanded across the latent space using generative adversarial networks. Subsequently, motion state latent codes are input into motion-aware neural radiance fields for dynamic appearance and geometry reconstruction. To deduce motion attributes from the continuously generated motions, we adopt a cluster-based strategy. We thoroughly evaluate and validate our method on both synthesized and real data, demonstrating superior fidelity in appearances, geometries, and motion attributes of articulated objects compared to state-of-the-art methods.

Abstract:
Recall impairment in a different environmental context from learning is called context-dependent forgetting. Two learning methods have been proposed to prevent context-dependent forgetting: reinstatement and decontextualization. Reinstatement matches the environmental context between learning and retrieval, whereas decontextualization involves repeated learning in various environmental contexts and eliminates the context dependency of memory. Conventionally, these methods have been validated by switching between physical rooms. However, in this study, we use immersive virtual environments (IVEs) as the environmental context assisted by virtual reality (VR), which is known for its low cost and high reproducibility compared to traditional manipulation. Whereas most existing studies using VR have failed to reveal the reinstatement effect, we test its occurrence using a 360-degree video-based IVE with improved familiarity and realism instead of a computer graphics-based IVE. Furthermore, we are the first to address decontextualization using VR. Our experiment showed that repeated learning in the same constant IVE as retrieval did not significantly reduce forgetting compared to repeated learning in different constant IVEs. Conversely, repeated learning in various IVEs significantly reduced forgetting than repeated learning in constant IVEs. These findings contribute to the design of IVEs for VR-based applications, particularly in educational settings.

Affiliations: Faculty of Information Technology, Beijing University of Technology, Chaoyang, China; MOE-Key Laboratory of Pervasive Computing, Department of Computer Science and Technology, Tsinghua University, Beijing, China; Institution of Data and Information, Shenzhen International Graduate School, Tsinghua University, Shenzhen, China; School of Computer Science and Technology, Beijing Institute of Technology, Beijing, China; School of Electronic Information and Electrical Engineering, Shanghai Jiao Tong University, Shanghai, China; College of Intelligence and Computing, Tianjin University, Tianjin, China

Abstract:
Some robust point cloud registration approaches with controllable pose refinement magnitude, such as ICP and its variants, are commonly used to improve 6D pose estimation accuracy. However, the effectiveness of these methods gradually diminishes with the advancement of deep learning techniques and the enhancement of initial pose accuracy, primarily due to their lack of specific design for pose refinement. In this paper, we propose Point Cloud Completion and Keypoint Refinement with Fusion Data (PCKRF), a new pose refinement pipeline for 6D pose estimation. The pipeline consists of two steps. First, it completes the input point clouds via a novel pose-sensitive point completion network. The network uses both local and global features with pose information during point completion. Then, it registers the completed object point cloud with the corresponding target point cloud by our proposed Color supported Iterative KeyPoint (CIKP) method. The CIKP method introduces color information into registration and registers a point cloud around each keypoint to increase stability. The PCKRF pipeline can be integrated with existing popular 6D pose estimation methods, such as the full flow bidirectional fusion network, to further improve their pose estimation accuracy. Experiments demonstrate that our method exhibits superior stability compared to existing approaches when optimizing initial poses with relatively high precision. Notably, the results indicate that our method effectively complements most existing pose estimation techniques, leading to improved performance in most cases. Furthermore, our method achieves promising results even in challenging scenarios involving textureless and symmetrical objects.

Abstract:
Extensive research has been done in haptic feedback for texture simulation in virtual reality (VR). However, it is challenging to modify the perceived tactile texture of existing physical objects which usually serve as anchors for virtual objects in mixed reality (MR). In this article, we present ViboPneumo, a finger-worn haptic device that uses vibratory-pneumatic feedback to modulate (i.e., increase and decrease) the perceived roughness of the material surface contacted by the user's fingerpad while supporting the perceived sensation of other haptic properties (e.g., temperature or stickiness) in MR. Our device includes a silicone-based pneumatic actuator that can lift the user's fingerpad on the physical surface to reduce the contact area for roughness decreasing, and an on-finger vibrator for roughness increasing. The results of our perceptual study showed that the participants could perceive changes in roughness, both increasing and decreasing, compared to the original material surface. We also observed the overlapping roughness ratings among certain haptic stimuli (i.e., vibrotactile and pneumatic) and the originally perceived roughness of some materials without any haptic feedback. This suggests the potential to alter the perceived texture of one type of material to another in terms of roughness (e.g., modifying the perceived texture of ceramics as glass). Lastly, a user study of MR experience showed that ViboPneumo could significantly improve the MR user experience, particularly for visual-haptic matching, compared to the condition of a bare finger. We also demonstrated a few application scenarios for ViboPneumo.

Abstract:
Ordering has been extensively studied in many visualization applications, such as axis and matrix reordering, for the simple reason that the order will greatly impact the perceived pattern of data. Many quality metrics concerning data pattern, perception, and aesthetics are proposed, and respective optimization algorithms are developed. However, the optimization problems related to ordering are often difficult to solve (e.g., TSP is NP-complete), and developing specialized optimization algorithms is costly. In this paper, we propose Versatile Ordering Network (VON), which automatically learns the strategy to order given a quality metric. VON uses the quality metric to evaluate its solutions, and leverages reinforcement learning with a greedy rollout baseline to improve itself. This keeps the metric transparent and allows VON to optimize over different metrics. Additionally, VON uses the attention mechanism to collect information across scales and reposition the data points with respect to the current context. This allows VONs to deal with data points following different distributions. We examine the effectiveness of VON under different usage scenarios and metrics. The results demonstrate that VON can produce comparable results to specialized solvers.

Abstract:
Urban segregation refers to the physical and social division of people, often driving inequalities within cities and exacerbating socioeconomic and racial tensions. While most studies focus on residential spaces, they often neglect segregation across “activity spaces” where people work, socialize, and engage in leisure. Human mobility data offers new opportunities to analyze broader segregation patterns, encompassing both residential and activity spaces, but challenges existing methods in capturing the complexity and local nuances of urban segregation. This work introduces InclusiViz, a novel visual analytics system for multi-level analysis of urban segregation, facilitating the development of targeted, data-driven interventions. Specifically, we developed a deep learning model to predict mobility patterns across social groups using environmental features, augmented with explainable AI to reveal how these features influence segregation. The system integrates innovative visualizations that allow users to explore segregation patterns from broad overviews to fine-grained detail and evaluate urban planning interventions with real-time feedback. We conducted a quantitative evaluation to validate the model’s accuracy and efficiency. Two case studies and expert interviews with social scientists and urban analysts demonstrated the system’s effectiveness, highlighting its potential to guide urban planning toward more inclusive cities.

Abstract:
The rising incidence of stroke has created a significant global public health challenge. The immersive qualities of virtual reality (VR) technology, along with its distinct advantages, make it a promising tool for stroke rehabilitation. To address this challenge, developing VR-based upper limb rehabilitation systems has become a critical research focus. This study developed and evaluated an innovative ArmVR system that combines VR technology with rehabilitation hardware to improve recovery outcomes for stroke patients. Through comprehensive assessments, including neurofeedback, pressure feedback, and subjective feedback, the results suggest that VR technology has the potential to positively support the recovery of cognitive and motor functions. Different VR environments affect rehabilitation outcomes: forest scenarios aid emotional relaxation, while city scenarios better activate motor centers in stroke patients. The study also identified variations in responses among different user groups. Normal users showed significant changes in cognitive function, whereas stroke patients primarily experienced motor function recovery. These findings suggest that VR-integrated rehabilitation systems possess great potential, and personalized design can further enhance recovery outcomes, meet diverse patient needs, and ultimately improve quality of life.

Abstract:
The integration of gaze/eye tracking into virtual and augmented reality devices has unlocked new possibilities, offering a novel human-computer interaction (HCI) modality for on-device extended reality (XR). Emerging applications in XR, such as low-effort user authentication, mental health diagnosis, and foveated rendering, demand real-time eye tracking at high frequencies, a capability that current solutions struggle to deliver. To address this challenge, we present EX-Gaze, an event-based real-time eye tracking system designed for on-device extended reality. EX-Gaze achieves a high tracking frequency of 2KHz, providing decent accuracy and low tracking latency. The exceptional tracking frequency of EX-Gaze is achieved through the use of event cameras, cutting-edge, bio-inspired vision hardware that delivers event-stream output at high temporal resolution. We have developed a lightweight tracking framework that enables real-time pupil region localization and tracking on mobile devices. To effectively leverage the sparse nature of event-streams, we introduce the sparse event-patch representation and the corresponding sparse event patches transformer as key components to reduce computational time. Implemented on Jetson Orin Nano, a low-cost, small-sized mobile device with hybrid GPU and CPU components capable of parallel processing of multiple deep neural networks, EX-Gaze maximizes the computation power of Jetson Orin Nano through sophisticated computation scheduling and offloading between GPUs and CPUs. This enables EX-Gaze to achieve real-time tracking at 2KHz without accumulating latency. Evaluation on public datasets demonstrates that EX-Gaze outperforms other event-based eye tracking methods by striking the best balance between accuracy and efficiency on mobile devices. These results highlight EX-Gaze's potential as a groundbreaking technology to support XR applications that require high-frequency and real-time eye tracking. The code is available at https://github.com/Ningreka/EX-Gaze.

Abstract:
Creating a compelling sense of presence and embodiment can enhance the user experience in virtual reality (VR). One method to accomplish this is through self-representation with embodied personalized avatars or video self-avatars. However, these approaches require external hardware and primarily evaluate hand representations in VR across various tasks. We therefore present in this paper an alternative approach: video Pass-Through Embodiment (PTE), which utilizes the per-eye real-time depth map from Head-Mounted Displays (HMDs) traditionally used for Augmented Reality features. This method allows the user's real body to be cut out of the pass-through video stream and be represented in the VR environment without the need for additional hardware. To evaluate our approach, we conducted a between-subjects study involving 40 participants who completed a seated object sorting task using either PTE or a customized avatar. The results show that PTE, despite its limited depth resolution that leads to some visual artifacts, significantly enhances the user's sense of presence and embodiment. In addition, PTE does not negatively affect task performance, cognitive load, or cause VR sickness. These findings imply that video pass-through embodiment offers a practical and efficient alternative to traditional avatar-based methods in VR.

Abstract:
The building industry and the design of interior environments are increasingly focusing on the user experience, incorporating sensory analysis to reconsider how office environments can be optimized. New immersive technologies offer significant opportunities for sensory science, enhancing our understanding of human perception and enabling the collection of multi-sensory data under controlled laboratory conditions. While the potential of Virtual Reality (VR) for these types of studies is well recognized, certain limitations still need to be addressed, including the lack of standardized research practices and the challenge of ensuring the simulated environment closely mirrors the real world. In this study, we compare 360° and 3D formats, to real-life settings in order to determine which format offers greater ecological validity for visual perception and immersion. Additionally, we examine the effects of vibroacoustic stimuli with different levels of intensity on perception and cognition of 30 participants. Subjective, physiological and cognitive data was collected throughout the test to tackle the participant's experience. This preliminary study introduces an immersive methodology that leverages advanced techniques to gain deeper insights into multisensory user experience in VR, marking a significant step forward in the optimization of VR for building evaluation.

Abstract:
Virtual reality (VR) enables beyond-real interactions (BRI) that transcend physical constraints, offering effective user experiences like extending a hand to grasp distant objects. However, adapting to novel mappings of BRI often reduces performance and the sense of embodiment. To address this, we propose using noisy tendon electrical stimulation (n-TES) to decrease proprioceptive precision. Previous studies have suggested that attenuating proprioceptive precision is crucial for sensory-motor adaptations. Thus, we hypothesize that n-TES, which has been shown to reduce proprioceptive precision and induce visual-dependent perception in VR, can enhance user adaptation to BRI. We conducted a user study using go-go interaction, a BRI technique for interacting with distant objects, to assess the effects of n-TES. Given the individual variability in n-TES response, participants first underwent a proprioceptive precision test to determine the optimal stimulation intensity to lower the proprioceptive precision from 5 levels (\sigma=0.25-125\textmA). Reaching tasks using a 2x2 within-participants design evaluated the effects of go-go interaction and n-TES on performance, subjective task load, and embodiment. Results from 24 participants showed that go-go interaction increased reaching time and task load while decreasing the sense of embodiment. Contrary to our hypothesis, n-TES did not significantly mitigate most of these negative effects of go-go interaction, except that perceived agency was higher with n-TES during go-go interaction. The limited effectiveness of n-TES may be due to participants' habituation or sensory adaptation during the tasks. Future research should consider the adaptation process to BRI and investigate different BRI scenarios.

Abstract:
Typing with a keyboard is a common task in content production in the workplace. For simulation purposes in VR environments, it is important for users to perform this task accurately, with minimal performance loss, and without distraction. One common approach is using mid-air typing on virtual keyboards. However, this method presents challenges, particularly due to the lack of haptic feedback and spatial awareness. Various solutions have been suggested in the literature to address these challenges, but several design factors that influence performance, behavior, and user experience still need to be explored. This paper investigates the effects of two types of visual feedback (hover and keypress) and three passive haptic feedback conditions (physical keyboard, physical surface, and a mid-air virtual keyboard with no haptic feedback) and the possible interactions between these factors on typing using the two index fingers in VR. Results show that keypress visual feedback enhanced typing speed and reduced workload, while hover feedback lowered error rates but negatively impacted typing speed. Additionally, using a physical keyboard to provide passive haptic feedback increased the error rate. This increase in the error rate could be attributed to inaccuracies in finger and keyboard tracking, which may have caused a misalignment between the physical and virtual environments. Regarding eye gaze behavior, participants spent more time looking at the keyboard with the keypress visual feedback and when no haptic feedback was provided. Finally, participants rated the physical keyboard as the least usable option.

Abstract:
Integrating biofeedback technology, such as real-time eye-tracking, has revolutionized the landscape of virtual reality (VR) rehabilitation games, offering new opportunities for personalized therapy. Motivated to increase patient focus during rehabilitation, the Focus-Driven Augmented Feedback (FDAF) system was developed to enhance focus and maintain engagement during upper limb VR rehabilitation. This novel approach dynamically adjusts augmented visual feedback based on a patient's gaze, creating a personalised rehabilitation experience tailored to individual needs. This research aims to develop and comprehensively evaluate the FDAF system to enhance patient focus and maintain engagement in VR rehabilitation environments. The methodology involved three experimental studies, which tested varying levels of augmented feedback with 71 healthy participants and 17 patients requiring upper limb rehabilitation. The results demonstrated that a 30% augmented level was optimal for healthy participants, while a 20% was most effective for patients, ensuring sustained engagement without inducing discomfort. The research's findings highlight the potential of eye-tracking technology to dynamically customise feedback in VR rehabilitation, leading to more effective therapy and improved patient outcomes. This research contributes significant advancements in developing personalised VR rehabilitation techniques, offering valuable insights for future therapeutic applications.

Abstract:
The integration of Large Language Models (LLMs) like GPT-4 with Extended Reality (XR) technologies offers the potential to build truly immersive XR environments that interact with human users through natural language, e.g., generating and animating 3D scenes from audio inputs. However, the complexity of XR environments makes it difficult to accurately extract relevant contextual data and scene/object parameters from an overwhelming volume of XR artifacts. It leads to not only increased costs with pay-per-use models, but also elevated levels of generation errors. Moreover, existing approaches focusing on coding script generation are often prone to generation errors, resulting in flawed or invalid scripts, application crashes, and ultimately a degraded user experience. To overcome these challenges, we introduce LLMER, a novel framework that creates interactive XR worlds using JSON data generated by LLMs. Unlike prior approaches focusing on coding script generation, LLMER translates natural language inputs into JSON data, significantly reducing the likelihood of application crashes and processing latency. It employs a multi-stage strategy to supply only the essential contextual information adapted to the user's request and features multiple modules designed for various XR tasks. Our preliminary user study reveals the effectiveness of the proposed system, with over 80% reduction in consumed tokens and around 60% reduction in task completion time compared to state-of-the-art approaches. The analysis of users' feedback also illuminates a series of directions for further optimization.

Abstract:
As social VR grows in popularity, understanding how to optimise interactions becomes increasingly important. Interpersonal distance–the physical space people maintain between each other–is a key aspect of user experience. Previous work in psychology has shown that breaches of personal space cause stress and discomfort. Thus, effectively managing this distance is crucial in social VR, where social interactions are frequent. Teleportation, a commonly used locomotion method in these environments, involves distinct cognitive processes and requires users to rely on their ability to estimate distance. Despite its widespread use, the effect of teleportation on proximity remains unexplored. To investigate this, we measured the interpersonal distance of 70 participants during interactions with embodied conversational agents, comparing teleportation to natural walking. Our findings revealed that participants maintained closer proximity from the agents during teleportation. Female participants kept greater distances from the agents than male participants, and natural walking was associated with higher agency and body ownership, though co-presence remained unchanged. We propose that differences in spatial perception and spatial cognitive load contribute to reduced interpersonal distance with teleportation. These findings emphasise that proximity should be a key consideration when selecting locomotion methods in social VR, highlighting the need for further research on how locomotion impacts spatial perception and social dynamics in virtual environments.

Abstract:
The aim of this study is to develop an understanding of how virtual objects can be transitioned between 3D Augmented Reality and 2D standard monitor display spaces. The increased availability of Augmented Reality devices, in combination with the prevalence of conventional desktop setups with mouse and keyboard input, gives rise to future hybrid setups in which users may need to transition virtual objects between display spaces. We developed three virtual object transition techniques: Mouse-based, Hand-based, and a Modality Switch (where users can only use the input methods in their respective display spaces). The three techniques were evaluated in a user study (N=24) alongside a fourth condition in which participants could freely switch between Hand and Mouse-based techniques. Participants were tasked with transitioning virtual bricks from 3D space onto the screen, then using the mouse to make fine adjustments, such as choosing the colour of the brick and placing decorations, to then transition them back into 3D space to build with. The Modality Switch technique was not preferred due to higher mental demand. Participants preferred the mouse-based technique, which allowed them to transition the virtual bricks faster.

Abstract:
Humans experience social interactions partly through nonverbal communication, including proxemic behaviors and haptic sensations. Body language, facial expressions, personal spaces, and social touch are multiple factors influencing how a stranger's approach is experienced. Furthermore, the rise of virtual social platforms raises concerns about virtual harassment and the perception of personal space in VR: harassment is felt much more strongly in virtual spaces, and the psychological effects can be just as severe. While most virtual platforms have a ‘personal bubble’ feature that keeps strangers at a distance, it does not seem to suffice: personal space violations seem influenced by more than simply distance. With this paper, we aim to further clarify the variability of personal spaces. We focus on haptic stimulation, elaborating our hypotheses on the relationship between social touch and the perception of personal spaces. Users wore a haptic compression belt and were immersed in a virtual dark alley. Virtual agents approached them while exhibiting either neutral or threatening body language. In half of all trials, as the agent advanced, the compression belt tightened around the users' torsos with three different pressures. Participants could press a response button when uncomfortable with the agent's proximity. Peripersonal space violations occurred 31% earlier on average when the agent was visibly angry and the compression belt activated. A greater tightening pressure also slightly increased the personal sphere radius by up to 13%. Overall, our results are consistent with previous works on peripersonal spaces. They help further define our relationship to personal space boundaries and encourage using haptic devices during simulated social interactions in VR.

Abstract:
Paranormal visual effects, such as spirits and miracles, are frequently depicted in visual games and media design. However, current methods do not express paranormal experiences as aspects of the sixth sense. We propose utilizing binocular rivalry to provide a new visual presentation method by displaying different images in each eye. In this study, we conducted two experiments. Experiment 1 assessed paranormal sensation, color perception controllability, and visual discomfort caused by mismatched colors in each eye in relation to color difference. Experiment 2 assessed our proposed visual presentation method in three application scenarios. The results indicate that our proposed method improves the visual experience of more realistic paranormal phenomena. Moreover, the sensation of paranormal activity, color perception controllability, and visual discomfort increase as the color difference between the colors displayed in the two eyes increases.

Abstract:
In mixed reality (MR) telepresence applications, the differences between participants' physical environments can interfere with effective collaboration. For asymmetric tasks, users might need to access different resources (information, objects, tools) distributed throughout their room. Existing intersection methods do not support such interactions, because a large portion of the telepresence participants' rooms become inaccessible, along with the relevant task resources. We propose MRUnion, a Mixed Reality Telepresence pipeline for asymmetric task-aware 3D mutual scene generation. The key concept of our approach is to enable a user in an asymmetric telecollaboration scenario to access the entire room, while still being able to communicate with remote users in a shared space. For this purpose, we introduce a novel mutual room layout called Union. We evaluated 882 space combinations quantitatively involving two, three, and four combined remote spaces and compared it to a conventional Intersect room layout. The results show that our method outperforms existing intersection methods and enables a significant increase in space and accessibility to resources within the shared space. In an exploratory user study (N=24), we investigated the applicability of the synthetic mutual scene in both MR and VR setups, where users collaborated on an asymmetric remote assembly task. The study results showed that our method achieved comparable results to the intersect method but requires further investigation in terms of social presence, safety and support of collaboration. From this study, we derived design implications for synthetic mutual spaces.

Abstract:
As avatars have evolved from simple digital representations into extensions of our identities, they offer unprecedented opportunities for self-expression and customization beyond the physical world limitations. While virtual platforms foster new forms of identity exploration, social norms still play a crucial role in defining what is considered appropriate in these environments. In this study, we surveyed 150 participants to investigate social norms surrounding avatar modifications, examining how perspectives, contexts, and personality traits influence attitudes toward appropriateness. Our findings reveal that avatar modifications are generally viewed as more appropriate when considered from a partner's perspective, especially for changeable attributes. However, these modifications are perceived as less acceptable in professional settings such as workplaces. Additionally, individuals with high self-monitoring tendencies tend to be more resistant to changes, while those scoring higher on Machiavellianism are more accepting of changes, particularly regarding unchangeable attributes and emotional expressions. These findings provide valuable insights for platform developers and designers, highlighting the importance of implementing context-aware customization options that balance core identity elements with personality-driven preferences, thereby enhancing user experiences while respecting social norms.

Abstract:
Virtual co-embodiment in virtual reality (VR) allows two users to share an avatar, enabling skill transfer from teachers to learners and influencing their Sense of Ownership (SoO) and Sense of Agency (SoA). However, mismatches between actual movements and displayed actions in VR can impair user experience, posing challenges to learning effectiveness. Although previous studies have addressed the influence of virtual bodies' visual factors on SoO and SoA, the impact of co-embodied hands' appearances remains underexplored. We conducted two user studies to examine the effects of virtual self-hands' existence and their visual factors (transparency and congruency) on SoO, SoA, and social presence. Study One showed significant improvements in SoO and SoA with the existence of virtual self-hands. In Study Two, we kept the self-hands and further focused on hand transparency and congruency. We found that identical appearances between self-hands and co-embodied hands significantly enhanced SoO. These findings stressed the importance of visual factors for virtual hands, offering valuable insights for VR co-embodiment design.

Abstract:
Single-view clothing reconstruction usually relies on topologically fixed clothing templates to reduce the problem complexity, but this strategy also makes the reconstructed clothing shape contours simple and lack diversity. In this article, we propose a novel clothing reconstruction method to generate complex shape contours and open clothing mesh from a single image. At the heart of our work is an implicit unsigned distance field condition on clothing-oriented and pose-stable spatial shape features to represent the clothing from the image. This feature can provide spatially aligned clothing shape priors to improve the pose robustness. It is based on a type-generic clothing template derived from the mainstream clothing generative model to avoid tedious template design and switching. To output open clothing mesh results from noisy clothing unsigned distance fields, we develop a two-stage clothing mesh extraction method. It takes the point clouds as an intermediate representation and produces smooth, plausible and editable clothing mesh results. To provide effective supervision, we construct a pose-rich and shape-complete clothing scan dataset by enhancing clothing pose diversity and complementing missing clothing geometry caused by occlusion. Extensive experiments demonstrate that our method achieves state-of-the-art levels. More importantly, we provide a simple but effective, and low-cost way to reconstruct complex shape contours clothing from a single image.

Abstract:
Virtual displays enabled through head-worn augmented reality have unique characteristics that can yield extensive amounts of screen space. Existing research has shown that increasing the space on a computer screen can enhance usability. Since virtual displays offer the unique ability to present content without rigid physical space constraints, they provide various new design possibilities. Therefore, we must understand the trade-offs of layout choices when structuring that space. We propose a single Canvas approach that eliminates boundaries from traditional multi-monitor approaches and instead places windows in one large, unified space. Our user study compared this approach against a multi-monitor setup, and we considered both purely virtual systems and hybrid systems that included a physical monitor. We looked into usability factors such as performance, accuracy, and overall window management. Results show that Canvas displays can cause users to compact window layouts more than multiple monitors with snapping behavior, even though such optimizations may not lead to longer window management times. We did not find conclusive evidence of either setup providing a better user experience. Multi-Monitor displays offer quick window management with snapping and a structured layout through subdivisions. However, Canvas displays allow for more control in placement and size, lowering the amount of space used and, thus, head rotation. Multi-Monitor benefits were more prominent in the hybrid configuration, while the Canvas display was more beneficial in the purely virtual configuration.

Abstract:
When visualising data, chart designers have the freedom to choose the upper and lower limits of numerical axes. Axis limits can determine the physical characteristics of plotted values, such as the physical position of data points in dot plots. In two experiments (total N=300), we demonstrate that axis limits affect viewers’ interpretations of the magnitudes of plotted values. Participants did not simply associate values presented at higher vertical positions with greater magnitudes. Instead, participants considered the relative positions of data points within the axis limits. Data points were considered to represent larger values when they were closer to the end of the axis associated with greater values, even when they were presented at the bottom of a chart. This provides further evidence of framing effects in the display of data, and offers insight into the cognitive mechanisms involved in assessing magnitude in data visualisations.

Abstract:
High-dimensional data, characterized by many features, can be difficult to visualize effectively. Dimensionality reduction techniques, such as PCA, UMAP, and t-SNE, address this challenge by projecting the data into a lower-dimensional space while preserving important relationships. TopoMap is another technique that excels at preserving the underlying structure of the data, leading to interpretable visualizations. In particular, TopoMap maps the high-dimensional data into a visual space, guaranteeing that the 0-dimensional persistence diagram of the Rips filtration of the visual space matches the one from the high-dimensional data. However, the original TopoMap algorithm can be slow and its layout can be too sparse for large and complex datasets. In this paper, we propose three improvements to TopoMap: 1) a more space-efficient layout, 2) a significantly faster implementation, and 3) a novel TreeMap-based representation that makes use of the topological hierarchy to aid the exploration of the projections. These advancements make TopoMap, now referred to as TopoMap++, a more powerful tool for visualizing high-dimensional data which we demonstrate through different use case scenarios.

Abstract:
In recent years, the global adoption of electric vehicles (EVs) has surged, prompting a corresponding rise in the installation of charging stations. This proliferation has underscored the importance of expediting the deployment of charging infrastructure. Both academia and industry have thus devoted to addressing the charging station location problem (CSLP) to streamline this process. However, prevailing algorithms addressing CSLP are hampered by restrictive assumptions and computational overhead, leading to a dearth of comprehensive evaluations in the spatiotemporal dimensions. Consequently, their practical viability is restricted. Moreover, the placement of charging stations exerts a significant impact on both the road network and the power grid, which necessitates the evaluation of the potential post-deployment impacts on these interconnected networks holistically. In this study, we propose CSLens, a visual analytics system designed to inform charging station deployment decisions through the lens of coupled transportation and power networks. CSLens offers multiple visualizations and interactive features, empowering users to delve into the existing charging station layout, explore alternative deployment solutions, and assess the ensuring impact. To validate the efficacy of CSLens, we conducted two case studies and engaged in interviews with domain experts. Through these efforts, we substantiated the usability and practical utility of CSLens in enhancing the decision-making process surrounding charging station deployment. Our findings underscore CSLens's potential to serve as a valuable asset in navigating the complexities of charging infrastructure planning.

Abstract:
Acute stroke demands prompt diagnosis and treatment to achieve optimal patient outcomes. However, the intricate and irregular nature of clinical data associated with acute stroke, particularly blood pressure (BP) measurements, presents substantial obstacles to effective visual analytics and decision-making. Through a year-long collaboration with experienced neurologists, we developed PhenoFlow, a visual analytics system that leverages the collaboration between human and Large Language Models (LLMs) to analyze the extensive and complex data of acute ischemic stroke patients. PhenoFlow pioneers an innovative workflow, where the LLM serves as a data wrangler while neurologists explore and supervise the output using visualizations and natural language interactions. This approach enables neurologists to focus more on decision-making with reduced cognitive load. To protect sensitive patient information, PhenoFlow only utilizes metadata to make inferences and synthesize executable codes, without accessing raw patient data. This ensures that the results are both reproducible and interpretable while maintaining patient privacy. The system incorporates a slice-and-wrap design that employs temporal folding to create an overlaid circular visualization. Combined with a linear bar graph, this design aids in exploring meaningful patterns within irregularly measured BP data. Through case studies, PhenoFlow has demonstrated its capability to support iterative analysis of extensive clinical datasets, reducing cognitive load and enabling neurologists to make well-informed decisions. Grounded in long-term collaboration with domain experts, our research demonstrates the potential of utilizing LLMs to tackle current challenges in data-driven clinical decision-making for acute ischemic stroke patients.

Abstract:
Large Language Models (LLMs) have shown great potential in intelligent visualization systems, especially for domain-specific applications. Integrating LLMs into visualization systems presents challenges, and we categorize these challenges into three alignments: domain problems with LLMs, visualization with LLMs, and interaction with LLMs. To achieve these alignments, we propose a framework and outline a workflow to guide the application of fine-tuned LLMs to enhance visual interactions for domain-specific tasks. These alignment challenges are critical in education because of the need for an intelligent visualization system to support beginners' self-regulated learning. Therefore, we apply the framework to education and introduce Tailor-Mind, an interactive visualization system designed to facilitate self-regulated learning for artificial intelligence beginners. Drawing on insights from a preliminary study, we identify self-regulated learning tasks and fine-tuning objectives to guide visualization design and tuning data construction. Our focus on aligning visualization with fine-tuned LLM makes Tailor-Mind more like a personalized tutor. Tailor-Mind also supports interactive recommendations to help beginners better achieve their learning goals. Model performance evaluations and user studies confirm that Tailor-Mind improves the self-regulated learning experience, effectively validating the proposed framework.

Abstract:
We present a visual analytics approach for multi-level visual exploration of users' interaction strategies in an interactive digital environment. The use of interactive touchscreen exhibits in informal learning environments, such as museums and science centers, often incorporate frameworks that classify learning processes, such as Bloom's taxonomy, to achieve better user engagement and knowledge transfer. To analyze user behavior within these digital environments, interaction logs are recorded to capture diverse exploration strategies. However, analysis of such logs is challenging, especially in terms of coupling interactions and cognitive learning processes, and existing work within learning and educational contexts remains limited. To address these gaps, we develop a visual analytics approach for analyzing interaction logs that supports exploration at the individual user level and multi-user comparison. The approach utilizes algorithmic methods to identify similarities in users' interactions and reveal their exploration strategies. We motivate and illustrate our approach through an application scenario, using event sequences derived from interaction log data in an experimental study conducted with science center visitors from diverse backgrounds and demographics. The study involves 14 users completing tasks of increasing complexity, designed to stimulate different levels of cognitive learning processes. We implement our approach in an interactive visual analytics prototype system, named VISID, and together with domain experts, discover a set of task-solving exploration strategies, such as “cascading” and “nested-loop”, which reflect different levels of learning processes from Bloom's taxonomy. Finally, we discuss the generalizability and scalability of the presented system and the need for further research with data acquired in the wild.

Abstract:
Inspired by recent advances in digital fabrication, artists and scientists have demonstrated that physical data encodings (i.e., data physicalizations) can increase engagement with data, foster collaboration, and in some cases, improve data legibility and analysis relative to digital alternatives. However, prior empirical studies have only investigated abstract data encoded in physical form (e.g., laser cut bar charts) and not continuously sampled spatial data fields relevant to climate and medical science (e.g., heights, temperatures, densities, and velocities sampled on a spatial grid). This paper presents the design and results of the first study to characterize human performance in 3D spatial data analysis tasks across analogous physical and digital visualizations. Participants analyzed continuous spatial elevation data with three visualization modalities: (1) 2D digital visualization; (2) perspective-tracked, stereoscopic “fishtank” virtual reality; and (3) 3D printed data physicalization. Their tasks included tracing paths downhill, looking up spatial locations and comparing their relative heights, and identifying and reporting the minimum and maximum heights within certain spatial regions. As hypothesized, in most cases, participants performed the tasks just as well or better in the physical modality (based on time and error metrics). Additional results include an analysis of open-ended feedback from participants and discussion of implications for further research on the value of data physicalization. All data and supplemental materials are available at https://osf.io/7xdq4/.

Abstract:
Genomics experts rely on visualization to extract and share insights from complex and large-scale datasets. Beyond off-the-shelf tools for data exploration, there is an increasing need for platforms that aid experts in authoring customized visualizations for both exploration and communication of insights. A variety of interactive techniques have been proposed for authoring data visualizations, such as template editing, shelf configuration, natural language input, and code editors. However, it remains unclear how genomics experts create visualizations and which techniques best support their visualization tasks and needs. To address this gap, we conducted two user studies with genomics researchers: (1) semi-structured interviews (n=20) to identify the tasks, user contexts, and current visualization authoring techniques and (2) an exploratory study (n=13) using visual probes to elicit users' intents and desired techniques when creating visualizations. Our contributions include (1) a characterization of how visualization authoring is currently utilized in genomics visualization, identifying limitations and benefits in light of common criteria for authoring tools, and (2) generalizable design implications for genomics visualization authoring tools based on our findings on task- and user-specific usefulness of authoring techniques. All supplemental materials are available at https://osf.io/bdj4v/

Abstract:
We present ProvenanceWidgets, a Javascript library of UI control elements such as radio buttons, checkboxes, and dropdowns to track and dynamically overlay a user's analytic provenance. These in situ overlays not only save screen space but also minimize the amount of time and effort needed to access the same information from elsewhere in the UI. In this paper, we discuss how we design modular UI control elements to track how often and how recently a user interacts with them and design visual overlays showing an aggregated summary as well as a detailed temporal history. We demonstrate the capability of ProvenanceWidgets by recreating three prior widget libraries: (1) Scented Widgets, (2) Phosphor objects, and (3) Dynamic Query Widgets. We also evaluated its expressiveness and conducted case studies with visualization developers to evaluate its effectiveness. We find that ProvenanceWidgets enables developers to implement custom provenance-tracking applications effectively. ProvenanceWidgets is available as open-source software at https://github.com/ProvenanceWidgets to help application developers build custom provenance-based systems.

Abstract:
Sense of embodiment (SoE) refers to the participant’s perception of a virtual avatar as an extension of their own body, involving both physical and functional aspects. Research has highlighted the importance of SoE for the effectiveness of virtual rehabilitation. Virtual co-embodiment technology, an emerging virtual reality (VR) application, has the potential to enhance users’ engagement and SoE, which demonstrates significant promise for motor rehabilitation. However, the exploration of factors influencing embodiment in virtual co-embodiment is still limited, particularly regarding both internal and external factors, which constrains its rehabilitation applications. This study investigates factors influencing SoE changes in Virtual Co-embodiment Rehabilitation by developing a theoretical model, based on 859 valid trials collected from 40 healthy participants, and analyzing the data using Structural Equation Modeling (SEM) and Partial Least Squares Structural Equation Modeling (PLS-SEM). The results suggest that both “visual consistency” (external factor) and “individual sensitivity” (internal factor) may influence changes in SoE, with “visual consistency” appearing to have a predominant effect. These findings contribute to understanding changes in SoE in virtual co-embodiment and offer insights for optimizing the technology in rehabilitation.

Abstract:
Current behavioral authentication systems in Virtual Reality (VR) require sustained focused interaction during task execution - an assumption frequently incompatible with real-world constraints across two factors: (1) physical limitations (e.g., restricted hand/eye mobility), and (2) psychological barriers (e.g., task-switching fatigue or break-in-presence). To address this attentional gap, we propose a design framework bridging focused and casual interactions in behavior-based VR authentication (Casual-VRAuth). Based on this framework, we designed an authentication prototype using a modified ball-and-tunnel task (propelling a ball along a circular path), supporting three interaction modes: baseline Touch, and two eyes-free options (Hover/Tapping). Experimental results demonstrate that our framework effectively guides the design of authentication systems with varying interaction engagement levels (Touch > Hover > Tapping) to accommodate scenarios requiring casual interaction (e.g., multitasking or eyes-free operation). Furthermore, we revealed that reducing interaction engagement enhances resistance to mimicry attacks while decreasing cognitive workload and error rates in multitasking or eyes-free environments. However, this approach compromises the average classification accuracy of Interaction behavior under different algorithms (including InceptionTime, FCN, ResNet, CNN, MLP, and MCDCNN). Notably, moderate reduction of interaction engagement enhances authentication speed, while excessive reduction may conversely slow it down. Overall, our work establishes a novel design paradigm for VR authentication that supports casual interactions and offers valuable insights into balancing usability and security.

Abstract:
As Extended Reality (XR) environments increasingly involve large amounts of data and content, designing effective exploration techniques has become critical. Depth-based object exploration is a common but underexplored task in XR environments, especially in settings without physical devices. Prior studies have largely focused on horizontal or planar interactions, leaving depth-oriented exploration relatively overlooked. To bridge this gap, we propose three linearly aligned layer transition techniques (Continuous Push, Push&Return, and Tilt&Return) specifically designed to support efficient, precise, and continuous object exploration along the depth axis within depth-based UIs. In a user study with 30 participants, we compared their performance, usability, and user preference across two different layer configurations (8-layer vs. 16-layer). The results highlight that Continuous Push enables faster exploration with lower effort, while Push&Return provides the highest accuracy and is most preferred by users. Based on these findings, we discuss design implications for depth-based interaction techniques in controller-free XR environments.

Abstract:
Augmented reality (AR) technology has great potential in the applications of training, exhibition, and visual guidance, all of which demand precise virtual-real registration in perceived depth. Many AR applications such as navigation and tourism guidance are usually implemented in outdoor environments. However, prior research on depth perception in AR predominantly focused on the indoor environment, characterized by a lower illumination level and more confined space compared to outdoor settings. To address this gap, this paper presented a systematic investigation into the depth perception in outdoor environments. Two experiments were conducted in this study: the first one aimed to explore how to eliminate the bias induced by the floating object and how the knowledge of object height influences the perceived depth. The second experiment examined how ambient luminance affects depth estimation in AR. Our findings revealed an overestimation of perceived depth when participants were unaware of the actual height of the floating object, but an underestimation when they were informed of this information prior to the experiment. Additionally, shadows effectively reduced depth errors regardless of whether participants were informed of the object's height. The second experiment further indicated that, in outdoor environments, reducing ambient luminance significantly improves the accuracy of depth perception in AR.

Abstract:
Immersive technologies such as virtual and augmented reality (VR/AR) allow remote users to meet and interact in a shared virtual space using embodied virtual avatars, creating a sense of co-presence. However, asynchronous communication-essential in many real-world contexts-remains underexplored in these environments. Traditional playback-based systems lack interactivity and often fail to preserve critical contextual cues necessary for effective asynchronous communication. In this paper, we introduce AVAGENTs, AI-powered virtual avatars that replicate users' verbal and nonverbal cues from recordings of past meetings. Avagents can interpret meeting context and generate appropriate responses to questions posed by asynchronous viewers. Through a user study (N = 30), we evaluated Avagents against a traditional playback method and a voice-based AI assistant across two asynchronous meeting scenarios: analytic reasoning and affective resonance. Results showed that Avagents enhance the asynchronous communication experience by increasing social presence, sense of belonging, emotional intimacy, and other user perceptions. We discuss the findings and their implications for designing effective AI-driven asynchronous communication tools in VR/AR environments.

Affiliations: Interactive Media Lab Dresden, TUD Dresden University of Technology, Dresden, Germany; Interactive Media Lab Dresden and the Center for Scalable Data Analytics and Artificial Intelligence (ScaDS.AI) Dresden/Leipzig, TUD Dresden University of Technology, Dresden, Germany; Interactive Media Lab Dresden, the Center for Scalable Data Analytics and Artificial Intelligence (ScaDS.AI) Dresden/Leipzig, and the Centre for Tactile Internet with Human-in-the-Loop (CeTI), TUD Dresden University of Technology, Dresden, Germany

Abstract:
In recent years, data-driven stories have found a firm footing in journalism, business, and education. They leverage visualization and storytelling to convey information to broader audiences. Likewise, immersive technologies, like augmented and virtual reality devices, provide excellent potential for exploring and explaining data, thus inviting research on how data-driven storytelling transfers to immersive environments. To gain a better understanding of this exciting novel research area, we conducted a scoping review on the emerging notion of immersive data-driven storytelling, extended by surveying immersive data journalism and by analyzing immersive games, selected based on community reviews and tags. We present our methodology for the survey and discuss prominent themes that coalesce the knowledge we extracted from the literature, journalism, and games. These themes include, among others, the spatial embodiment of narration, the incorporation of the users and their context into narratives, and the balance between guiding the user versus promoting serendipity. Our discussion of these themes reveals research opportunities and challenges that will inform the design of immersive data-driven stories in the future.

Abstract:
Convincing crowd behavior simulation is becoming essential in many application domains, including video games, cinematography, urban planning, safety simulations, and training. In this article, we propose a novel and lightweight mesoscopic system for personality-based crowd simulation in immersive virtual reality (iVR). We use the Big Five personality framework, also known as OCEAN, to model a synthetic personality for each autonomous agent. Agents can autonomously aggregate in formations using machine learning-based clustering techniques operating on OCEAN. Moreover, agents can also externalize their personality traits by performing peculiar behavioral animations. To choose which animations to perform, we adopt a probabilistic approach that considers each OCEAN dimension as a continuous spectrum with two extremes linked to pairs of animations. Our system is designed to be flexible and suitable for different applications. Flexibility is achieved by using graphs to store agent and map topology data that control how the agents move and behave at runtime. In a within-subjects study with 40 users, we compare our personality-based system against a basic system that does not use personality. Results show that introducing personality into iVR crowd simulation enhances users’ social presence and experienced realism. Introducing personality also increases the perceived match between the agents and the virtual environment where the simulation takes place.

Abstract:
Multi-person motion capture over sparse angular observations is a challenging problem under interference from both self- and mutual-occlusions. Existing works produce accurate 2D joint detection, however, when these are triangulated and lifted into 3D, available solutions all struggle in selecting the most accurate candidates and associating them to the correct joint type and target identity. As such, in order to fully utilize all accurate 2D joint location information, we propose to independently triangulate between all same-typed 2D joints from all camera views regardless of their target ID, forming the Joint Cloud. Joint Cloud consist of both valid joints lifted from the same joint type and target ID, as well as falsely constructed ones that are from different 2D sources. These redundant and inaccurate candidates are processed over the proposed Joint Cloud Selection and Aggregation Transformer (JCSAT) involving three cascaded encoders which deeply explore the trajectile, skeletal structural, and view-dependent correlations among all 3D point candidates in the cross-embedding space. An Optimal Token Attention Path (OTAP) module is proposed which subsequently selects and aggregates informative features from these redundant observations for the final prediction of human motion. To demonstrate the effectiveness of JCSAT, we build and publish a new multi-person motion capture dataset BUMocap-X with complex interactions and severe occlusions. Comprehensive experiments over the newly presented as well as benchmark datasets validate the effectiveness of the proposed framework, which outperforms all existing state-of-the-art methods, especially under challenging occlusion scenarios.

Abstract:
The escalating frequency and intensity of heat-related climate events, particularly heatwaves, emphasize the pressing need for advanced heat risk management strategies. Current approaches, primarily relying on numerical models, face challenges in spatial-temporal resolution and in capturing the dynamic interplay of environmental, social, and behavioral factors affecting heat risks. This has led to difficulties in translating risk assessments into effective mitigation actions. Recognizing these problems, we introduce a novel approach leveraging the burgeoning capabilities of Large Language Models (LLMs) to extract rich and contextual insights from news reports. We hence propose an LLM-empowered visual analytics system, Havior, that integrates the precise, data-driven insights of numerical models with nuanced news report information. This hybrid approach enables a more comprehensive assessment of heat risks and better identification, assessment, and mitigation of heat-related threats. The system incorporates novel visualization designs, such as “thermoglyph” and news glyph, enhancing intuitive understanding and analysis of heat risks. The integration of LLM-based techniques also enables advanced information retrieval and semantic knowledge extraction that can be guided by experts’ analytics needs. We conducted an experiment on information extraction, a case study on the 2022 China Heatwave, and an expert survey & interview collaborated with six domain experts, demonstrating the usefulness of our system in providing in-depth and actionable insights for heat risk management.

Abstract:
Virtual content in augmented reality (AR) applications can be tailored to a designer’s specifications. However, real-world environments are challenging to control precisely or replicate fully. Consequently, prototyping AR applications for specific environments is often difficult. One potential solution is employing mixed reality (MR) to simulate an AR system, enabling controlled experiments. Nevertheless, the effectiveness of using MR to simulate AR office work remains underexplored. In this paper, we report the results of a user study (N = 40) that investigated the impact of an MR simulation of an AR office on participants’ task performance and cognitive workload (CWL). Participants completed several office tasks in both an AR scene featuring a virtual monitor and an MR-simulated AR scene. During these tasks, CWL was measured using electroencephalography (EEG) and a subjective questionnaire. The results show that the performance of the pass-through window is a major constraint on the effectiveness of the MR simulation office. Finally, we discuss the study’s limitations and directions for future research.

Abstract:
Augmented Reality (AR) has been proven beneficial to External Ventricular Drain (EVD) surgery by providing in-situ visual guidance during operations. During this procedure, the key challenge is estimating the spatial relationship between pre-operative images and actual patient anatomy accurately and efficiently. Previous works have revealed conflicts between tracking accuracy, workflow efficiency, and non-invasiveness in tracking pipelines. This research fully utilizes the capabilities of Time of Flight (ToF) depth sensors, including retro-reflective tool tracking and dense surface information, to construct a convenient and accurate EVD guiding pipeline. As previous studies have proven significant depth errors in ToF depth sensors, we first evaluated the feasibility of using ToF sensors in surgical guidance by estimating its accuracy under different conditions and corrected this error in our pipeline. Our results show \text7.580\pm \text1.488\,\textmm7.580±1.488mm depth value errors on human skin under HoloLens 2 depth camera, indicating the significance of depth correction. This error was reduced by over 85% using proposed depth correction method on head phantoms in different materials. The corrected depth information can then be utilized to reconstruct the head surface with sub-millimeter accuracy, validated on a series of 3D-printed models and a sheep head. To demonstrate the effectiveness of the proposed framework, we conducted a case study simulating EVD surgery. Five surgeons were involved in this study, each performing nine k-wire insertions on a head phantom under virtual guidance without tracking for surgical tools. The results revealed \text2.09 \pm \text1.00\,\textmm2.09±1.00mm translational and \text2.97\pm \text1.95^\circ2.97±1.95∘ orientational guidance accuracy, demonstrating competitive performance with previous research.

Abstract:
Effective chart summary can significantly reduce the time and effort decision makers spend interpreting charts, enabling precise and efficient communication of data insights. Previous studies have faced challenges in generating accurate and semantically rich summaries of time-series data charts. In this paper, we identify summary elements and common hallucination types in the generation of time-series chart summaries, which serve as our guidelines for automatic generation. We introduce ChartInsighter, which automatically generates chart summaries of time-series data, effectively reducing hallucinations in chart summary generation. Specifically, we assign multiple agents to generate the initial chart summary and collaborate iteratively, during which they invoke external data analysis modules to extract insights and compile them into a coherent summary. Additionally, we implement a self-consistency test method to validate and correct our summary. We create a high-quality benchmark of charts and summaries, with hallucination types annotated on a sentence-by-sentence basis, facilitating the evaluation of the effectiveness of reducing hallucinations. Our evaluations using our benchmark show that our method surpasses state-of-the-art models, and that our summary hallucination rate is the lowest, which effectively reduces various hallucinations and improves summary quality.

Abstract:
With the growing computational power available for high-resolution ensemble simulations in scientific fields such as cosmology and oceanology, storage and computational demands present significant challenges. Current surrogate models fall short in the flexibility of point- or region-based predictions as the entire field reconstruction is required for each parameter setting, hence hindering the efficiency of parameter space exploration. Limitations exist in capturing physical attribute distributions and pinpointing optimal parameter configurations. In this work, we propose Explorable INR, a novel implicit neural representation-based surrogate model, designed to facilitate exploration and allow point-based spatial queries without computing full-scale field data. In addition, to further address computational bottlenecks of spatial exploration, we utilize probabilistic affine forms (PAFs) for uncertainty propagation through Explorable INR to obtain statistical summaries, facilitating various ensemble analysis and visualization tasks that are expensive with existing models. Furthermore, we reformulate the parameter exploration problem as optimization tasks using gradient descent and KL divergence minimization that ensures scalability. We demonstrate that the Explorable INR with the proposed approach for spatial and parameter exploration can significantly reduce computation and memory costs while providing effective ensemble analysis.

Abstract:
Investigating auditory perception and cognition in realistic, controlled environments is made possible by virtual reality (VR). However, when visual information is presented, sound localization results from multimodal integration. Additionally, using head-mounted displays leads to a distortion of visual egocentric distances. With two different paradigms, we investigated the extent to which different visual scenes influence auditory distance perception, and secondary presence and realism. To be more precise, different room models were displayed via HMD while participants had to localize sounds emanating from real loudspeakers. In the first paradigm, we manipulated whether a room was congruent or incongruent to the physical room. In a second paradigm, we manipulated room visibility - displaying either an audiovisual congruent room or a scene containing almost no spatial information- and localization task. Participants indicated distances either by placing a virtual loudspeaker, walking, or verbal report. While audiovisual room incongruence had a detrimental effect on distance perception, no main effect of room visibility was found but an interaction with the task. Overestimation of distances was higher using the placement task in the non-spatial scene. The results suggest an effect of visual scene on auditory perception in VR implying a need for consideration e.g. in virtual acoustics research.

Abstract:
Mixed Reality (MR) systems enable users to perform augmented superpowers that transcend real-world limitations. However, it remains unclear what types of action-outcome mappings can enable users to easily learn, control, and feel a sense of ownership of these augmented superpowers. Humans develop a set of sensorimotor regularities (i.e., image schemas and lawful relations between them) from recurring bodily experiences since early infancy, and use them to predict the outcome of our actions, or choose actions based on the desired outcome. We investigate whether sensorimotor regularities (SRs) can serve as effective design constraints for superpower interactions, by comparing three temporal manipulation methods in MR games: (1) mid-air button control; (2) gestures incongruent with SRs embedded in the concept of temporal manipulation; and (3) gestures congruent with these SRs. A within-subject study with 18 participants reveals that the SRs-congruent method enables significantly improved task performance, lower overall workload, and a greater sense of agency and presence compared to both an SRs-incongruent method and a mid-air button-based method. The SRs-congruent method also enabled faster mastery of the augmented superpower. No significant difference was observed in any of the above-mentioned metrics between the SRs-incongruent and mid-air button-based methods. These results empirically demonstrate multiple benefits of SRs as design constraints for superpower controls in MR, and encourage future research to explore their wider applicability in superpower interaction design.

Abstract:
Trust in agents within Virtual Reality is becoming increasingly important, as they provide advice and influence people's decision-making. However, previous studies show that encountering speech recognition errors can reduce users' trust in agents. Such errors lead users to ignore the agent's advice and make suboptimal decisions. While agents can offer an apology to repair trust, its effectiveness is often limited because it fails to fully repair the original level of trust. Therefore, we examined the use of social touch, a social interaction involving physical interaction between users and the virtual agent, to enhance the effect of an apology on trust repair and to increase the acceptability of its advice. In a controlled experiment (N=24), participants experienced a robotic arm touching the back of their hands while interacting with the agent before decision-making. The results showed that social touch did not repair participants' trust in agents. However, participants were more likely to accept the agent's advice when they experienced touch with physical feedback, regardless of the level of trust in the agent. We discuss the role of presenting physical haptic feedback and its influence on human-agent interactions in VR.

Abstract:
Virtual Reality Video Quality Assessment (VR-VQA) aims to evaluate the perceptual quality of 360-degree videos, which is crucial for ensuring a distortion-free user experience. Traditional VR-VQA methods trained on static datasets with limited distortion diversity struggle to balance correlation and precision. This becomes particularly critical when generalizing to diverse VR content and continually adapting to dynamic and evolving video distribution variations. To address these challenges, we propose a novel approach for assessing the perceptual quality of VR videos, Adaptive Score Alignment Learning (ASAL). ASAL integrates correlation loss with error loss to enhance alignment with human subjective ratings and precision in predicting perceptual quality. In particular, ASAL can naturally adapt to continually changing distributions through a feature space smoothing process that enhances generalization to unseen content. To further improve continual adaptation to dynamic VR environments, we extend ASAL with adaptive memory replay as a novel Continual Learning (CL) framework. Unlike traditional CL models, ASAL utilizes key frame extraction and feature adaptation to address the unique challenges of non-stationary variations with both the computation and storage restrictions of VR devices. We establish a comprehensive benchmark for VR-VQA and its CL counterpart, introducing new data splits and evaluation metrics. Our experiments demonstrate that ASAL outperforms recent strong baseline models, achieving overall correlation gains of up to 4.78% in the static joint training setting and 12.19% in the dynamic CL setting on various datasets. This validates the effectiveness of ASAL in addressing the inherent challenges of VR-VQA. Our code is available at https://github.com/ZhouKanglei/ASAL_CVQA.

Abstract:
Markerless hand-tracking has become increasingly common on commercially available virtual and mixed reality headsets to improve the naturalness of interaction and immersivity of virtual environments. However, there has been limited examination of the performance of markerless hand-tracking on commercial head-mounted displays. Here, we propose an evaluation methodology that leverages a robotic manipulator to measure the positional accuracy, jitter, and latency of such systems and provides a standardized characterization framework of markerless hand-tracking. We apply this methodology to evaluate the hand-tracking performance of two recent mixed reality devices from Meta: the Quest Pro and Quest 3. Results demonstrate the influence of proximity to the headset, rotation of hand, and joint selected as the tracking feature on hand-tracking performance. We found that hand-tracking error and jitter were lowest for both headsets in conditions where the knuckle was the tracking point compared to the fingertip. Regarding positional accuracy, in best-performing conditions, the Quest Pro outperformed the Quest 3 with 1.22 cm of average error compared to 1.73 cm. The opposite result was true concerning jitter, with results of 1.77 cm and 1.11 cm for the Quest Pro and Quest 3, respectively. We found latency highly variable for the Quest Pro (15.8 - 229.2 ms) and Quest 3 (14.4 - 220.5 ms). This work provides a testing framework for highly systematic and repeatable performance measurements of markerless hand-tracking systems embedded in headsets.

Abstract:
Seamless transition between public dialogue and private talks is essential in everyday conversations. Social Virtual Reality (VR) has revolutionized interpersonal communication by creating a sense of closeness over distance through virtual avatars. However, existing social VR platforms are not successful in providing safety and supporting private conversations, thereby hindering self-disclosure and limiting the potential for meaningful experiences. We approach this problem by exploring the factors affecting private conversations in social VR applications, including the usability of different interaction methods and the awareness with respect to the virtual world. We conduct both expert interviews and a controlled experiment with a social VR prototype we realized. We then leverage the outcomes of the two studies to establish a design space that considers diverse dimensions (including privacy levels, social awareness, and modalities), laying the groundwork for more intuitive and meaningful experiences of private conversation in social VR.

Abstract:
Mesh saliency aims to empower artificial intelligence with strong adaptability to highlight regions that naturally attract visual attention. Existing advances primarily emphasize the crucial role of geometric shapes in determining mesh saliency, but it remains challenging to flexibly sense the unique visual appeal brought by the realism of complex texture patterns. To investigate the interaction between geometric shapes and texture features in visual perception, we establish a comprehensive mesh saliency dataset, capturing saliency distributions for identical 3D models under both non-textured and textured conditions. Additionally, we propose a unified saliency prediction model applicable to various mesh types, providing valuable insights for both detailed modeling and realistic rendering applications. This model effectively analyzes the geometric structure of the mesh while seamlessly incorporating texture features into the topological framework, ensuring coherence throughout appearance-enhanced modeling. Through extensive theoretical and empirical validation, our approach not only enhances performance across different mesh types, but also demonstrates the model's scalability and generalizability, particularly through cross-validation of various visual features.

Abstract:
Considering the issue of privacy leakage and motivating more sophisticated protection methods for air-typing with XR devices, in this paper, we propose AirtypeLogger, a new approach towards practical video-based attacks on the air-typing activities of XR users in virtual space. Different from the existing approaches, AirtypeLogger considers a scenario in which the users are typing a short text fragment with semantic meaning occasionally under the spy of video cameras. It detects and localizes the air-typing events in video streams and proposes the spatial-temporal representation to encode the keystrokes' relative positions and temporal order. Then, high-precision inference can be achieved by applying a Transformer-based network to the spatial and temporal encodings of the keystroke sequences. Finally, according to our extensive real-world experiments, AirtypeLogger can achieve a Character Error Rate (CER) of less than 0.1 as long as 7 air-typing events are observed, which is impossible for previous approaches that require long-term observation of the typing activities online before launching inference attacks. The implementation details and source codes can be found at https://github.com/ztysdu/AirtypeLogger.

Abstract:
The emergence of holographic media drives the standardization of Geometry-based Point Cloud Compression (G-PCC) to sustain networked service provisioning. However, G-PCC inevitably introduces visually annoying artifacts, degrading the quality of experience (QoE). This work focuses on restoring G-PCC compressed point cloud attributes, e.g., RGB colors, to which fully data-driven and rules-unrolling-based post-processing filters are studied. At first, as compressed attributes exhibit nested blockiness, we develop a learning-based sample adaptive offset (NeuralSAO), which leverages a neural model using multiscale feature aggregation and embedding to characterize local correlations for quantization error compensation. Later, given statistically Gaussian distributed quantization noise, we suggest the utilization of a bilateral filter with Gaussian kernels to weigh neighbors by jointly considering their geometric and photometric contributions for restoration. Since local signals often present varying distributions, we propose estimating the smoothing parameters of the bilateral filter using an ultra-lightweight neural model. Such a bilateral filter with learnable parameters is called NeuralBF. The proposed NeuralSAO demonstrates the state-of-art restoration quality improvement, e.g., > >20% BD-BR (Bjøntegaard delta rate) reduction over G-PCC on solid points clouds. However, NeuralSAO is computationally intensive and may suffer from poor generalization. On the other hand, although NeuralBF only achieves half of the gains of NeuralSAO, it is lightweight and exhibits impressive generalization across various samples. This comparative study between the data-driven large-scale NeuralSAO and the rules-unrolling-based small-scale NeuralBF helps to understand the capacity (i.e., performance, complexity, generalization) of underlying filters in terms of the quality restoration for compressed point cloud attribute.

Abstract:
Informal learners of computational skills often find it difficult to self-direct their learning pursuits, which may be spread across different mediums and study sessions. Inspired by self-monitoring interventions from domains such as health and productivity, we investigate key requirements for helping informal learners better self-reflect on their learning experiences. We carried out two elicitation studies with article-based and interactive probes to explore a range of manual, automatic, and semi-automatic design approaches for capturing and presenting a learner's data. We found that although automatically generated visual overviews of learning histories are initially promising for increasing awareness, learners prefer having controls to manipulate overviews through personally relevant filtering options to better reflect on their past, plan for future sessions, and communicate with others for feedback. To validate our findings and expand our understanding of designing self-monitoring tools for use in real settings, we gathered further insights from experts, who shed light on factors to consider in terms of data collection techniques, designing for reflections, and carrying out field studies. Our findings have several implications for designing learner-centered self-monitoring interventions that can be both useful and engaging for informal learners.

Abstract:
In medical diagnostics of both early disease detection and routine patient care, particle-based contamination of in-vitro diagnostics consumables poses a significant threat to patients. Objective data-driven decision-making on the severity of contamination is key for reducing patient risk, while saving time and cost in quality assessment. Our collaborators introduced us to their quality control process, including particle data acquisition through image recognition, feature extraction, and attributes reflecting the production context of particles. Shortcomings in the current process are limitations in exploring thousands of images, data-driven decision making, and ineffective knowledge externalization. Following the design study methodology, our contributions are a characterization of the problem space and requirements, the development and validation of DaedalusData, a comprehensive discussion of our study's learnings, and a generalizable framework for knowledge externalization. DaedalusData is a visual analytics system that enables domain experts to explore particle contamination patterns, label particles in label alphabets, and externalize knowledge through semi-supervised label-informed data projections. The results of our case study and user study show high usability of DaedalusData and its efficient support of experts in generating comprehensive overviews of thousands of particles, labeling of large quantities of particles, and externalizing knowledge to augment the dataset further. Reflecting on our approach, we discuss insights on dataset augmentation via human knowledge externalization, and on the scalability and trade-offs that come with the adoption of this approach in practice.

Abstract:
This paper presents a novel end-to-end framework for closed-form computation and visualization of critical point uncertainty in 2D uncertain scalar fields. Critical points are fundamental topological descriptors used in the visualization and analysis of scalar fields. The uncertainty inherent in data (e.g., observational and experimental data, approximations in simulations, and compression), however, creates uncertainty regarding critical point positions. Uncertainty in critical point positions, therefore, cannot be ignored, given their impact on downstream data analysis tasks. In this work, we study uncertainty in critical points as a function of uncertainty in data modeled with probability distributions. Although Monte Carlo (MC) sampling techniques have been used in prior studies to quantify critical point uncertainty, they are often expensive and are infrequently used in production-quality visualization software. We, therefore, propose a new end-to-end framework to address these challenges that comprises a threefold contribution. First, we derive the critical point uncertainty in closed form, which is more accurate and efficient than the conventional MC sampling methods. Specifically, we provide the closed-form and semianalytical (a mix of closed-form and MC methods) solutions for parametric (e.g., uniform, Epanechnikov) and nonparametric models (e.g., histograms) with finite support. Second, we accelerate critical point probability computations using a parallel implementation with the VTK-m library, which is platform portable. Finally, we demonstrate the integration of our implementation with the ParaView software system to demonstrate near-real-time results for real datasets.

Abstract:
There is increased interest in understanding the interplay between text and visuals in the field of data visualization. However, this attention has predominantly been on the use of text in standalone visualizations (such as text annotation overlays) or augmenting text stories supported by a series of independent views. In this paper, we shift from the traditional focus on single-chart annotations to characterize the nuanced but crucial communication role of text in the complex environment of interactive dashboards. Through a survey and analysis of 190 dashboards in the wild, plus 13 expert interview sessions with experienced dashboard authors, we highlight the distinctive nature of text as an integral component of the dashboard experience, while delving into the categories, semantic levels, and functional roles of text, and exploring how these text elements are coalesced by dashboard authors to guide and inform dashboard users. Our contributions are threefold. First, we distill qualitative and quantitative findings from our studies to characterize current practices of text use in dashboards, including a categorization of text-based components and design patterns. Second, we leverage current practices and existing literature to propose, discuss, and validate recommended practices for text in dashboards, embodied as a set of 12 heuristics that underscore the semantic and functional role of text in offering navigational cues, contextualizing data insights, supporting reading order, among other concerns. Third, we reflect on our findings to identify gaps and propose opportunities for data visualization researchers to push the boundaries on text usage for dashboards, from authoring support and interactivity to text generation and content personalization. Our research underscores the significance of elevating text as a first-class citizen in data visualization, and the need to support the inclusion of textual components and their interactive affordances in dashboard design.

Abstract:
The visualization community has a rich history of reflecting upon visualization design flaws. Although research in this area has remained lively, we believe it is essential to continuously revisit this classic and critical topic in visualization research by incorporating more empirical evidence from diverse sources, characterizing new design flaws, building more systematic theoretical frameworks, and understanding the underlying reasons for these flaws. To address the above gaps, this work investigated visualization design flaws through the lens of the public, constructed a framework to summarize and categorize the identified flaws, and explored why these flaws occur. Specifically, we analyzed 2227 flawed data visualizations collected from an online gallery and derived a design task-associated taxonomy containing 76 specific design flaws. These flaws were further classified into three high-level categories (i.e., misinformation, uninformativeness, unsociability) and ten subcategories (e.g., inaccuracy, unfairness, ambiguity). Next, we organized five focus groups to explore why these design flaws occur and identified seven causes of the flaws. Finally, we proposed a research agenda for combating visualization design flaws and summarize nine research opportunities.

Abstract:
Differential privacy ensures the security of individual privacy but poses challenges to data exploration processes because the limited privacy budget incapacitates the flexibility of exploration and the noisy feedback of data requests leads to confusing uncertainty. In this study, we take the lead in describing corresponding exploration scenarios, including underlying requirements and available exploration strategies. To facilitate practical applications, we propose a visual analysis approach to the formulation of exploration strategies. Our approach applies a reinforcement learning model to provide diverse suggestions for exploration strategies according to the exploration intent of users. A novel visual design for representing uncertainty in correlation patterns is integrated into our prototype system to support the proposed approach. Finally, we implemented a user study and two case studies. The results of these studies verified that our approach can help develop strategies that satisfy the exploration intent of users.

Abstract:
This paper discusses challenges and design strategies in responsive design for thematic maps in information visualization. Thematic maps pose a number of unique challenges for responsiveness, such as inflexible aspect ratios that do not easily adapt to varying screen dimensions, or densely clustered visual elements in urban areas becoming illegible at smaller scales. However, design guidance on how to best address these issues is currently lacking. We conducted design sessions with eight professional designers and developers of web-based thematic maps for information visualization. Participants were asked to redesign a given map for various screen sizes and aspect ratios and to describe their reasoning for when and how they adapted the design. We report general observations of practitioners' motivations, decision-making processes, and personal design frameworks. We then derive seven challenges commonly encountered in responsive maps, and 17 strategies to address them, such as repositioning elements, segmenting the map, or using alternative visualizations. We compile these challenges and strategies into an illustrated cheat sheet targeted at anyone designing or learning to design responsive maps. The cheat sheet is available online: responsive-vis.github.io/map-cheat-sheet.

Abstract:
Optical see-through head-mounted displays (OHMDs) blend digital content with the physical world, presenting unique color management challenges. Previous literature suggests using green as the main color, but this severely limits creative freedom. To address this, we conducted an empirical study with 30 participants, evaluating 216 colors under various OHMD usage conditions. Based on the results, we propose color guidelines indicating each hue’s clear and comfortable saturation and brightness ranges, along with clarity and comfort scores across hues for different devices and lighting conditions. Our color guidelines expand the usable color palette, offering designers a wider range of color options. These guidelines were used and iteratively refined through feedback in a workshop with 12 designers, integrating them into practical design workflows. The resulting comprehensive color guide provides a valuable resource for OHMD interface designers, enhancing both the aesthetic possibilities and functional effectiveness of augmented reality experiences.

Abstract:
Visualization recommendation systems make data exploration less tedious by automating the process of visualization generation. They are particularly helpful for non-expert users who may not be familiar with a data set or the process of visualization specification. These systems allow users to input their preferences in the form of partial specifications to steer the recommendations made. However, the interaction approaches for partial specification input and their trade-offs have not been explored in prior work. In this article, we compare three different combinations of interaction approaches and granularities for users to indicate a preferred partial specification: 1) manual input, 2) inferring preferred partial specifications from binary like/dislike ratings for a visualization as a whole, or 3) inferring preferred partial specifications from binary like/dislike ratings for granular components of a visualization specification. In a between-subjects study, participants were assigned to one of three conditions and asked to complete a data exploration task. Our results indicate that manual input led to a greater coverage of data dimensions, while like/dislike ratings led to a greater diversity of marks and channels used. Qualitative participant feedback also reveals differences in user strategy and visualization comprehension across the three interaction conditions. Finally, we conclude with a discussion on implications for multiplicity and visualization comprehension during visual data exploration.

Abstract:
Recent advances in eye-tracking technology have positioned gaze as an efficient and intuitive input method for Virtual Reality (VR), offering a natural and immersive user experience. As a result, gaze input is now leveraged for fundamental interaction tasks such as selection, manipulation, crossing, and steering. Although several studies have modeled user steering performance across various path characteristics and input methods, our understanding of gaze-based steering in VR remains limited. This gap persists because the unique qualities of eye movements-involving rapid, continuous motions-and the variability in eye-tracking make findings from other input modalities nontransferable to a gaze-based context, underscoring the need for a dedicated investigation into gaze-based steering behaviors and performance. To bridge this gap, we present two user studies to explore and model gaze-based steering. In the first one, user behavior data are collected across various path characteristics and eye-tracking conditions. Based on this data, we propose four refined models that extend the classic Steering Law to predict users' movement time in gaze-based steering tasks, explicitly incorporating the impact of tracking quality. The best-performing model achieves an adjusted R2 of 0.956, corresponding to a 16% improvement in movement time prediction. This model also yields a substantial reduction in AIC (from 1550 to 1132) and BIC (from 1555 to 1142), highlighting improved model quality and better balance between goodness of fit and model complexity. Finally, data from a second study with varied settings, such as a different eye-tracking sampling rate, illustrate the strong robustness and predictability of our models. Finally, we present scenarios and applications that demonstrate how our models can be used to design enhanced gaze-based interactions in VR systems.

Abstract:
AR head-mounted displays (HMDs) facilitate pedestrian navigation by integrating AR guidance into users' field of view (FOV). Displaying AR guidance using a body-fixed coordinate system has the potential to further leverage this integration by enabling users to control when the guidance appears in their FOV. However, it remains unclear how to effectively position AR guidance within this coordinate system during pedestrian navigation. Therefore, we explored the effects of three AR guidance positions (top, middle, and bottom) within a body-fixed coordinate system on pedestrian navigation in a virtual environment. Our results showed that AR guidance position significantly influenced eye movements, walking behaviors, and subjective evaluations. The top position resulted in the shortest duration of fixations on the guidance compared to the middle and bottom positions, and lower mental demand than the bottom position. The middle position had the smallest rate of vertical eye movement during gaze shifts between the guidance and the environment, and the smallest relative difference in walking speed between fixations on the guidance and the environment compared to the top and bottom positions. The bottom position led to the shortest duration and smallest amplitude of gaze shifts between the guidance and the environment compared to the top and middle positions, and lower frustration than the top position. Based on these findings, we offer design implications for AR guidance positioning within a body-fixed coordinate system during pedestrian navigation.

Abstract:
Hand-tracking technologies allow us to use our own hands to interact with real and virtual objects in Augmented Reality (AR) environments. This enables us to explore the interplay between hand-visualizations and hand-object interactions. We present a user study that examines the effect of different hand visualizations (invisible, transparent, opaque) on manipulation performance when interacting with real and virtual objects. For this, we implemented video-see-through (VST) AR-based virtual building blocks and hot wire tasks with real one-to-one counterparts that require participants to use gross and fine motor hand movements. To evaluate manipulation performance, we considered three measures: task completion time, number of collisions (hot wire task), and percentage of object displacement (building block task). Additionally, we explored the sense of agency and subjective impressions (preference, ease of interaction, successful and awkwardness) evoked by the different hand-visualizations. The results show that (1) manipulation performance is significantly higher when interacting with real objects compared to virtual ones, (2) invisible hands lead to fewer errors, higher agency, higher perceived success and ease of interaction during fine manipulation tasks with real objects, and (3) having some visualization of the virtual hands (transparent or opaque) overlayed on the real hands is preferred when manipulating virtual objects even when there are no significant performance improvements. Our empirical findings about the differences when interacting with real and virtual objects can aid hand visualization choices for manipulation tasks in AR.

Abstract:
This study explores how human and AI visual attention differ in a short-term prediction task, particularly in the moments before an accident is about to happen. Since real-world studies of this kind would pose ethical and safety risks, we employed virtual reality (VR) to simulate an accident scenario. In the scenario, the driver approaches a fork in the road, knowing that one path would lead off a cliff crashing the car fatally-as the fork comes closer, the other, safe, path is suddenly blocked by trees, forcing the driver to make a split-second decision where to go. A total of N=71 drivers completed the task, and we asked another N=30 observers to watch short video clips leading up to the final event and to predict which way the driver would take. We then compared both prediction accuracy as well as attention patterns-how focus is distributed across objects-with AI systems, including vision language models (VLMs) and vision-only models. We found that overall, prediction performance increased as the accident time point approached; interestingly, humans fared better than AI systems overall except for the final time period just before the event. We also found that humans adapted their attention dynamically, shifting focus to important scene elements before an event, whereas AI attention remained static, overlooking key details of the scene. Importantly, as the accident time point approached, human-AI attentional alignment decreased, even though both types of models improved in prediction accuracy. Despite distinct temporal trajectories-vision-only models declining from an early advantage and VLMs peaking in the middle-both models achieved low to zero alignment with human attention. These findings highlight a critical dissociation: AI models make accurate predictions, but rely on visual strategies diverging from human processing, underscoring a gap between explainability and task performance.

Abstract:
Point cloud perceptual quality assessment plays a critical role in many applications, including compression and communication. We propose PKT-PCQA, a point-based no-reference point cloud quality assessment deep learning network that emulates the human visual system by using progressive knowledge transfer to convert coarse-grained quality classification knowledge into a fine-grained quality prediction task. PKT-PCQA exploits local and global features, as well as an attention mechanism based on spatial and channel attention modules. Experiments on three large and independent point cloud assessment datasets show that PKT-PCQA outperforms existing no-reference and reduced-reference point cloud quality assessment methods and achieves better or similar performance compared to several State-of-the-Art full-reference methods.

Abstract:
In this article, we assess the visualization literacy of two prominent Large Language Models (LLMs): OpenAI’s Generative Pretrained Transformers (GPT), the backend of ChatGPT, and Google’s Gemini, previously known as Bard, to establish benchmarks for assessing their visualization capabilities. While LLMs have shown promise in generating chart descriptions, captions, and design suggestions, their potential for evaluating visualizations remains under-explored. Collecting data from humans for evaluations has been a bottleneck for visualization research in terms of both time and money, and if LLMs were able to serve, even in some limited role, as evaluators, they could be a significant resource. To investigate the feasibility of using LLMs in the visualization evaluation process, we explore the extent to which LLMs possess visualization literacy—a crucial factor for their effective utility in the field. We conducted a series of experiments using a modified 53-item Visualization Literacy Assessment Test (VLAT) for and . Our findings indicate that the LLMs we explored currently fail to achieve the same levels of visualization literacy when compared to data from the general public reported in VLAT, and LLMs heavily relied on their pre-existing knowledge to answer questions instead of utilizing the information provided by the visualization when answering questions.

Abstract:
Animatable and relightable 3D facial generation has fundamental applications in computer vision and graphics. Although animation and relighting are highly correlated, previous methods usually address them separately. Effectively combining animation methods and relighting methods is nontrivial. In terms of explicit shading models, animatable methods cannot be easily extended to achieve realistic relighting results, such as shadow effects, due to prohibitive computational training costs. Regarding implicit lighting representations, current animatable methods cannot be incorporated due to their inharmonious animation representations, i.e., deforming spatial points. This paper, armed with a lightweight but effective lighting representation, presents a compatible animation representation to achieve a disentangled generative model of 3D animatable and relightable heads. Our represented animation allows for updating and control of realistic lighting effects. Due to the disentangled nature of our representations, we learn the animation and relighting from large-scale, in-the-wild videos instead of relying on a morphable model. We show that our method can synthesize geometrically consistent and detailed motion along with the disentangled control of lighting conditions. We further show that our method is still compatible with morphable models for driving generated avatars. Our method can also be extended to domains without video data by domain transfer to achieve a broader range of animatable and relightable head synthesis. We will release the code for reproducibility and facilitating future research.

Abstract:
Over the past few years, the manufacturing industry has increasingly embraced Augmented Reality (AR) for inspecting real products, yet faces challenges in visualization modalities. In fact, AR content presentation significantly impacts user performance, especially when virtual object colors lack real-world context. Additionally, the lack of studies in this area compounds uncertainty about visualization effects on user performance in inspection tasks. This study introduces a novel AR recoloring technique to enhance user performance during industrial assembly inspection tasks. This technique automatically recolors virtual components based on their physical counterparts, improving distinctiveness. Experimental comparisons with AR experts and representative users, using objective and subjective metrics, demonstrate the proposed AR recoloring technique enhances task performance and reduces mental burden during inspection activities. This innovative approach outperforms established methods like CAD and random modes, showcasing its potential for advancing AR applications in manufacturing, particularly in the inspection of products.

Abstract:
The rapid development of large language models (LLMs), such as ChatGPT, has revolutionized the efficiency of creating programming tutorials. LLMs can be instructed with text prompts to generate comprehensive text descriptions for code snippets provided by users. However, the lack of transparency in the end-to-end generation process has hindered the understanding of model behavior and limited user control over the generated results. To tackle this challenge, we introduce a novel approach that breaks down the programming tutorial creation task into actionable steps. By employing the tree-of-thought method, LLMs engage in an exploratory process to generate diverse and faithful programming tutorials. We then present SPROUT, an authoring tool equipped with a series of interactive visualizations that empower users to have greater control and understanding of the programming tutorial creation process. A formal user study demonstrated the effectiveness of SPROUT, showing that our tool assists users to actively participate in the programming tutorial creation process, leading to more reliable and customizable results. By providing users with greater control and understanding, SPROUT enhances the user experience and improves the overall quality of programming tutorial.

Abstract:
Mixed reality (MR) technologies have a high potential to enhance obstacle negotiation training beyond the capabilities of existing physical systems. Despite such potential, the feasibility of using MR for obstacle negotiation on typical training treadmill systems and its effects on obstacle negotiation performance remains largely unknown. This research bridges this gap by developing an MR obstacle negotiation training system deployed on a treadmill, and implementing two MR systems with a video see-through (VST) and an optical see-through (OST) Head Mounted Displays (HMDs). We investigated the obstacle negotiation performance with virtual and real obstacles. The main outcomes show that the VST MR system significantly changed the parameters of the leading foot in cases of Box obstacle (approximately 22 cm to 30 cm for stepping over 7cm-box), which we believe was mainly attributed to the latency difference between the HMDs. In the condition of OST MR HMD, users tended to not lift their trailing foot for virtual obstacles (approximately 30 cm to 25 cm for stepping over 7cm-box). Our findings indicate that the low-latency visual contact with the world and the user's body is a critical factor for visuo-motor integration to elicit obstacle negotiation.

Abstract:
In this study, by taking advantage of the benefit of virtual reality (VR), we propose a new learning method that employs multiple embodied self-avatars during learning. Based on the multiple-context effect, which posits that learning in diverse situations can prevent forgetting and enhance memory retention, we conducted a between-participants study under two conditions: the varied avatar condition, in which participants learned sign languages with different self-avatars in six iterations, and the constant avatar condition, in which the same self-avatar was used consistently. Initially, the varied avatar condition performed worse than the constant avatar condition. However, in a test conducted after one week in the real world, the varied avatar condition showed significantly less forgetting and better retention than the constant avatar condition. Furthermore, our results suggested a positive correlation between the degree of embodiment toward the avatars and the effectiveness of the proposed method. This study presents an innovative design approach for the use of self-avatars in VR-based education.

Abstract:
Performing tasks in a close range using augmented content or instructions visualized on a 2D display can be difficult because of missing visual information in the third dimension. This is because the world on the screen is rendered from the perspective of a single camera, typically on the device itself. However, when performing tasks using hands, haptic feedback supports vision, and prior knowledge and visual context affect task performance. This study rendered the world on a display from the user’s perspective to re-enable depth cues from motion parallax and compared it with the conventional device perspective during haptic interactions. We conducted a user study involving 20 subjects and two experiments. First, the accuracy of touchpoint and depth estimation was measured under the conditions of a visual context and perspective rendering on a magic-lens display. We found that user-perspective rendering slightly improved the touch accuracy of targets on a physical surface; however, it significantly improved interactions without tactile feedback. This effect is relatively large when contextual information from the environment is absent, and it diminishes with increased haptic interactions. In the second experiment, we used a user-perspective magic lens to validate the proposed method in a practical needle injection scenario and confirm that the initial injections to virtual targets were more accurate. The results indicate that user-perspective rendering on magic lenses improves immediate performance in haptic tasks, suggesting they are particularly advantageous for frequently changing environments or short-duration tasks.

Abstract:
Understanding visual attention is key to designing efficient human-computer interaction, especially for virtual reality (VR) and augmented reality (AR) applications. However, the relationship between 3D spatial attributes of visual stimuli and visual attention is still underexplored. Thus, we design an experiment to collect a gaze dataset in VR, and use it to quantitatively model the probability of first attention between two stimuli. First, we construct the dataset by presenting subjects with a synthetic VR scene containing varying spatial configurations of two spheres. Second, we formulate their selective attention based on a probability model that takes as input two view-specific stimuli attributes: their eccentricities in the field of view and their sizes as visual angles. Third, we train two models using our gaze dataset to predict the probability distribution of a user’s preferences of visual stimuli within the scene. We evaluate our method by comparing model performance across two challenging synthetic scenes in VR. Our application case study demonstrates that VR designers can utilize our models for attention prediction in two-foreground-object scenarios, which are common when designing 3D content for storytelling or scene guidance. We make the dataset and the source code to visualize it available alongside this work.

Abstract:
Dynamic collaboration is a common form of human interaction in everyday life. With the increasing support of virtual reality (VR) applications for such collaborative joint actions, understanding how visualization design choices affect collaboration becomes crucial. However, the impact of two key elements — avatar visibility and perspective — remains underexplored in dynamic collaborative tasks. To address this, we conducted two user studies representing distinct collaborative contexts: a proximal task and a spatially distributed task. Within these contexts, we investigated how varying levels of avatar visibility (full-body, upper-body, and head-and-hands) and perspectives (first-person perspective, 1PP, and third-person perspective, 3PP) influenced social presence and collaborative performance, including both task completion and team coordination aspects. Our findings revealed that 3PP significantly enhanced both social presence and team coordination compared to 1PP. The impact of avatar visibility was context-dependent: in environments with low dynamic complexity, even low-visibility avatars maintained effective collaboration, while complex spatial tasks benefited from increased avatar visibility. Furthermore, our research demonstrated that 3PP mitigated the negative effects of low-visibility avatars in 1PP, lessening coordination conflicts in spatially complex scenarios. Based on these findings, we propose design recommendations for avatar visibility and perspective choices in dynamic collaborative environments. Our research advances the understanding of how these fundamental visual design elements shape VR collaboration, informing the design of future interactive environments.

Abstract:
Analysts often have to work with and make sense of large complex networks. One possible solution is to make visualisations interactive, providing users with a way to control visual clutter. Although several interactive methods have been proposed, there may be situations where some of them are too specific to be directly applicable. We have therefore identified several underlying low-level visual transformations, steered by group structures in the networks, and investigated their individual effects on user performance. This may both facilitate the development of further methods and support the generation of new hypotheses. We conducted an exploratory online experiment with 300 participants, involving five tasks, one control condition, and five group-based visual transformations: de-emphasising groups by opacity, position or size, aggregating groups, and hiding groups. The results for the three tasks that were specifically referring to groups show a high usage of the visual transformations by participants and several positive effects of the latter on accuracy, completion time, and mental effort spent. On the other hand, the two tasks that were not directly referring to groups show a lower usage of the visual transformations and the results regarding effects are rather mixed.

Affiliations: Thomas Lord Department of Computer Science, Viterbi School of Engineering, University of Southern California (USC), Los Angeles, CA, USA; Alfred E. Mann Department of Biomedical Engineering, Viterbi School of Engineering, University of Southern California (USC), Los Angeles, CA, USA; Department of Aerospace and Mechanical Engineering, Viterbi School of Engineering, University of Southern California (USC), Los Angeles, CA, USA; Division of Biokinesiology and Physical Therapy, Viterbi School of Engineering, University of Southern California (USC), Los Angeles, CA, USA

Abstract:
Virtual Reality (VR) systems have increasingly leveraged multisensory feedback to enrich user experience and mitigate cybersickness. With a similar goal in focus, this paper presents an in-depth exploration of integrating airflow with visual and kinesthetic cues in a VR surfing simulation. Utilizing a custom-designed airflow system and a physical surfboard mounted on a 6-Degree of Freedom (DoF) motion platform, we present two studies that evaluate the effect of the different feedback modalities. The first study assesses the impact of variable airflow, which dynamically adjusts to the user's speed (wind speed) in VR, compared to constant airflow conditions, under both active and passive user engagement scenarios. Results demonstrate that variable airflow significantly enhances immersion and reduces cybersickness, particularly when users are actively engaged in the simulation. The second study evaluates the individual and combined effects of vision, motion, and airflow on acceleration perception, user immersion, and cybersickness, revealing that the integration of all feedback modalities yields the most immersive and comfortable VR experience. This study underscores the importance of synchronized multisensory feedback in dynamic VR environments and provides valuable insights for the design of more immersive and realistic virtual simulations, particularly in aquatic, interactive, and motion-intensive scenarios.

Abstract:
Virtual reality, which enables users to engage in physical activities in ways distinct from those in the real world, is increasingly recognized for its potential to enhance motor skill acquisition. Research on co-embodiment learning, in which instructors and learners utilize a single avatar that represents a weighted average of their movements, has demonstrated its efficacy in facilitating motor skill development. However, the current implementation of co-embodiment learning necessitates the real-time participation of instructors proficient in both virtual reality and co-embodiment, which poses challenges for its widespread adoption. To address this limitation, this study proposed a method for developing instructors trained on human motor data to effectively support motor skill learning through co-embodiment. The AI model was trained using supervised learning on data obtained from human motor learning sessions that employed co-embodiment. To evaluate the performance of the AI instructor, we compared the learning performance in co-embodiment learning with that of the AI instructor, recorded human instructor data, and a human instructor as well as in solo learning. The results showed that practicing with the AI instructor significantly improved learning efficiency compared with practicing alone or with recorded data and was comparable to that achieved by practicing with a human instructor.

Abstract:
Immersive authoring tools have emerged as key enablers for trainers-designers to create Virtual Reality Learning Systems (VRLS) without requiring extensive programming skills. However, the design of such tools presents significant challenges in terms of interaction, usability, and interface complexity. These challenges underscore the need to validate an appropriate design interface to ensure these tools can be effectively utilized by non-technical users. This study evaluates the usability of an immersive authoring tool for VRLS using interactive panoramic videos. Two types of interfaces were compared: one that integrates storyboarding and rendering visualization, and one that separates these functionalities. Quantitative and qualitative data were collected from 24 participants divided into two groups. The results indicated that the separated interface was more effective, efficient, and satisfactory, particularly for more complex scenarios. Moreover, the integrated interface led to more pronounced symptoms of cybersickness while motivation levels did not differ significantly between the two groups. These findings highlight that integrating all functionalities into a single interface may not always be the best approach, especially for complex tasks, as it can lead to decreased usability and increased physical discomfort.

Abstract:
Music Performance Anxiety (MPA) is highly prevalent among musicians and often debilitating, associated with changes in cognitive, emotional, behavioral, and physiological responses to performance situations. Efforts have been made to create simulated performance environments in conservatoires and Virtual Reality (VR) to assess their effectiveness in managing MPA. Despite these advances, results have been mixed, underscoring the need for controlled experimental designs and joint analyses of performance, physiology, and subjective ratings in these settings. Furthermore, the broader application of simulated performance environments for at-home use and laboratory studies on MPA remains limited. We designed VR scenarios to induce MPA in pianists and embedded them within a controlled within-subject experimental design to systematically assess their effects on performance, physiology, and anxiety ratings. Twenty pianists completed a performance task under two conditions: a public ‘Audition’ and a private ‘Studio’ rehearsal. Participants experienced VR pre-performance settings before transitioning to live piano performances in the real world. We measured subjective anxiety, performance (MIDI data), and heart rate variability (HRV). Compared to the Studio condition, pianists in the Audition condition reported higher somatic anxiety ratings and demonstrated an increase in performance accuracy over time, with a reduced error rate. Additionally, their performances were faster and featured increased note intensity. No concurrent changes in HRV were observed. These results validate the potential of VR to induce MPA, enhancing pitch accuracy and invigorating tempo and dynamics. We discuss the strengths and limitations of this approach to develop VR-based interventions to mitigate the debilitating effects of MPA.

Abstract:
Resetting in redirected walking (RDW) allows users to maintain a continuous, collision-free walking experience in virtual reality (VR), even in a limited physical space. Since frequent resets reduce the user's sense of immersion, extensive research has been conducted to develop resetters that provide optimal reset directions. Various visual reset user interfaces (UIs) have been proposed to help users perform the correct reset direction according to the improved resetter, but their effectiveness has not been sufficiently verified. In addition, expert interviews conducted to identify the problems in the current reset process revealed that users sometimes fail to recognize the visual reset UI in time. Therefore, we propose a novel visual reset UI using Gauge, which is expected to provide users with an effective and high-quality experience. In Study 1, we demonstrate the effectiveness of the Gauge UI by comparing it to existing UIs (Direction, End Point, and Arrow Alignment). Users of various locomotion techniques, including RDW, inevitably need to perform resets, and in this work we propose a novel paradigm: a combined multimodal reset interface.

Abstract:
The integration of immersive Virtual Reality (I-VR) technology in education has emerged as a promising approach for enhancing learning experiences. There is a handful of research done to study the impact of I-VR on learning outcomes, comparison of learning using I-VR and other traditional learning methods, and the impact of values such as haptic sensation, and verbal and non-verbal cues on the learning outcomes. However, there is a dearth of research on understanding how learning is happening from the perspective of the behavior of the learners in the Virtual Reality Learning Environment (VRLE). To address this gap, we developed an Interaction Behavioral Data (IBD) logging mechanism to log all the interaction traces that constitute the behavior of the learners in a Virtual Reality Learning Environment (VRLE). We deployed the IBD logging mechanism in a VRLE used to learn electromagnetic induction concepts and conducted a study with 30 undergraduate computer science students. We extract the learners' actions from the logged data and contextualize them based on the action features such as duration (Long and Short), and frequency of occurrence (First and Repeated occurrence). In this paper, we investigate the actions extracted from logged interaction trace data to understand the behaviors that lead to high and low performance in the VRLE. Using Epistemic Network Analysis (ENA), we identify differences in prominent actions and co-occurring actions between high and low performers. Additionally, we apply Differential Sequence Mining (DSM) to uncover significant action patterns, involving multiple actions, that are differentially frequent between these two groups. Our findings demonstrate that high performers engage in structured, iterative patterns of experimentation and evaluation, while low performers exhibit less focused exploration patterns. The insights gained from ENA and DSM highlight the behavioral variations between high and low performers in the VRLE, providing valuable information for enhancing learning experiences in VRLEs. These insights gained can be further utilized by the VR content developers to develop adaptive VR learning content by providing personalized scaffolding leading to the enhancement in the learning process via I-VR.

Abstract:
The development of AR and VR technologies is enhancing users' online shopping experiences in various ways. However, in existing VR shopping applications, shopping contexts merely refer to the products and virtual malls or metaphorical scenes where users select products. This leads to the defect that users can only imagine rather than intuitively feel whether the selected products are suitable for their real usage scenes, resulting in a significant discrepancy between their expectations before and after the purchase. To address this issue, we propose PaRUS, a VR shopping approach that focuses on the context between products and their real usage scenes. PaRUS begins by rebuilding the virtual scenario of the products' real usage scene through a new semantic scene reconstruction pipeline (manual operation needed), which preserves both the structured scene and textured object models in the scene. Afterwards, intuitive visualization of how the selected products fit the reconstructed virtual scene is provided. We conducted two user studies to evaluate how PaRUS impacts user experience, behavior, and satisfaction with their purchase. The results indicated that PaRUS significantly reduced the perceived performance risk and improved users' trust and expectation with their results of purchase.

Abstract:
Previous studies introduced an avatar body control sharing system known as “virtual co-embodiment,” where control over bodily movements and external events, or agency, of a single avatar is shared among multiple individuals. However, how this virtual co-embodiment experience influences users' perception of agency, both explicitly and implicitly, and the extent to which they are willing to take responsibility for successful or failed outcomes, remains an imminent problem. In this research, we addressed this issue using: (1) explicit agency questionnaires, (2) implicit intentional binding (IB) effect, (3) responsibility attribution measured through financial gain/loss distribution, and (4) interview to evaluate this experience where agency over the right hand's fingers was fully transferred to a human partner. Given the distinction between two layers of agency (body agency: control over actions, and external agency: action's effect on external events), we also investigated the impact of sharing only the body-level of agency. In a ball-throwing task involving 24 participants, results showed that sharing body agency over the fingers negatively affected the feeling of having control over both the fingers and the entire right upper limb, as measured by the questionnaire. However, sharing external agency did not significantly diminish the participants' perceived control over the ball-throwing, as indicated by IB. Interestingly, while IB demonstrated that participants felt greater causality for failed ball-throwing attempts, they were reluctant to take responsibility and accept financial penalties. Additionally, responsibility attribution was found to be linked to the participants' personal trait—Locus of Control.

Abstract:
Spatial computing is set to change the way we work. It will enable both focused work through a higher degree of immersion and collaborative work through enhanced integration of shared interaction spaces or interaction partners. With the Apple Vision Pro, the level of immersion can be adjusted seamlessly. So far, there have been no systematic studies on how this adjustability affects work performance when working alone or together. The present empirical study fills this research gap by varying the level of immersion across three stages (high, medium, low) while solving various tasks with the Apple Vision Pro. The results show that selective attention improves significantly with increasing immersion levels. In contrast, social presence decreases with increasing immersion. In general, participants performed better in the individual task than in the collaborative task. However, the degree of immersion did not influence the collaborative performance. In addition, we could not determine any adverse effects on depth perception or user experience after use. The present study provides initial contributions to the future of spatial computing in professional settings and highlights the importance of balancing immersion and social interaction in a world where digital and physical spaces seamlessly coexist.

Abstract:
When walking, our head does not travel on a straight path but oscillates in a swaying pattern. This pattern has been implemented in Virtual Reality (VR) as “viewpoint oscillations” - which can be defined as periodic changes in position and/or orientation of the point of view to enhance walking simulations and make them feel closer to real walking. Viewpoint oscillations are especially beneficial when users cannot physically walk because of limitations of space or hardware, disability, or to avoid fatigue. In this paper, we provide new experimental work on the effects of viewpoint oscillations on walking sensation, as well as cybersickness and virtual embodiment, since such results are scarce in immersive VR, especially when using an avatar in first-person view. To do so, we also propose a technical improvement of viewpoint oscillations in embodied VR. Our technique makes use of an HMD-embedded gaze tracker to artificially add rotations that stabilize the target of the gaze in the users' field of view. We conducted a user study on 24 participants, which showed that our implementation of viewpoint oscillations successfully increased walking sensation and did not impact cybersickness or agency, compared to a linear motion without oscillations. In addition, a novel positive effect of stabilized viewpoint oscillations was found on virtual body ownership. As such, this study demonstrates the feasibility and viability of implementing gaze tracking-based stabilization with standard commercial HMDs, and, taken together, our results promote the use of viewpoint oscillations during walking simulations in embodied VR with an HMD.

Abstract:
Virtual reality (VR) has become a popular tool for studying navigation, providing the experimental control of a laboratory setting but also the potential for immersive and natural experiences that resemble the real world. For VR to be an effective tool to study navigation and be used for training or rehabilitation, it is important to establish whether performance is similar across virtual and real environments. Much of the existing navigation research has focused on young adult performance either in a virtual or a real environment, resulting in an open question regarding the validity of VR for studying age-related effects on spatial navigation. In this paper, young (18–30 years old) and older adults (60 years and older) performed the same navigation task in similar real and virtual environments. They completed a homing task, requiring walking along two legs of a triangle and returning to a home location, under three sensory conditions: visual cues (environmental landmarks present), body-based self-motion cues, and the combination of both cues. Our findings reveal that homing performance in VR demonstrates the same age-related differences as those observed in the real-world task. That said, within-age group differences arise when comparing cue use across environment types. In particular, young adults are less accurate and more variable with self-motion cues than visual cues in VR, while older adults show similar deficits with both cues. However, when both age groups can access multiple sensory cues, navigation performance does not differ between environment types. These results demonstrate that VR effectively captures age-related differences, with navigation performance most closely resembling performance in the real world when navigators can rely on an array of sensory information. Such findings have implications for future research on the aging population, highlighting that VR can be a valuable tool, particularly when multisensory cues are available.

Abstract:
Touch interaction is one of the fundamental interaction paradigms in XR, as users have become very familiar with touch interactions on physical touchscreens. However, users typically need to perform extensive arm movements for engaging with XR user interfaces much larger than mobile device touchscreens. We propose the SummonBrush technique to facilitate easy access to hidden windows while interacting with large XR user interfaces, requiring minimal arm movements. The SummonBrush technique adds a virtual brush to the index fingertip of a user's hand. Upon making contact with a virtual user interface, the brush bends and diverges and ink starts to diffuse in it. The more the brush bends and diverges, the more the ink diffuses. The user can summon hidden windows or background applications in situ, which is achieved by firstly pressing the brush against the user interface to make ink fully fill the brush and then perform swipe gestures. Also, the user can press the brush against the thumbtails of background applications in situ to quickly cycle them through. Ecological studies showed that SummonBrush significantly reduced the arm movement time by 39% and 34% in summoning hidden windows and activating/closing background applications, respectively, leading to a significant decrease in reported physical demand.

Abstract:
In virtual reality (VR) education, particularly in creative fields like film production, the role of different virtual environments in shaping learning outcomes remains underexplored. This study investigates how three distinct environments-baseline, a dynamic beach setting, and a familiar office space-affect students' ability to learn film lighting techniques and whether team-based learning offers advantages over individual learning. We conducted a 3×2 factorial experiment with 36 participants to examine the effects of these environments on learning performance. Our results show for individual learners, the dynamic and potentially distracting beach environment increased frustration and effort but also heightened their sense of engagement and perceived performance. In contrast, team-based learning in familiar environments like the office significantly reduced frustration and fostered collaboration, leading to improved performance. Interestingly, team-based learning excelled in the baseline environment, whereas individual learners performed better in more challenging settings like the beach. These findings provide practical insights into optimizing virtual environments to enhance both individual and collaborative learning in VR education.

Abstract:
In the early stages of learning a technical skill, trainees require guidance from a mentor through augmented feedback to develop higher expertise. However, the impact of such feedback and the different modalities used to communicate it remain underexplored in immersive virtual environments (IVE). This paper presents a study in which 27 participants were divided into three groups to learn a tool manipulation trajectory in an IVE. Two experimental groups received guidance from an expert using visual and/or haptic augmented feedback, while the control group received no feedback. The results indicate that both experimental groups showed significantly greater improvement in tool trajectory performance than the control group from pre- to post-test, with no significant differences between them. Analysis of their learning curves revealed similar performance improvements in tool trajectory across trials, outperforming the control group. Additionally, the visual-haptic feedback condition was linked to lower task load in three out of six dimensions of the NASA-TLX and a higher perceived interdependence with the expert's actions. These findings suggest that augmented feedback from an expert enhances the learning of tool manipulation skills. Although adding haptic feedback did not lead to better learning outcomes compared to visual feedback alone, it did enhance the overall user experience. These results offer valuable insights for designing IVEs that support mentor-trainee interactions through augmented feedback.

Abstract:
Studies have shown that prolonged Human-Computer Interaction (HCI) fatigue can increase the risk of mental illness and lead to a higher probability of errors and accidents during operations. Virtual Reality (VR) technology can simultaneously stimulate multiple senses such as visual, auditory, and tactile, providing an immersive experience that enhances cognition and understanding. Therefore, this study collects multimodal data to develop evaluation methods for HCI fatigue and further explores the fatigue-relieving effects of VR technology by comparing it with flat video. Using a modular design, electroencephalogram (EEG) and functional near-infrared spectroscopy (fNIRS) data in the resting, fatigue-induced, and recovery states, eye movement data in the resting and fatigue-induced states, as well as subjective scale results after each state were collected from the participants. Preprocessing and statistical analysis are performed through data flow architecture. After fatigue induction, it was found that the degree of activation of brain areas, especially the Theta band of prefrontal cortex, occurred significantly higher, the effective connectivity in the Alpha and Theta bands occurred significantly lower, the subjects' pupil diameters decreased, the blink frequency increased, and subjective questionnaire scores increased, which verified the validity of the multimodal data for assessing HCI fatigue. Analyzing fatigue relief through subgroups, it was found that when using the natural grassland scene with soothing music, both flat video and VR had the ability to alleviate fatigue, which was manifested as a significant decrease in the Alpha band in the LPFC brain area and a decrease in the questionnaire score. Moreover, during the recovery state, it was found that compared to the video group, the VR group had significantly higher activation in the Alpha and Theta bands of the prefrontal cortex, while the video group had significantly higher effective connectivity than the VR group in the Alpha band. This study delved deeply into the multidimensional characterization of fatigue and investigated new scenarios for the use of VR, which can help to promote the use of VR and can be migrated to scenarios that require fatigue management and productivity enhancement.

Affiliations: Network Control Laboratory at the Gordon Center for Medical Imaging, Harvard Medical School (HMS), the Precision Neuroscience and Neuromodulation Program (PNN), Massachusetts General Hospital (MGH), Boston, USA; Department of Information Engineering (DII), Università Politecnica delle Marche, Italy; Department of Political Sciences, Communication and International Relations, University of Macerata, Italy; Department of Civil Engineering Building and Architecture (DICEA), Università Politecnica delle Marche, Italy

Abstract:
The convergence of Artificial Intelligence (AI), Computer Vision (CV), Computer Graphics (CG), and Extended Reality (XR) is driving innovation in immersive environments. A key challenge in these environments is the creation of personalized 3D assets, traditionally achieved through manual modeling, a time-consuming process that often fails to meet individual user needs. More recently, Generative AI (GenAI) has emerged as a promising solution for automated, context-aware content generation. In this paper, we present MineVRA (Multimodal generative artificial iNtelligence for contExt-aware Virtual Reality Assets), a novel Human-In-The-Loop (HITL) XR framework that integrates GenAI to facilitate coherent and adaptive 3D content generation in immersive scenarios. To evaluate the effectiveness of this approach, we conducted a comparative user study analyzing the performance and user satisfaction of GenAI-generated 3D objects compared to those generated by Sketchfab in different immersive contexts. The results suggest that GenAI can significantly complement traditional 3D asset libraries, with valuable design implications for the development of human-centered XR environments.

Abstract:
To relieve perceived pain in patients undergoing colonoscopy, we developed an endoscopist-patient co-participative VR tool (EPC-VR) based on A Neurocognitive Model of Attention to Pain. It allows the patient to play a VR game actively and supports the endoscopist in triggering a distraction mechanism to divert the patient's attention away from the medical procedure. We performed a comparative clinical study with 40 patients. Patients' perception of pain and affective responses were evaluated, and the results support the effectiveness of EPC-VR: active VR playing with endoscopists' participation can help relieve the perceived pain and scare of patients undergoing colonoscopy. Finally, 87.5% of patients opt to use the VR application in the next colonoscopy.

Abstract:
Digital twin models are of high interest to Head and Neck Cancer (HNC) oncologists, who have to navigate a series of complex treatment decisions that weigh the efficacy of tumor control against toxicity and mortality risks. Evaluating individual risk profiles necessitates a deeper understanding of the interplay between different factors such as patient health, spatial tumor location and spread, and risk of subsequent toxicities that can not be adequately captured through simple heuristics. To support clinicians in better understanding tradeoffs when deciding on treatment courses, we developed DITTO, a digital-twin and visual computing system that allows clinicians to analyze detailed risk profiles for each patient, and decide on a treatment plan. DITTO relies on a sequential Deep Reinforcement Learning digital twin (DT) to deliver personalized risk of both long-term and short-term disease outcome and toxicity risk for HNC patients. Based on a participatory collaborative design alongside oncologists, we also implement several visual explainability methods to promote clinical trust and encourage healthy skepticism when using our system. We evaluate the efficacy of DITTO through quantitative evaluation of performance and case studies with qualitative feedback. Finally, we discuss design lessons for developing clinical visual XAI applications for clinical end users.

Abstract:
The concept of an intelligent augmented reality (AR) assistant has significant, wide-ranging applications, with potential uses in medicine, military, and mechanics domains. Such an assistant must be able to perceive the environment and actions, reason about the environment state in relation to a given task, and seamlessly interact with the task performer. These interactions typically involve an AR headset equipped with sensors which capture video, audio, and haptic feedback. Previous works have sought to facilitate the development of intelligent AR assistants by visualizing these sensor data streams in conjunction with the assistant's perception and reasoning model outputs. However, existing visual analytics systems do not focus on user modeling or include biometric data, and are only capable of visualizing a single task session for a single performer at a time. Moreover, they typically assume a task involves linear progression from one step to the next. We propose a visual analytics system that allows users to compare performance during multiple task sessions, focusing on non-linear tasks where different step sequences can lead to success. In particular, we design visualizations for understanding user behavior through functional near-infrared spectroscopy (fNIRS) data as a proxy for perception, attention, and memory as well as corresponding motion data (acceleration, angular velocity, and gaze). We distill these insights into embedding representations that allow users to easily select groups of sessions with similar behaviors. We provide two case studies that demonstrate how to use these visualizations to gain insights about task performance using data collected during helicopter copilot training tasks. Finally, we evaluate our approach through an in-depth examination of a think-aloud experiment with five domain experts.

Affiliations: Department of Computer Science and Information Engineering, National Chung Cheng University, Chiayi, Taiwan; Department of Information Management, National Taiwan University, Taipei, Taiwan; Graduate School of Interdisciplinary Information Studies, University of Tokyo, Tokyo, Japan; Department of Computer Science, University of Michigan, Ann Arbor, MI, USA; Department of Computer Science and Information Engineering, Department of Information Management, Graduate Institute of Networking and Multimedia, and Trans-disciplinary Bachelor Degree (TBD) Program, National Taiwan University, National Science and Technology Council, Taipei, Taiwan

Abstract:
Rekindling old social bonds with individuals who were once a part of our lives but have since faded away is crucial for our well-being. Such connections with dormant ties help us overcome loneliness and provide social support. Recently, virtual reality (VR) emerged as a promising tool for facilitating social interactions, such as online gatherings for formal or casual activities. VR can offer immersive and shared experiences, facilitating genuine connections between people. This provides a unique advantage over traditional computer-mediated communication methods. However, while prior research has explored how VR can aid in forming new social connections, its potential to reconnect dormant ties is largely unexplored. This paper aims to bridge this gap by examining how different features of VR, specifically avatar appearance and virtual environments, influence reactivations of dormant ties. We conducted an experiment involving 24 dyads to investigate the effect of different avatar-self similarities and virtual environments on the perceptions and interactions between dormant ties. Our findings indicate that avatars resembling oneself and dormant ties promote social closeness. Familiar virtual environments evoke shared memories, while unfamiliar ones stimulate more conversations. We discuss the impact of VR features on reconnecting dormant ties and provide implications for re-connecting relationships in VR.

Abstract:
The preservation of cultural heritage (CH) is a complex and promising field. Driven by technological advancements, digitization has emerged as a crucial approach for revitalizing tangible/intangible cultural heritage (TCH/ICH). However, current research and practice remain limited in their exploration of abstract forms of ICH, such as traditional philosophies and ideologies. In this study, utilizing Zen as a context, we designed an immersive mixed reality (MR) experience system, Flowing with Zen, based on formative study and cultural symbol analysis. The MR system integrates multi-modal interfaces, motion capture, environmental sensing, and generative computing, enabling users to engage with four scenarios through meditation, life appreciation, and experiential Zen practice, providing the embodied experience of Zen. Comparative user evaluation (N = 51) revealed that the MR system has significant advantages in eliciting engagement and interest from users, enhancing their aesthetic appreciation and cultural understanding, and increasing the accessibility of Zen. Our research proposes a novel approach and design inspiration for the digital inheritance of abstract ICH.

Abstract:
Neurodivergent individuals, particularly those with Autism and Attention Deficit Hyperactivity Disorder (ADHD), frequently experience anxiety, panic attacks, meltdowns, and emotional dysregulation due to societal pressures and inadequate accommodations. These challenges are especially pronounced for neurodivergent women and non-binary individuals navigating intersecting barriers of neurological differences and gender expectations. This research investigates virtual reality (VR) as a portable safe space for emotional regulation, addressing challenges of sensory overload and motion sickness while enhancing relaxation capabilities. Our mixed-methods approach included an online survey (N=223) and an ideation workshop (N=32), which provided key design elements for creating effective calming VR environments. Based on these findings, we developed and iteratively tested VR prototypes with neurodivergent women and non-binary participants (N=12), leading to a final version offering enhanced adaptability to individual sensory needs. This final prototype underwent a comprehensive evaluation with 25 neurodivergent participants to assess its effectiveness as a regulatory tool. This research contributes to the development of inclusive, adaptive VR environments that function as personalized “portable silent rooms” offering neurodivergent individuals on-demand access to sensory regulation regardless of physical location.

Abstract:
The development of radiance fields (RF), such as 3D Gaussian Splatting (3DGS) and Neural Radiance Fields (NeRF), has revolutionized interactive photorealistic view synthesis and presents enormous opportunities for XR research and applications. However, despite the exponential growth of RF research, RF-related contributions to the XR community remain sparse. To better understand this research gap, we performed a systematic survey of current RF literature to analyze (i) how RF is envisioned for XR applications, (ii) how they have already been implemented, and (iii) the remaining research gaps. We collected 365 RF contributions related to XR from computer vision, computer graphics, robotics, multimedia, human-computer interaction, and XR communities, seeking to answer the above research questions. Among the 365 papers, we performed an analysis of 66 papers that already addressed a detailed aspect of RF research for XR. With this survey, we extended and positioned XR-specific RF research topics in the broader RF research field and provide a helpful resource for the XR community to navigate within the rapid development of RF research.

Abstract:
Promoting public health is challenging owing to its abstract nature, and individuals may be apprehensive about confronting it. Recently, there has been an increasing interest in using the metaverse and gamification as novel educational techniques to improve learning experiences related to the immune system. Thus, we present MetaRoundWorm, an immersive virtual reality (VR) escape room game designed to enhance the understanding of parasitic infections and host immune responses through interactive, gamified learning. The application simulates the lifecycle of Ascaris lumbricoides and corresponding immunological mechanisms across anatomically accurate environments within the human body. Integrating serious game mechanics with embodied learning principles, MetaRoundWorm offers players a task-driven experience combining exploration, puzzle-solving, and immune system simulation. To evaluate the educational efficacy and user engagement, we conducted a controlled study comparing MetaRoundWorm against a traditional approach, i.e., interactive slides. Results indicate that MetaRoundWorm significantly improves immediate learning outcomes, cognitive engagement, and emotional experience, while maintaining knowledge retention over time. Our findings suggest that immersive VR gamification holds promise as an effective pedagogical tool for communicating complex biomedical concepts and advancing digital health education.

Abstract:
Handheld haptic devices often face challenges in delivering stiffness feedback with both high force output and good backdrivability, especially under practical constraints on power consumption, size, and weight. These difficulties stem from the inherent performance limitations of conventional actuation mechanisms. To address this issue, we propose a lightweight, low-power handheld device that provides wide-range stiffness feedback through a novel dual actuation design composed of two key components. A programmable spring (PS), implemented via an adjustable lever arm, enables tunable physical stiffness. Two electrostatic clutches (ECs) are integrated to compensate for the inherent limitations of PS-based interactions in stiffness display range, rendered object size, and free motion capability. The feedback force arises passively from the reaction of the PS and ECs to user input, effectively lowering both power consumption and actuator torque demands. A fully integrated prototype was developed, incorporating wireless communication, control, and power modules. The results of the evaluation experiments and user studies demonstrate that the device effectively renders stiffness across the full range, from free motion to full rigidity, and delivers more realistic elastic feedback compared to conventional electric motor-based systems.

Abstract:
Collaborative work in social virtual reality often requires an interplay of loosely coupled collaboration from different virtual locations and tightly coupled face-to-face collaboration. Without appropriate system mediation, however, transitioning between these phases requires high navigation and coordination efforts. In this paper, we present an interaction system that allows collaborators in virtual reality to seamlessly switch between different collaboration models known from related work. To this end, we present collaborators with functionalities that let them work on individual sub-tasks in different virtual locations, consult each other using asymmetric interaction patterns while keeping their current location, and temporarily or permanently join each other for face-to-face interaction. We evaluated our methods in a user study with 32 participants working in teams of two. Our quantitative results indicate that delegating the target selection process for a long-distance teleport significantly improves placement accuracy and decreases task load within the team. Our qualitative user feedback shows that our system can be applied to support flexible collaboration. In addition, the proposed interaction sequence received positive evaluations from teams with varying VR experiences.

Abstract:
As Differential Privacy (DP) transitions from theory to practice, visualization has surfaced as a catalyst in promoting acceptance and usage. Despite the potential of visualization tools to support differential privacy implementation, their development is limited by a lack of understanding of the overall deployment process, practitioner challenges, and the role of visual tools in real-world deployments. To narrow this gap, we interviewed 18 professionals from various backgrounds who regularly engage with differential privacy in their work. Our objectives were to understand the differential privacy implementation process and associated challenges; explore the actors (individuals involved in differential privacy implementation), how they use or struggle to use visualization; and identify the benefits and challenges of using visualization in the implementation process. Our results delineate the differential privacy implementation process into five distinct stages and highlight the main actors alongside the diverse visualization applications and shortcomings. We find that visualizations can be used to build foundational differential privacy knowledge, describe implementation parameters, and evaluate private outputs. However, the visualization strategies described often fail to address the diverse technical backgrounds and varied privacy and accuracy concerns of users, hindering effective communication between the different actors involved in the implementation process. From our findings, we propose three research directions: visualizations for setting and evaluating noise addition, evaluation of uncertainty visualization related to trust in differential privacy, and research focused on pedagogical visualizations for complex data science topics.

Abstract:
This study is the first to explore the interplay between haptic interaction and avatar representation in Shared Virtual Environments (SVEs). Specifically, how these factors shape users’ sense of social presence during dyadic collaborations, while assessing potential effects on task performance. In a series of experiments, participants performed the collaborative task with haptic interaction under four avatar representation conditions: avatars of both participant and partner were displayed, only the participant’s avatar was displayed, only the partner’s avatar was displayed, and no avatars were displayed. The study finds that avatar representation, especially of the partner, significantly enhances the perception of social presence, which haptic interaction alone does not fully achieve. However, neither the presence nor the type of avatar representation impacts the task performance or participants’ force effort of the task, suggesting that haptic interaction provides sufficient interaction cues for the execution of the task. These results underscore the significance of integrating both visual and haptic modalities to optimize remote collaboration experiences in virtual environments, ensuring effective communication and a strong sense of social presence.

Abstract:
The burgeoning online video game industry has sparked intense competition among providers to both expand their user base and retain existing players, particularly within social interaction genres. To anticipate player churn, there is an increasing reliance on machine learning (ML) models that focus on social interaction dynamics. However, the prevalent opacity of most ML algorithms poses a significant hurdle to their acceptance among domain experts, who often view them as “opaque models”. Despite the availability of eXplainable Artificial Intelligence (XAI) techniques capable of elucidating model decisions, their adoption in the gaming industry remains limited. This is primarily because non-technical domain experts, such as product managers and game designers, encounter substantial challenges in deciphering the “explicit” and “implicit” features embedded within computational models. This study proposes a reliable, interpretable, and actionable solution for predicting player churn by restructuring model inputs into explicit and implicit features. It explores how establishing a connection between explicit and implicit features can assist experts in understanding the underlying implicit features. Moreover, it emphasizes the necessity for XAI techniques that not only offer implementable interventions but also pinpoint the most crucial features for those interventions. Two case studies, including expert feedback and a within-subject user study, demonstrate the efficacy of our approach.

Abstract:
Future augmented reality (AR) glasses empower users to view personal applications and services anytime and anywhere without being restricted by physical locations and the availability of physical screens. In typical everyday activities, people move around to carry out different tasks and need a variety of information on the go. Existing interfaces in AR do not support these use cases well, especially when the number of applications increases. We explore the usability of three world-referenced approaches that move AR applications with users as they transition among different locations, featuring different levels of AR app availability: (1) always using a menu to manually open an app when needed; (2) automatically suggesting a relevant subset of all apps; and (3) carrying all apps with the users to the new location. Through a controlled study and a relatively more ecologically-valid study in AR, we reached better understandings on the performance trade-offs and observed the impact of various everyday contextual factors on these interfaces in more realistic AR settings. Our results shed light on how to better support the mobile information needs of users in everyday life in future AR interfaces.

Abstract:
Augmented Reality (AR) is a growing technology that enables interaction with both virtual and real objects. However, in order to support the future development of efficient and usable AR interactions, there is still a lack of systematic knowledge establishing basic interaction performance across different conditions. Therefore, in this paper, we report a user study measuring the impact of objects' physicalization (object's set composed of (i) virtual, (ii) real, or (iii) a composite mix of real and virtual objects) and hand appearance (hand's appearance displayed as (i) the real hand, (ii) an avatar, or (iii) dynamically adapting to the surrounding objects' physicalization) on the speed performance of a pick-and-place task. Overall, our results reveal that objects' physicalization plays a significant role in interaction performance, with the more real objects in a set the better the performance. Moreover, our results also suggest that pick-and-place interaction performances are mostly unaffected by the hand appearance. Interestingly, we also observed that interactions with real objects were less efficient as the object condition required the user to alternate between interactions with virtual and real objects (object condition (iii)), which provides novel insights into an important - mostly AR-specific - factor to consider for designing future AR interactions. Taken together, our results provide a rich characterization of different factors influencing different phases of a pick-and-place interaction, which could be employed to improve the design of future AR applications.

Abstract:
While 3D user interfaces are often designed with efficiency and accuracy in mind, artistic performances have their own very specific constraints and criteria for a successful interaction in mixed or virtual reality, which have yet to be fully understood. In this paper, we study the design of 3D interactions for a Spatial Augmented-Reality display in the context of a cyber-opera. We perform an analysis of design decisions taken during the multiple residencies and lab sessions, and we conduct a reflexive thematic analysis of interviews with the director and actors, which highlight how they integrated the story, the actors' performance, the audience experience and the technology. We identify four main criteria for designing 3D interactions in performing arts, namely their efficiency, expressiveness, contextualisation and legibility, and we derive guidelines for future research and creation.

Abstract:
Working with scatterplots is a classic everyday task for data analysts, which gets increasingly complex the more plots are required to form an understanding of the underlying data. To help analysts retrieve relevant plots more quickly when they are needed, immersive virtual environments (iVEs) provide them with the option to freely arrange scatterplots in the 3D space around them. In this paper, we investigate the impact of different virtual environments on the users' ability to quickly find and retrieve individual scatterplots from a larger collection. We tested three different scenarios, all having in common that users were able to position the plots freely in space according to their own needs, but each providing them with varying numbers of landmarks serving as visual cues: an Empty scene as a baseline condition, a single landmark condition with one prominent visual cue being a Desk, and a multiple landmarks condition being a virtual Office. Results from a between-subject investigation with 45 participants indicate that the time and effort users invest in arranging their plots within an iVE had a greater impact on memory performance than the design of the iVE itself. We report on the individual arrangement strategies that participants used to solve the task effectively and underline the importance of an active arrangement phase for supporting the spatial memorization of scatterplots in iVEs.

Abstract:
Virtual reality training simulations to acquire surgical skills are important for increasing patient safety and save valuable resources, e.g., cadavers, supervision and operating room time. However, as surgery is a craft, simulators must not only provide a high degree of visual realism, but especially a realistic haptic behavior. While such simulators exist for surgeries like laparoscopy or arthroscopy, other surgical fields, especially where large forces need to be exerted, like total hip arthroplasty (THA; implantation of a hip joint protheses), lack realistic VR training simulations. In this paper we present for the first time a novel VR training simulation for the five steps of THA (from femur head resection to stem implantation) with realis-tic haptic feedback. To achieve this, a novel haptic hammering device, an upgraded version of the Virtuose 6D haptic device from Haption, novel algorithms for collision detection, haptic rendering, and material removal are introduced. In a study with 17 surgeons of diverse experience levels, we confirmed the realism, usefulness and usability of our novel methods.

Abstract:
State-of-the-art Virtual Reality (VR) Head-Mounted Displays (HMDs) enable users to interact with virtual objects using their hands via built-in camera systems. However, the accuracy of the hand movement detection algorithm is often affected by limitations in both camera hardware and software, including image processing & machine learning algorithms used for hand skeleton detection. In this work, we investigated a visual feedback mechanism to create an early warning system that detects hand skeleton recognition failures in VR HMDs and warns users in advance. We conducted two user studies to evaluate the system's effectiveness. The first study involved a cup stacking task, where participants stacked virtual cups. In the second study, participants performed a ball sorting task, picking and placing colored balls into corresponding baskets. During both of the studies, we monitored the built-in hand tracking confidence of the VR HMD system and provided visual feedback to the user to warn them when the tracking confidence is ‘low’. The results showed that warning users before the hand tracking algorithm fails improved the system's usability while reducing frustration. The impact of our results extends beyond VR HMDs, any system that uses hand tracking, such as robotics, can benefit from this approach.

Abstract:
Despite the recent surge of research efforts to make data visualizations accessible to people who are blind or have low vision (BLV), how to support BLV people's data analysis remains an important and challenging question. As refreshable tactile displays (RTDs) become cheaper and conversational agents continue to improve, their combination provides a promising approach to support BLV people's interactive data exploration and analysis. To understand how BLV people would use and react to a system combining an RTD with a conversational agent, we conducted a Wizard-of-Oz study with 11 BLV participants, where they interacted with line charts, bar charts, and isarithmic maps. Our analysis of participants' interactions led to the identification of nine distinct patterns. We also learned that the choice of modalities depended on the type of task and prior experience with tactile graphics, and that participants strongly preferred the combination of RTD and speech to a single modality. In addition, participants with more tactile experience described how tactile images facilitated a deeper engagement with the data and supported independent interpretation. Our findings will inform the design of interfaces for such interactive mixed-modality systems.

Abstract:
User experience in data visualization is typically assessed through post-viewing self-reports, but these overlook the dynamic cognitive processes during interaction. This study explores the use of mind wandering- a phenomenon where attention spontaneously shifts from a primary task to internal, task-related thoughts or unrelated distractions- as a dynamic measure during visualization exploration. Participants reported mind wandering while viewing visualizations from a pre-labeled visualization database and then provided quantitative ratings of trust, engagement, and design quality, along with qualitative descriptions and short-term/long-term recall assessments. Results show that mind wandering negatively affects short-term visualization recall and various post-viewing measures, particularly for visualizations with little text annotation. Further, the type of mind wandering impacts engagement and emotional response. Mind wandering also functions as an intermediate process linking visualization design elements to post-viewing measures, influencing how viewers engage with and interpret visual information over time. Overall, this research underscores the importance of incorporating mind wandering as a dynamic measure in visualization design and evaluation, offering novel avenues for enhancing user engagement and comprehension.

Abstract:
The literature has demonstrated the advantages of Mixed Reality (MR) for product configuration by providing a more engaging and effective end-user experience. While collaborative and remote design tools in MR have been widely explored in previous studies, a noticeable gap remains in the exploration of co-located product configuration for couples. This gap is noteworthy since in many industries, couples (e.g., friends, partners) often make purchasing decisions together in physical retail environments. In this paper, we introduce MR-CoCo, an open MR testbed designed to explore collaborative configurations by co-located couples, both in the role of customers. The testbed is developed in Unity and features: (i) a shared MR space with virtual product 3D model anchoring, (ii) shared visualization of the current configuration, (iii) a versatile UI for selecting configuration areas, (iv) hand gestures for 3D drag and drop of colors and materials from 3D catalog to the product. A case study of the personalization of a sailboat is provided as proof of concept. The user study involved 24 couples (48 participants in total), simulating a purchasing experience and the related configuration using MR-CoCo. We assessed usability through post-experience evaluations, with the System Usability Scale (SUS) and the Co-Presence Configuration Questionnaire (CCQ) to measure collaboration and decision-making. The results demonstrated a high level of usability and perceived quality of collaboration. We also explore guidelines that can be used for remote collaboration applications, enabling configuration across a wide range of industries (e.g., automotive and clothing).

Abstract:
Sensorimotor contingencies (SC) refer to the rules by which we use our body to perceive. It has been argued that to the extent that a virtual reality (VR) application affords natural SC so the greater likelihood that participants will experience Place Illusion (PI), the illusion of ‘being there’ (a component of presence) in the virtual environment. However, notwithstanding numerous studies this only has anecdotal support. Here we used a reinforcement learning (RL) paradigm where 26 participants experienced a VR scenario where the RL agent could sequentially propose changes to 5 binary factors: mono or stereo vision, 3 or 6 degrees of freedom head tracking, mono or spatialised sound, low or high display resolution, or one of two color schemes. The first 4 are SC, whereas the last is not. Participants could reject or accept each change proposed by the RL, until convergence. Participants were more likely to accept changes from low to high SC than changes to the color. Additionally, theory suggests that increased PI should be associated with lower eye scanpath entropy. Our results show that mean entropy did decrease over time and the final level of entropy was negatively correlated with a post exposure questionnaire-based assessment of PI.

Abstract:
This paper explores the potential for human-AI collaboration in the context of data storytelling for data workers. Data storytelling communicates insights and knowledge from data analysis. It plays a vital role in data workers’ daily jobs since it boosts team collaboration and public communication. However, to make an appealing data story, data workers need to spend tremendous effort on various tasks, including outlining and styling the story. Recently, a growing research trend has been exploring how to assist data storytelling with advanced artificial intelligence (AI). However, existing studies focus more on individual tasks in the workflow of data storytelling and do not reveal a complete picture of humans’ preference for collaborating with AI. To address this gap, we conducted an interview study with 18 data workers to explore their preferences for AI collaboration in the planning, implementation, and communication stages of their workflow. We propose a framework for expected AI collaborators’ roles, categorize people's expectations for the level of automation for different tasks, and delve into the reasons behind them. Our research provides insights and suggestions for the design of future AI-powered data storytelling tools.

Abstract:
Viewpoint oscillations are periodic changes in the position and/or orientation of the point of view in a virtual environment. They can be implemented in Virtual Reality (VR) walking simulations to make them feel closer to real walking. This is especially useful in simulations where users remain in place because of space or hardware constraints. As for today, it remains unclear what exact benefit they bring to user experience during walking simulations, and with what characteristics they should be implemented. To answer these questions, we conduct a systematic literature review focusing on five main dimensions of user experience (walking sensation, vection, cybersickness, presence and embodiment) and discuss 44 articles from the fields of VR, Vision, and Human-Computer Interaction. Overall, the literature suggests that viewpoint oscillations benefit vection, and with less evidence, walking sensation and presence. As for cybersickness, the literature contains contrasted results. Based on these results, we recommend using viewpoint oscillations in applications that require accurate distance or speed perception, or that aim to provide compelling walking simulations without a walking avatar, and a particular attention should be paid to cybersickness. Taken together, this work gives recommendations for enhancing walking simulations in VR, which may be applied to entertainment, virtual visits, and medical rehabilitation.

Abstract:
Visual Parameter Space Analysis (VPSA) enables domain scientists to explore input-output relationships of computational models. Existing VPSA applications often feature multi-view visualizations designed by visualization experts for a specific scenario, making it hard for domain scientists to adapt them to their problems without professional help. We present RSVP, the Rapid Suggestive Visualization Prototyping system encoding VPSA knowledge to enable domain scientists to prototype custom visualization dashboards tailored to their specific needs. The system implements a task-oriented, multi-view visualization recommendation strategy over a visualization design space optimized for VPSA to guide users in meeting their analytical demands. We derived the VPSA knowledge implemented in the system by conducting an extensive meta design study over the body of work on VPSA. We show how this process can be used to perform a data and task abstraction, extract a common visualization design space, and derive a task-oriented VisRec strategy. User studies indicate that the system is user-friendly and can uncover novel insights.

Abstract:
Augmented reality (AR) magic-lens (ML) displays, such as handheld devices, offer a convenient and accessible way to enrich our environment using virtual imagery. Several display technologies, including conventional monocular, less common stereoscopic, and varifocal displays, are currently being used. Vergence and accommodation effects on depth perception, as well as vergence–accommodation conflict, have been studied, where users interact only with the content on the display. However, little research exists on how vergence and accommodation influence user performance and cognitive-task load when users interact with the content on a display and its surroundings in a short timeframe. Examples of this are validating augmented instructions before making an incision and performing general hand-eye coordinated tasks such as grasping augmented objects. To improve interactions with future AR displays in such scenarios, we must improve our understanding of this influence. To this end, we conducted two fundamental visual-acuity user studies with 28 and 27 participants, while investigating eye vergence and accommodation distances on four ML displays. Our findings show that minimizing the accommodation difference between the display and its surroundings is crucial when the gaze between the display and its surroundings shifts rapidly. Minimizing the difference in vergence is more important when viewing the display and its surroundings as a single context without shifting the gaze. Interestingly, the vergence–accommodation conflict did not significantly affect the cognitive-task load nor play a pivotal role in the accuracy of interactions with AR ML content and its physical surroundings.

Abstract:
The interest and dissemination of Virtual Reality (VR) is still expanding across multiple domains. While VR has the capacity to revolutionize many different industries and fields, the recent Covid-19 pandemic has also increased awareness of hygiene and safety associated with VR usage. Despite the growing commercial availability of both VR headsets and preventive and disinfection solutions, confirmatory studies required to validate both the efficacy and safety of the different solutions are severely lacking. This paper presents the findings of a survey aimed at gathering information about current hygiene practices in various domains, along with the perception of research availability. Cleaning methods varied among respondents (n=42), but most popular methods consisted of several consecutive solutions. Respondents primarily used anti-bacterial or alcohol disinfection wipes (81%), permanent face covers (leather/silicone) (43%), disposable cover/mask (26%), and UVC light disinfection (26%). 65% of the respondents stated that the Covid-19 pandemic made them change their practices. A majority of respondents remarked that there was a scarcity of research, yet, most respondents were fairly or completely confident that their cleaning protocols were sufficient, despite remarking that it was sometimes not adhered to. The efficacy of VR hygiene solutions and practices remains largely understudied despite the urgent need to establish validated and efficacious cleaning protocols and practices. Current solutions and practices primarily focuses on the inside of the headset, although the outside of the headset may be far more exposed to contaminants through e.g. hand-contact. Further research is needed to define and evaluate context-dependent risk-assessments as well as suitable cleaning protocols for VR-headsets.

Abstract:
This research investigated how the similarity of the rendering parameters of background and foreground objects affected egocentric depth perception in indoor virtual and augmented environments. We refer to the similarity of the rendering parameters as visual ‘congruence’. Study participants manipulated the depth of a sphere to match the depth of a designated target peg. In the first experiment, the sphere and peg were both virtual, while in the second experiment, the sphere is virtual and the peg is real. In both experiments, depth perception accuracy was found to depend on the levels of realism and congruence between the sphere, pegs, and background. In Experiment 1, realistic backgrounds lead to overestimation of depth, but resulted in underestimation when the background was virtual, and when depth cues were applied to the sphere and target peg. In Experiment 2, background and target pegs were real but matched with the virtual sphere; in comparison to Experiment 1, realistically rendered targets prompted an underestimation and more accuracy with the manipulated object. These findings suggest that congruence can affect distance estimation and the underestimation effect in the AR environment resulted from increased graphical fidelity of the foreground target and background.

Abstract:
Large Language Models (LLMs) have been adopted for a variety of visualizations tasks, but how far are we from perceptually aware LLMs that can predict human takeaways? Graphical perception literature has shown that human chart takeaways are sensitive to visualization design choices, such as spatial layouts. In this work, we examine the extent to which LLMs exhibit such sensitivity when generating takeaways, using bar charts with varying spatial layouts as a case study. We conducted three experiments and tested four common bar chart layouts: vertically juxtaposed, horizontally juxtaposed, overlaid, and stacked. In Experiment 1, we identified the optimal configurations to generate meaningful chart takeaways by testing four LLMs, two temperature settings, nine chart specifications, and two prompting strategies. We found that even state-of-the-art LLMs struggled to generate semantically diverse and factually accurate takeaways. In Experiment 2, we used the optimal configurations to generate 30 chart takeaways each for eight visualizations across four layouts and two datasets in both zero-shot and one-shot settings. Compared to human takeaways, we found that the takeaways LLMs generated often did not match the types of comparisons made by humans. In Experiment 3, we examined the effect of chart context and data on LLM takeaways. We found that LLMs, unlike humans, exhibited variation in takeaway comparison types for different bar charts using the same bar layout. Overall, our case study evaluates the ability of LLMs to emulate human interpretations of data and points to challenges and opportunities in using LLMs to predict human chart takeaways.

Abstract:
Written language is a useful tool for non-visual creative activities like composing essays and planning searches. This paper investigates the integration of written language into the visualization design process. We create the idea of a 'writing rudder,’ which acts as a guiding force or strategy for the designer. Via an interview study of 24 working visualization designers, we first established that only a minority of participants systematically use writing to aid in design. A second study with 15 visualization designers examined four different variants of written rudders: asking questions, stating conclusions, composing a narrative, and writing titles. Overall, participants had a positive reaction; designers recognized the benefits of explicitly writing down components of the design and indicated that they would use this approach in future design work. More specifically, two approaches - writing questions and writing conclusions/takeaways - were seen as beneficial across the design process, while writing narratives showed promise mainly for the creation stage. Although concerns around potential bias during data exploration were raised, participants also discussed strategies to mitigate such concerns. This paper contributes to a deeper understanding of the interplay between language and visualization, and proposes a straightforward, lightweight addition to the visualization design process.

Abstract:
Healthcare artificial intelligence (HAI) robots could alleviate the current healthcare provision crisis. However, their successful deployment rests on achieving and responsibly calibrating appropriate patient trust in them. Over two user studies, we systematically investigate trust formation in and usage intentions for HAI robots. We first investigate manipulations of visual appearance in a passive observation study without healthcare-specific situational context (n=87) and then in a realistic VR scenario depicting a medical consultation with a HAI robot (n=177) that introduces such situational context (diagnosis and treatment severity). Results show that within a healthcare context, a unique combination of (i) situational (diagnosis and treatment severity), (ii) robot (perceived competence) and (iii) user characteristics (personality and attitudes) determine trust and ultimately influence usage intentions for HAI robots. Furthermore, our results emphasize the absence of HAI visual appearance effects on trust. Our findings support and inform a human centred design approach of HAI robots.

Abstract:
Through a human-subject experiment, we investigated the effects of focal distance on depth perception and accommodative response in an optical see-through augmented reality (AR) display. The display was able to provide focus cues and was rigorously calibrated. The near-field distances ranging between 3 diopters and 1 diopter were considered as target distance. In the experiment, it was found that the perceived depth of a virtual object was significantly biased along with the focal distance of virtual image plane of the display. In addition, the experimental results implied that the perceived depth of a virtual object would be potentially more accurate in the condition where the focal distance of virtual image plane was consistent with the target distance than in the conditions where it could deviate from the target distance. Regarding accommodative response, it was found that the response to a virtual object changed along with the focal distance of virtual image plane as well as the target distance. However, the changing rate depending on target distance was less steep in the conditions where the focal distance could be mismatched with the target distance than in the condition where it was consistent with the target distance. In the consistent condition, the changing rate of accommodative responses to virtual objects were similar to that for their physical counterparts.

Abstract:
Although access to sufficient screen space is crucial to knowledge work, workers often find themselves with limited access to display infrastructure in remote or public settings. While virtual displays can be used to extend the available screen space through extended reality (XR) head-worn displays (HWD), we must better understand the implications of working with them in public settings from both users’ and bystanders’ viewpoints. To this end, we conducted two user studies. We first explored the usage of a hybrid AR display across real-world settings and tasks. We focused on how users take advantage of virtual displays and what social and environmental factors impact their usage of the system. A second study investigated the differences between working with a laptop, an AR system, or a VR system in public. We focused on a single location and participants performed a predefined task to enable direct comparisons between the conditions while also gathering data from bystanders. The combined results suggest a positive acceptance of XR technology in public settings and show that virtual displays can be used to accompany existing devices. We highlighted some environmental and social factors. We saw that previous XR experience and personality can influence how people perceive the use of XR in public. In addition, we confirmed that using XR in public still makes users stand out and that bystanders are curious about the devices, yet have no clear understanding of how they can be used.

Abstract:
User-Avatar interaction within augmented reality applications is rapidly increasing in frequency. Applications routinely place users in rooms with other, remote users embodied by photorealistic avatars, or require users to work with an avatar of a remote user to complete a task. During these types of interactions, it is often required to modify or redirect the posture of an avatar to achieve goals such as contact with or pointing at an object or maintaining eye gaze with the local user. A key limitation of modern redirection techniques is successfully preserving body posture, a critical component of nonverbal communication. This article presents a new pose-preserving objective function to be used in the multi-objective optimization of an avatar's kinematic configuration. This objective function not only mimics the correct placement of body joints, but also preserves their orientation in space. We have tested this approach against several commonly used and current state-of-the-art redirection techniques and have found that our new approach achieves a significant reduction in targeted redirection error while simultaneously reducing body posture error. Additionally, human subject testing has shown that our new technique provides both a significantly more natural looking redirection and a significantly more realistic and believable overall body posture.

Abstract:
This paper explores the impact of an augmented reality (AR) virtual dog, designed to mimic user behavior, on the exercise experience in both solo and group settings. Focusing on the virtual pet's role as a companion during physical activity, we conducted a human-subject experiment comparing three conditions: a mimicking virtual dog, a randomly behaving virtual dog, and no virtual dog. Participants exercised either solo or in groups, specifically in pairs, allowing for a detailed analysis of how the behavior and physical presence of the virtual dog influenced users' exercise experience and social connections. The findings demonstrate that the mimicking virtual dog significantly enhanced the exercise experience, especially in solo settings, by fostering a stronger sense of companionship. In group exercises, the virtual dog acted as a social facilitator, improving group cohesion and interaction. This research highlights the potential of behavior-mimicking virtual pets to enhance both individual and group exercise experiences and offers valuable insights for developing AR-based fitness applications.

Abstract:
Harassment in Social Virtual Reality (SVR) is a growing concern. The current SVR landscape features inconsistent access to non-standardised safety features, with minimal empirical evidence on their real-world effectiveness, usage and impact. We examine the use and effectiveness of safety tools across 12 popular SVR platforms by surveying 100 users about their experiences of different types of harassment and their use of features like muting, blocking, personal spaces and safety gestures. While harassment remained common-including hate speech, virtual stalking, and physical harassment-many find safety features insufficient or inconsistently applied. Reactive tools like muting and blocking are widely used, largely driven by users' familiarity from other platforms. Safety tools are also used to proactively curate individual virtual experiences, protecting users from harassment, but inadvertently leading to fragmented social spaces. We advocate for standardising proactive, rather than reactive, anti-harassment tools across platforms, and present insights into future safety feature development.

Abstract:
Virtual Reality (VR) can support effective and scalable training for procedures presented as step-by-step processes in machinery use, such as in an engineering room. Currently, many procedures are trained in closed experiences that do not allow for human errors or interaction with real subjects or machinery. Our goal is to address this gap by using VR training tools for guided procedural learning. To achieve this, we applied an existing methodology for designing such systems, called ProtoColVR, to develop a VR training simulation prototype for emergency procedures within an engineering room on a ship. The simulator, called OPVSim, employs end-to-end instruction, including a VR controls tutorial, step-by-step guided training based on graphs, offering different paths to the same goal, with feedback on correct and incorrect actions. We conducted a study with two groups of participants-cadets in training and experienced officers-to assess the effectiveness of the system developed using the described methodology. The study results reveal positive trends, including increased knowledge scores after using the simulator, and favorable outcomes for the prototype in terms of usability, presence, and workload. We discuss our findings, limitations, and the implications for designing VR training systems for guided procedural learning.

Abstract:
In immersive virtual environments (IVEs), accurate size perception is critical, especially in training simulations designed to mimic real-world tasks, such as, nuclear power plant control room or medical procedures. These simulations have dials or instruments of varying sizes. Visual information of the objects alone, often fails to capture subtle size differences in virtual reality (VR). However, integrating haptic and hand-avatars may potentially improve accuracy and performance. This improvement could be especially beneficial for real-world scenarios where hand(s) are intermittently visible or obscured. To investigate how this intermittent presence or absence of body-scaled hand-avatars affects size perception when integrated with haptic information, we conducted 2x2 mixed-factorial experiment design using a near-field, size-estimation task in VR. The experiment conditions compared size estimations with or without virtual hand visibility in the perception and reporting phases. The task involved 16 graspable objects of varying sizes and randomly repeated 3 times across 48 trials per participant (total 80 participants). We employed Linear Mixed Models (LMMs) analysis to objective measures: perceived size, residual error and proportional errors. Results revealed that as the tangible-graspable size increases, overestimation reduces if the hand-avatars are visible in the reporting phase. Also, overestimation reduces as the number of trials increases, if the hand-avatars are visible in the reporting phase. Thus, the presence of hand-avatars facilitated perceptual calibration. This novel study, with different combinations of hand-avatar visibility, taking perception and reporting of size as two separate phases, could open future research directions in more complex scenarios for refined integration of sensory modalities and consequently enhance real-world application performance.

Abstract:
This study explores how people interact in dyadic social eXtended Reality (XR), focusing on two main factors: the animation type of a conversation partner's avatar and how immersed the user feels in the virtual environment. Specifically, we investigate how 1) idle behavior, 2) AI-generated gestures, and 3) motion-captured movements from a confederate (a controlled partner in the study) influence the quality of conversation and how that partner is perceived. We examined these effects in both symmetric interactions (where both participants use VR headsets and controllers) and asymmetric interactions (where one participant uses a desktop setup). We developed a social XR platform that supports asymmetric device configurations to provide varying levels of immersion. The platform also supports a modular avatar animation system providing idle behavior, real-time AI-generated co-speech gestures, and full-body motion capture. Using a 2×3 mixed design with 39 participants, we measured users' sense of spatial presence, their perception of the confederate, and the overall conversation quality. Our results show that users who were more immersed felt a stronger sense of presence and viewed their partner as more human-like and believable. Surprisingly, however, the type of avatar animation did not significantly affect conversation quality or how the partner was perceived. Participants often reported focusing more on what was said rather than how the avatar moved.

Abstract:
Augmented Reality (AR) is increasingly positioned as a tool for knowledge work, providing beneficial affordances such as a virtually limitless display space that integrates digital information with the user’s physical surroundings. However, for AR to supplant traditional screen-based devices in knowledge work, it must support prolonged usage across diverse contexts. Until now, few studies have explored the effects, opportunities, and challenges of working in AR outside a controlled laboratory setting and for an extended duration. This gap in research limits our understanding of how users may adapt its affordances to their daily workflows and what barriers hinder its adoption. In this article, we present findings from a longitudinal diary study examining how participants incorporated an AR laptop — Sightful’s Spacetop EA — into their daily work routines. 14 participants used the device for 40-minute daily sessions over two weeks, collectively completing 103 hours of AR-based work. Through survey responses, workspace photographs, and post-study interviews, we analyzed usage patterns, workspace configurations, and evolving user perceptions. Our findings reveal key factors influencing participants’ usage of AR, including task demands, environmental constraints, social dynamics, and ergonomic considerations. We highlight how participants leveraged and configured AR’s virtual display space, along with emergent hybrid workflows that involved physical screens and tasks. Based on our results, we discuss both overlaps with current literature and new considerations and challenges for the future design of AR systems for pervasive and productive use.

Abstract:
Recent studies utilized virtual reality (VR) as an “empathy machine” to mitigate bias towards various social groups. However, studies addressing bias against physical disabilities remain scarce, with inconsistent results based on VR experience design. Moreover, most studies assumed the universal effects of VR simulation on bias reduction, ignoring the potential moderating effects of individual characteristics. This study investigated how experience design components and individual characteristics moderate VR simulation's effect on changes in bias towards physical disabilities. We designed a VR wheelchair experience, manipulating the situational context (negative, neutral) and whole-body avatar visualization (visible, invisible). Participants' implicit and explicit bias levels were assessed to examine the changes according to VR design components and individual characteristics (gender, preexisting bias level). Results indicated that following the VR intervention, implicit bias was reduced in the group with higher preexisting bias but rather increased in the group with lower preexisting bias. In addition, gender interacted with avatar visualization such that male participants' implicit bias was reduced with invisible avatars but increased with visible avatars. Explicit bias, in contrast, was reduced regardless of conditions, suggesting the potential response bias in self-report measures. These findings underscore the importance of considering the complex interplay between experience design and individual characteristics in understanding VR's efficacy as an empathy-inducing tool. This study provides insights and guidelines for developing more effective VR interventions to alleviate implicit bias towards physical disabilities.

Abstract:
This study aims to explore the use of Augmented Reality (AR) visual assets to convey procedural instructions, specifically for conveying information about component orientation. We focused on assembly scenarios where no affordance is provided for orientation while maintaining a consistently high affordance for how components are mounted. This information is recurrent in tasks where users are familiar with components that fit together without needing specialized tools but lack knowledge of the specific orientations required for the assembly. A typical example is placing rubber gaskets that fit smoothly into grooves but where no markings indicate the correct sealing side. We evaluated six different AR presentation modes for conveying component orientation: image, video, static side-by-side product model, animated side-by-side product model, static in-situ product model, and animated in-situ product model. The literature provides no clear agreement on which is the most effective. To fill this gap, we conducted a user study with 36 participants, measuring completion time, accuracy, and cognitive load across the six AR presentation modes. We also analyzed how users interacted with each of them and collected user subjective feedback. Our findings revealed that the animated side-by-side product model ensures better completion time, demanding less cognitive load and being favored by users.

Abstract:
Telerehabilitation is a cost-effective alternative to in-clinic rehabilitation. Although convenient, it lacks immersive and free-viewpoint patient visualization. Current research explores two solutions to this issue. Mesh-based methods use 3D models and motion capture for AR visualization. However, they are labor-intensive and less photorealistic than 2D images. Microsoft's Holoportation generates photorealistic 3D models with eight RGBD cameras in real time. However, it requires complex setups, high GPU power, and high-speed communication infrastructure, making deployment challenging. This article presents a Real-Time Free-Viewpoint Holographic Patient Rendering (RT-FVHP) system for telerehabilitation. Unlike traditional methods that require manually crafted assets such as 3D meshes, texture maps, and skeletal rigging, our data-driven approach eliminates the need for explicit asset definitions. Inspired by the HumanNeRF framework, we retarget dynamic human poses to a canonical pose and leverage 3D Gaussian Splatting to train a neural network in canonical space for patient representation. The trained model generates 2D RGB\sigmaσ outputs via Gaussian Splatting rasterization, guided by camera parameters and human pose inputs. Compatible with HoloLens 2 and web-based platforms, RT-FVHP operates effectively under real-world conditions, including handling occlusions caused by treadmills. Occlusion handling is accomplished using our Shape-Enforced Gaussian Density Control (SGDC), which initializes and densifies 3D Gaussians in occluded regions using estimated SMPL human body priors. This approach minimizes manual intervention while ensuring complete body reconstruction. With efficient Gaussian rasterization, the model delivers real-time performance of up to 400 FPS at 1080p resolution on a dedicated RTX6000 GPU.

Abstract:
The way users interact with Virtual Reality (VR) environments plays a crucial role in shaping their experience when embodying an avatar. How avatars are perceived by users significantly influences their behavior based on stereotypes, a phenomenon known as the Proteus effect. The psychological concept of affordances may also appear relevant when it comes to interact through avatars and is yet underexplored. Indeed, understanding how virtual representations suggest possibilities for action has attracted considerable attention in the human-computer interaction community, but only few studies clearly address the use of affordances. Of particular interest is the fact aesthetic features of avatars may signify false affordances, conflicting with users' expectations and impacting perceived plausibility of the depicted situations. Recent models of congruence and plausibility suggest altering the latter may result in unexpected consequences on other qualia like presence and embodiment. The proposed research initially aimed at exploring the operationalization of affordances as a tool to investigate the impact of congruence and plausibility manipulations on the sense of embodiment. In spite of a long and careful endeavor materialized by a preliminary assessment and two user studies, it appears our participants were primed by other internal processes that took precedence over the perception of the affordances we selected. However, we unexpectedly manipulated the internal congruence following repeated exposures (mixed design), causing a rupture in plausibility and significantly lowering scores of embodiment and task performance. The present research then constitutes a direct proof of a relationship between a break in plausibility and a break in embodiment.

Abstract:
With the rapid development of advanced machine learning methods for spatial reconstruction, it becomes important to understand the psychological and emotional impacts of such technologies on autobiographical memories. In a within-subjects study, we found that allowing users to walk through old spaces reconstructed from their videos significantly enhances their sense of traveling into past memories, increases the vividness of those memories, and boosts their emotional intensity compared to simply viewing videos of the same past events. These findings highlight that, regardless of the technological advancements, the immersive experience of VR can profoundly affect memory phenomenology and emotional engagement. As systems enabling immersive memory reconstruction become more ubiquitous, it is crucial to critically examine their effects on human cognition and perception of reality.

Abstract:
In the real world, the body typically remains within the downward field of view (FOV) irrespective of whether it is consciously or unconsciously observed. However, with head-mounted displays (HMDs) that have restricted viewing angles, users are unable to see their own avatars directly unless they tilt their heads significantly downward. To overcome this, virtual mirrors are commonly employed in virtual environments (VEs) to provide users with a full-body view of their avatars, which ultimately enhances the viewer's sense of embodiment (SoE). However, positioning users directly in front of a virtual mirror may restrict their movement and reduce their sense of presence. To overcome this limitation, we developed an HMD with an extended downward field of view (FOV) integrated with eye-tracking. We compared the impact of using virtual mirrors alone versus the extended downward FOV in terms of several parameters, including the head pitch angle, perceived SoE, presence, and task difficulty during a reaching task that involves both hand and foot movements. The results demonstrated that the extended downward FOV offered by the proposed HMD improved the users' sense of agency toward their avatars and enhanced their sense of presence compared to when only virtual mirrors are used. In addition, the participants tended to use the virtual mirror for hand movements and relied on the extended downward FOV for foot movements, often maintaining a stable head angle. Both the virtual mirror and extended downward FOV facilitated easier and faster completion of a reaching task while encouraging an upward head angle. However, the combined use of virtual mirrors and extended downward FOV did not have a significant impact on the SoE or presence. Notably, the extended downward FOV alone, without the integration of the virtual mirror, was more effective in increasing both SoE and presence.

Abstract:
Members of the LGBTQIA+ community are more likely to face mental health challenges. However, stigma and the fear of being outed often prevent them from seeking professional support. To address this, we collaborated with mental health professionals and LGBTQIA+ communities in Japan to develop a multi-user Virtual Reality (VR) platform that facilitates access to group therapy sessions. The system allows users to participate using personalized avatars and customized voices, preserving anonymity while enabling them to present themselves as they wish. We conducted a user study with 21 LGBTQIA+ participants and two qualified counselors to evaluate their experiences with VR-based therapy. Findings revealed that the created avatars enabled participants to express their chosen gender identity and increase confidence, acting as protective intermediaries. However, participants also noted how anonymity could affect trust, and suggested that better representation of body language and the introduction of trust-building activities could help compensate for such ambivalence. Overall, the platform fostered a strong sense of co-presence, and both counselors and LGBTQIA+ members felt that, with some ergonomic adjustment to improve the comfort of the headset during longer sessions, VR platforms could offer substantial opportunities for safe and representative access to mental health services.

Abstract:
Virtual museums are increasingly used to present cultural and educational content, often relying on textual descriptions to convey essential information. However, when users interact with text objects in virtual environments, the optimal text segment quantity for readability and comprehension remains unclear, especially when social distractions such as other visitors are present. This study investigated the effects of text segment quantity and the presence of other visitors on text readability and comprehension in the context of virtual museums. Participants read exhibit descriptions under four text-segment length conditions (1, 2, 4, and 8 lines) with or without simulated visitor agents. The results indicated that readability and comprehension were maximized when text was presented in intermediate segmentation lengths (2 and 4 Lines), while both excessively short (1 line) and long (8 lines) text segments hindered reading performance. Additionally, a significant interaction between text length and the presence of visitors was observed. Specifically, the presence of visitors led to increased comprehension task completion times only in the intermediate segmentation conditions, suggesting that social presence imposes an additional cognitive demand as social distractions in optimal text segment conditions. These findings provide empirical guidelines for designing effective text-based information systems in virtual museums, optimizing both user engagement and learning outcomes in immersive cultural environments.