Abstract Paper Portal of IEEE Transactions on Neural Networks and Learning Systems (TNNLS) 2025

PaperID: 1,   

Authors:  Siddharth Barve, Rashmi Jha

Title: Dynamic Self-Organizing Neurons

Abstract:
Many currently available deep neural network (DNN) accelerators are highly application specific and have focused on supervised learning. In addition, many accelerators have rigid architectures and algorithms that prevent adapting to dynamic environments. In this work, we propose a neuromorphic architecture implementing a self-organizing feature map (SOFM) using ferroelectric field-effect transistors (FeFETs) for in-memory error computation. The neuromorphic architecture takes inspiration from biological networks and is able to grow neurons to adapt to the application. Furthermore, it is able to modulate the distance between neurons to provide more fluidity to its topography. We demonstrate that the ability of the network to adapt to various datasets and even exhibit lifelong learning and self-repair. We further demonstrate the architecture’s efficiency in terms of both power and speed as well as its robustness to device variability.

---

PaperID: 2,   

Authors:  Man-Sheng Chen, Jia-Qi Lin, Chang-Dong Wang, Dong Huang, Jian-Huang Lai

Title: Contrastive Ensemble Clustering

Abstract:
Ensemble clustering aims to combine different base clusterings into a better clustering than that of the individual one. In general, a co-association matrix depicting the pairwise affinity between different data samples is constructed by average fusion or weighted fusion of the connective matrices from multiple base clusterings. Despite the significant success, the existing works fail to capture the global structure information from multiple noisy connective matrices. Meanwhile, the locality property of the resulting representation matrix could not be explicitly preserved. In this article, we propose a novel contrastive ensemble clustering (CEC) method. Specifically, a consensus mapping model is designed for the discovery of the latent representation from the noisy observations with distinct confidences. Furthermore, a contrastive regularizer is dexterously formulated to refine the latent representation while preserving its locality property. Extensive experiments conducted on several benchmark datasets demonstrate the superiority of the proposed CEC method. To the best of our knowledge, it is the first time to explore the potential of latent representation learning and contrastive components for the ensemble clustering task.

---

PaperID: 3,   

Authors:  Fuzhi Wu, Jiasong Wu, Chen Zhang, Youyong Kong, Chunfeng Yang, Guanyu Yang, Huazhong Shu, Guy Carrault, Lotfi Senhadji

Title: Wavelet-Based Dual-Task Network

Abstract:
In image processing, wavelet transform (WT) offers multiscale image decomposition, generating a blend of low-resolution approximation images and high-resolution detail components. Drawing parallels to this concept, we view feature maps in convolutional neural networks (CNNs) as a similar mix, but uniquely within the channel domain. Inspired by multitask learning (MTL) principles, we propose a wavelet-based dual-task (WDT) framework. This novel framework employs WT in the channel domain to split a single task into two parallel tasks, thereby reforming traditional single-task CNNs into dynamic dual-task networks. Our WDT framework integrates seamlessly with various popular network architectures, enhancing their versatility and efficiency. It offers a more rational approach to resource allocation in CNNs, balancing between low-frequency and high-frequency information. Rigorous experiments on Cifar10, ImageNet, HMDB51, and UCF101 validate our approach’s effectiveness. Results reveal significant improvements in the performance of traditional CNNs on classification tasks, and notably, these enhancements are achieved with fewer parameters and computations. In summary, our work presents a pioneering step toward redefining the performance and efficiency of CNN-based tasks through WT.

---

PaperID: 4,   

Authors:  Amelia Sorrenti, Giovanni Bellitto, Federica Proietto Salanitri, Matteo Pennisi, Simone Palazzo, Concetto Spampinato

Title: Wake-Sleep Consolidated Learning

Abstract:
We propose wake-sleep consolidated learning (WSCL), a learning strategy leveraging complementary learning system (CLS) theory and the wake-sleep phases of the human brain to improve the performance of deep neural networks (DNNs) for visual classification tasks in continual learning (CL) settings. Our method learns continually via the synchronization between distinct wake and sleep phases. During the wake phase, the model is exposed to sensory input and adapts its representations, ensuring stability through a dynamic parameter freezing mechanism and storing episodic memories in a short-term temporary memory (similar to what happens in the hippocampus). During the sleep phase, the training process is split into nonrapid eye movement (NREM) and rapid eye movement (REM) stages. In the NREM stage, the model’s synaptic weights are consolidated using replayed samples from the short-term and long-term memory and the synaptic plasticity mechanism is activated, strengthening important connections and weakening unimportant ones. In the REM stage, the model is exposed to previously-unseen realistic visual sensory experience, and the dreaming process is activated, which enables the model to explore the potential feature space, thus preparing synapses for future knowledge. We evaluate the effectiveness of our approach on four benchmark datasets: CIFAR-10, CIFAR-100, Tiny-ImageNet, and FG-ImageNet. In all cases, our method outperforms the baselines and prior work, yielding a significant performance gain on continual visual classification tasks. Furthermore, we demonstrate the usefulness of all processing stages and the importance of dreaming to enable positive forward transfer (FWT). The code is available at: https://github.com/perceivelab/wscl.

---

PaperID: 5,   

Authors:  Ling Huang, Dong Huang, Han Zou, Yuefang Gao, Chang-Dong Wang, Philip S. Yu

Title: Knowledge-Reinforced Cross-Domain Recommendation

Abstract:
Over the past few years, cross-domain recommendation has gained great attention to resolve the cold-start issue. Many existing cross-domain recommendation methods model a preference bridge between the source and target domains to transfer preferences by the overlapping users. However, when there are insufficient cross-domain users available to bridge the two domains, it will negatively impact the recommender system’s accuracy (ACC) and performance. Therefore, in this article, we propose to create a link between the source and the target domains by leveraging knowledge graph (KG) as the auxiliary information, and propose a novel knowledge-reinforced cross-domain recommendation (KR-CDR) method. First of all, we construct a new cross-domain KG (CDKG) by using the KGs that represent the source and target domains, respectively. Additionally, we employ reinforcement learning (RL) with meta learning on CDKG to discover meta-paths between the source and target domains. With these meta-paths, we obtain meta-path aggregated embedding vectors for cold-start users. Ultimately, the predicted rating can be acquired from the user meta-path aggregated embedding vector and item embedding vector. Experiments carried out on five real-world datasets show that the proposed method performs better than the state-of-the-art methods.

---

PaperID: 6,   

Authors:  Hongkun Dou, Junzhe Lu, Zeyu Li, Xiaoqing Zhong, Wen Yao, Lijun Yang, Yue Deng

Title: Score-Based Neural Processes

Abstract:
Neural processes (NPs) have recently emerged as a powerful meta-learning framework capable of making predictions based on an arbitrary number of context points. However, the learning of NPs and their variants is hindered by the need for explicit reliance on the log-likelihood of predictive distributions, which complicates the training process. To tackle this problem, we introduce score-based NP (SNP) models, drawing inspiration from recently developed score-based generative models (SGMs) that restore data from noise by reversing a perturbation process. With denoising score matching (DSM) techniques, the SNPs bypass the intractable log-likelihood calculations, learning parameterized score functions instead. We also demonstrate that score functions possess excellent attributes that enable us to represent a wide family of conditional distributions naturally. Moreover, as data points are inherently unordered, it is crucial to incorporate appropriate inductive biases into SNPs. To this end, we propose building blocks for parameterizing permutation equivariant score functions, which induce the SNPs with the desired properties. Through extensive experimentation on both synthetic and real-world datasets, our SNPs exhibit remarkable performance and outperform existing state-of-the-art NP approaches.

---

PaperID: 7,   

Authors:  Baochang Zhang, Runqi Wang, Xiaodi Wang, Jungong Han, Rongrong Ji

Title: Modulated Convolutional Networks

Abstract:
While the deep convolutional neural network (DCNN) has achieved overwhelming success in various vision tasks, its heavy computational and storage overhead hinders the practical use of resource-constrained devices. Recently, compressing DCNN models has attracted increasing attention, where binarization-based schemes have generated great research popularity due to their high compression rate. In this article, we propose modulated convolutional networks (MCNs) to obtain binarized DCNNs with high performance. We lead a new architecture in MCNs to efficiently fuse the multiple features and achieve a similar performance as the full-precision model. The calculation of MCNs is theoretically reformulated as a discrete optimization problem to build binarized DCNNs, for the first time, which jointly consider the filter loss, center loss, and softmax loss in a unified framework. Our MCNs are generic and can decompose full-precision filters in DCNNs, e.g., conventional DCNNs, VGG, AlexNet, ResNets, or Wide-ResNets, into a compact set of binarized filters which are optimized based on a projection function and a new updated rule during the backpropagation. Moreover, we propose modulation filters (M-Filters) to recover filters from binarized ones, which lead to a specific architecture to calculate the network model. Our proposed MCNs substantially reduce the storage cost of convolutional filters by a factor of 32 with a comparable performance to the full-precision counterparts, achieving much better performance than other state-of-the-art binarized models.

---

PaperID: 8,   

Authors:  Di Wang, Ping Wang, Zhong Ji, Xiaojun Yang, Hong-Yue Li

Title: Conformal Loss-Controlling Prediction

Abstract:
Conformal prediction (CP) is a learning framework controlling prediction coverage of prediction sets, which can be built on any learning algorithm for point prediction. This work proposes a learning framework named conformal loss-controlling prediction, which extends CP to the situation where the value of a loss function needs to be controlled. Different from existing works about risk-controlling prediction sets and conformal risk control with the purpose of controlling the expected values of loss functions, the proposed approach in this article focuses on the loss for any test object, which is an extension of CP from miscoverage loss to some general loss. The controlling guarantee is proved under the assumption of exchangeability of data in finite-sample cases and the framework is tested empirically for classification with a class-varying loss and statistical postprocessing of numerical weather forecasting applications, which are introduced as point-wise classification and point-wise regression problems. All theoretical analysis and experimental results confirm the effectiveness of our loss-controlling approach.

---

PaperID: 9,   

Authors:  Meng Liu, Ke Liang, Hao Yu, Lingyuan Meng, Siwei Wang, Sihang Zhou, Xinwang Liu

Title: Multiview Temporal Graph Clustering

Abstract:
As an emerging task, temporal graph clustering (TGC) is committed to clustering nodes on temporal graphs through interaction sequence-based batch-processing patterns. These patterns allow for more flexibility in finding a balance between time and space requirements than adjacency matrix-based static graph clustering. However, as a new task, TGC still has important unresolved challenges, such as insufficient information. This challenge manifests itself in a variety of problems in real-world datasets, including missing features (eigenvalues are missing or even nonexistent), long-tail nodes (most inactive nodes have little interaction), and noisy data (data is subject to anomalies, errors, and sparsity). These problems occur before training, making it difficult for the model to train well with insufficient information. To solve the challenge, we propose a method that introduces multiview clustering (MVC) into TGC, called MVTGC. Our method aims to perform data augmentation on the temporal graph by constructing multiple views to increase the information richness. In particular, we utilize different techniques to model a certain part of the temporal graph to generate enhanced views focusing on different angles. These views are combined into training through early fusion and late fusion and ultimately enhance the model’s receptive field and information richness. Comparative experiments and a case study on real-world datasets demonstrate the significance and effectiveness of MVTGC, which achieves at most 10.48% performance improvement. The code and data are available at https://github.com/MGitHubL/MVTGC

---

PaperID: 10,   

Authors:  Nunzio A. Letizia, Nicola Novello, Andrea M. Tonello

Title: Copula Density Neural Estimation

Abstract:
Probability density estimation from observed data constitutes a central task in statistics. In this brief, we focus on the problem of estimating the copula density associated with any observed data, as it fully describes the dependence between random variables. We separate univariate marginal distributions from the joint dependence structure in the data, the copula itself, and we model the latter with a neural network-based method referred to as copula density neural estimation (CODINE). Results show that the novel learning approach is capable of modeling complex distributions and can be applied for mutual information estimation and data generation.

---

PaperID: 11,   

Authors:  Chao Ren, Rudai Yan, Huihui Zhu, Han Yu, Minrui Xu, Yuan Shen, Yan Xu, Ming Xiao, Zhao Yang Dong, Mikael Skoglund, Dusit Niyato, Leong Chuan Kwek

Title: Toward Quantum Federated Learning

Abstract:
Quantum federated learning (QFL) is an emerging interdisciplinary field that merges the principles of quantum computing (QC) and federated learning (FL), with the goal of leveraging quantum technologies to enhance privacy, security, and efficiency in the learning process. Currently, there is no comprehensive survey for this interdisciplinary field. This review offers a thorough, holistic examination of QFL. We aim to provide a comprehensive understanding of the principles, techniques, and emerging applications of QFL. We discuss the current state of research in this rapidly evolving field, identify challenges and opportunities associated with integrating these technologies, and outline future directions and open research questions. We propose a unique taxonomy of QFL techniques, categorized according to their characteristics and the quantum techniques employed. As the field of QFL continues to progress, we can anticipate further breakthroughs and applications across various industries, driving innovation and addressing challenges related to data privacy, security, and resource optimization. This review serves as a first-of-its-kind comprehensive guide for researchers and practitioners interested in understanding and advancing the field of QFL.

---

PaperID: 12,   

Authors:  Chaoyang Li, Heyan Chai, Yan Jia, Ning Hu, Qing Liao

Title: Multitask Causal Contrastive Learning

Abstract:
Multitask learning (MTL) aims to improve the performance of multiple tasks by sharing knowledge among multiple different tasks, which has attracted increasing interest and shown success in various fields. However, MTL often suffers from negative transfer since the model may utilize useless features and face interference among tasks’ optimization objectives. The utilization of useless features can be attributed to the confounding factors in multitask features, while the interference among tasks’ optimization objectives is due to inadequate measurement of the relationship among tasks. This article proposes a novel multitask causal contrastive learning (MT-CCL) approach to address the above problem. First, we propose a multitask causal inference method, which removes the confounding factors in features via task-aware causal intervention (TACI) and measures the relationship among tasks from a novel causal perspective by quantifying the intertask causal affinity. Then, we build a dual contrastive learning objective to help the model better learn useful features via intratask contrast (Intra-TCS) and mitigate interference among tasks’ optimization objectives via intertask contrast (Inter-TCS). Experiments demonstrate that MT-CCL achieves improved performance over state-of-the-art methods on Multi-MNIST, NYU-v2, CityScapes, and CelebA, verifying the effectiveness of intertask causal affinity.

---

PaperID: 13,   

Authors:  Shuyi Ji, Yifan Feng, Donglin Di, Shihui Ying, Yue Gao

Title: Mode Hypergraph Neural Network

Abstract:
The hypergraph neural network (HGNN) is an emerging powerful tool for modeling and learning complex, high-order correlations among entities upon hypergraph structures. While existing HGNN-based approaches excel in modeling high-order correlations among data using hyperedges, they often have difficulties in distinguishing diverse semantics (e.g., bioactivities between drug and target in biological networks) of different correlations, making it challenging to learn accurate final representations. The underlying reason is that the specific semantic information of each hyperedge cannot be captured and distinguished during the modeling and learning process. To address this, we propose a mode HGNN ( \textsf MHGNN ) framework that extends the vanilla hypergraph structure by endowing hyperedges with mode information for encapsulating their semantics and then performs mode-aware high-order message passing upon mode hypergraph for achieving comprehensive node representations. Extensive evaluations on four real-world datasets under two representative tasks have demonstrated the outstanding performance of \textsf MHGNN against the state of the arts.

---

PaperID: 14,   

Authors:  Yueyi Zhang, Jin Wang, Wenming Weng, Xiaoyan Sun, Zhiwei Xiong

Title: EGVD: Event-Guided Video Deraining

Abstract:
Recent research has explored leveraging event cameras, known for their prowess in capturing scenes with nonuniform motion, for video deraining, leading to performance improvements. However, the existing event-based method still faces the challenge that the complex spatiotemporal distribution disrupts temporal information fusion and complicates feature separation. This article proposes a novel end-to-end learning framework for video deraining that effectively extracts the rich dynamic information provided by the event stream. Our framework incorporates two key modules: an event-aware motion detection (EAMD) module that adaptively aggregates multiframe motion information using event-driven masks and a pyramidal adaptive selection module that separates background and rain layers by leveraging contextual priors from both event and conventional camera data. To facilitate efficient training, we introduce a real-world dataset of synchronized rainy videos and event streams. Extensive evaluations on both synthetic and real-world datasets demonstrate the superiority of our proposed method compared to state-of-the-art approaches. The code is available at https://github.com/booker-max/EGVD.

---

PaperID: 15,   

Authors:  Hyo-Seok Hwang, Yoojoong Kim, Junhee Seok

Title: Generative Adversarial Soft Actor-Critic

Abstract:
In deep reinforcement learning (RL), learning stochastic and multimodal policies are crucial for various tasks, but most continuous control algorithms model the policy using a deterministic or unimodal Gaussian distribution. Despite being designed to inherently learn a stochastic policy under the maximum entropy RL framework, soft actor–critic (SAC) also uses a factorized Gaussian policy for tractable optimization, which not only restricts the expressiveness of the policy but also ignores correlations among the components of the action vector. In this article, we revisit the approach of employing normalizing flow for SAC policy, justified by the change of variable theorem, and then propose a state-dependent nonvolume preserving (SD-NVP) architecture suitable for SAC learning. In addition, we introduce a generative adversarial SAC (GASAC) that implicitly defines and optimizes various divergences without calculating the normalization constant through a generative adversarial loss. Experimental results on multigoal environment and MuJoCo continuous control tasks suite demonstrate that GASAC model multimodal policy and learn policy more stably in terms of cumulative return.

---

PaperID: 16,   

Authors:  Jun Chen, Jingyang Xiang, Tianxin Huang, Xiangrui Zhao, Yong Liu

Title: Hyperbolic Binary Neural Network

Abstract:
Binary neural network (BNN) converts full-precision weights and activations into their extreme 1-bit counterparts, making it particularly suitable for deployment on lightweight mobile devices. While BNNs are typically formulated as a constrained optimization problem and optimized in the binarized space, general neural networks are formulated as an unconstrained optimization problem and optimized in the continuous space. This article introduces the hyperbolic BNN (HBNN) by leveraging the framework of hyperbolic geometry to optimize the constrained problem. Specifically, we transform the constrained problem in hyperbolic space into an unconstrained one in Euclidean space using the Riemannian exponential map. On the other hand, we also propose the exponential parametrization cluster (EPC) method, which, compared with the Riemannian exponential map, shrinks the segment domain based on a diffeomorphism. This approach increases the probability of weight flips, thereby maximizing the information gain in BNNs. Experimental results on CIFAR10, CIFAR100, and ImageNet classification datasets with VGGsmall, ResNet18, and ResNet34 models illustrate the superior performance of our HBNN over state-of-the-art methods.

---

PaperID: 17,   

Authors:  Yesong Xu, Shuo Chen, Jun Li, Jian Yang

Title: Metric Learning-Based Subspace Clustering

Abstract:
The self-expressive strategy has shown excellent capabilities in realizing low-dimensional representations of high-dimensional data for subspace clustering algorithms. The existing designs, however, are formulated on the linearization assumptions of the data, neglecting the precise characterization of linear relationships within samples. Considering that real-world data adheres to diverse distribution forms, it becomes impractical to first treat the samples as existing in a uniform linear space before finding an appropriate manifold space. To handle this challenge, we propose a novel self-expressive-based learning framework termed metric learning-based subspace clustering (MLSC). Particularly, we smoothly incorporate metric learning into the subspace clustering framework by introducing adaptive neighbors learning and defining a linearity-aware distance to discover the linear manifold space of the original data. We simultaneously utilize the generated representation of the linear structure as input for self-expressiveness to pursue an ideal similarity matrix, which establishes an essential connection with the linearization assumption of the self-expressive strategy. Furthermore, we theoretically demonstrate that our measure can accurately describe the level of linear correlation between instances. Finally, our tests demonstrate that the proposed MLSC attains competitive clustering results compared to state-of-the-art approaches on benchmark datasets.

---

PaperID: 18,   

Authors:  Tenghai Qiu, Shiguang Wu, Zhen Liu, Zhiqiang Pu, Jianqiang Yi, Yuqian Zhao, Biao Luo

Title: Cognition-Oriented Multiagent Reinforcement Learning

Abstract:
Inspired by psychological insights into individual behavior, we propose a novel cognition-oriented multiagent reinforcement learning (CORL) framework. CORL equips agents with two distinct types of cognition—situational and self-cognition—derived from local observations. To enhance the informativeness and precision of these cognition types, we introduce two information-theoretical regularizers: one to align situational cognition with the global state and the other to align self-cognition with each agent’s identity for improved role differentiation and team coordination. In addition, the centralized training and decentralized execution framework is adopted to train the policy network. Our simulations demonstrate that CORL effectively harnesses local observations for enriched cooperation, leading to pronounced performance improvements, particularly in challenging tasks.

---

PaperID: 19,   

Authors:  Dai Shi, Zhiqi Shao, Junbin Gao, Zhiyong Wang, Yi Guo

Title: Frameless Graph Knowledge Distillation

Abstract:
Knowledge distillation (KD) has shown great potential for transferring knowledge from a complex teacher model to a simple student model in which the heavy learning task can be accomplished efficiently and without losing too much prediction accuracy. Recently, many attempts have been made by applying the KD mechanism to graph representation learning models such as graph neural networks (GNNs) to accelerate the model’s inference speed via student models. However, many existing KD-based GNNs utilize multilayer perceptron (MLP) as a universal approximator in the student model to imitate the teacher model’s process without considering the graph knowledge from the teacher model. In this work, we provide a KD-based framework on multiscaled GNNs, known as graph framelet, and prove that by adequately utilizing the graph knowledge in a multiscaled manner provided by graph framelet decomposition, the student model is capable of adapting both homophilic and heterophilic graphs and has the potential of alleviating the oversquashing issue with a simple yet effective graph surgery. Furthermore, we show how the graph knowledge supplied by the teacher is learned and digested by the student model via both algebra and geometry. Comprehensive experiments show that our proposed model can generate learning accuracy identical to or even surpass the teacher model while maintaining the high speed of inference.

---

PaperID: 20,   

Authors:  Chao Wang, Jiaxuan Zhao, Lingling Li, Licheng Jiao, Fang Liu, Shuyuan Yang

Title: Automatic Graph Topology-Aware Transformer

Abstract:
Existing efforts are dedicated to designing many topologies and graph-aware strategies for the graph Transformer, which greatly improve the model’s representation capabilities. However, manually determining the suitable Transformer architecture for a specific graph dataset or task requires extensive expert knowledge and laborious trials. This article proposes an evolutionary graph Transformer architecture search (EGTAS) framework to automate the construction of strong graph Transformers. We build a comprehensive graph Transformer search space with the micro-level and macro-level designs. EGTAS evolves graph Transformer topologies at the macro level and graph-aware strategies at the micro level. Furthermore, a surrogate model based on generic architectural coding is proposed to directly predict the performance of graph Transformers, substantially reducing the evaluation cost of evolutionary search. We demonstrate the efficacy of EGTAS across a range of graph-level and node-level tasks, encompassing both small-scale and large-scale graph datasets. Experimental results and ablation studies show that EGTAS can construct high-performance architectures that rival state-of-the-art manual and automated baselines.

---

PaperID: 21,   

Authors:  Qin Zhao, Peihan Wu, Gang Liu, Dongdong An, Jie Lian, MengChu Zhou

Title: Sociological-Theory-Based Multitopic Self-Supervised Recommendation

Abstract:
Social relationships offer crucial supplementary information for recommendations by leveraging users’ social connections to gain insights into their preferences. However, prevalent social recommendation methods often grapple with the issues of sparsity and noise, which curtail their effectiveness. In addition, these methods overlook the intricacies of user interactions within social networks, which could provide invaluable information. Addressing their deficiencies, this article introduces a novel sociological-theory-based multitopic self-supervised recommendation method (SMSR). This method integrates user attitude information into the construction of social relationships and utilizes dynamic routing to identify and categorize topics, thereby mitigating the impact of social noise on recommendation accuracy. Furthermore, we reveal sophisticated higher order user relations within these topics by using motifs. By combining the light graph convolutional network with balance theory, SMSR efficiently aggregates information from diverse social relations to gain its outstanding performance. Moreover, we have devised and integrated four self-supervised signals, inspired by social theory and derived from heterogeneous graph analysis, to more effectively exploit the rich structural and semantic information inherent in social relationship graphs. Empirical results from extensive experiments on publicly available datasets underscore SMSR’s superiority over the state of the art.

---

PaperID: 22,   

Authors:  Naoyuki Terashita, Hiroki Ohashi, Satoshi Hara

Title: Data Cleansing for GANs

Abstract:
As the application of generative adversarial networks (GANs) expands, it becomes increasingly critical to develop a unified approach that improves performance across various generative tasks. One effective strategy that applies to any machine learning task is identifying harmful instances, whose removal improves the performance. While previous studies have successfully estimated these harmful training instances in supervised settings, their approaches are not easily applicable to GANs. The challenge lies in two requirements of the previous approaches that do not apply to GANs. First, previous approaches require that the absence of a training instance directly affects the parameters. However, in the training for GANs, the instances do not directly affect the generator’s parameters since they are only fed into the discriminator. Second, previous approaches assume that the change in loss directly quantifies the harmfulness of the instance to a model’s performance, while common types of GAN losses do not always reflect the generative performance. To overcome the first challenge, we propose influence estimation methods that use the Jacobian of the generator’s gradient with respect to the discriminator’s parameters (and vice versa). Such a Jacobian represents the indirect effect between two models: how removing an instance from the discriminator’s training changes the generator’s parameters. Second, we propose an instance evaluation scheme that measures the harmfulness of each training instance based on how a GAN evaluation metric [e.g., inception score (IS)] is expected to change by the instance’s removal. Furthermore, we demonstrate that removing the identified harmful instances significantly improves the generative performance on various GAN evaluation metrics. The code is available at https://github.com/hitachi-rd-cv/data-cleansing-for-gans.

---

PaperID: 23,   

Authors:  Zhongyi Que, Chih-Jen Lin

Title: One-Class SVM Probabilistic Outputs

Abstract:
One-class support vector machine (SVM) is an extension of SVM to handle unlabeled data. As a mature technique for outlier detection, one-class SVM has been widely used in many applications. However, similar to standard two-class SVM, the design of one-class SVM does not give probabilistic outputs. For two-class SVM, some methods have been proposed to effectively obtain probabilistic outputs, but due to the difficulty of no-label information, less attention has been paid to one-class SVM. Our aim in this work is to propose some practically viable techniques to generate probabilistic outputs for one-class SVM. We investigate existing methods for two-class SVM and explain why they may not be suitable for one-class SVM. Due to the lack of label information, we think a feasible setting is to have probabilities mimic to the decision values of training data. Based on this principle, we propose several new methods. Detailed experiments on both artificial and real-world data demonstrate the effectiveness of the proposed methods.

---

PaperID: 24,   

Authors:  Luca Cosmo, Giorgia Minello, Alessandro Bicciato, Michael M. Bronstein, Emanuele Rodolà, Luca Rossi, Andrea Torsello

Title: Graph Kernel Neural Networks

Abstract:
The convolution operator at the core of many modern neural architectures can effectively be seen as performing a dot product between an input matrix and a filter. While this is readily applicable to data such as images, which can be represented as regular grids in the Euclidean space, extending the convolution operator to work on graphs proves more challenging, due to their irregular structure. In this article, we propose to use graph kernels, i.e., kernel functions that compute an inner product on graphs, to extend the standard convolution operator to the graph domain. This allows us to define an entirely structural model that does not require computing the embedding of the input graph. Our architecture allows to plug-in any type of graph kernels and has the added benefit of providing some interpretability in terms of the structural masks that are learned during the training process, similar to what happens for convolutional masks in traditional convolutional neural networks (CNNs). We perform an extensive ablation study to investigate the model hyperparameters’ impact and show that our model achieves competitive performance on standard graph classification and regression datasets.

---

PaperID: 25,   

Authors:  Zebiao Hu, Jian Wang, Kai Zhang, Witold Pedrycz, Nikhil R. Pal

Title: Bi-Level Spectral Feature Selection

Abstract:
Unsupervised feature selection (UFS) aims to learn an indicator matrix relying on some characteristics of the high-dimensional data to identify the features to be selected. However, traditional unsupervised methods perform only at the feature level, i.e., they directly select useful features by feature ranking. Such methods do not pay any attention to the interaction information with other tasks such as classification, which severely degrades their feature selection performance. In this article, we propose an UFS method which also takes into account the classification level, and selects features that perform well both in clustering and classification. To achieve this, we design a bi-level spectral feature selection (BLSFS) method, which combines classification level and feature level. More concretely, at the classification level, we first apply the spectral clustering to generate pseudolabels, and then train a linear classifier to obtain the optimal regression matrix. At the feature level, we select useful features via maintaining the intrinsic structure of data in the embedding space with the learned regression matrix from the classification level, which in turn guides classifier training. We utilize a balancing parameter to seamlessly bridge the classification and feature levels together to construct a unified framework. A series of experiments on 12 benchmark datasets are carried out to demonstrate the superiority of BLSFS in both clustering and classification performance.

---

PaperID: 26,   

Authors:  Ping He, Xiaohua Xu, Suquan Chen

Title: Robust Supervised Spline Embedding

Abstract:
High-dimensional data present significant challenges such as inadequate sample size, abundance of noise, and the curse of dimensionality, which make many traditional classification algorithms inapplicable. To provide valid inference for such data, it requires finding a noise-free low-dimensional representation that preserves both the underlying manifold structure and discriminative information. However, the existing methods often fail to take full consideration of these requirements. In this article, we introduce a robust supervised spline embedding (RS2E) algorithm for high-dimensional classification. The proposed approach is highlighted in four aspects: 1) it preserves the class-aware submanifold structure in the thin plate spline embedding space; 2) it eliminates noise and outliers to recover the clean manifold by exploiting its intrinsic low complexity; 3) it separates the class-aware submanifolds by maximizing the distance between each data point and the marginal data points of other class-aware submanifolds; and 4) it applies the alternating direction method of multipliers with generalized power iteration to solve the objective function. Promising experimental results on the real-world, generative adversarial network (GAN)-generated and artificially corrupted datasets demonstrate that RS2E outperforms other supervised dimensionality reduction algorithms in terms of classification accuracy.

---

PaperID: 27,   

Authors:  Saeed Anwar, Nick Barnes, Lars Petersson

Title: Attention-Based Real Image Restoration

Abstract:
Deep convolutional neural networks perform better on images containing spatially invariant degradations, also known as synthetic degradations; however, their performance is limited on real-degraded photographs and requires multiple-stage network modeling. To advance the practicability of restoration algorithms, this article proposes a novel single-stage blind real image restoration network ( \textR^2 Net) by employing a modular architecture. We use a residual on the residual structure to ease low-frequency information flow and apply feature attention to exploit the channel dependencies. Furthermore, the evaluation in terms of quantitative metrics and visual quality for four restoration tasks, i.e., denoising, super-resolution, raindrop removal, and JPEG compression on 11 real degraded datasets against more than 30 state-of-the-art algorithms, demonstrates the superiority of our \textR^2 Net. We also present the comparison on three synthetically generated degraded datasets for denoising to showcase our method’s capability on synthetics denoising. The codes, trained models, and results are available on https://github.com/saeed-anwar/R2Net.

---

PaperID: 28,   

Authors:  Ben Yang, Xuetao Zhang, Jinghan Wu, Feiping Nie, Zhiping Lin, Fei Wang, Badong Chen

Title: Fast Multiview Anchor-Graph Clustering

Abstract:
Due to its high computational complexity, graph-based methods have limited applicability in large-scale multiview clustering tasks. To address this issue, many accelerated algorithms, especially anchor graph-based methods and indicator learning-based methods, have been developed and made a great success. Nevertheless, since the restrictions of the optimization strategy, these accelerated methods still need to approximate the discrete graph-cutting problem to a continuous spectral embedding problem and utilize different discretization strategies to obtain discrete sample categories. To avoid the loss of effectiveness and efficiency caused by the approximation and discretization, we establish a discrete fast multiview anchor graph clustering (FMAGC) model that first constructs an anchor graph of each view and then generates a discrete cluster indicator matrix by solving the discrete multiview graph-cutting problem directly. Since the gradient descent-based method makes it hard to solve this discrete model, we propose a fast coordinate descent-based optimization strategy with linear complexity to solve it without approximating it as a continuous one. Extensive experiments on widely used normal and large-scale multiview datasets show that FMAGC can improve clustering effectiveness and efficiency compared to other state-of-the-art baselines.

---

PaperID: 29,   

Authors:  Ren Togo, Nao Nakagawa, Takahiro Ogawa, Miki Haseyama

Title: ConcVAE: Conceptual Representation Learning

Abstract:
Disentangled representation learning aims at obtaining an independent latent representation without supervisory signals. However, the independence of a representation does not guarantee interpretability to match human intuition in the unsupervised settings. In this article, we introduce conceptual representation learning, an unsupervised strategy to learn a representation and its concepts. An antonym pair forms a concept, which determines the semantically meaningful axes in the latent space. Since the connection between signifying words and signified notions is arbitrary in natural languages, the verbalization of data features makes the representation make sense to humans. We thus construct Conceptual VAE (ConcVAE), a variational autoencoder (VAE)-based generative model with an explicit process in which the semantic representation of data is generated via trainable concepts. In visual data, ConcVAE utilizes natural language arbitrariness as an inductive bias of unsupervised learning by using a vision-language pretraining, which can tell an unsupervised model what makes sense to humans. Qualitative and quantitative evaluations show that the conceptual inductive bias in ConcVAE effectively disentangles the latent representation in a sense-making manner without supervision. Code is available at https://github.com/ganmodokix/concvae.

---

PaperID: 30,   

Authors:  Xiangyu Li, Hunter Belcher, Hua Wang

Title: Robust Spherical Laplacian Embedding

Abstract:
Laplacian embedding (LE) aims to project high-dimensional input data samples, which often contain nonlinear structures, into a low-dimensional space. However, existing distance functions used in the embedding space fail to provide discriminative representations for real-world datasets, especially those related to text analysis or image processing. Cosine similarity measurements are advantageous in dealing with sparse data but are fragile to the impact of outliers and noise from samples or data features. In this work, we propose robust spherical LE (RS-LE), a powerful method that builds the LE on a novel metric that unifies the Euclidean distance and cosine similarity in a spherical space using a robust \ell _p,p -norm. We introduce an efficient iterative algorithm, the proximal alternating linearized minimization (ALM) method, to solve the difficult optimization problem that arises from the new metric, which is neither convex nor smooth. This algorithm allows the solution to achieve both global and objective convergence. Our findings are presented in rigorous theoretical analysis and validated in experiments.

---

PaperID: 31,   

Authors:  Yanshan Xiao, Bo Liu, Zhifeng Hao

Title: Multi-Instance Nonparallel Tube Learning

Abstract:
In multi-instance nonparallel plane learning (NPL), the training set is comprised of bags of instances and the nonparallel planes are trained to classify the bags. Most of the existing multi-instance NPL methods are proposed based on a twin support vector machine (TWSVM). Similar to TWSVM, they use only a single plane to generalize the data occurrence of one class and do not sufficiently consider the boundary information, which may lead to the limitation of their classification accuracy. In this article, we propose a multi-instance nonparallel tube learning (MINTL) method. Distinguished from the existing multi-instance NPL methods, MINTL embeds the boundary information into the classifier by learning a large-margin-based \epsilon -tube for each class, such that the boundary information can be incorporated into refining the classifier and further improving the performance. Specifically, given a K -class multi-instance dataset, MINTL seeks K~\epsilon -tubes, one for each class. In multi-instance learning, each positive bag contains at least one positive instance. To build up the \epsilon _k -tube of class k , we require that each bag of class k should have at least one instance included in the \epsilon _k -tube. Moreover, except for one instance included in the \epsilon _k -tube, the remaining instances in the positive bag may include positive instances or irrelevant instances, and their labels are unavailable. A large margin constraint is presented to assign the remaining instances either inside the \epsilon _k -tube or outside the \epsilon _k -tube with a large margin. Substantial experiments on real-world datasets have shown that MINTL obtains significantly better classification accuracy than the existing multi-instance NPL methods.

---

PaperID: 32,   

Authors:  Zhangkai Wu, Longbing Cao, Lei Qi

Title: eVAE: Evolutionary Variational Autoencoder

Abstract:
Variational autoencoders (VAEs) are challenged by the imbalance between representation inference and task fitting caused by surrogate loss. To address this issue, existing methods adjust their balance by directly tuning their coefficients. However, these methods suffer from a tradeoff uncertainty, i.e., nondynamic regulation over iterations and inflexible hyperparameters for learning tasks. Accordingly, we make the first attempt to introduce an evolutionary VAE (eVAE), building on the variational information bottleneck (VIB) theory and integrative evolutionary neural learning. eVAE integrates a variational genetic algorithm (VGA) into VAE with variational evolutionary operators, including variational mutation (V-mutation), crossover, and evolution. Its training mechanism synergistically and dynamically addresses and updates the learning tradeoff uncertainty in the evidence lower bound (ELBO) without additional constraints and hyperparameter tuning. Furthermore, eVAE presents an evolutionary paradigm to tune critical factors of VAEs and addresses the premature convergence and random search problem in integrating evolutionary optimization into deep learning. Experiments show that eVAE addresses the KL-vanishing problem for text generation with low reconstruction loss, generates all the disentangled factors with sharp images, and improves image generation quality. eVAE achieves better disentanglement, generation performance, and generation–inference balance than its competitors. Code available at: https://github.com/amasawa/eVAE.

---

PaperID: 33,   

Authors:  Kahou Tam, Li Li, Bo Han, Cheng-Zhong Xu, Huazhu Fu

Title: Federated Noisy Client Learning

Abstract:
Federated learning (FL) collaboratively trains a shared global model depending on multiple local clients, while keeping the training data decentralized to preserve data privacy. However, standard FL methods ignore the noisy client issue, which may harm the overall performance of the shared model. We first investigate the critical issue caused by noisy clients in FL and quantify the negative impact of the noisy clients in terms of the representations learned by different layers. We have the following two key observations: 1) the noisy clients can severely impact the convergence and performance of the global model in FL and 2) the noisy clients can induce greater bias in the deeper layers than the former layers of the global model. Based on the above observations, we propose federated noisy client learning (Fed-NCL), a framework that conducts robust FL with noisy clients. Specifically, Fed-NCL first identifies the noisy clients through well estimating the data quality and model divergence. Then robust layerwise aggregation is proposed to adaptively aggregate the local models of each client to deal with the data heterogeneity caused by the noisy clients. We further perform label correction on the noisy clients to improve the generalization of the global model. Experimental results on various datasets demonstrate that our algorithm boosts the performances of different state-of-the-art systems with noisy clients. Our code is available at https://github.com/TKH666/Fed-NCL.

---

PaperID: 34,   

Authors:  Gairui Bai, Wei Xi, Xiaopeng Hong, Xinhui Liu, Yang Yue, Songwen Zhao

Title: Robust and Rotation-Equivariant Contrastive Learning

Abstract:
Contrastive learning (CL) methods achieve great success by learning the invariant representation from various transformations. However, rotation transformations are considered harmful to CL and are rarely used, which results in failure when the objects show unseen orientations. This article proposes a representation focus shift network (RefosNet), which adds the rotation transformations to CL methods to improve the robustness of representation. First, the RefosNet constructs the rotation-equivariant mapping between the features of the original image and the rotated ones. Then, the RefosNet learns semantic-invariant representations (SIRs) based on explicitly decoupling the rotation-invariant features and the rotation-equivariant features. Moreover, an adaptive gradient passivation strategy is introduced to gradually shift the representation focus to invariant representations. This strategy can prevent catastrophic forgetting of the rotation equivariance, which is beneficial to the generalization of representations in both seen and unseen orientations. We adapt the baseline methods (i.e., “SimCLR” and “momentum contrast (MoCo) v2”) to work with RefosNet to verify the performance. Extensive experimental results show that our method achieves significant improvements on the task of recognition. On ObjectNet-13 with unseen orientations, RefosNet gains 7.12% in terms of classification accuracy compared with SimCLR. On datasets in seen orientation, the performance improves by 5.5% on ImageNet-100, 7.29% on STL10, and 1.93% on CIFAR10. In addition, RefosNet has strong generalization on Place205, PASCAL VOC, and Caltech 101. Our method has also achieved satisfactory results in image retrieval tasks.

---

PaperID: 35,   

Authors:  Yang Liu, Fang Liu, Licheng Jiao, Qianyue Bao, Shuo Li, Lingling Li, Xu Liu, Puhua Chen, Wenping Ma

Title: Chain-of-Situation Aware Progressive Inference Learning

Abstract:
The grounded situation recognition (GSR) task aims to recognize the structured semantics of an image to achieve “human-like” event understanding. Most previous studies primarily focus on the visual features of the situation, overlooking the step-by-step cognitive reasoning process that humans employ in complex task settings. Recently, the emergence of multimodal large language models (MLLMs) has provided novel directions for addressing complex problems. However, directly deploying MLLMs on the GSR task is suboptimal due to their tendency to exhibit “hallucination” issues. Additionally, fine-tuning MLLMs for the GSR task incurs high training costs. To address these challenges, inspired by human cognitive theory and the chain-of-thought (CoT) strategy, we propose the chain-of-situation progressive inference learning (CoS-PIL) framework, a lightweight approach that progressively completes verb prediction, noun prediction, and role grounding. The prediction of each step depends on the historical information of the previous step. Specifically, we first design situation prompts tailored to the GSR task and utilize MLLMs to analyze the input image and language prompts, generating heuristic response text for the current situation in the image. Instead of fine-tuning the MLLM, we activate the reasoning capabilities of the frozen MLLM and adapt its generated responses into three lightweight modules: CoS-Verb, CoS-Noun, and CoS-Ground. Considering that MLLMs may generate redundant content, we carefully design the chain-of-interest predictor (CoI-Predictor) to extract key information from the extensive response text and inject it into the model as prompts to enhance the performance. Extensive experiments on the challenging SWiG benchmark demonstrate that CoS-PIL outperforms other state-of-the-art methods. The code is publically available at https://github.com/XDLiuyyy/CoS-PIL

---

PaperID: 36,   

Authors:  Jian Chen, Wenlong Shi, Wanyu Lin, Chen Wang, Wei Liu, Hailong Sun, Gaoyang Liu

Title: Unlearning Attacks for Regression Learning

Abstract:
Recently, the machine unlearning has emerged as a popular method for efficiently erasing the impact of personal data in machine learning (ML) models upon the data owner’s removal request. However, few studies take into consideration the security concerns that may exist in the unlearning process. In this article, we propose the first unlearning attack dubbed unlearning attack for regression learning (UnAR) to deliberately influence the predictive behavior of the target sample against regression learning models. The central concept of UnAR revolves around misleading the regression model into erasing the information associated with the influential samples for the target sample. Observing that the influential samples for target data are generally located far away from the regression plane, we thus propose two novel methods, known as influential sample selection (ISS) and influential sample unlearning (ISU), to identify and subsequently eliminate the lineage of the influential samples. By doing so, we can substantially introduce bias into the prediction pertaining to the target sample, yielding the deliberate manipulation for the user adversely. We extensively evaluate UnAR on five public datasets, and the experimental results indicate our attacks can achieve prediction deviations over 35% by unlearning only 0.5% data as the influential samples.

---

PaperID: 37,   

Authors:  Xiaotong Li, Licheng Jiao, Lingling Li, Fang Liu, Hao Zhu, Xin Zhang, Xu Liu, Shuyuan Yang

Title: Complex Dual-Tree Pyramid Scattering Transformer

Abstract:
Attention-based transformer networks have recently played an increasingly important role in computer vision tasks. However, since pixel-by-pixel attention multiplication does not involve constraint assumptions such as spatial invariance, the computational complexity grows quadratically with the increase of input pixels. Therefore, this article proposes a complex pyramid scattering Transformer in dense scale space, which introduces sparse scattering constraints with a small number of wavelet basis parameters. It enhances the Transformer’s flexibility and sparsity in multiscale space and, to a certain extent, slows down the increase in computational complexity caused by multiresolution input. In addition, compared with the general single-tree real wavelet transform, the dual-tree complex scattering method improves the aliasing of the scattering attention layer and helps obtain a more robust feature representation. At the same time, the multihead stepwise pyramid scattering coupling mechanism helps increase the abundance of directional priors. We conduct experiments in image classification and video tracking scenarios and verify the reliability and superiority of our dual-tree complex pyramid scattering Transformer for visual tasks with different scale requirements. The performance is better than that of the baseline Transformer and other advanced wavelet scattering networks at the same parameter scale. The code is available at https://github.com/Dawn5786/CPSTFormer

---

PaperID: 38,   

Authors:  Chuanbin Zhang, Long Chen, Weiping Ding, Kai Zhao, Zhaoyin Shi, Yingxu Wang, Chun-Lung Philip Chen

Title: Scale-Driven Tensor Representation-Based Multiview Clustering

Abstract:
Real-world data tends to exhibit an inherent hierarchical structure, providing a natural multiview perspective where features at different scales can be treated as distinct views. However, most existing multiview clustering algorithms primarily focus on the inter-sample relationships at a single level. These methods overlook the hierarchical structures present in the data and are specifically designed for native multiview data. This article introduces a comprehensive multiview clustering framework that transforms both typical data and images into a unified multiview feature representation. The framework allows for extracting multiscale features from the raw data and clustering different types of data with the same algorithm. A novel scale-driven pre-processing approach unifies the feature structure across various data types and explores local relationships among samples at multiple scales. Features at larger scales delineate the global cluster contours, while features at smaller scales reveal fine-grained local details. Subsequently, the proposed method learns the view-specific partitions from different scales of views and derives consensus features through tensor low-rank representation. By optimizing these consensus features, the approach effectively captures the precise cluster shapes from coarse to fine-grained levels. The final label indicator matrix is directly obtained from these consensus features. To demonstrate the effectiveness and versatility of the proposed method, we conducted experimental comparisons with state-of-the-art (SOTA) algorithms in both multiview clustering and image segmentation across diverse datasets. The source code and datasets are released at https://github.com/ChuanbinZhang/SDTR

---

PaperID: 39,   

Authors:  Jie Chen, Lingling Li, Licheng Jiao, Fang Liu, Xu Liu, Yuwei Guo, Puhua Chen, Wenping Ma

Title: Heterogeneous Riemannian Few-Shot Learning Network

Abstract:
How to learn and accurately distinguish new concepts from few samples, as humans do, is a long-standing concern in artificial intelligence (AI). Studies in brain science and neuroscience have shown that human brain perception is based on nonlinear manifolds, and high-dimensional manifolds can facilitate concept learning in neural circuits. Based on this inspiration, in this paper, we propose a heterogeneous Riemannian few-shot learning network (HRFL-Net), which is the first few-shot learning method to perform end-to-end deep learning on heterogeneous Riemannian manifolds. Specifically, to enhance the geometric invariance of the image representation, the image features are projected into three heterogeneous Riemannian manifold spaces. Then, the implicit Riemannian kernel function maps the manifolds to the separable high-dimensional reproducing Hilbert space. It is assumed that the embedded kernel features of the complementary manifolds are mapped to the same common subspace. Thus, a novel neural network-based Riemannian metric learning method is designed to solve the subspace feature vectors by imposing orthogonal normalized projection, which overcomes the data extension limitation of the Riemannian metric. Finally, with the optimization objective of increasing the interclass distance and decreasing the intraclass distance in Hilbert space, the HRFL-Net is trained with end-to-end stochastic optimization, and the optimal aggregation subspace is learned during the gradient descent process. Thus, the proposed HRFL-Net can be easily generalized to challenging nonconvex data. The evaluation of four public datasets shows that the proposed HRFL-Net has significant superiority and also achieves competitive results compared with the state-of-the-art methods.

---

PaperID: 40,   

Authors:  Yuebin Xu, C. L. Philip Chen, Mengqi Wu, Tong Zhang

Title: SGB-Net: Scalable Graph Broad Network

Abstract:
Due to the complexity and self-evolutionary property of graph data in reality, graph learning methods require both validity to represent unstructured data and scalability to adapt to evolving graphs. However, current works have representation learning limitations on optimizable graph feature space due to the bottleneck of the structure depth. Moreover, they encounter a complete retraining process when graphs evolve, especially in the case without the assistance of new labels. To address the above issues, we propose a scalable graph broad network (SGB-Net), which contains three proposed modules: the graph feature broad transformation layer (GFBT layer) for enhancing graph embedding and two update algorithms (SGB-Net-U, SGB-Net-S) for endowing scalability. The GFBT layer aims to explicitly expand the graph feature space and broadly build the model. It constructs two expandable feature spaces in various graph scales to embed graphs discriminatively. SGB-Net-U is an exploratory method designed to tackle the label-free graph incremental learning (GIL) problem by leveraging unsupervised incremental knowledge to expand graph representation. SGB-Net-S endows scalability in classical incremental learning scenarios involving labels. Benefiting from its broad construction framework, SGB-Net not only enhances graph embeddings but also seamlessly adapts and improves performance in response to graph expansion without requiring retraining. In the experiments conducted on 15 benchmark datasets, SGB-Net outperforms state-of-the-art GNNs in terms of both effectiveness and scalability.

---

PaperID: 41,   

Authors:  Jingkai Ma, Shuang Bai

Title: Multi-View Part-Based Few-Shot Object Detection

Abstract:
Few-shot object detection (FSOD) aims to detect new object categories with a few annotated samples and has achieved great development. However, compared with general object detection, object misclassification remains a serious issue in FSOD. This is because, in scenarios with few samples, the model struggles to learn sufficient discriminative information to represent the corresponding classes. To address this issue, we propose a multi-view part-based FSOD network (MPFSOD), which extracts more discriminative information from multiple views to generate highly discriminative parts that effectively characterize objects, thereby promoting the accurate detection of new categories. Specifically, we first propose a part-based detector (PBD), where task-aware object-level parts are generated and used to enhance the discriminativeness of the object representations. Based on the PBD, we then introduce an image-level multi-view fusion module (Img-MVF) and an instance-level multi-view modulation module (Inst-MVM). These two modules extract richer discriminative information hidden in multiple views of the target, further facilitating the generation of discriminative parts in PBD. Extensive experiments on PASCAL VOC and MS COCO demonstrate that our method significantly outperforms a strong baseline (up to 11.2%) and previous state-of-the-art methods (4.3% in average).

---

PaperID: 42,   

Authors:  Huiqiang Chen, Tianqing Zhu, Wanlei Zhou, Wei Zhao

Title: AFed: Algorithmic Fair Federated Learning

Abstract:
Federated learning (FL) has gained significant attention as it facilitates collaborative machine learning among multiple clients without centralizing their data on a server. FL ensures the privacy of participating clients by locally storing their data, which creates new challenges in fairness. Traditional debiasing methods assume centralized access to sensitive information, rendering them impractical for the FL setting. Additionally, FL is more susceptible to fairness issues than centralized machine learning due to the diverse client data sources that may be associated with group information. Therefore, training a fair model in FL without access to client local data is important and challenging. This article presents AFed, a straightforward, yet effective framework for promoting group fairness in FL. The core idea is to circumvent restricted data access by learning the global data distribution. This article proposes two approaches: AFed-G, which uses a conditional generator trained on the server side, and AFed-GAN, which improves upon AFed-G by training a conditional GAN on the client side. We augment the client data with the generated samples to help remove bias. Our theoretical analysis justifies the proposed methods, and empirical results on multiple real-world datasets demonstrate a substantial improvement in AFed over several baselines.

---

PaperID: 43,   

Authors:  Puhong Duan, Tianci Shan, Xudong Kang, Shutao Li

Title: Spectral Super-Resolution in Frequency Domain

Abstract:
Spectral super-resolution aims to reconstruct a hyperspectral image (HSI) from its corresponding RGB image, which has drawn much more attention in remote sensing field. Recent advances in the application of deep learning models for spectral super-resolution have demonstrated great potential. However, these methods only work in spectral-spatial domain while rarely explore the potential property in the frequency domain. In this work, we first attempt to address spectral super-resolution in the frequency domain. To well merge the frequency information into the super-resolution network, a spectral-spatial–frequency domain fusion network (SSFDF) is designed, which consists of three key parts: frequency-domain feature learning, spectral-spatial domain feature learning, and feature fusion module. In more detail, a frequency-domain feature learning network is first exploited to dig the frequency-domain information of the input data. Then, a symmetric convolutional neural network (CNN) is developed to acquire the spectral-spatial features of the input data, where a parameter-sharing strategy is utilized to reduce network parameters. Finally, a feature fusion module is proposed to reconstruct HSI. Comprehensive experiments on several datasets reveal that our method can attain state-of-the-art reconstruction result with respect to other spectral super-resolution techniques.

---

PaperID: 44,   

Authors:  Yanbei Liu, Yu Zhao, Zhitao Xiao, Lei Geng, Xiao Wang, Yanwei Pang, Jerry Chun-Wei Lin

Title: Multiscale Subgraph Adversarial Contrastive Learning

Abstract:
Graph contrastive learning (GCL), as a typical self-supervised learning paradigm, has been able to achieve promising performance without labels and gradually attracts much attention. Graph-level method aims to learn representations of each graph by contrasting two augmented graphs. Previous studies usually simply apply contrastive learning to keep the embeddings of augmented views from the same anchor graph (positive pairs) close to each other, as well as separate the embeddings of augmented views from different anchor graphs (negative pairs). However, it is well-known that the structure of graph is always complex and multiscale, which gives rise to a fundamental question: after graph augmentation, will the previous assumption still hold in reality? Through experimental analytics, we find that the semantic information of two augmented graphs from the same anchor graph may be not consistent, and whether two augmented graphs are positive or negative sample pairs is highly correlated with the multiscale structure of the graph. Based on this observation, we then propose a multiscale subgraph contrastive learning method, named MSSGCL, which can characterize the fine-grained semantic information. Specifically, we generate global and local views at different scales based on subgraph sampling and construct multiple contrastive relationships according to their semantic associations to provide richer self-supervised information. Furthermore, to further improve the generalization performance of the model, we propose an extended model called MSSGCL++. It adopts an asymmetric structure to avoid pushing semantically similar negative samples far away. We further introduce adversarial training to perturb the augmented view and thus construct a more difficult self-supervised training task. Finally, a min-max saddle point problem is optimized and the “free” strategy is used to speed up the training process. Extensive experiments and parametric analysis on 16 real-world graph classification datasets confirm the effectiveness of our proposed approach. Compared with state of the art (SOTA) method, our method achieves improvements of 2% and 1.6% in unsupervised and transfer learning settings, respectively.

---

PaperID: 45,   

Authors:  Xingfu Wang, Wenxia Qi, Wenjie Yang, Wei Wang

Title: Cholesky Space for Brain-Computer Interfaces

Abstract:
Brain–computer interfaces (BCIs) based on electroencephalogram (EEG) enable direct interactions between the brain and external environments, with applications in medical rehabilitation, motor substitution, gaming, and entertainment. Traditional methods that model the non-Euclidean characteristics of EEG signals demonstrate robustness and high performance, but they suffer from significant computational costs and are typically restricted to a single BCI paradigm. This article addresses these limitations by utilizing a diffeomorphism from Riemannian manifolds to the Cholesky space, which simplifies the solution process and enables application across multiple BCI paradigms. Our proposed Cholesky space-based model, CSNet, achieves state-of-the-art (SOTA) performance in motor imagery (MI) decoding and emotion recognition and demonstrates competitive performance in error-related negativity (ERN) decoding, all without the need for data preprocessing. Furthermore, our runtime comparison shows that the Cholesky space method is more efficient than the method based on the Riemannian manifold as the matrix dimension increases. To enhance the interpretability of CSNet, we perform t-distributed stochastic neighbor embedding (t-SNE) visualization for MI, frequency band energy visualization for emotion recognition, and temporal importance visualization for ERN. The results indicate that CSNet effectively learns discriminative features, identifies important frequency bands, and focuses on important temporal features. The CSNet effectively captures the non-Euclidean characteristics of EEG signals across various BCI paradigms, while mitigating high computational costs, making it a promising candidate for future BCI algorithms. The code for this study is publicly available at: https://github.com/XingfuWang/CSNet.

---

PaperID: 46,   

Authors:  Nian Wang, Zhigao Cui, Aihua Li, Yihang Lu, Rong Wang, Feiping Nie

Title: Structured Doubly Stochastic Graph-Based Clustering

Abstract:
Graph-based clustering is a hot topic in machine learning, whose effectiveness highly relies on the quality of the learned graph. Recent researches preferred to learn the nearest doubly stochastic approximation of a graph to suppress intercluster connections and enhance intracluster connections and thus improve clustering performance. While current paradigm is limited by three key problems: 1) it is restricted by a predefined graph; 2) the separated stages of spectral decomposition-based way (graph learning, spectral embedding learning, and cluster assignment by k-means) cause mismatched problems and randomness; and 3) the optimization of doubly stochastic conditions is generally achieved by von Neumann successive projection (VNSP) lemma, which separates the conditions to form two subproblems for alternative optimization, converging only to a feasible solution. To solve these problems, in this article, a novel structured doubly stochastic graph-based clustering model termed SDSGC is proposed, which learns a structured doubly stochastic graph from data to directly provide cluster indicators. For optimization, a simple but effective augmented Lagrangian multiplier (ALM)-based method is proposed, which optimizes all the doubly stochastic conditions simultaneously to obtain the optimal solution. Experiments on one toy dataset and eight ad hoc noised face datasets have demonstrated that the proposed SDSGC is more robust to noise. Furthermore, a quantitative comparison of ten benchmarks has verified our SDSGC achieves better clustering performance when compared with SOTA methods. The code is available at https://github.com/NianWang-HJJGCDX/SDSGC.git.

---

PaperID: 47,   

Authors:  Luca Savant Aira, Diego Valsesia, Enrico Magli

Title: Modeling Uncertainty for Gaussian Splatting

Abstract:
We present stochastic Gaussian splatting (SGS): the first framework for uncertainty estimation using Gaussian splatting (GS). GS recently advanced the novel-view synthesis field by achieving impressive reconstruction quality at a fraction of the computational cost of neural radiance fields (NeRFs). However, contrary to the latter, it still lacks the ability to provide information about the confidence associated with their outputs. To address this limitation, in this brief, we introduce a variational inference (VI)-based approach that seamlessly integrates uncertainty prediction into the common rendering pipeline of GS. In addition, we introduce the area under sparsification error (AUSE) as a new term in the loss function, enabling optimization of uncertainty estimation alongside image reconstruction. Experimental results on the three different datasets demonstrate that our method outperforms existing approaches in terms of both image rendering quality and uncertainty estimation accuracy. Overall, our framework equips practitioners with valuable insights into the reliability of synthesized views, facilitating safer decision-making in real-world applications.

---

PaperID: 48,   

Authors:  Yazhou Ren, Jingyu Pu, Zhimeng Yang, Jie Xu, Guofeng Li, Xiaorong Pu, Philip S. Yu, Lifang He

Title: Deep Clustering: A Comprehensive Survey

Abstract:
Cluster analysis plays an indispensable role in machine learning and data mining. Learning a good data representation is crucial for clustering algorithms. Recently, deep clustering (DC), which can learn clustering-friendly representations using deep neural networks (DNNs), has been broadly applied in a wide range of clustering tasks. Existing surveys for DC mainly focus on the single-view fields and the network architectures, ignoring the complex application scenarios of clustering. To address this issue, in this article, we provide a comprehensive survey for DC in views of data sources. With different data sources, we systematically distinguish the clustering methods in terms of methodology, prior knowledge, and architecture. Concretely, DC methods are introduced according to four categories, i.e., traditional single-view DC, semi-supervised DC, deep multiview clustering (MVC), and deep transfer clustering. Finally, we discuss the open challenges and potential future opportunities in different fields of DC.

---

PaperID: 49,   

Authors:  Jing Sun, Shuo Chen, Cong Zhang, Yining Ma, Jie Zhang

Title: Decision-Making With Speculative Opponent Models

Abstract:
Opponent modeling has proven effective in enhancing the decision-making of the controlled agent by constructing models of opponent agents. However, existing methods often rely on access to the observations and actions of opponents, a requirement that is infeasible when such information is either unobservable or challenging to obtain. To address this issue, we introduce distributional opponent-aided multiagent actor–critic (DOMAC), the first speculative opponent modeling algorithm that relies solely on local information (i.e., the controlled agent’s observations, actions, and rewards). Specifically, the actor maintains a speculated belief about the opponents using the tailored speculative opponent models that predict the opponents’ actions using only local information. Moreover, DOMAC features distributional critic models that estimate the return distribution of the actor’s policy, yielding a more fine-grained assessment of the actor’s quality. This thus more effectively guides the training of the speculative opponent models that the actor depends upon. Furthermore, we formally derive a policy gradient theorem with the proposed opponent models. Extensive experiments under eight different challenging multiagent benchmark tasks within the MPE, Pommerman, and starcraft multiagent challenge (SMAC) demonstrate that our DOMAC successfully models opponents’ behaviors and delivers superior performance against state-of-the-art (SOTA) methods with a faster convergence speed.

---

PaperID: 50,   

Authors:  Zheng Wang, Zhenwei Gao, Yang Yang, Guoqing Wang, Chengbo Jiao, Heng Tao Shen

Title: Geometric Matching for Cross-Modal Retrieval

Abstract:
Despite its significant progress, cross-modal retrieval still suffers from one-to-many matching cases, where the multiplicity of semantic instances in another modality could be acquired by a given query. However, existing approaches usually map heterogeneous data into the learned space as deterministic point vectors. In spite of their remarkable performance in matching the most similar instance, such deterministic point embedding suffers from the insufficient representation of rich semantics in one-to-many correspondence. To address the limitations, we intuitively extend a deterministic point into a closed geometry and develop geometric representation learning methods for cross-modal retrieval. Thus, a set of points inside such a geometry could be semantically related to many candidates, and we could effectively capture the semantic uncertainty. We then introduce two types of geometric matching for one-to-many correspondence, i.e., point-to-rectangle matching (dubbed P2RM) and rectangle-to-rectangle matching (termed R2RM). The former treats all retrieved candidates as rectangles with zero volume (equivalent to points) and the query as a box, while the latter encodes all heterogeneous data into rectangles. Therefore, we could evaluate semantic similarity among heterogeneous data by the Euclidean distance from a point to a rectangle or the volume of intersection between two rectangles. Additionally, both strategies could be easily employed for off-the-self approaches and further improve the retrieval performance of baselines. Under various evaluation metrics, extensive experiments and ablation studies on several commonly used datasets, two for image-text matching and two for video-text retrieval, demonstrate our effectiveness and superiority.

---

PaperID: 51,   

Authors:  Yongsik Jin, Sang-Moon Lee

Title: Sampled-Data State Estimation for LSTM

Abstract:
This article first introduces a sampled-data state estimator design method for continuous-time long short-term memory (LSTM) neural networks with irregularly sampled output. To this end, the structure of the LSTM is addressed to obtain its dynamic equation. As a result, the LSTM neural network is modeled as a continuous-time linear parameter-varying system that is dependent on the gate units. For this system, the sampled-data Luenberger- and Arcak-type state estimator design methods are presented in terms of linear matrix inequalities (LMIs) by using the properties of the gate units. Lastly, the proposed method not only provides a numerical example for analyzing absolute stability but also demonstrates it in practice by applying a pre-trained behavior generation model of a robot manipulator.

---

PaperID: 52,   

Authors:  Kun Hu, Yingyuan Xiao, Wenguang Zheng, Wenxin Zhu, Ching-Hsien Hsu

Title: Multiview Large Margin Distribution Machine

Abstract:
Margin distribution has been proven to play a crucial role in improving generalization ability. In recent studies, many methods are designed using large margin distribution machine (LDM), which combines margin distribution with support vector machine (SVM), such that a better performance can be achieved. However, these methods are usually proposed based on single-view data and ignore the connection between different views. In this article, we propose a new multiview margin distribution model, called MVLDM, which constructs both multiview margin mean and variance. Besides, a framework is proposed to achieve multiview learning (MVL). MVLDM provides a new way to explore the utilization of complementary information in MVL from the perspective of margin distribution and satisfies both the consistency principle and the complementarity principle. In the theoretical analysis, we used Rademacher complexity theory to analyze the consistency error bound and generalization error bound of the MVLDM. In the experiments, we constructed a new performance metric, the view consistency rate (VCR), for the characteristics of multiview data. The effectiveness of MVLDM was evaluated using both VCR and other traditional performance metrics. The experimental results show that MVLDM is superior to other benchmark methods.

---

PaperID: 53,   

Authors:  Yang Tan, Enming Zhang, Yang Li, Shao-Lun Huang, Xiao-Ping Zhang

Title: Transferability-Guided Cross-Domain Cross-Task Transfer Learning

Abstract:
We propose two novel transferability metrics fast optimal transport-based conditional entropy (F-OTCE) and joint correspondence OTCE (JC-OTCE) to evaluate how much the source model (task) can benefit the learning of the target task and to learn more generalizable representations for cross-domain cross-task transfer learning. Unlike the original OTCE metric that requires evaluating the empirical transferability on auxiliary tasks, our metrics are auxiliary-free such that they can be computed much more efficiently. Specifically, F-OTCE estimates transferability by first solving an optimal transport (OT) problem between source and target distributions and then uses the optimal coupling to compute the negative conditional entropy (NCE) between the source and target labels. It can also serve as an objective function to enhance downstream transfer learning tasks including model finetuning and domain generalization (DG). Meanwhile, JC-OTCE improves the transferability accuracy of F-OTCE by including label distances in the OT problem, though it incurs additional computation costs. Extensive experiments demonstrate that F-OTCE and JC-OTCE outperform state-of-the-art auxiliary-free metrics by 21.1% and 25.8%, respectively, in correlation coefficient with the ground-truth transfer accuracy. By eliminating the training cost of auxiliary tasks, the two metrics reduce the total computation time of the previous method from 43 min to 9.32 and 10.78 s, respectively, for a pair of tasks. When applied in the model finetuning and DG tasks, F-OTCE shows significant improvements in the transfer accuracy in few-shot classification experiments, with up to 4.41% and 2.34% accuracy gains, respectively.

---

PaperID: 54,   

Authors:  Yu Duan, Zhoumin Lu, Rong Wang, Xuelong Li, Feiping Nie

Title: Toward Balance Deep Semisupervised Clustering

Abstract:
The goal of balanced clustering is partitioning data into distinct groups of equal size. Previous studies have attempted to address this problem by designing balanced regularizers or utilizing conventional clustering methods. However, these methods often rely solely on classic methods, which limits their performance and primarily focuses on low-dimensional data. Although neural networks exhibit effective performance on high-dimensional datasets, they struggle to effectively leverage prior knowledge for clustering with a balanced tendency. To overcome the above limitations, we propose deep semisupervised balanced clustering, which simultaneously learns clustering and generates balance-favorable representations. Our model is based on the autoencoder paradigm incorporating a semisupervised module. Specifically, we introduce a balance-oriented clustering loss and incorporate pairwise constraints into the penalty term as a pluggable module using the Lagrangian multiplier method. Theoretically, we ensure that the proposed model maintains a balanced orientation and provides a comprehensive optimization process. Empirically, we conducted extensive experiments on four datasets to demonstrate significant improvements in clustering performance and balanced measurements. Our code is available at https://github.com/DuannYu/BalancedSemi-TNNLS.

---

PaperID: 55,   

Authors:  Ekanut Sotthiwat, Liangli Zhen, Chi Zhang, Zengxiang Li, Rick Siow Mong Goh

Title: Generative Image Reconstruction From Gradients

Abstract:
In this article, we propose a method, generative image reconstruction from gradients (GIRG), for recovering training images from gradients in a federated learning (FL) setting, where privacy is preserved by sharing model weights and gradients rather than raw training data. Previous studies have shown the potential for revealing clients’ private information or even pixel-level recovery of training images from shared gradients. However, existing methods are limited to low-resolution images and small batch sizes (BSs) or require prior knowledge about the client data. GIRG utilizes a conditional generative model to reconstruct training images and their corresponding labels from the shared gradients. Unlike previous generative model-based methods, GIRG does not require prior knowledge of the training data. Furthermore, GIRG optimizes the weights of the conditional generative model to generate highly accurate “dummy” images instead of optimizing the input vectors of the generative model. Comprehensive empirical results show that GIRG is able to recover high-resolution images with large BSs and can even recover images from the aggregation of gradients from multiple participants. These results reveal the vulnerability of current FL practices and call for immediate efforts to prevent inversion attacks in gradient-sharing-based collaborative training.

---

PaperID: 56,   

Authors:  Shifei Ding, Chao Li, Xiao Xu, Lili Guo, Ling Ding, Xindong Wu

Title: Horizontal Federated Density Peaks Clustering

Abstract:
Density peaks clustering (DPC) is a popular clustering algorithm, which has been studied and favored by many scholars because of its simplicity, fewer parameters, and no iteration. However, in previous improvements of DPC, the issue of privacy data leakage was not considered, and the “Domino” effect caused by the misallocation of noncenters has not been effectively addressed. In view of the above shortcomings, a horizontal federated DPC (HFDPC) is proposed. First, HFDPC introduces the idea of horizontal federated learning and proposes a protection mechanism for client parameter transmission. Second, DPC is improved by using similar density chain (SDC) to alleviate the “Domino” effect caused by multiple local peaks in the flow pattern dataset. Finally, a novel data dimension reduction and image encryption are used to improve the effectiveness of data partitioning. The experimental results show that compared with DPC and some of its improvements, HFDPC has a certain degree of improvement in accuracy and speed.

---

PaperID: 57,   

Authors:  Renyao Chen, Junye Lei, Hong Yao, Tailong Li, Shengwen Li

Title: Anchor-Enhanced Geographical Entity Representation Learning

Abstract:
Geographical entity representation learning (GERL) aims to embed geographical entities into a low-dimensional vector space, which provides a generalized approach for utilizing geographical entities to serve various geographical intelligence applications. In practice, the spatial distribution of geographical entities is highly unbalanced; thus, it is challenging to embed them accurately. Previous GERL models treated all geographical entities uniformly, resulting in insufficient entity representations. To address this issue, this article proposes an anchor-enhanced GERL (AE-GERL) model, which utilizes the key informative entities as anchors to improve the representations of geographical entities. Specifically, AE-GERL develops an anchor selection algorithm to identify anchors from large-scale geographical entities based on their spatial distribution and entity types. To utilize anchors to guide geographical entities, AE-GERL constructs an anchor-enhanced graph to establish explicit connections between anchors and nonanchor entities. Finally, a graph neural network (GNN) based anchor to nonanchor node learning model is designed to impute missing information of nonanchor entities. Extensive experiments are conducted on four datasets, and the experimental results demonstrate that AE-GERL outperforms the baseline models in both sparse and dense scenarios. This study provides a methodological reference for embedding geographical entities in various geographical applications and also provides an effective approach to improve the performance of message-passing-based GNN models.

---

PaperID: 58,   

Authors:  Mohammad Askarizadeh, Alireza Morsali, Kim Khoa Nguyen

Title: Resource-Constrained Multisource Instance-Based Transfer Learning

Abstract:
In today’s machine learning (ML), the need for vast amounts of training data has become a significant challenge. Transfer learning (TL) offers a promising solution by leveraging knowledge across different domains/tasks, effectively addressing data scarcity. However, TL encounters computational and communication challenges in resource-constrained scenarios, and negative transfer (NT) can arise from specific data distributions. This article presents a novel focus on maximizing the accuracy of instance-based TL in multisource resource-constrained environments while mitigating NT, a key concern in TL. Previous studies have overlooked the impact of resource consumption in addressing the NT problem. To address these challenges, we introduce an optimization model named multisource resource-constrained optimized TL (MSOPTL), which employs a convex combination of empirical sources and target errors while considering feasibility and resource constraints. Moreover, we enhance one of the generalization error upper bounds in domain adaptation setting by demonstrating the potential to substitute the H \Delta H divergence with the Kullback–Leibler (KL) divergence. We utilize this enhanced error upper bound as one of the feasibility constraints of MSOPTL. Our suggested model can be applied as a versatile framework for various ML methods. Our approach is extensively validated in a neural network (NN)-based classification problem, demonstrating the efficiency of MSOPTL in achieving the desired trade-offs between TL’s benefits and associated costs. This advancement holds tremendous potential for enhancing edge artificial intelligence (AI) applications in resource-constrained environments.

---

PaperID: 59,   

Authors:  Chuan Ma, Yingwei Zhang, Chun-Yi Su

Title: Graph-Based Multicentroid Nonnegative Matrix Factorization

Abstract:
Nonnegative matrix factorization (NMF) is a widely recognized approach for data representation. When it comes to clustering, NMF fails to handle data points located in complex geometries, as each sample cluster is represented by a centroid. In this article, a novel multicentroid-based clustering method called graph-based multicentroid NMF (MCNMF) is proposed. Because the method constructs the neighborhood connection graph between data points and centroids, each data point is represented by adjacent centroids, which preserves the local geometric structure. Second, because the method constructs an undirected connected graph with centroids as nodes, in which the centroids are divided into different centroid clusters, a novel data clustering method based on MCNMF is proposed. In addition, the membership index matrix is reconstructed based on the obtained centroid clusters, which solves the problem of membership identification of the final sample. Extensive experiments conducted on synthetic datasets and real benchmark datasets illustrate the effectiveness of the proposed MCNMF method. Compared with single-centroid-based methods, the MCNMF can obtain the best experimental results.

---

PaperID: 60,   

Authors:  Chunjiang Ge, Rui Huang, Mixue Xie, Zihang Lai, Shiji Song, Shuang Li, Gao Huang

Title: Domain Adaptation via Prompt Learning

Abstract:
Unsupervised domain adaptation (UDA) aims to adapt models learned from a well-annotated source domain to a target domain, where only unlabeled samples are given. Current UDA approaches learn domain-invariant features by aligning source and target feature spaces through statistical discrepancy minimization or adversarial training. However, these constraints could lead to the distortion of semantic feature structures and loss of class discriminability. In this article, we introduce a novel prompt learning paradigm for UDA, named domain adaptation via prompt learning (DAPrompt). In contrast to prior works, our approach learns the underlying label distribution for target domain rather than aligning domains. The main idea is to embed domain information into prompts, a form of representation generated from natural language, which is then used to perform classification. This domain information is shared only by images from the same domain, thereby dynamically adapting the classifier according to each domain. By adopting this paradigm, we show that our model not only outperforms previous methods on several cross-domain benchmarks but also is very efficient to train and easy to implement.

---

PaperID: 61,   

Authors:  Zhiqiang Kou, Jing Wang, Yuheng Jia, Biao Liu, Xin Geng

Title: Instance-Dependent Inaccurate Label Distribution Learning

Abstract:
Label distribution learning (LDL) is a novel learning paradigm that assigns each instance with a label distribution. Although many specialized LDL algorithms have been proposed, few of them have noticed that the obtained label distributions are generally inaccurate with noise due to the difficulty of annotation. Besides, existing LDL algorithms overlooked that the noise in the inaccurate label distributions generally depends on instances. In this article, we identify the instance-dependent inaccurate LDL (IDI-LDL) problem and propose a novel algorithm called low-rank and sparse LDL (LRS-LDL). First, we assume that the inaccurate label distribution consists of the ground-truth label distribution and instance-dependent noise. Then, we learn a low-rank linear mapping from instances to the ground-truth label distributions and a sparse mapping from instances to the instance-dependent noise. In the theoretical analysis, we establish a generalization bound for LRS-LDL. Finally, in the experiments, we demonstrate that LRS-LDL can effectively address the IDI-LDL problem and outperform existing LDL methods.

---

PaperID: 62,   

Authors:  Mehran Mazandarani, Jianfei Pan

Title: The Q-Fractionalism Reasoning Learning Method

Abstract:
As the title suggests, in this work, a modern machine learning method called the Q-fractionalism reasoning is introduced. The proposed method is founded upon a synergy of the Q-learning and fractional fuzzy inference systems (FFISs). Unlike other approaches, the Q-fractionalism reasoning not only incorporates the knowledge base to understand how to perform but also explores a reasoning mechanism from the fractional order to justify what it has performed. This method suggests that the agent choose actions aimed at the characterization of reasoning. In fact, the agent deals with states termed as primary and secondary fuzzy states. The primary fuzzy states are unobservable and uncertain, for which the agent chooses actions. However, the projection of primary fuzzy states onto the knowledge base results in secondary fuzzy states, which are observable by the agent, allowing it to detect primary fuzzy states with degrees of detectability. With a practical experiment implemented on a linear switched reluctance motor (LSRM), the results demonstrate that the application of the Q-fractionalism reasoning in the real-time position control of the LSRM leads to a remarkable improvement of about 70% in the accuracy of the control objective compared with a typical fuzzy inference system (FIS) under the same setting.

---

PaperID: 63,   

Authors:  Yen-Lung Lai, Zhe Jin

Title: Wormhole Dynamics in Deep Neural Networks

Abstract:
This work investigates the generalization behavior of deep neural networks (DNNs), focusing on the phenomenon of “fooling examples,” where DNNs confidently classify inputs that appear random or unstructured to humans. To explore this phenomenon, we introduce an analytical framework based on maximum likelihood estimation (MLE), without adhering to conventional numerical approaches that rely on gradient-based optimization and explicit labels. Our analysis reveals that DNNs operating in an overparameterized regime exhibit a collapse in the output feature space. While this collapse improves network generalization, adding more layers eventually leads to a state of degeneracy, where the model learns trivial solutions by mapping distinct inputs to the same output, resulting in zero loss. Further investigation demonstrates that this degeneracy can be bypassed using our newly derived “wormhole” solution. The wormhole solution, when applied to arbitrary fooling examples, reconciles meaningful labels with random ones and provides a novel perspective on shortcut learning. These findings offer deeper insights into DNN generalization and highlight directions for future research on learning dynamics in unsupervised settings to bridge the gap between theory and practice.

---

PaperID: 64,   

Authors:  Yaxuan Hu, Jie Hua, Zhen Han, Hua Zou, Gang Wu, Zhongyuan Wang

Title: DiffusionMOT: A Diffusion-Based Multiple Object Tracker

Abstract:
Recently, researchers have introduced diffusion models into multiple object tracking (MOT) tasks. However, existing diffusion-based MOT methods, such as DiffusionTrack, have significant limitations, including frequent ID switching, reduced performance when tracking nonlinear motion objects, and long inference time. To this end, we propose a more effective diffusion-based multiple object tracker named DiffusionMOT. In particular, we propose a mixed intersection over union (IoU) and Re-Identification (ReID) method for trajectory matching, which effectively reduces incorrect matches. Meanwhile, we propose a secondary calibration method for trajectory boxes, improving the accuracy of the generated detection boxes. Moreover, we introduce the parallel sampling technique from the field of image generation into object tracking and propose a parallel sampling module to enhance the model’s inference speed while maintaining tracking accuracy. Furthermore, we design a pair-based two-stage matching (PTM) pipeline to more effectively utilize potential detection information. Extensive experiments on several public MOT benchmarks, including DanceTrack, SportsMOT, MOT20, and MOT17, demonstrate that our approach achieves state-of-the-art (SOTA) performance. The code and models are available at https://github.com/sad123-yx/DiffusionMOT

---

PaperID: 65,   

Authors:  Xinyu Su, Xiwen Wang, Dezhong Peng, Xiaomin Song, Huiming Zheng, Zhong Yuan

Title: Identifying Outliers via Local Granular-Ball Density

Abstract:
Existing density-based outlier detection methods process data at the single-granularity level of individual samples, requiring pairwise distance calculations between all samples and exhibiting high sensitivity to noise. The single-granularity-based processing paradigm fails to mine the information at multiple levels of granularity in data, and most of these methods ignore the potential uncertainty information in data, such as fuzziness, resulting in an inability to effectively detect potential outliers in data. As a novel granular computing method, Granular-Ball Computing (GBC) is characterized by its multi-granularity and robustness, which makes it able to make up for the above drawbacks well. In this study, we propose local Granular-Ball Density-based Outlier (GBDO) detection to improve the performance of the density-based methods. In GBDO, we first identify the k\text - similarity Granular-Ball (GB) neighborhoods of each GB via the fuzzy relations among them. Subsequently, the local reachability similarity density of the GBs is calculated through the reachability similarity we defined. Finally, the local GB outlier factors of the samples are calculated based on the local reachability similarity density of the GBs. We adopt a multi-granularity processing paradigm using GBs as the basic units, which reduces computational complexity and improves robustness to noisy data by leveraging the multi-granularity nature of GBs. The experimental results demonstrate the effectiveness of GBDO by comparing it with state-of-the-art methods. The source code and datasets are publicly available at https://github.com/Mxeron/GBDO.

---

PaperID: 66,   

Authors:  Rudolf J. Szadkowski, Jan Faigl

Title: Lifelong Active Inference of Gait Control

Abstract:
Sustaining the robot’s longevity becomes challenging in dynamic deployments characterized by new unknown environments and embodiments outside of the prior knowledge. Hence, the knowledge of robot-environment interactions needs to be continually updated for system adaptation. It can be implemented through self-verification as a continual comparison of predictions with observations using the predictive coding (PC) principle. The principle has been further extended into the active inference control (AIC) in biomimetic robotics to drive the control, state estimation, and model update. However, continually updating one model leads to catastrophic forgetting in the long term. Therefore, we propose an autonomously expanding self-verifying world model (WM) of sensorimotor dynamics utilized in model-based gait control. The model combines PC with the incremental knowledge representation based on the internal model (IM) principle. The proposed method is experimentally validated in virtual and real scenarios, where the hexapod walking robot has to recognize and adapt to leg paralysis and then recognize the recovery. The method generates novel behaviors in real time, improving the performance and outperforming the examined state-of-the-art methods. Furthermore, the robot’s decisions and gained knowledge are interpretable and promise further functional scalability.

---

PaperID: 67,   

Authors:  Jiaping Xiao, Rangya Zhang, Yuhang Zhang, Mir Feroskhan

Title: Vision-Based Learning for Drones: A Survey

Abstract:
Drones, as advanced cyber-physical systems (CPSs), are undergoing a transformative shift with the advent of vision-based learning, a field that is rapidly gaining prominence due to its profound impact on drone autonomy and functionality. Unlike existing task-specific surveys, this work offers a comprehensive overview of vision-based learning for drones, emphasizing its pivotal role in enhancing their operational capabilities across various scenarios. First, the fundamental principles of vision-based learning are elucidated, demonstrating how it significantly improves drones’ visual perception and decision-making processes. Vision-based control methods are then categorized into indirect, semidirect, and end-to-end approaches from the perception-control perspective. Various applications of vision-based drones with learning capabilities are further explored, ranging from single-agent systems to more complex multiagent and heterogeneous system scenarios, while highlighting the challenges and innovations characterizing each domain. Finally, open questions and potential solutions are discussed to guide future research and development in this dynamic and rapidly evolving field. With the growth of large language models (LLMs) and embodied intelligence, vision-based learning for drones provides a promising yet challenging road toward achieving artificial general intelligence (AGI) in the 3-D physical world.

---

PaperID: 68,   

Authors:  Runhao Li, Yongming Chen, Zhenyu Weng, Zhiping Lin, Yap-Peng Tan

Title: Class-Specific Prompt Learning for Vision-Language Models

Abstract:
The use of learning prompts to adapt pretrained vision–language models (VLMs) for downstream tasks has gained significant attention due to its potential to reduce training costs compared to model fine-tuning through few-shot learning. Most existing methods rely on a universal prompt for all classes, as it generally delivers consistent performance across various datasets. However, a universal prompt cannot capture class-specific discriminative information. To overcome this limitation, we propose class-specific prompt learning (CPL). CPL represents the context of a prompt using two components: a base vector shared among all classes and a class-specific vector designed for individual classes. This method combines the generalization ability of the base context with the adaptability of the class-specific context. Furthermore, we introduce contrastive CPL, which enhances the ability of the prompt to capture discriminative features unique to each class. Also, we adopt the self-consistency loss to regularize the base context, enhancing its generalization ability. As a result, CPL effectively learns tailored prompts for each class. Extensive experiments demonstrate that CPL achieves superior performance over existing methods in both base-class classification and new class generalization.

---

PaperID: 69,   

Authors:  Shuo Chen, Chen Gong, Jun Li, Jian Yang

Title: Similarity-Agnostic Contrastive Learning With Alterable Self-Supervision

Abstract:
Self-supervised contrastive learning (CL) seeks to learn generalizable feature representations via the self-supervision of pairwise similarities, where existing CL approaches usually build definite similarity labels (e.g., positive or negative) for model training. Yet in practice, the same pair of instances may have opposite similarity labels in different scenarios, e.g., two interclass images from CIFAR-100 can be a similar pair in CIFAR-20. Learning with definite similarities can hardly obtain an ideal representation that simultaneously characterizes the similar and dissimilar patterns (e.g., the contexts and details) between each two instances. Therefore, pairwise similarities used for CL should be agnostic, and we argue that simultaneously considering both the similarity and dissimilarity for each data pair could learn more generalizable representations. To this end, we propose similarity-agnostic CL (SACL), which generalizes the instance discrimination strategy of conventional CL to a new multiobjective programming (MOP) form. In SACL, we build multiple projection layers with corresponding regularizers to constrain the distance matrix to have different sparsity in different objectives so that we can obtain alterable pairwise distances to capture both the similarity and dissimilarity between each pair of instances. We show that SACL can be equivalently converted to a single learning objective, easily solved by stochastic optimization with convergence guarantees. Theoretically, we prove a tighter error bound than conventional CL approaches; empirically, our method improves the downstream task performance for image, text, and graph data.

---

PaperID: 70,   

Authors:  Yongcan Luo, Jiahao Zheng, Zhengjie Yang, Ning Chen, Dapeng Oliver Wu

Title: Pleno-Alignment Framework for Stock Trend Prediction

Abstract:
Predicting stock trends is a highly rewarding but high-risk endeavor due to the complex interplay of market dynamics, irrational behaviors, and diverse sentiments. Previous studies have used time-series analysis on historical prices or sentiment analysis on textual information. However, these methods often fail to capture the dynamic interactions between text and time-series modalities and overlook the different perspectives embedded in textual data. To address these limitations, we propose the pleno-alignment framework (PAFrame) that enhances multimodal stock information through intermodal and intramodal alignment to capture market dynamics. Our framework first integrates textual and time-series data in a shared representation space to learn modal-invariant information. To tackle divergent sentiments in textual data, we employ a contrastive learning approach to extract abstract semantic meanings from objective and subjective perspectives, thereby improving the robustness of language representations. Finally, we use a hybrid approach that explicitly combines cross-attention mechanisms to create a unified representation and utilizes prompts to implicitly guide language models with numerical financial indicators for final prediction. Our comprehensive experiments on five real-world datasets show that PAFrame outperforms existing methods in predicting stock trends.

---

PaperID: 71,   

Authors:  Xiaoye Chen, Wensheng Gan, Zefeng Chen, Jian Zhu, Ruichu Cai, Philip S. Yu

Title: Toward Targeted Mining of RFM Patterns

Abstract:
In today’s era of information overload, leveraging data mining techniques to understand and analyze customer behavior has become essential for businesses. Among these techniques, the recency, frequency, and monetary value analysis model serves as a powerful tool for customer segmentation, enabling companies to identify high-value customers. However, traditional recency, frequency, and monetary (RFM) models do not focus on user-specific targets, often struggling to meet the increasing demands for personalization and efficiency. To address this challenge, this article introduces the concept of target RFM patterns, which must satisfy the three dimensions of recency, frequency, and utility while aligning with user interests. Based on this concept, we formulate the problem of mining target RFM patterns. More importantly, we define a mining order, called TaRFM order, and propose an efficient algorithm called TaRFM. This new algorithm is optimized through three pruning strategies based on the TaRFM order, which not only eliminates a significant number of invalid operations, thereby reducing pattern generation, but also accurately extracts all TaRFM patterns without requiring postprocessing techniques. Finally, extensive experiments conducted on multiple datasets demonstrate the accuracy and efficiency of the TaRFM algorithm.

---

PaperID: 72,   

Authors:  Yuning Cui, Mingyu Liu, Wenqi Ren, Alois Knoll

Title: Modumer: Modulating Transformer for Image Restoration

Abstract:
Image restoration aims to recover clean images from degraded counterparts. While Transformer-based approaches have achieved significant advancements in this field, they are limited by high complexity and their inability to capture omni-range dependencies, hindering their overall performance. In this work, we develop Modumer for effective and efficient image restoration by revisiting the Transformer block and modulation design, which processes input through a convolutional block and projection layers and fuses features via elementwise multiplication. Specifically, within each unit of Modumer, we integrate the cascaded modulation design with the downsampled Transformer block to build the attention layers, enabling omni-kernel modulation and mapping inputs into high-dimensional feature spaces. Moreover, we introduce a bioinspired parameter-sharing mechanism to attention layers, which not only enhances efficiency but also improves performance. In addition, a dual-domain feed-forward network (DFFN) strengthens the representational power of the model. Extensive experimental evaluations demonstrate that the proposed Modumer achieves state-of-the-art performance across ten datasets in five single-degradation image restoration tasks, including image motion deblurring, deraining, dehazing, desnowing, and low-light enhancement. Moreover, the model exhibits strong generalization capabilities in all-in-one image restoration tasks. Additionally, it demonstrates competitive performance in composite-degradation image restoration.

---

PaperID: 73,   

Authors:  Chongsheng Zhang, George Almpanidis, Gaojuan Fan, Binquan Deng, Yanbo Zhang, Ji Liu, Aouaidjia Kamel, Paolo Soda, João Gama

Title: A Systematic Review on Long-Tailed Learning

Abstract:
Long-tailed data are a special type of multiclass imbalanced data with a very large amount of minority/tail classes that have a very significant combined influence. Long-tailed learning (LTL) aims to build high-performance models on datasets with long-tailed distributions that can identify all the classes with high accuracy, in particular the minority/tail classes. It is a cutting-edge research direction that has attracted a remarkable amount of research effort in the past few years. In this article, we present a comprehensive survey of the latest advances in long-tailed visual learning. We first propose a new taxonomy for LTL, which consists of eight different dimensions, including data balancing, neural architecture, feature enrichment, logits adjustment, loss function, bells and whistles, network optimization, and posthoc processing techniques. Based on our proposed taxonomy, we present a systematic review of LTL methods, discussing their commonalities and alignable differences. We also analyze the differences between imbalance learning and LTL. Finally, we discuss prospects and future directions in this field.

---

PaperID: 74,   

Authors:  Renhao Huang, Hao Xue, Maurice Pagnucco, Flora D. Salim, Yang Song

Title: Vision-Based Multi-Future Trajectory Prediction: A Survey

Abstract:
Vision-based trajectory prediction is an important task that supports safe and intelligent behaviors in autonomous systems. Many advanced approaches have been proposed over the years with improved spatial and temporal feature extraction. However, human behavior is naturally diverse and uncertain. Given the past trajectory and surrounding environment information, an agent can have multiple plausible trajectories in the future. To tackle this problem, an essential task named multi-future trajectory prediction (MTP) has recently been studied. This task aims to generate a diverse, acceptable, and explainable distribution of future predictions for each agent. In this article, we present the first survey for MTP with our unique taxonomies and a comprehensive analysis of frameworks, datasets, and evaluation metrics. We also compare models on existing MTP datasets and conduct experiments on the ForkingPath dataset. Finally, we discuss multiple future directions that can help researchers develop novel MTP systems and other diverse learning tasks similar to MTP.

---

PaperID: 75,   

Authors:  Yabo Wang, Bo Qi, Xin Wang, Tongliang Liu, Daoyi Dong

Title: Power Characterization of Noisy Quantum Kernels

Abstract:
Quantum kernel methods have been widely recognized as one of the promising quantum machine learning (QML) algorithms that have the potential to achieve quantum advantages. However, their capabilities may be severely degraded by inevitable noises in the current noisy intermediate-scale quantum (NISQ) era. In this article, we theoretically characterize the power of noisy quantum kernels and demonstrate that under depolarizing noise, quantum kernel methods may only have very poor prediction capability, even when the generalization error is small. Specifically, we quantitatively describe the decreasing of the prediction capability of noisy quantum kernels in terms of the rate of quantum noise, the size of training samples, the number of qubits, and the number of layers affected by quantum noises. Our results clearly demonstrate that for a given number of training samples, once the number of layers affected by noise exceeds some threshold, the prediction capability of noisy kernels is very poor. Thus, we provide a crucial warning to employ noisy quantum kernel methods for quantum computation and the theoretical results can also serve as guidelines when developing practical quantum kernel algorithms for achieving quantum advantages.

---

PaperID: 76,   

Authors:  Liang Gao, Li Li, Yingwen Chen, Shaojing Fu, Dongsheng Wang, Siwei Wang, Cheng-Zhong Xu, Ming Xu

Title: Noise-Robust Federated Learning via Interclient Co-Distillation

Abstract:
Federated learning (FL) is a new learning paradigm that enables multiple clients to collaboratively train a high-performance model while preserving user privacy. However, the effectiveness of FL heavily relies on the availability of accurately labeled data, which can be challenging to obtain in real-world scenarios. To address this issue and robustly train shared models using distributed noisy labeled data, we propose FedDQ, a noise-robust FL framework that utilizes co-distillation and quality-aware aggregation techniques. FedDQ incorporates two key features: a noise-adaptive training strategy and an efficient label-correcting mechanism. The noise-adaptive training strategy relies on the estimation of labels’ noise levels to dynamically adjust clients’ training engagement, which mitigates the impact of wrong labels while efficiently exploring features from clean data. In addition, FedDQ designs a two-head network and employs it for co-distillation. The co-distillation strategy facilitates knowledge transfer among clients to share the representational capabilities. Besides, FedDQ enhances label correction to rectify improper labels through co-filtering and label correction. The experimental results demonstrate the effectiveness of FedDQ in improving model performance and handling noisy data challenges in FL settings. On the CIFAR-100 dataset with noisy labels, FedDQ exhibits a notable improvement of up to 32.4% compared to the baseline method.

---

PaperID: 77,   

Authors:  Arik Reuter, Anton Thielmann, Christoph Weisser, Benjamin Säfken, Thomas Kneib

Title: Probabilistic Topic Modeling With Transformer Representations

Abstract:
The field of topic modelling was mostly dominated by Bayesian graphical models during the last decade. With the rise of transformers in natural language processing, however, several successful models that rely on straightforward clustering approaches in transformer-based embedding spaces have emerged and consolidated the notion of topics as clusters of embedding vectors. We propose the transformer-representation neural topic model (TNTM), which combines the benefits of topic representations in transformer-based embedding spaces and probabilistic modeling. Therefore, this approach unifies the powerful and versatile notion of topics based on transformer embeddings with fully probabilistic modeling, as in models such as latent Dirichlet allocation (LDA). We utilize the variational autoencoder (VAE) framework for improved inference speed and modeling flexibility. Experimental results show that our proposed model achieves results on par with various state-of-the-art approaches in terms of embedding coherence while maintaining almost perfect topic diversity. The corresponding source code is available at: https://github.com/ArikReuter/TNTM.

---

PaperID: 78,   

Authors:  Yanru Sun, Zongxia Xie, Haoyu Xing, Hualong Yu, Qinghua Hu

Title: PPGF: Probability Pattern-Guided Time Series Forecasting

Abstract:
Time series forecasting (TSF) is an essential branch of machine learning with various applications. Most methods for TSF focus on constructing different networks to extract better information and improve performance. However, practical application data contain different internal mechanisms, resulting in a mixture of multiple patterns. That is, the model’s ability to fit different patterns is different and generates different errors. In order to solve this problem, we propose an end-to-end framework, namely probability pattern-guided time series forecasting (PPGF). PPGF reformulates the TSF problem as a forecasting task guided by probabilistic pattern classification. First, we propose the grouping strategy to approach forecasting problems as classification and alleviate the impact of data imbalance on classification. Second, we predict the corresponding class interval to guarantee the consistency of classification and forecasting. In addition, true class probability (TCP) is introduced to pay more attention to the difficult samples to improve the classification accuracy. Detailedly, PPGF classifies the different patterns to determine which one the target value may belong to and estimates it accurately in the corresponding interval. To demonstrate the effectiveness of the proposed framework, we conduct extensive experiments on real-world datasets, and PPGF achieves significant performance improvements over several baseline methods. Furthermore, the effectiveness of TCP and the necessity of consistency between classification and forecasting are proved in the experiments. All data and codes are available online: https://github.com/syrGitHub/PPGF.

---

PaperID: 79,   

Authors:  Ting Hu, Xiaotong Liu, Kai Ji, Yunwen Lei

Title: Convergence of Adaptive Stochastic Mirror Descent

Abstract:
In this article, we present a family of adaptive stochastic optimization methods, which are associated with mirror maps that are widely used to capture the geometry properties of optimization problems during iteration processes. The well-known adaptive moment estimation (Adam)-type algorithm falls into the family when the mirror maps take the form of temporal adaptation. In the context of convex objective functions, we show that with proper step sizes and hyperparameters, the average regret can achieve the convergence rate \mathcal O(T^-(1/2)) after T iterations under some standard assumptions. We further improve it to O(T^-1\log T) when the objective functions are strongly convex. In the context of smooth objective functions (not necessarily convex), based on properties of the strongly convex differentiable mirror map, our algorithms achieve convergence rates of order \mathcal O(T^-(1/2)) up to a logarithmic term, requiring large or increasing hyperparameters that are coincident with practical usage of Adam-type algorithms. Thus, our work gives explanations for the selection of the hyperparameters in Adam-type algorithms’ implementation.

---

PaperID: 80,   

Authors:  Guanxiong He, Zheng Wang, Liaoyuan Tang, Weizhong Yu, Feiping Nie, Xuelong Li

Title: Reweighted-Boosting: A Gradient-Based Boosting Optimization Framework

Abstract:
Boosting is a well-established ensemble learning approach that aims to enhance overall performance by combining multiple weak learners with a linear combination structure. It operates on the principle of using new learners to compensate for the shortcomings of previous learners and is known for its ability to reduce computational resource requirements while mitigating the risks of overfitting. However, from the perspective of convex optimization, it becomes apparent that classical boosting methods often converge to local optima rather than global optima when minimizing the target loss due to its greedy strategy. In this article, we address the issue and propose a novel optimization framework for the boosting paradigm. Our framework focuses on refining the ensemble model by further minimizing loss function through the reallocation of base learner weights, which results in a more robust and powerful learner. We have conducted experiments on various real-world and synthetic datasets, and our findings confirm that our Reweighted-Boosting model consistently outperforms its counterparts. It also exhibits an increased classification margin for the data, making it a valuable enhancement to original boosting algorithms.

---

PaperID: 81,   

Authors:  Yao Zhu, Yuefeng Chen, Xiaofeng Mao, Xiu Yan, Yue Wang, Wang Lu, Jindong Wang, Xiangyang Ji

Title: Enhancing Few-Shot CLIP With Semantic-Aware Fine-Tuning

Abstract:
Learning generalized representations from limited training samples is crucial for applying deep neural networks in low-resource scenarios. Recently, methods based on contrastive language-image pretraining (CLIP) have exhibited promising performance in few-shot adaptation tasks. To avoid catastrophic forgetting and overfitting caused by few-shot fine-tuning, existing works usually freeze the parameters of CLIP pretrained on large-scale datasets, overlooking the possibility that some parameters might not be suitable for downstream tasks. To this end, we revisit CLIP’s visual encoder with a specific focus on its distinctive attention pooling layer, which performs a spatial weighted-sum of the dense feature maps. Given that dense feature maps contain meaningful semantic information, and different semantics hold varying importance for diverse downstream tasks (such as prioritizing semantics like ears and eyes in pet classification tasks rather than side mirrors), using the same weighted-sum operation for dense features across different few-shot tasks might not be appropriate. Hence, we propose fine-tuning the parameters of the attention pooling layer during the training process to encourage the model to focus on task-specific semantics. In the inference process, we perform residual blending between the features pooled by the fine-tuned and the original attention pooling layers to incorporate both the few-shot knowledge and the pretrained CLIP’s prior knowledge. We term this method as semantic-aware fine-tuning (SAFE). SAFE is effective in enhancing the conventional few-shot CLIP and is compatible with the existing adapter approach (termed SAFE-A). Extensive experiments on 11 benchmarks demonstrate that both SAFE and SAFE-A significantly outperform the second-best method by +1.51% and +2.38% in the one-shot setting and by +0.48% and +1.37% in the four-shot setting, respectively.

---

PaperID: 82,   

Authors:  Abdul Quadir, M. Sajid, Mohammad Tanveer

Title: Granular Ball Twin Support Vector Machine

Abstract:
Twin support vector machine (TSVM) is an emerging machine learning model with versatile applicability in classification and regression endeavors. Nevertheless, TSVM confronts noteworthy challenges: 1) the imperative demand for matrix inversions presents formidable obstacles to its efficiency and applicability on large-scale datasets; 2) the omission of the structural risk minimization (SRM) principle in its primal formulation heightens the vulnerability to overfitting risks; and 3) the TSVM exhibits a high susceptibility to noise and outliers and also demonstrates instability when subjected to resampling. In view of the aforementioned challenges, we propose the granular ball TSVM (GBTSVM). GBTSVM takes granular balls (GBs), rather than individual data points, as inputs to construct a classifier. These GBs, characterized by their coarser granularity, exhibit robustness to resampling and reduced susceptibility to the impact of noise and outliers. We further propose a novel large-scale GBTSVM (LS-GBTSVM). LS-GBTSVM’s optimization formulation ensures two critical facets: 1) it eliminates the need for matrix inversions, streamlining the LS-GBTSVM’s computational efficiency; and 2) it incorporates the SRM principle through the incorporation of regularization terms, effectively addressing the issue of overfitting. The proposed LS-GBTSVM exemplifies efficiency, scalability for large datasets, and robustness against noise and outliers. We conduct a comprehensive evaluation of the GBTSVM and LS-GBTSVM models on benchmark datasets from UCI and KEEL, both with and without the addition of label noise, and compared with existing baseline models. Furthermore, we extend our assessment to the large-scale NDC datasets to establish the practicality of the proposed models in such contexts. Our experimental findings and rigorous statistical analyses affirm the superior generalization prowess of the proposed GBTSVM and LS-GBTSVM models compared to the baseline models. The source code of the proposed GBTSVM and LS-GBTSVM models are available at https://github.com/mtanveer1/GBTSVM.

---

PaperID: 83,   

Authors:  Huiliang Zhang, Ping Nie, Lijun Sun, Benoit Boulet

Title: Nearest Neighbor Multivariate Time Series Forecasting

Abstract:
Multivariate time series (MTS) forecasting has a wide range of applications in both industry and academia. Recently, spatial-temporal graph neural networks (STGNNs) have gained popularity as MTS forecasting methods. However, current STGNNs can only use the finite length of MTS input data due to the computational complexity. Moreover, they lack the ability to identify similar patterns throughout the entire dataset and struggle with data that exhibit sparsely and discontinuously distributed correlations among variables over an extensive historical period, resulting in only marginal improvements. In this article, we introduce a simple yet effective k-nearest neighbor MTS forecasting (kNN-MTS) framework, which forecasts with a nearest neighbor retrieval mechanism over a large datastore of cached series, using representations from the MTS model for similarity search. This approach requires no additional training and scales to give the MTS model direct access to the whole dataset at test time, resulting in a highly expressive model that consistently improves performance, and has the ability to extract sparse distributed but similar patterns span over multivariables from the entire dataset. Furthermore, a hybrid spatial-temporal encoder (HSTEncoder) is designed for kNN-MTS which can capture both long-term temporal and short-term spatial-temporal dependencies and is shown to provide accurate representation for kNN-MTS for better forecasting. Experimental results on several real-world datasets show a significant improvement in the forecasting performance of kNN-MTS. The quantitative analysis also illustrates the interpretability and efficiency of kNN-MTS, showing better application prospects and opening up a new path for efficiently using the large dataset in MTS models.

---

PaperID: 84,   

Authors:  Weiqing Yan, Shuochen Yao, Chang Tang, Wujie Zhou

Title: Multiview Representation Learning via Information-Theoretic Optimization

Abstract:
Multiview data, characterized by rich features, are crucial in many machine learning applications. However, effectively extracting intraview features and integrating interview information present significant challenges in multiview learning (MVL). Traditional deep network-based approaches often involve learning multiple layers to derive latent. In these methods, the features of different classes are typically implicitly embedded rather than systematically organized. This lack of structure makes it challenging to explicitly map classes to independent principal subspaces in the feature space, potentially causing class overlap and confusion. Consequently, the capability of these representations to accurately capture the intrinsic structure of the data remains uncertain. In this article, we introduce an innovative multiview representation learning (MVRL) by maximizing two information-theoretic metrics: intraview coding rate reduction and interview mutual information. Specifically, in the intraview representation learning, we aim to optimize feature representations by maximizing the coding rate difference between the entire dataset and individual classes. This process expands the feature representation space while compressing the representations within each class, resulting in more compact feature representations within each viewpoint. Subsequently, we align and fuse these view-specific features through space transformation and cross-sample fusion to achieve consistent representation across multiple views. Finally, we maximize information transmission to maintain consistency and correlation among data representations across views. By maximizing mutual information between the consensus representations and view-specific representations, our method ensures that the learned representations capture more concise intrinsic features and correlations among different views, thereby enhancing the performance and generalization ability of MVL. Experiments show that the proposed methods have achieved excellent performance.

---

PaperID: 85,   

Authors:  Yang Yue, Bingyi Kang, Xiao Ma, Qisen Yang, Gao Huang, Shiji Song, Shuicheng Yan

Title: Decoupled Prioritized Resampling for Offline RL

Abstract:
Offline reinforcement learning (RL) is challenged by the distributional shift problem. To tackle this issue, existing works mainly focus on designing sophisticated policy constraints between the learned policy and the behavior policy. However, these constraints are applied equally to well-performing and inferior actions through uniform sampling, which might negatively affect the learned policy. In this article, we propose offline decoupled prioritized resampling (ODPR), which designs specialized priority functions for the suboptimal policy constraint issue in offline RL and employs unique decoupled resampling for training stability. Through theoretical analysis, we show that the distinctive priority functions induce a provable improved behavior policy by modifying the distribution of the original behavior policy, and when constrained to this improved policy, a policy-constrained offline RL algorithm is likely to yield a better solution. We provide two practical implementations to balance computation and performance: one estimates priorities based on a fit value network [advantage-based ODPR (ODPR-A)] and the other utilizes trajectory returns [return-based ODPR (ODPR-R)] for quick computation. As a highly compatible plug-and-play component, ODPR is evaluated with five prevalent offline RL algorithms: behavior cloning (BC), twin delayed deep deterministic policy gradient + BC (TD3 + BC), OnestepRL, conservative Q-learning (CQL), and implicit Q-learning (IQL). Our experiments confirm that both ODPR-A and ODPR-R significantly improve performance across all baseline methods. Moreover, ODPR-A can be effective in some challenging settings, i.e., without trajectory information. Code and pretrained weights are available at https://github.com/yueyang130/ODPR.

---

PaperID: 86,   

Authors:  Marco Buzzelli, Simone Bianco

Title: A Convolutional Framework for Color Constancy

Abstract:
We introduce a convolutional framework (CF) for computational color constancy, building upon the established low-level image feature-based framework, which utilized simple image statistics for illuminant estimation. Our framework expands upon this through an end-to-end learnable neural architecture. This adaptation enables the learning and usage of advanced filters that are not restricted to Gaussian kernels operating on individual color channels, thus generalizing the capabilities of the original framework. Additionally, our general framework supports deeper convolutional architectures, thus increasing its computational power. It can also be efficiently applied to estimate multiple spatially varying illuminants within a single scene. Our experimental results on standard datasets demonstrate that the CF outperforms the best methods in the low-level framework, improving the illuminant estimation accuracy by up to 34% for single illuminant estimation and 30% for multiple illuminants estimation. Additionally, our framework exhibits superior performance even when the number of training images is reduced. Finally, we document the inference speedup of our implementation reaching up to 30× , making the CF especially suitable for applications where efficiency is critical. Source code and trained models available at: https://github.com/MarcoBauzz/convolutional-color-constancy

---

PaperID: 87,   

Authors:  Lifeng Zhang, Xiangwei Zheng, Xuanchi Chen, Lizhen Cui

Title: Three-Dimensional View Relationship-Based Context-Aware Emotion Recognition

Abstract:
Context-aware emotion recognition (CAER) leverages comprehensive scene information, including facial expressions, body postures, and contextual background. However, current studies predominantly rely on facial expressions, body postures, and global contextual features; the interaction between the agents (target individuals) and other objects in the scene is usually absent or incomplete. In this article, a three-dimensional view relationship-based CAER (TDRCer) method is proposed, which comprises two branches: the personal emotional branch (PEB) and the contextual emotional branch (CEB). First, PEB is designed for the extraction of facial expression features and body posture features from the agent. A vision transformer (ViT), pretrained by contrastive learning with a novel loss function combining Euclidean distance and cosine similarity, is applied to enhance the robustness of facial expression features. Meanwhile, the human body contour images extracted by semantic segmentation are fed into another ViT to extract body posture features. Second, CEB is constructed for the extraction of global contextual features and interactive relationships among objects in the scene. The images masked by the agents’ bodies are fed into a ViT to extract global contextual features. By leveraging both the gaze angle and depth map, a three-dimensional view graph (3DVG) is constructed to represent the interactive relationships between agents and objects in the scene. Then, a graph convolutional network is employed to extract interactive relationship features from the 3DVG. Finally, the multiplicative fusion strategy is applied to fuse the features of two branches, and the fused features are utilized to classify the emotions. TDRCer achieves an accuracy of 89.90% on the CAER-S dataset and a mean average precision (mAP) of 36.02% on the EMOTIons in context (EMOTIC) dataset. The code can be accessed at https://github.com/mengTender/TDRCer.

---

PaperID: 88,   

Authors:  Seung Park, Yong-Goo Shin

Title: Rethinking Image Skip Connections in StyleGAN2

Abstract:
Various models based on StyleGAN have gained significant traction in the field of image synthesis, attributed to their robust training stability and superior performances. Within the StyleGAN framework, the adoption of image skip connection is favored over the traditional residual connection. However, this preference is just based on empirical observations; there has not been any in-depth mathematical analysis on it yet. To rectify this situation, this brief aims to elucidate the mathematical meaning of the image skip connection and introduce a groundbreaking methodology, termed the image squeeze connection, which significantly improves the quality of image synthesis. Specifically, we analyze the image skip connection technique to reveal its problem and introduce the proposed method which not only effectively boosts the GAN performance but also reduces the required number of network parameters. Extensive experiments on various datasets demonstrate that the proposed method consistently enhances the performance of state-of-the-art models based on StyleGAN. We believe that our findings represent a vital advancement in the field of image synthesis, suggesting a novel direction for future research and applications.

---

PaperID: 89,   

Authors:  Jiatai Wang, Zhiwei Xu, Xin Wang, Tao Li

Title: Toward Generalized Multistage Clustering: Multiview Self-Distillation

Abstract:
Existing multistage clustering methods independently learn the salient features from multiple views and then perform the clustering task. Particularly, multiview clustering (MVC) has attracted a lot of attention in multiview or multimodal scenarios. MVC aims at exploring common semantics and pseudo-labels from multiple views and clustering in a self-supervised manner. However, limited by noisy data and inadequate feature learning, such a clustering paradigm generates overconfident pseudo-labels that misguide the model to produce inaccurate predictions. Therefore, it is desirable to have a method that can correct this pseudo-label mistraction in multistage clustering to avoid bias accumulation. To alleviate the effect of overconfident pseudo-labels and improve the generalization ability of the model, this article proposes a novel multistage deep MVC framework where multiview self-distillation (DistilMVC) is introduced to distill dark knowledge of label distribution. Specifically, in the feature subspace at different hierarchies, we explore the common semantics of multiple views through contrastive learning and obtain pseudo-labels by maximizing the mutual information between views. Additionally, a teacher network is responsible for distilling pseudo-labels into dark knowledge, supervising the student network and improving its predictive capabilities to enhance its robustness. Extensive experiments on real-world multiview datasets show that our method has better clustering performance than the state-of-the-art (SOTA) methods.

---

PaperID: 90,   

Authors:  Cheng Qian, Xiaoxian Lao, Chunguang Li

Title: Reconstruction-Based Anomaly Localization via Knowledge-Informed Self-Training

Abstract:
Anomaly localization, which involves localizing anomalous regions within images, is a significant industrial task. Reconstruction-based methods are widely adopted for anomaly localization because of their low complexity and high interpretability. Most existing reconstruction-based methods only use normal samples to construct model. If anomalous samples are appropriately utilized in the process of anomaly localization, the localization performance can be improved. However, usually only weakly labeled anomalous samples are available, which limits the improvement. In many cases, we can obtain some knowledge of anomalies summarized by domain experts. Taking advantage of such knowledge can help us better utilize the anomalous samples and thus further improve the localization performance. In this article, we propose a novel reconstruction-based method named knowledge-informed self-training (KIST) which integrates knowledge into a reconstruction model through self-training. Specifically, KIST utilizes weakly labeled anomalous samples in addition to the normal ones and exploits knowledge to yield pixel-level pseudolabels of the anomalous samples. Based on the pseudolabels, a novel loss that promotes the reconstruction of normal pixels while suppressing the reconstruction of anomalous pixels is used. We conduct experiments on different datasets and demonstrate the advantages of KIST over the existing reconstruction-based methods.

---

PaperID: 91,   

Authors:  Xiaoxie Zhu, Jinfeng Yi, Lijun Zhang

Title: Continual Learning With Unknown Task Boundary

Abstract:
Most existing studies on continual learning (CL) consider the task-based setting, where task boundaries are known to learners during training. However, they may be impractical for real-world problems, where new tasks arrive with unnotified distribution shifts. In this article, we introduce a new boundary-unknown continual learning scenario called continuum incremental learning (CoIL), where the incremental unit may be a concatenation of several tasks or a subset of one task. To identify task boundaries, we design a continual out-of-distribution (OOD) detection method based on softmax probabilities, which can detect OOD samples for the latest learned task. Then, we incorporate it with continual learning approaches to solve the CoIL problem. Furthermore, we investigate the more challenging task-reappear setting and propose a method named continual learning with unknown task boundary (CLUTaB). CLUTaB first adopts in-distribution detection and OOD loss to determine whether a set of data is sampled from any learned distribution. Then, a two-step inference technique is designed to improve the continual learning performance. Experiments show that our methods work well with existing continual learning approaches and achieve good performance on CIFAR-100 and mini-ImageNet datasets.

---

PaperID: 92,   

Authors:  Zijian Ying, Qianmu Li, Zhichao Lian, Jun Hou, Tong Lin, Tao Wang

Title: Understanding Convolutional Neural Networks From Excitations

Abstract:
Saliency maps have proven to be a highly efficacious approach for explicating the decisions of convolutional neural networks (CNNs). However, extant methodologies predominantly rely on gradients, which constrain their ability to explicate complex models. Furthermore, such approaches are not fully adept at leveraging negative gradient information to improve interpretive veracity. In this study, we present a novel concept, termed positive and negative excitation (PANE), which enables the direct extraction of PANE for each layer, thus enabling complete layer-by-layer information utilization sans gradients. To organize these excitations into final saliency maps, we introduce a double-chain backpropagation procedure. A comprehensive experimental evaluation, encompassing both binary classification and multiclassification tasks, was conducted to gauge the effectiveness of our proposed method. Encouragingly, the results evince that our approach offers a significant improvement over the state-of-the-art methods in terms of salient pixel removal, minor pixel removal, and inconspicuous adversarial perturbation generation guidance. In addition, we verify the correlation between PANEs.

---

PaperID: 93,   

Authors:  Muhammad Anwar Ma'sum, Mahardhika Pratama, Edwin Lughofer, Lin Liu, Habibullah, Ryszard Kowalczyk

Title: Few-Shot Continual Learning via Flat-to-Wide Approaches

Abstract:
The existing approaches on continual learning (CL) call for a lot of samples in their training processes. Such approaches are impractical for many real-world problems having limited samples because of the overfitting problem. This article proposes a few-shot CL approach, termed flat-to-wide approach (FLOWER), where a flat-to-wide learning process finding the flat-wide minima is proposed to address the catastrophic forgetting (CF) problem. The issue of data scarcity is overcome with a data augmentation approach making use of a ball-generator concept to restrict the sampling space into the smallest enclosing ball. Our numerical studies demonstrate the advantage of FLOWER achieving significantly improved performances over prior arts notably in the small base tasks. For further study, source codes of FLOWER, competitor algorithms, and experimental logs are shared publicly in https://github.com/anwarmaxsum/FLOWER.

---

PaperID: 94,   

Authors:  Hui Lin, Xiaopeng Hong, Zhiheng Ma, Yaowei Wang, Deyu Meng

Title: Multidimensional Measure Matching for Crowd Counting

Abstract:
This article addresses the challenge of scale variations in crowd-counting problems from a multidimensional measure-theoretic perspective. We start by formulating crowd counting as a measure-matching problem, based on the assumption that discrete measures can express the scattered ground truth and the predicted density map. In this context, we introduce the Sinkhorn counting loss and extend it to the semi-balanced form, which alleviates the problems including entropic bias, distance destruction, and amount constraints. We then model the measure matching under the multidimensional space, in order to learn the counting from both location and scale. To achieve this, we extend the traditional 2-D coordinate support to 3-D, incorporating an additional axis to represent scale information, where a pyramid-based structure will be leveraged to learn the scale value for the predicted density. Extensive experiments on four challenging crowd-counting datasets, namely, ShanghaiTech A, UCF-QNRF, JHU++, and NWPU have validated the proposed method. Code is released at https://github.com/LoraLinH/Multidimensional-Measure-Matching-for-Crowd-Counting.

---

PaperID: 95,   

Authors:  Jie Jiang, Xingjian He, Weining Wang, Hanqing Lu, Jing Liu

Title: Hierarchical Contrastive Learning for Semantic Segmentation

Abstract:
Recently, pixel-to-pixel contrastive learning in single-scale feature space has been widely studied in semantic segmentation to learn a unified feature expression for pixels of the same category. However, the unified representation is too extreme, and the receptive field of each single-scale pixel is limited, which is insufficient to reflect the representative features of the category. To address these problems, this article extends the single-scale feature space to that of multiscale and proposes a hierarchical contrastive learning (Hi-CL) method to explore pixel-to-component semantic relationships. First, we generate multiscale candidate samples by applying several pooling windows with different sizes on a feature map, where different windows may represent different parts of the objects in the image. Then, we prune the sample set through threshold-based criteria to select appropriate samples for feature representation learning. Finally, Hi-CL is performed to learn the pixel-to-component consistency with the pruned samples. Our method is easy to be applied on existing semantic segmentation models and obtains consistent improvement. Furthermore, we achieve state-of-the-art results on three popular benchmarks, including Cityscapes, ADE20K, and COCO Stuff datasets.

---

PaperID: 96,   

Authors:  Zhiyuan Zhang, Haoxuan Li, Chengjie Ke, Jun Chen, Xin Tian

Title: Deep Variational Network for Blind Pansharpening

Abstract:
Deep-learning-based methods play an important role in pansharpening that uses panchromatic images to enhance the spatial resolution of multispectral images while maintaining spectral features. However, most existing methods mainly consider only one fixed degradation in the training process. Therefore, their performance may drop significantly when the degradation of testing data is unknown (blind) and different from the training data, which is common in real-world applications. To address this issue, we proposed a deep variational network for blind pansharpening, named VBPN, which integrates degradation estimation and image fusion into a whole Bayesian framework. First, by taking the noise and blurring parameters of the multispectral image with the noise parameters of the panchromatic image as hidden variables, we parameterize the approximate posterior distribution for the fusion problem using neural networks. Since all parameters in this posterior distribution are explicitly modeled, the degradation parameters of the multispectral image and the panchromatic image are easily estimated. Furthermore, we designed VPBN composed of degradation estimation and image fusion subnetworks, which can optimize the fusion results guided by the variational inference according to the testing data. As a result, the blind pansharpening performance can be improved. In general, VPBN has good interpretability and generalization ability by combining the advantages of model-based and deep-learning-based approaches. Experiments on simulated and real datasets prove that VPBN can achieve state-of-the-art fusion results.

---

PaperID: 97,   

Authors:  Jingwei Xin, Nannan Wang, Xinrui Jiang, Jie Li, Xiaoyu Wang, Xinbo Gao

Title: Rectified Binary Network for Single-Image Super-Resolution

Abstract:
Binary neural network (BNN) is an effective approach to reduce the memory usage and the computational complexity of full-precision convolutional neural networks (CNNs), which has been widely used in the field of deep learning. However, there are different properties between BNNs and real-valued models, making it difficult to draw on the experience of CNN composition to develop BNN. In this article, we study the application of binary network to the single-image super-resolution (SISR) task in which the network is trained for restoring original high-resolution (HR) images. Generally, the distribution of features in the network for SISR is more complex than those in recognition models for preserving the abundant image information, e.g., texture, color, and details. To enhance the representation ability of BNN, we explore a novel activation-rectified inference (ARI) module that achieves a more complete representation of features by combining observations from different quantitative perspectives. The activations are divided into several parts with different quantification intervals and are inferred independently. This allows the binary activations to retain more image detail and yield finer inference. In addition, we further propose an adaptive approximation estimator (AAE) for gradually learning the accurate gradient estimation interval in each layer to alleviate the optimization difficulty. Experiments conducted on several benchmarks show that our approach is able to learn a binary SISR model with superior performance over the state-of-the-art methods. The code will be released at https://github.com/jwxintt/Rectified-BSR.

---

PaperID: 98,   

Authors:  Rui Zhu, Yalong Bai, Ting Yao, Jingen Liu, Zhenglong Sun, Tao Mei, Chang Wen Chen

Title: Teaching Masked Autoencoder With Strong Augmentations

Abstract:
Masked autoencoder (MAE) has been regarded as a capable self-supervised learner for various downstream tasks. Nevertheless, the model still lacks high-level discriminability, which results in poor linear probing performance. In view of the fact that strong augmentation plays an essential role in contrastive learning, can we capitalize on strong augmentation in MAE? The difficulty originates from the pixel uncertainty caused by strong augmentation that may affect the reconstruction, and thus, directly introducing strong augmentation into MAE often hurts the performance. In this article, we delve into the potential of strong augmented views to enhance MAE while maintaining MAE’s advantages. To this end, we propose a simple yet effective masked Siamese autoencoder (MSA) model, which consists of a student branch and a teacher branch. The student branch derives MAE’s advanced architecture, and the teacher branch treats the unmasked strong view as an exemplary teacher to impose high-level discrimination onto the student branch. We demonstrate that our MSA can improve the model’s spatial perception capability and, therefore, globally favors interimage discrimination. Empirical evidence shows that the model pretrained by MSA provides superior performances across different downstream tasks. Notably, linear probing performance on frozen features extracted from MSA leads to 6.1% gains over MAE on ImageNet-1k. Fine-tuning (FT) the network on VQAv2 task finally achieves 67.4% accuracy, outperforming 1.6% of the supervised method DeiT and 1.2% of MAE. Codes and models are available at https://github.com/KimSoybean/MSA.

---

PaperID: 99,   

Authors:  Hantao Zhou, Rui Yang, Yachao Zhang, Haoran Duan, Yawen Huang, Runze Hu, Xiu Li, Yefeng Zheng

Title: UniHead: Unifying Multi-Perception for Detection Heads

Abstract:
The detection head constitutes a pivotal component within object detectors, tasked with executing both classification and localization functions. Regrettably, the commonly used parallel head often lacks omni perceptual capabilities, such as deformation perception (DP), global perception (GP), and cross-task perception (CTP). Despite numerous methods attempting to enhance these abilities from a single aspect, achieving a comprehensive and unified solution remains a significant challenge. In response to this challenge, we develop an innovative detection head, termed UniHead, to unify three perceptual abilities simultaneously. More precisely, our approach: 1) introduces DP, enabling the model to adaptively sample object features; 2) proposes a dual-axial aggregation transformer (DAT) to adeptly model long-range dependencies, thereby achieving GP; and 3) devises a cross-task interaction transformer (CIT) that facilitates interaction between the classification and localization branches, thus aligning the two tasks. As a plug-and-play method, the proposed UniHead can be conveniently integrated with existing detectors. Extensive experiments on the COCO dataset demonstrate that our UniHead can bring significant improvements to many detectors. For instance, the UniHead can obtain +2.7 AP gains in RetinaNet, +2.9 AP gains in FreeAnchor, and +2.1 AP gains in GFL. The code is available at https://github.com/zht8506/UniHead.

---

PaperID: 100,   

Authors:  Yuzhong Chen, Zhenxiang Xiao, Yi Pan, Lin Zhao, Haixing Dai, Zihao Wu, Changhe Li, Tuo Zhang, Changying Li, Dajiang Zhu, Tianming Liu, Xi Jiang

Title: Mask-Guided Vision Transformer for Few-Shot Learning

Abstract:
Learning with little data is challenging but often inevitable in various application scenarios where the labeled data are limited and costly. Recently, few-shot learning (FSL) gained increasing attention because of its generalizability of prior knowledge to new tasks that contain only a few samples. However, for data-intensive models such as vision transformer (ViT), current fine-tuning-based FSL approaches are inefficient in knowledge generalization and, thus, degenerate the downstream task performances. In this article, we propose a novel mask-guided ViT (MG-ViT) to achieve an effective and efficient FSL on the ViT model. The key idea is to apply a mask on image patches to screen out the task-irrelevant ones and to guide the ViT focusing on task-relevant and discriminative patches during FSL. Particularly, MG-ViT only introduces an additional mask operation and a residual connection, enabling the inheritance of parameters from pretrained ViT without any other cost. To optimally select representative few-shot samples, we also include an active learning-based sample selection method to further improve the generalizability of MG-ViT-based FSL. We evaluate the proposed MG-ViT on classification, object detection, and segmentation tasks using gradient-weighted class activation mapping (Grad-CAM) to generate masks. The experimental results show that the MG-ViT model significantly improves the performance and efficiency compared with general fine-tuning-based ViT and ResNet models, providing novel insights and a concrete approach toward generalizing data-intensive and large-scale deep learning models for FSL.

---

PaperID: 101,   

Authors:  Yan Zeng, Ruichu Cai, Fuchun Sun, Libo Huang, Zhifeng Hao

Title: A Survey on Causal Reinforcement Learning

Abstract:
While reinforcement learning (RL) achieves tremendous success in sequential decision-making problems of many domains, it still faces key challenges of data inefficiency and the lack of interpretability. Interestingly, many researchers have leveraged insights from the causality literature recently, bringing forth flourishing works to unify the merits of causality and address well the challenges from RL. As such, it is of great necessity and significance to collate these causal RL (CRL) works, offer a review of CRL methods, and investigate the potential functionality from causality toward RL. In particular, we divide the existing CRL approaches into two categories according to whether their causality-based information is given in advance or not. We further analyze each category in terms of the formalization of different models, ranging from the Markov decision process (MDP), partially observed MDP (POMDP), multiarmed bandits (MABs), imitation learning (IL), and dynamic treatment regime (DTR). Each of them represents a distinct type of causal graphical illustration. Moreover, we summarize the evaluation matrices and open sources, while we discuss emerging applications, along with promising prospects for the future development of CRL.

---

PaperID: 102,   

Authors:  Yiwei Ma, Jiayi Ji, Xiaoshuai Sun, Yiyi Zhou, Xiaopeng Hong, Yongjian Wu, Rongrong Ji

Title: Image Captioning via Dynamic Path Customization

Abstract:
This article explores a novel dynamic network for vision and language (V&L) tasks, where the inferring structure is customized on the fly for different inputs. Most previous state-of-the-art (SOTA) approaches are static and handcrafted networks, which not only heavily rely on expert knowledge but also ignore the semantic diversity of input samples, therefore resulting in suboptimal performance. To address these issues, we propose a novel Dynamic Transformer Network (DTNet) for image captioning, which dynamically assigns customized paths to different samples, leading to discriminative yet accurate captions. Specifically, to build a rich routing space and improve routing efficiency, we introduce five types of basic cells and group them into two separate routing spaces according to their operating domains, i.e., spatial and channel. Then, we design a Spatial-Channel Joint Router (SCJR), which endows the model with the capability of path customization based on both spatial and channel information of the input sample. To validate the effectiveness of our proposed DTNet, we conduct extensive experiments on the MS-COCO dataset and achieve new SOTA performance on both the Karpathy split and the online test server. The source code is publicly available at https://github.com/xmu-xiaoma666/DTNet.

---

PaperID: 103,   

Authors:  Claire M. Postlethwaite, Peter Ashwin, Matthew D. Egbert

Title: A Continuous Time Dynamical Turing Machine

Abstract:
Continuous time recurrent neural networks (CTRNNs) are systems of coupled ordinary differential equations (ODEs) inspired by the structure of neural networks in the brain. CTRNNs are known to be universal dynamical approximators: given a large enough system, the parameters of a CTRNN can be tuned to produce output that is arbitrarily close to that of any other dynamical system. However, in practice, both designing systems of CTRNN to have a certain output, and the reverse—understanding the dynamics of a given system of CTRNN—can be nontrivial. In this article, we describe a method for embedding any specified Turing machine in its entirety into a CTRNN. As such, we describe in detail a continuous time dynamical system that performs arbitrary discrete-state computations. We suggest that in acting as both a continuous time dynamical system and as a computer, the study of such systems can help refine and advance the debate concerning the Computational Hypothesis that cognition is a form of computation and the Dynamical Hypothesis that cognitive systems are dynamical systems.

---

PaperID: 104,   

Authors:  Xusheng Zhao, Qiong Dai, Xu Bai, Jia Wu, Hao Peng, Huailiang Peng, Zhengtao Yu, Philip S. Yu

Title: Reinforced GNNs for Multiple Instance Learning

Abstract:
Multiple instance learning (MIL) trains models from bags of instances, where each bag contains multiple instances, and only bag-level labels are available for supervision. The application of graph neural networks (GNNs) in capturing intrabag topology effectively improves MIL. Existing GNNs usually require filtering low-confidence edges among instances and adapting graph neural architectures to new bag structures. However, such asynchronous adjustments to structure and architecture are tedious and ignore their correlations. To tackle these issues, we propose a reinforced GNN framework for MIL (RGMIL), pioneering the exploitation of multiagent deep reinforcement learning (MADRL) in MIL tasks. MADRL enables the flexible definition or extension of factors that influence bag graphs or GNNs and provides synchronous control over them. Moreover, MADRL explores structure-to-architecture correlations while automating adjustments. Experimental results on multiple MIL datasets demonstrate that RGMIL achieves the best performance with excellent explainability. The code and data are available at https://github.com/RingBDStack/RGMIL.

---

PaperID: 105,   

Authors:  Jifeng Guo, Zhulin Liu, C. L. Philip Chen

Title: An Incremental-Self-Training-Guided Semi-Supervised Broad Learning System

Abstract:
The broad learning system (BLS) has recently been applied in numerous fields. However, it is mainly a supervised learning system and thus not suitable for specific practical applications with a mixture of labeled and unlabeled data. Despite a manifold regularization-based semi-supervised BLS, its performance still requires improvement, because its assumption is not always applicable. Therefore, this article proposes an incremental-self-training-guided semi-supervised BLS (ISTSS-BLS). Distinctive to traditional self-training, where all unlabeled data are labeled simultaneously, incremental self-training (IST) obtains unlabeled data incrementally from an established sorted list based on the distance between the data and their cluster center. During iterative learning, a small portion of labeled data is first used to train BLS. The system recursively self-updates its structure and meta-parameters using: 1) the double-restricted mechanism and 2) the dynamic neuron-incremental mechanism. The double-restricted mechanism is beneficial to preventing the introduction of incorrect pseudo-labeled samples, and the dynamic neuron-incremental mechanism guides the self-updating of the network structure effectively based on the training accuracy of the labeled data. These strategies guarantee a parsimonious model during the update. Besides, a novel metric, the accuracy-time ratio (A/T), is proposed to evaluate the model’s performance comprehensively regarding time and accuracy. In experimental verifications, ISTSS-BLS performs outstandingly on 11 datasets. Specifically, the IST is compared with the traditional one on three scales data, saving up to 52.02% learning time. In addition, ISTSS-BLS is compared with different state-of-the-art alternatives, and all results indicate that it possesses significant advantages in performance.

---

PaperID: 106,   

Authors:  Keke Tang, Yuexin Ma, Dingruibo Miao, Peng Song, Zhaoquan Gu, Zhihong Tian, Wenping Wang

Title: Decision Fusion Networks for Image Classification

Abstract:
Convolutional neural networks, in which each layer receives features from the previous layer(s) and then aggregates/abstracts higher level features from them, are widely adopted for image classification. To avoid information loss during feature aggregation/abstraction and fully utilize lower layer features, we propose a novel decision fusion module (DFM) for making an intermediate decision based on the features in the current layer and then fuse its results with the original features before passing them to the next layers. This decision is devised to determine an auxiliary category corresponding to the category at a higher hierarchical level, which can, thus, serve as category-coherent guidance for later layers. Therefore, by stacking a collection of DFMs into a classification network, the generated decision fusion network is explicitly formulated to progressively aggregate/abstract more discriminative features guided by these decisions and then refine the decisions based on the newly generated features in a layer-by-layer manner. Comprehensive results on four benchmarks validate that the proposed DFM can bring significant improvements for various common classification networks at a minimal additional computational cost and are superior to the state-of-the-art decision fusion-based methods. In addition, we demonstrate the generalization ability of the DFM to object detection and semantic segmentation.

---

PaperID: 107,   

Authors:  Bingrong Xu, Zhigang Zeng, Cheng Lian, Zhengming Ding

Title: Generative Mixup Networks for Zero-Shot Learning

Abstract:
Zero-shot learning casts light on lacking unseen class data by transferring knowledge from seen classes via a joint semantic space. However, the distributions of samples from seen and unseen classes are usually imbalanced. Many zero-shot learning methods fail to obtain satisfactory results in the generalized zero-shot learning task, where seen and unseen classes are all used for the test. Also, irregular structures of some classes may result in inappropriate mapping from visual features space to semantic attribute space. A novel generative mixup networks with semantic graph alignment is proposed in this article to mitigate such problems. To be specific, our model first attempts to synthesize samples conditioned with class-level semantic information as the prototype to recover the class-based feature distribution from the given semantic description. Second, the proposed model explores a mixup mechanism to augment training samples and improve the generalization ability of the model. Third, triplet gradient matching loss is developed to guarantee the class invariance to be more continuous in the latent space, and it can help the discriminator distinguish the real and fake samples. Finally, a similarity graph is constructed from semantic attributes to capture the intrinsic correlations and guides the feature generation process. Extensive experiments conducted on several zero-shot learning benchmarks from different tasks prove that the proposed model can achieve superior performance over the state-of-the-art generalized zero-shot learning.

---

PaperID: 108,   

Authors:  Shijie Li, Yun Liu, Juergen Gall

Title: Rethinking 3-D LiDAR Point Cloud Segmentation

Abstract:
Many point-based semantic segmentation methods have been designed for indoor scenarios, but they struggle if they are applied to point clouds that are captured by a light detection and ranging (LiDAR) sensor in an outdoor environment. In order to make these methods more efficient and robust such that they can handle LiDAR data, we introduce the general concept of reformulating 3-D point-based operations such that they can operate in the projection space. While we show by means of three point-based methods that the reformulated versions are between 300 and 400 times faster and achieve higher accuracy, we furthermore demonstrate that the concept of reformulating 3-D point-based operations allows to design new architectures that unify the benefits of point-based and image-based methods. As an example, we introduce a network that integrates reformulated 3-D point-based operations into a 2-D encoder-decoder architecture that fuses the information from different 2-D scales. We evaluate the approach on four challenging datasets for semantic LiDAR point cloud segmentation and show that leveraging reformulated 3-D point-based operations with 2-D image-based operations achieves very good results for all four datasets.

---

PaperID: 109,   

Authors:  Zhuang Shao, Jungong Han, Demetris Marnerides, Kurt Debattista

Title: Region-Object Relation-Aware Dense Captioning via Transformer

Abstract:
Dense captioning provides detailed captions of complex visual scenes. While a number of successes have been achieved in recent years, there are still two broad limitations: 1) most existing methods adopt an encoder–decoder framework, where the contextual information is sequentially encoded using long short-term memory (LSTM). However, the forget gate mechanism of LSTM makes it vulnerable when dealing with a long sequence and 2) the vast majority of prior arts consider regions of interests (RoIs) equally important, thus failing to focus on more informative regions. The consequence is that the generated captions cannot highlight important contents of the image, which does not seem natural. To overcome these limitations, in this article, we propose a novel end-to-end transformer-based dense image captioning architecture, termed the transformer-based dense captioner (TDC). TDC learns the mapping between images and their dense captions via a transformer, prioritizing more informative regions. To this end, we present a novel unit, named region-object correlation score unit (ROCSU), to measure the importance of each region, where the relationships between detected objects and the region, alongside the confidence scores of detected objects within the region, are taken into account. Extensive experimental results and ablation studies on the standard dense-captioning datasets demonstrate the superiority of the proposed method to the state-of-the-art methods.

---

PaperID: 110,   

Authors:  Yu Xue, Xiaolong Han, Ferrante Neri, Jiafeng Qin, Danilo Pelusi

Title: A Gradient-Guided Evolutionary Neural Architecture Search

Abstract:
Neural architecture search (NAS) is a popular method that can automatically design deep neural network structures. However, designing a neural network using NAS is computationally expensive. This article proposes a gradient-guided evolutionary NAS (GENAS) to design convolutional neural networks (CNNs) for image classification. GENAS is a hybrid algorithm that combines evolutionary global and local search operators to evolve a population of subnets sampled from a supernet. Each candidate architecture is encoded as a table describing which operations are associated with the edges between nodes signifying feature maps. Besides, evolutionary optimization uses novel crossover and mutation operators to manipulate the subnets using the proposed tabular encoding. Every n generations, the candidate architectures undergo a local search inspired by differentiable NAS. GENAS is designed to overcome the limitations of both evolutionary and gradient descent NAS. This algorithmic structure enables the performance assessment of the candidate architecture without retraining, thus limiting the NAS calculation time. Furthermore, subnet individuals are decoupled during evaluation to prevent strong coupling of operations in the supernet. The experimental results indicate that the searched structures achieve test errors of 2.45%, 16.86%, and 23.9% on CIFAR-10/100/ImageNet datasets and it costs only 0.26 GPU days on a graphic card. GENAS can effectively expedite the training and evaluation processes and obtain high-performance network structures.

---

PaperID: 111,   

Authors:  Archer Moore, Heejung Shim, Jingge Zhu, Mingming Gong

Title: Semi-Supervised Learning Under General Causal Models

Abstract:
Semi-supervised learning (SSL) aims to train a machine learning (ML) model using both labeled and unlabeled data. While the unlabeled data have been used in various ways to improve the prediction accuracy, the reason why unlabeled data could help is not fully understood. One interesting and promising direction is to understand SSL from a causal perspective. In light of the independent causal mechanisms (ICM) principle, the unlabeled data can be helpful when the label causes the features but not vice versa. However, the causal relations between the features and labels can be complex in real world applications. In this article, we propose an SSL framework that works with general causal models in which the variables have flexible causal relations. More specifically, we explore the causal graph structures and design corresponding causal generative models which can be learned with the help of unlabeled data. The learned causal generative model can generate synthetic labeled data for training a more accurate predictive model. We verify the effectiveness of our proposed method by empirical studies on both simulated and real data.

---

PaperID: 112,   

Authors:  Xi Xiao, Hailong Ma, Guojun Gan, Qing Li, Bin Zhang, Shutao Xia

Title: Robust k-Means-Type Clustering for Noisy Data

Abstract:
Data clustering is a fundamental machine learning task that seeks to categorize a dataset into homogeneous groups. However, real data usually contain noise, which poses significant challenges to clustering algorithms. In this article, motivated by how the k-means algorithm is derived from a Gaussian mixture model (GMM), we propose a robust k-means-type algorithm, named k-means-type clustering based on t-distribution (KMTD), by assuming that the data points are drawn from a special multivariate t-mixture model (TMM). Compared to the Gaussian distribution, the t-distribution has a fatter tail. The proposed algorithm is more robust to noise. Like the k-means algorithm, the proposed algorithm is simpler than those based on a full TMM. Both synthetic and actual data are used to illustrate the proposed algorithm’s performance and efficiency. The experimental results demonstrated that the proposed algorithm operates more quickly than other sophisticated algorithms and, in most cases, achieves higher accuracy than the other algorithms.

---

PaperID: 113,   

Authors:  Andrea Ceni, Claudio Gallicchio

Title: Edge of Stability Echo State Network

Abstract:
Echo state networks (ESNs) are time series processing models working under the echo state property (ESP) principle. The ESP is a notion of stability that imposes an asymptotic fading of the memory of the input. On the other hand, the resulting inherent architectural bias of ESNs may lead to an excessive loss of information, which in turn harms the performance in certain tasks with long short-term memory requirements. To bring together the fading memory property and the ability to retain as much memory as possible, in this article, we introduce a new ESN architecture called the Edge of Stability ESN (ES2N). The introduced ES2N model is based on defining the reservoir layer as a convex combination of a nonlinear reservoir (as in the standard ESN), and a linear reservoir that implements an orthogonal transformation. In virtue of a thorough mathematical analysis, we prove that the whole eigenspectrum of the Jacobian of the ES2N map can be contained in an annular neighborhood of a complex circle of controllable radius. This property is exploited to tune the ES2N’s dynamics close to the edge-of-chaos regime by design. Remarkably, our experimental analysis shows that ES2N model can reach the theoretical maximum short-term memory capacity (MC). At the same time, in comparison to conventional reservoir approaches, ES2N is shown to offer an excellent trade-off between memory and nonlinearity, as well as a significant improvement of performance in autoregressive nonlinear modeling and real-world time series modeling.

---

PaperID: 114,   

Authors:  Aleksandr Cariow, Galina Cariowa

Title: Reduced-Complexity Algorithms for Tessarine Neural Networks

Abstract:
The brief presents the results of synthesizing efficient algorithms for implementing the basic data-processing macro operations used in tessarine-valued neural networks. These macro operations primarily include the macro operation of multiplication of two tessarines: the macro operation of calculating the inner product of two tessarine-valued vectors and the macro operation of multiple multiplications of one tessarine by the set of different tessarines. When synthesizing the discussed algorithms, we use the fact that tessarine multiplications can be interpreted as matrix–vector products. In each of these cases, the matrices have a specific block structure, which allows them to be efficiently factorized. This factorization provides a reduction in the multiplicative complexity of computing the product of two tessarines. In what follows, we use the optimized tessarine multiplication algorithm to synthesize reduced-complexity algorithms for the inner product of two tessarine-valued vectors as well as to compute multiple tessarine multiplication. Also, to further decrease the computational complexity of the inner product of two tessarine-valued vectors and multiple tessarine multiplication, we take into account that the selective sets of operations in the computations of all partial matrix–vector multiplications participating in these macro operations are the same. Accounting for this fact provides an additional reduction in computation complexity.

---

PaperID: 115,   

Authors:  Ainur Zhaikhan, Ali H. Sayed

Title: Graph Exploration for Effective Multiagent Q-Learning

Abstract:
This article proposes an exploration technique for multiagent reinforcement learning (MARL) with graph-based communication among agents. We assume that the individual rewards received by the agents are independent of the actions by the other agents, while their policies are coupled. In the proposed framework, neighboring agents collaborate to estimate the uncertainty about the state–action space in order to execute more efficient explorative behavior. Different from existing works, the proposed algorithm does not require counting mechanisms and can be applied to continuous-state environments without requiring complex conversion techniques. Moreover, the proposed scheme allows agents to communicate in a fully decentralized manner with minimal information exchange. And for continuous-state scenarios, each agent needs to exchange only a single parameter vector. The performance of the algorithm is verified with theoretical results for discrete-state scenarios and with experiments for the continuous ones.

---

PaperID: 116,   

Authors:  Xinhang Wan, Jiyuan Liu, Xinbiao Gan, Xinwang Liu, Siwei Wang, Yi Wen, Tianjiao Wan, En Zhu

Title: One-Step Multi-View Clustering With Diverse Representation

Abstract:
Multi-View clustering has attracted broad attention due to its capacity to utilize consistent and complementary information among views. Although tremendous progress has been made recently, most existing methods undergo high complexity, preventing them from being applied to large-scale tasks. Multi-View clustering via matrix factorization is a representative to address this issue. However, most of them map the data matrices into a fixed dimension, limiting the model’s expressiveness. Moreover, a range of methods suffers from a two-step process, i.e., multimodal learning and the subsequent k-means, inevitably causing a suboptimal clustering result. In light of this, we propose a one-step multi-view clustering with diverse representation (OMVCDR) method, which incorporates multi-view learning and k-means into a unified framework. Specifically, we first project original data matrices into various latent spaces to attain comprehensive information and auto-weight them in a self-supervised manner. Then, we directly use the information matrices under diverse dimensions to obtain consensus discrete clustering labels. The unified work of representation learning and clustering boosts the quality of the final results. Furthermore, we develop an efficient optimization algorithm with proven convergence to solve the resultant problem. Comprehensive experiments on various datasets demonstrate the promising clustering performance of our proposed method. The code is publicly available at https://github.com/wanxinhang/OMVCDR.

---

PaperID: 117,   

Authors:  Maria Nareklishvili, Marius Geitle

Title: Deep Ensemble Transformers for Dimensionality Reduction

Abstract:
We propose deep ensemble transformers (DETs), a fast, scalable approach for dimensionality reduction problems. This method leverages the power of deep neural networks and employs cascade ensemble techniques as its fundamental feature extraction tool. To handle high-dimensional data, our approach employs a flexible number of intermediate layers sequentially. These layers progressively transform the input data into decision tree predictions. To further enhance prediction performance, the output from the final intermediate layer is fed through a feed-forward neural network architecture for final prediction. We derive an upper bound of the disparity between the generalization error and the empirical error and demonstrate that it converges to zero. This highlights the generalizability of our method to parameter estimation and feature selection problems. In our experimental evaluations, DETs outperform existing models in terms of prediction accuracy, representation learning ability, and computational time. Specifically, the method achieves over 95% accuracy in gene expression data and can be trained on average 50% faster than traditional artificial neural networks (ANNs).

---

PaperID: 118,   

Authors:  An Yang, Ying Liu, Chunguang Li, Qinyuan Ren

Title: Deeply Supervised Block-Wise Neural Architecture Search

Abstract:
Neural architecture search (NAS) has shown great promise in automatically designing neural network models. Recently, block-wise NAS has been proposed to alleviate deep coupling problem between architectures and weights existed in the well-known weight-sharing NAS, by training the huge weight-sharing supernet block-wisely. However, the existing block-wise NAS methods, which resort to either supervised distillation or self-supervised contrastive learning scheme to enable block-wise optimization, take massive computational cost. To be specific, the former introduces an external high-capacity teacher model, while the latter involves supernet-scale momentum model and requires a long training schedule. Considering this, in this work, we propose a resource-friendly deeply supervised block-wise NAS (DBNAS) method. In the proposed DBNAS, we construct a lightweight deeply-supervised module after each block to enable a simple supervised learning scheme and leverage ground-truth labels to indirectly supervise optimization of each block progressively. Besides, the deeply-supervised module is specifically designed as structural and functional condensation of the supernet, which establishes global awareness for progressive block-wise optimization and helps search for promising architectures. Experimental results show that the DBNAS method only takes less than 1 GPU day to search out promising architectures on the ImageNet dataset with less GPU memory footprint than the other block-wise NAS works. The best-performing model among the searched DBNAS family achieves 75.6% Top-1 accuracy on ImageNet, which is competitive with the state-of-the-art NAS models. Moreover, our DBNAS family models also achieve good transfer performance on CIFAR-10/100, as well as two downstream tasks: object detection and semantic segmentation.

---

PaperID: 119,   

Authors:  Pourya Shamsolmoali, Masoumeh Zareapoor, Swagatam Das, Eric Granger, Salvador García

Title: Hybrid Gromov-Wasserstein Embedding for Capsule Learning

Abstract:
Capsule networks (CapsNets) aim to parse images into a hierarchy of objects, parts, and their relationships using a two-step process involving part–whole transformation and hierarchical component routing. However, this hierarchical relationship modeling is computationally expensive, which has limited the wider use of CapsNet despite its potential advantages. The current state of CapsNet models primarily focuses on comparing their performance with capsule baselines, falling short of achieving the same level of proficiency as deep convolutional neural network (CNN) variants in intricate tasks. To address this limitation, we present an efficient approach for learning capsules that surpasses canonical baseline models and even demonstrates superior performance compared with high-performing convolution models. Our contribution can be outlined in two aspects: first, we introduce a group of subcapsules onto which an input vector is projected. Subsequently, we present the hybrid Gromov–Wasserstein (HGW) framework, which initially quantifies the dissimilarity between the input and the components modeled by the subcapsules, followed by determining their alignment degree through optimal transport (OT). This innovative mechanism capitalizes on new insights into defining alignment between the input and subcapsules, based on the similarity of their respective component distributions. This approach enhances CapsNets’ capacity to learn from intricate, high-dimensional data while retaining their interpretability and hierarchical structure. Our proposed model offers two distinct advantages: 1) its lightweight nature facilitates the application of capsules to more intricate vision tasks, including object detection; and 2) it outperforms baseline approaches in these demanding tasks. Our empirical findings illustrate that HGW capsules (HGWCapsules) exhibit enhanced robustness against affine transformations, scale effectively to larger datasets, and surpass CNN and CapsNet models across various vision tasks.

---

PaperID: 120,   

Authors:  Paul Jungmann, Julia Poray, Akash Kumar

Title: Analytical Uncertainty Propagation in Neural Networks

Abstract:
The usage of machine-learning techniques, such as neural networks, is common in a large variety of domains. Estimating the certainty of a predicted value is important when precise information is gained. Nevertheless, the forward propagation of uncertainty in machine-learning models is hardly understood. In general, providing error bars for measurements (measurement uncertainty) is crucial when high precision is needed for decision-making. The objective of this work is the development of an analytical method for aleatoric uncertainty forward propagation in neural networks, based on analytical uncertainty propagation well known from physics and engineering. With that, the method gives provable correct results. A benefit is that the method does not require a different training procedure, but only needs the weights and biases of the neural network and is computationally inexpensive. The analytical method is applied to real-world examples from the semiconductor industry (regression and image classification). Its usefulness is demonstrated by the provided examples, which show how meaningful error bars are when machine learning may be used for decision-making.

---

PaperID: 121,   

Authors:  Boyu Dong, Dong Chen, Yu Wu, Siliang Tang, Yueting Zhuang

Title: FADngs: Federated Learning for Anomaly Detection

Abstract:
With the increasing demand for data privacy, federated learning (FL) has gained popularity for various applications. Most existing FL works focus on the classification task, overlooking those scenarios where anomaly detection may also require privacy-preserving. Traditional anomaly detection algorithms cannot be directly applied to the FL setting due to false and missing detection issues. Moreover, with common aggregation methods used in FL (e.g., averaging model parameters), the global model cannot keep the capacities of local models in discriminating anomalies deviating from local distributions, which further degrades the performance. For the aforementioned challenges, we propose Federated Anomaly Detection with Noisy Global Density Estimation, and Self-supervised Ensemble Distillation (FADngs). Specifically, FADngs aligns the knowledge of data distributions from each client by sharing processed density functions. Besides, FADngs trains local models in an improved contrastive learning way that learns more discriminative representations specific for anomaly detection based on the shared density functions. Furthermore, FADngs aggregates capacities by ensemble distillation, which distills the knowledge learned from different distributions to the global model. Our experiments demonstrate that the proposed method significantly outperforms state-of-the-art federated anomaly detection methods. We also empirically show that the shared density function is privacy-preserving. The code for the proposed method is provided for research purposes https://github.com/kanade00/Federated_Anomaly_detection.

---

PaperID: 122,   

Authors:  Yi Liu, De Cheng, Dingwen Zhang, Shoukun Xu, Jungong Han

Title: Capsule Networks With Residual Pose Routing

Abstract:
Capsule networks (CapsNets) have been known difficult to develop a deeper architecture, which is desirable for high performance in the deep learning era, due to the complex capsule routing algorithms. In this article, we present a simple yet effective capsule routing algorithm, which is presented by a residual pose routing. Specifically, the higher-layer capsule pose is achieved by an identity mapping on the adjacently lower-layer capsule pose. Such simple residual pose routing has two advantages: 1) reducing the routing computation complexity and 2) avoiding gradient vanishing due to its residual learning framework. On top of that, we explicitly reformulate the capsule layers by building a residual pose block. Stacking multiple such blocks results in a deep residual CapsNets (ResCaps) with a ResNet-like architecture. Results on MNIST, AffNIST, SmallNORB, and CIFAR-10/100 show the effectiveness of ResCaps for image classification. Furthermore, we successfully extend our residual pose routing to large-scale real-world applications, including 3-D object reconstruction and classification, and 2-D saliency dense prediction. The source code has been released on https://github.com/liuyi1989/ResCaps.

---

PaperID: 123,   

Authors:  Vasco Lopes, Luís A. Alexandre

Title: Toward Less Constrained Macro-Neural Architecture Search

Abstract:
Networks found with neural architecture search (NAS) achieve the state-of-the-art performance in a variety of tasks, out-performing human-designed networks. However, most NAS methods heavily rely on human-defined assumptions that constrain the search: architecture’s outer skeletons, number of layers, parameter heuristics, and search spaces. In addition, common search spaces consist of repeatable modules (cells) instead of fully exploring the architecture’s search space by designing entire architectures (macro-search). Imposing such constraints requires deep human expertise and restricts the search to predefined settings. In this article, we propose less constrained macro-neural architecture search (LCMNAS), a method that pushes NAS to less constrained search spaces by performing macro-search without relying on predefined heuristics or bounded search spaces. LCMNAS introduces three components for the NAS pipeline: 1) a method that leverages information about well-known architectures to autonomously generate complex search spaces based on weighted directed graphs (WDGs) with hidden properties; 2) an evolutionary search strategy that generates complete architectures from scratch; and 3) a mixed-performance estimation approach that combines information about architectures at the initialization stage and lower fidelity estimates to infer their trainability and capacity to model complex functions. We present experiments in 14 different datasets showing that LCMNAS is capable of generating both cell and macro-based architectures with minimal GPU computation and state-of-the-art results. Moreover, we conduct extensive studies on the importance of different NAS components in both cell and macro-based settings. The code for reproducibility is publicly available at https://github.com/VascoLopes/LCMNAS.

---

PaperID: 124,   

Authors:  Alaa Tharwat, Wolfram Schenck

Title: Active Learning for Handling Missing Data

Abstract:
Recently, the massive growth of IoT devices and Internet data, which are widely used in many applications, including industry and healthcare, has dramatically increased the amount of free unlabeled data collected. However, this unlabeled data is useless if we want to learn supervised machine learning models. The expensive and time-consuming cost of labeling makes the problem even more challenging. Here, the active learning (AL) technique provides a solution by labeling small but highly informative and representative data, which guarantees a high degree of generalizability over space and improves classification performance with data we have never seen before. The task is more difficult when the active learner has no predefined knowledge, such as initial training data, and when the obtained data is incomplete (i.e., contains missing values). In previous studies, the missing data should first be imputed. Then, the active learner selects from the available unlabeled data, regardless of whether the points were originally observed or imputed. However, selecting inaccurate imputed data points would negatively affect the active learner and prevent it from selecting informative and/or representative points, thus reducing the overall classification performance of the prediction models. This motivated us to introduce a novel query selection strategy that accounts for imputation uncertainty when querying new points. For this purpose, we first introduce a novel multiple imputation method that considers feature importance in selecting the most promising feature groups for missing value estimation. This multiple imputation method provides the ability to quantify the imputation uncertainty of each imputed data point. Furthermore, in each of the two phases of the proposed active learner (exploration and exploitation), imputation uncertainty is taken into account to reduce the probability of selecting points with high imputation uncertainty. We tested the effectiveness of the proposed active learner on different binary and multiclass datasets with different missing rates.

---

PaperID: 125,   

Authors:  Zehua Sun, Yonghui Xu, Yong Liu, Wei He, Lanju Kong, Fangzhao Wu, Yali Jiang, Lizhen Cui

Title: A Survey on Federated Recommendation Systems

Abstract:
Federated learning has recently been applied to recommendation systems to protect user privacy. In federated learning settings, recommendation systems can train recommendation models by collecting the intermediate parameters instead of the real user data, which greatly enhances user privacy. In addition, federated recommendation systems (FedRSs) can cooperate with other data platforms to improve recommendation performance while meeting the regulation and privacy constraints. However, FedRSs face many new challenges such as privacy, security, heterogeneity, and communication costs. While significant research has been conducted in these areas, gaps in the surveying literature still exist. In this article, we: 1) summarize some common privacy mechanisms used in FedRSs and discuss the advantages and limitations of each mechanism; 2) review several novel attacks and defenses against security; 3) summarize some approaches to address heterogeneity and communication costs problems; 4) introduce some realistic applications and public benchmark datasets for FedRSs; and 5) present some prospective research directions in the future. This article can guide researchers and practitioners understand the research progress in these areas.

---

PaperID: 126,   

Authors:  Delong Chen, Zhao Wu, Fan Liu, Zaiquan Yang, Shaoqiu Zheng, Ying Tan, Erjin Zhou

Title: ProtoCLIP: Prototypical Contrastive Language Image Pretraining

Abstract:
Contrastive language image pretraining (CLIP) has received widespread attention since its learned representations can be transferred well to various downstream tasks. During the training process of the CLIP model, the InfoNCE objective aligns positive image–text pairs and separates negative ones. We show an underlying representation grouping effect during this process: the InfoNCE objective indirectly groups semantically similar representations together via randomly emerged within-modal anchors. Based on this understanding, in this article, prototypical contrastive language image pretraining (ProtoCLIP) is introduced to enhance such grouping by boosting its efficiency and increasing its robustness against the modality gap. Specifically, ProtoCLIP sets up prototype-level discrimination between image and text spaces, which efficiently transfers higher level structural knowledge. Furthermore, prototypical back translation (PBT) is proposed to decouple representation grouping from representation alignment, resulting in effective learning of meaningful representations under a large modality gap. The PBT also enables us to introduce additional external teachers with richer prior language knowledge. ProtoCLIP is trained with an online episodic training strategy, which means it can be scaled up to unlimited amounts of data. We trained our ProtoCLIP on conceptual captions (CCs) and achieved an +5.81% ImageNet linear probing improvement and an +2.01% ImageNet zero-shot classification improvement. On the larger YFCC-15M dataset, ProtoCLIP matches the performance of CLIP with 33% of training time.

---

PaperID: 127,   

Authors:  Kaihang Jiang, Wai Keung Wong, Xiaozhao Fang, Jiaxing Li, Jianyang Qin, Shengli Xie

Title: Random Online Hashing for Cross-Modal Retrieval

Abstract:
In the past decades, supervised cross-modal hashing methods have attracted considerable attentions due to their high searching efficiency on large-scale multimedia databases. Many of these methods leverage semantic correlations among heterogeneous modalities by constructing a similarity matrix or building a common semantic space with the collective matrix factorization method. However, the similarity matrix may sacrifice the scalability and cannot preserve more semantic information into hash codes in the existing methods. Meanwhile, the matrix factorization methods cannot embed the main modality-specific information into hash codes. To address these issues, we propose a novel supervised cross-modal hashing method called random online hashing (ROH) in this article. ROH proposes a linear bridging strategy to simplify the pair-wise similarities factorization problem into a linear optimization one. Specifically, a bridging matrix is introduced to establish a bidirectional linear relation between hash codes and labels, which preserves more semantic similarities into hash codes and significantly reduces the semantic distances between hash codes of samples with similar labels. Additionally, a novel maximum eigenvalue direction (MED) embedding method is proposed to identify the direction of maximum eigenvalue for the original features and preserve critical information into modality-specific hash codes. Eventually, to handle real-time data dynamically, an online structure is adopted to solve the problem of dealing with new arrival data chunks without considering pairwise constraints. Extensive experimental results on three benchmark datasets demonstrate that the proposed ROH outperforms several state-of-the-art cross-modal hashing methods.

---

PaperID: 128,   

Authors:  Chengzhi Cao, Xueyang Fu, Yurui Zhu, Zhijing Sun, Zheng-Jun Zha

Title: Event-Driven Video Restoration With Spiking-Convolutional Architecture

Abstract:
With high temporal resolution, high dynamic range, and low latency, event cameras have made great progress in numerous low-level vision tasks. To help restore low-quality (LQ) video sequences, most existing event-based methods usually employ convolutional neural networks (CNNs) to extract sparse event features without considering the spatial sparse distribution or the temporal relation in neighboring events. It brings about insufficient use of spatial and temporal information from events. To address this problem, we propose a new spiking-convolutional network (SC-Net) architecture to facilitate event-driven video restoration. Specifically, to properly extract the rich temporal information contained in the event data, we utilize a spiking neural network (SNN) to suit the sparse characteristics of events and capture temporal correlation in neighboring regions; to make full use of spatial consistency between events and frames, we adopt CNNs to transform sparse events as an extra brightness prior to being aware of detailed textures in video sequences. In this way, both the temporal correlation in neighboring events and the mutual spatial information between the two types of features are fully explored and exploited to accurately restore detailed textures and sharp edges. The effectiveness of the proposed network is validated in three representative video restoration tasks: deblurring, super-resolution, and deraining. Extensive experiments on synthetic and real-world benchmarks have illuminated that our method performs better than existing competing methods.

---

PaperID: 129,   

Authors:  Yunyun Wang, Weiwen Zheng, Qinghao Li, Songcan Chen

Title: Dual-Correction-Adaptation Network for Noisy Knowledge Transfer

Abstract:
Unsupervised domain adaptation (UDA) promotes target learning via a single -directional transfer from label-rich source domain to unlabeled target, while its reverse adaption from target to source has not been jointly considered yet. In real teaching practice, a teacher helps students learn and also gets promotion from students, and such a virtuous cycle inspires us to explore dual -directional transfer between domains. In fact, target pseudo-labels predicted by source commonly involve noise due to model bias; moreover, source domain usually contains innate noise, which inevitably aggravates target noise, leading to noise amplification. Transfer from target to source exploits target knowledge to rectify the adaptation, consequently enables better source transfer, and exploits a virtuous transfer circle. To this end, we propose a dual-correction–adaptation network (DualCAN), in which adaptation and correction cycle between domains, such that learning in both domains can be boosted gradually. To the best of our knowledge, this is the first naive attempt of dual-directional adaptation. Empirical results validate DualCAN with remarkable performance gains, particularly for extreme noisy tasks (e.g., approximately +10% on D \rightarrow A of Office-31 with 40% label corruption).

---

PaperID: 130,   

Authors:  Shiye Lei, Fengxiang He, Yancheng Yuan, Dacheng Tao

Title: Understanding Deep Learning via Decision Boundary

Abstract:
This article discovers that the neural network (NN) with lower decision boundary (DB) variability has better generalizability. Two new notions, algorithm DB variability and (\epsilon, \eta) -data DB variability, are proposed to measure the DB variability from the algorithm and data perspectives. Extensive experiments show significant negative correlations between the DB variability and the generalizability. From the theoretical view, two lower bounds based on algorithm DB variability are proposed and do not explicitly depend on the sample size. We also prove an upper bound of order \mathcal O\left ((1/\sqrt m)+\epsilon +\eta \log (1/\eta )\right) based on data DB variability. The bound is convenient to estimate without the requirement of labels and does not explicitly depend on the network size which is usually prohibitively large in deep learning.

---

PaperID: 131,   

Authors:  Rui Lin, Jason Chun Lok Li, Jiajun Zhou, Binxiao Huang, Jie Ran, Ngai Wong

Title: Lite It Fly: An All-Deformable-Butterfly Network

Abstract:
Most deep neural networks (DNNs) consist fundamentally of convolutional and/or fully connected layers, wherein the linear transform can be cast as the product between a filter matrix and a data matrix obtained by arranging feature tensors into columns. Recently proposed deformable butterfly (DeBut) decomposes the filter matrix into generalized, butterfly-like factors, thus achieving network compression orthogonal to the traditional ways of pruning or low-rank decomposition. This work reveals an intimate link between DeBut and a systematic hierarchy of depthwise and pointwise convolutions, which explains the empirically good performance of DeBut layers. By developing an automated DeBut chain generator, we show for the first time the viability of homogenizing a DNN into all DeBut layers, thus achieving extreme sparsity and compression. Various examples and hardware benchmarks verify the advantages of All-DeBut networks. In particular, we show it is possible to compress a PointNet to <5% parameters with <5% accuracy drop, a record not achievable by other compression schemes.

---

PaperID: 132,   

Authors:  Honghui Wang, Yifan Pu, Shiji Song, Gao Huang

Title: Advancing Generalization in PINNs Through Latent-Space Representations

Abstract:
Physics-informed neural networks (PINNs) have made significant strides in modeling dynamical systems governed by partial differential equations (PDEs). However, their generalization capabilities across varying scenarios remain limited. To overcome this limitation, we propose physics-informed dynamics representation learner (PiDo), a novel physics-informed neural PDE solver designed to generalize effectively across diverse PDE configurations, including varying initial conditions, PDE coefficients, and training-time horizons. PiDo exploits the shared underlying structure of dynamical systems with different properties by projecting PDE solutions into a latent space using auto-decoding. It then learns the dynamics of these latent representations, conditioned on the PDE coefficients. Despite its promise, integrating latent dynamics models within a physics-informed framework poses challenges due to the optimization difficulties associated with physics-informed losses. To address these challenges, we introduce a novel approach that diagnoses and mitigates these issues within the latent space. This strategy employs straightforward yet effective regularization techniques, enhancing both the temporal extrapolation performance and the training stability of PiDo. We validate PiDo on a range of benchmarks, including 1-D combined equations and 2-D Navier–Stokes equations. In addition, we demonstrate the transferability of its learned representations to downstream applications such as long-term integration and inverse problems.

---

PaperID: 133,   

Authors:  Guo-Sen Xie, Junyi Li, Ting Guo, Xiangbo Shu, Fang Zhao, Zheng Zhang, Ling Shao

Title: Attribute Prompt Alignment Network for Zero-Shot Learning

Abstract:
In the vanilla zero-shot learning (ZSL) paradigm, category attributes is the key for knowledge generalizable transfer from seen to unseen classes. By contrast, the current contrastive language-image pretraining (CLIP) model relies on the category names to achieve a more general ZSL-like prediction. When vanilla ZSL meets general CLIP, however, most existing methods on both sides struggle to benefit from each other. In this brief, we resort to attribute prompt tuning (APT) for improving the knowledge transferability from the pretrained CLIP model to the downstream ZSL framework for pursuing desirable feature representations. Our approach, termed as attribute prompt alignment network (APAN), leverages APT for cross-network feature alignment (CFA). In this way, we can investigate the effects of CLIP to vanilla ZSL task in the era of large model by the two branch APAN architecture. Specifically, APT takes as an input the templates of class attribute descriptions to produce attribute prompts, which are further used to both guide the localizations of visual regions across two frozen feature extraction networks, through a visual-semantic interaction attention. This enables APAN to progressively refine and align these cross-network features, thus resulting in generalizable feature representations that can capture fine-grained attribute information. For CFA, we simply introduce prediction alignment loss that constrains the predictions from these two cross-network visual features. Experimental results on three benchmark datasets well demonstrate that APAN outperforms the state-of-the-art methods by absorbing generalizable knowledge from CLIP models.

---

PaperID: 134,   

Authors:  Xun Wang, Jingmian Wang, Zhuzhong Qian, Bolei Zhang

Title: Online Adaptable Offline RL With Guidance Model

Abstract:
Reinforcement learning (RL) has emerged as a promising approach across various applications, yet its reliance on repeated trial-and-error learning to develop effective policies from scratch poses significant challenges for deployment in scenarios where interaction is costly or constrained. In this work, we investigate the offline-to-online RL paradigm, wherein policies are initially pretrained using offline historical datasets and subsequently fine-tuned with a limited amount of online interaction. Previous research has suggested that efficient offline pretraining is crucial for achieving optimal final performance. However, it is challenging to incorporate appropriate conservatism to prevent the overestimation of out-of-distribution (OOD) data while maintaining adaptability for online fine-tuning. To address these issues, we propose an effective offline RL algorithm that integrates a guidance model to introduce suitable conservatism and ensure seamless adaptability to online fine-tuning. Our rigorous theoretical analysis and extensive experimental evaluations demonstrate better performance of our novel algorithm, underscoring the critical role played by the guidance model in enhancing its efficacy.

---

PaperID: 135,   

Authors:  Pengfei Jiao, Yuanqi Liu, Yinghui Wang, Huijun Tang, Zhidong Zhao, Shirui Pan

Title: Interactive Graph Learning for Multilevel Network Alignment

Abstract:
The task of network alignment aims to identify corresponding nodes across multiple networks, with applications in various fields such as social network analysis and bioinformatics. Traditional methods typically focus on the topological structure of networks at a specific level, but they may overlook important properties exhibited by many networks, such as scale-free properties and specific power-law structures often found in social networks. Consequently, these methods fail to effectively capture and utilize such information, leading to misalignment. In this article, we propose a network alignment framework that incorporates both topological and attribute information from multiple levels in the network, including homogeneity, power-law, and higher order structures. We introduce a Euclidean hyperbolic interactive graph learning method specifically designed for modeling power-law structures in networks, aiming to improve the accuracy of network alignment. To evaluate the effectiveness of our proposed method, we conduct experiments on several real-world datasets. The results demonstrate that our approach achieves higher accuracy compared to other advanced baselines.

---

PaperID: 136,   

Authors:  Jiabin Liu, Bo Wang, Yuping Zhang, Huadong Wang, Biao Li, Xin Shen, Gang Kou

Title: Progressive Training for Learning From Label Proportions

Abstract:
Learning from label proportions (LLPs), which aims to learn an instance-level classifier using proportion-based grouped training data, has garnered increasing attention in the field of machine learning. Existing deep learning-based LLP methods employ end-to-end pipelines to derive proportional loss functions via the Kullback–Leibler (KL) divergence between bag-level prior and posterior class distributions. However, the optimal solutions of these methods often struggle to conform to the given proportions, inevitably leading to degradation in the final classification performance. In this article, we address this issue by proposing a novel progressive training method for LLP, termed PT-LLP, which strives to meet the proportion constraints from the bag level to the instance level. Specifically, we first train a model by using the existing KL-divergence-based LLP methods that are consistent with bag-level proportion information. Then, we impose additional constraints on strict proportion consistency to the classifier to further move toward a more ideal solution by reformulating it as a constrained optimization problem, which can be efficiently solved using optimal transport (OT) algorithms. In particular, the knowledge distillation is employed as a transition stage to transfer the bag-level information to the instance level using a teacher–student framework. Finally, our framework is model-agnostic and demonstrates significant performance improvements through extensive experiments on different datasets when incorporated into other deep LLP methods as the first training stage.

---

PaperID: 137,   

Authors:  Xiangmin Han, Rundong Xue, Jingxi Feng, Yifan Feng, Shaoyi Du, Jun Shi, Yue Gao

Title: Hypergraph Foundation Model for Brain Disease Diagnosis

Abstract:
The goal of the hypergraph foundation model (HGFM) is to learn an encoder based on the hypergraph computational paradigm through self-supervised pretraining on high-order correlation structures, enabling the encoder to rapidly adapt to various downstream tasks in scenarios, where no labeled data or only a small amount of labeled data are available. The initial exploratory work has been applied to brain disease diagnosis tasks. However, existing methods primarily rely on graph-based approaches to learn low-order correlation patterns between brain regions in brain networks, neglecting the modeling and learning of complex correlations between different brain diseases and patients. This article proposes an HGFM for brain disease diagnosis, which conducts multidimensional pretraining tasks to explore latent cross-dimensional high-order correlation patterns on various brain disease datasets. HGFM is a high-order correlation-driven foundation model for brain disease diagnosis and effectively improves prediction performance. Specifically, HGFM first performs brain functional network link prediction tasks on individual brain networks and group interaction network link prediction tasks on group brain networks, constructing an HGFM for brain disease diagnosis. In downstream tasks, it achieves predictions for different brain disease diagnosis tasks through few-shot learning fine-tuning methods. The proposed method is evaluated on functional magnetic resonance imaging (fMRI) data from 4409 patients across four brain diseases. Results show that it outperforms existing state-of-the-art methods in all brain disease diagnosis tasks, demonstrating its potential value in clinical applications.

---

PaperID: 138,   

Authors:  Mengzhou Gao, Xinxun Zhang, Pengfei Jiao, Tianpeng Li, Zhidong Zhao

Title: DVGMAE: Self-Supervised Dynamic Variational Graph Masked Autoencoder

Abstract:
Although contrastive self-supervised learning (SSL) on dynamic graphs has made significant success, the issue of heavy reliance on data augmentation and training tricks has been a persistent pain point. Generative SSL, especially masked autoencoders (MAEs) have recently produced promising results and can avoid these issues. However, the research on MAE in dynamic graphs remains largely unexplored due to the following challenges: 1) how to design an effective masking strategy for dynamic graphs? and 2) how to design a decoder to retain temporal dependency when graphs are perturbed? In this article, we propose DVGMAE, a novel dynamic variational graph masked autoencoder model to solve these challenges. DVGMAE simultaneously captures the evolving behaviors and topological features via an innovative masking strategy and an elaborate decoder. Specifically, we first implement a temporal-aware masking strategy on the edges of each snapshot based on the updated probabilities derived from historical mask information. This strategy mitigates potential masking bias in dynamic graphs. We then design a globally enhanced decoder to recover the temporal and spatial information of each snapshot. Extensive experiments demonstrate that DVGMAE outperforms the existing state-of-the-art on various tasks across different datasets.

---

PaperID: 139,   

Authors:  Feng Xiao, Youqing Wang, S. Joe Qin, Jicong Fan

Title: Semi-Supervised Anomaly Detection Using Restricted Distribution Transformation

Abstract:
Anomaly detection (AD) is typically regarded as an unsupervised learning task, where the training data either do not contain any anomalous samples or contain only a few unlabeled anomalous samples. In fact, in many real scenarios such as fault diagnosis and disease detection, a small number of anomalous samples labeled by domain experts are often available during the training phase, which makes semi-supervised AD (SAD) more appealing, though the related study is quite limited. Existing semi-supervised AD methods directly add optimization terms of anomalous samples to the optimization objective of unsupervised AD (UAD), where the effects of the limited labeled anomalous data on the optimization process become trivial and they cannot fully contribute to the detection task. To cover the shortage, in this work, we propose a novel semi-supervised AD method to fully use the limited labeled anomalous data and further to boost detection performance. The proposed method learns a nonlinear transformation to project normal data into a compact target distribution and simultaneously to project exposed anomalous samples into another target distribution, where the two target distributions do not overlap each other. The goal is difficult to achieve because of the scarcity of anomalous samples. To address this problem, we propose to generate a large number of intermediate samples interpolating between normal and anomalous data and project them into a third target distribution lying between the aforementioned two target distributions. Empirical results on multiple benchmarks with varying domains demonstrate the superiority of our method over existing supervised and semi-supervised AD methods.

---

PaperID: 140,   

Authors:  Shiben Liu, Huijie Fan, Qiang Wang, Xi'ai Chen, Zhi Han, Yandong Tang

Title: Diverse Representations Embedding for Lifelong Person Re-Identification

Abstract:
Lifelong person re-identification (LReID) aims to continuously learn from sequential data streams, enabling cross-camera matching of individuals over time. A critical challenge in LReID lies in balancing the preservation of previously acquired knowledge with the incremental acquisition of new information, due to task-level gaps and limited representation capacity. Conventional methods relying on CNN backbones struggle to fully capture the diverse perspectives of each instance, leading to suboptimal model performance. To tackle these limitations, we propose a diverse representation embedding (DRE) framework that balances preserving old knowledge with adapting to new information. Specifically, our DRE incorporates a robust Transformer-based backbone that utilizes maximum embedding (ME) and multiple class tokens to generate overlapping representations for each instance. To further enhance the model’s representation capacity, we design an adaptive constraint module (ACM), which performs integration and discrimination operations on overlapping representations to yield diverse yet diverse representations. Furthermore, we propose two strategies: knowledge update (KU) and knowledge preservation (KP), implemented within the adjustment and learner models, respectively. The KU strategy enhances the learner model’s ability to adapt to new information by leveraging prior knowledge from the adjustment model. The KP strategy ensures the retention of historical knowledge while maintaining the model’s adaptability. Extensive experiments validate that our DRE surpasses state-of-the-art approaches across large-scale, occluded, and holistic datasets, demonstrating significant performance gains. Our code is available at https://github.com/LiuShiBen/DRE

---

PaperID: 141,   

Authors:  Meng Xu, Xinhong Chen, Guanyi Zhao, Zihao Wen, Weiwei Fu, Jianping Wang

Title: Neighboring State-Aware Policy for Deep Reinforcement Learning

Abstract:
Deep reinforcement learning (DRL) methods, which train a policy to obtain the sequence of actions required to complete a task, have achieved remarkable success across diverse applications. It is a long-standing open issue in the DRL community to make the trained policy gradually approach the theoretically globally optimal policy, and existing research has also explored several challenges, such as exploration-exploitation, to improve the quality of the obtained policy. However, most DRL methods rely solely on the current state for decision-making, leading to short-sightedness and suboptimal learning. To overcome this, we propose a neighboring state-aware policy that enhances existing DRL methods by incorporating a neighboring state sequence in the decision-making process. Specifically, our approach saves multiple past and future states and concatenates them as the neighboring state sequence, along with the current state, and inputs them to the actor to generate an action during the training process. This global perspective, provided by neighboring states, is similar to human decision-making and helps the agent better understand state evolution, leading to improved policy learning. We present two specific implementations of our approach and demonstrate through extensive experiments that it effectively enhances ten representative DRL methods across nine tasks, based on three metrics, including return.

---

PaperID: 142,   

Authors:  Tengfei Yan, Jiankai Tu, Chunguang Li, Fan Zhang

Title: Topology-Preserved Information Bottleneck for Multiview Anomaly Detection

Abstract:
Anomaly detection (AD) techniques are widely used in various fields. Existing techniques primarily focus on learning a normal region from single-view data, which may be not suitable for multiview data that provides more comprehensive information from multiple perspectives. Therefore, AD techniques designed for multiview data are necessary. Straightforwardly, one can concatenate the features learned from multiple single-view data into a joint representation to conduct AD. However, this may overlook the inevitable overlaps between views, potentially masking view-specific information due to the repetitive calculations of these overlaps. Among the various possible methods, one way to address this is to compress redundant information while maintaining comprehensive information across views. Following this way, in this article, we leverage the principle of information bottleneck (IB) to extract concise and comprehensive representations for multiview data. But it is problematic to directly use these representations for AD, since the multiview fusion process may disturb the intrinsic structure of the original data. That is, samples distributed at the edges/center of the original normal data distribution are mapped closer to the center/edges. This might cause abnormal samples (close to the normal data at the edges) to be incorrectly mapped into the normal region during inference. In the AD scenario, the absence of abnormal training samples makes it unfeasible to preserve this structure using supervised information. In this article, we design a topology-preserved regularization that unsupervisedly constrains the latent representations to preserve the original data’s intrinsic structure, to improve the AD performance. Overall, we propose a topology-preserved multiview information bottleneck (TMVIB) feature extraction method to extract concise, comprehensive, and topology-preserved latent representations from multiview data. Interestingly, we find that the TMVIB feature extraction method itself can be viewed as a regularized anomaly detector, allowing it to output anomaly scores directly. Experiments on synthetic and real-world multiview datasets demonstrate the effectiveness of the proposed TMVIB.

---

PaperID: 143,   

Authors:  Longkun Guo, Chaoqi Jia, Kewen Liao, Zhigang Lu, Minhui Xue

Title: Near-Optimal Algorithms for Instance-Level Constrained k-Center Clustering

Abstract:
Many practical applications impose a new challenge of utilizing instance-level background knowledge (e.g., subsets of similar or dissimilar data points) within their input data to improve clustering results. In this work, we build on the widely adopted k-center clustering, modeling its input instance-level background knowledge as must-link (ML) and cannot-link (CL) constraint sets, and formulate the constrained k-center problem. Given the long-standing challenge of developing efficient algorithms for constrained clustering problems, we first derive an efficient approximation algorithm for constrained k-center at the best possible approximation ratio of 2 with linear programming (LP)-rounding technology. Recognizing the limitations of LP-rounding algorithms including high runtime complexity and challenges in parallelization, we subsequently develop a greedy algorithm that does not rely on the LP and can be efficiently parallelized. This algorithm also achieves the same approximation ratio 2 but with lower runtime complexity. Lastly, we empirically evaluate our approximation algorithm against baselines on various real datasets, validating our theoretical findings and demonstrating significant advantages of our algorithm in terms of clustering cost, quality, and runtime complexity.

---

PaperID: 144,   

Authors:  Like Xin, Wanqi Yang, Lei Wang, Ming Yang

Title: Multilevel Reliable Guidance for Unpaired Multiview Clustering

Abstract:
In this article, we address the challenging problem of unpaired multiview clustering (UMC), which aims to achieve effective joint clustering using unpaired samples observed across multiple views. Traditional incomplete multiview clustering (IMC) methods typically rely on paired samples to capture complementary information between views. However, such strategies become impractical in the UMC due to the absence of paired samples. Although some researchers have attempted to address this issue by preserving consistent cluster structures across views, effectively mining such consistency remains challenging when the cluster structures with low confidence. Therefore, we propose a novel method, multilevel reliable guidance for UMC (MRG-UMC), which integrates multilevel clustering and reliable view guidance to learn consistent and confident cluster structures from three perspectives. Specifically, inner view multilevel clustering exploits high-confidence sample pairs across different levels to reduce the impact of boundary samples, resulting in more confident cluster structures. Synthesized-view alignment leverages a synthesized view to mitigate cross-view discrepancies and promote consistency. Cross-view guidance employs a reliable view guidance strategy to enhance the clustering confidence of poorly clustered views. These three modules are jointly optimized across multiple levels to achieve consistent and confident cluster structures. Furthermore, theoretical analyses verify the effectiveness of MRG-UMC in enhancing clustering confidence. Extensive experimental results show that MRG-UMC outperforms state-of-the-art UMC methods, achieving an average NMI improvement of 12.95% on multiview datasets. The source code is available at https://anonymous.4open.science/r/MRG-UMC-5E20

---

PaperID: 145,   

Authors:  Hao Nan Sheng, Zhi-Yong Wang, Hing Cheung So

Title: Robust Rank-One Matrix Completion via Explicit Regularizer

Abstract:
In robust matrix completion (MC), the Welsch function, also referred to as the maximum correntropy criterion with Gaussian kernel, has been widely employed. However, it suffers from the drawback of down-weighing normal data. This work is the first to uncover the explicit regularizer (ER) for the Welsch function based on the multiplicative form of half-quadratic (HQ) minimization. Leveraging this discovery, we develop a new function called t-Welsch, also with ER, which provides unity weight to normal data and exhibits stronger robustness against large-magnitude outliers compared to Huber’s weight. We apply the t-Welsch to rank-one matching pursuit, enabling accurate and robust low-rank matrix recovery without the need of rank information and singular value decomposition (SVD). The resultant MC algorithm is realized via block coordinate descent (BCD), whose analyses of convergence and computational complexity are produced. Experiments are conducted using synthetic random data, as well as real-world images with salt-and-pepper noise and multiple-input multiple-output (MIMO) radar signals in the presence of Gaussian mixture disturbances. In all three scenarios, the proposed algorithm outperforms the state-of-the-art robust MC methods in terms of recovery accuracy. The code is available at https://github.com/ShuDun23/t-Welsch-and-RAR1MC.

---

PaperID: 146,   

Authors:  Jialong Wang, Zheng Wang, Zhiguo Gong

Title: Multihop Reconstruction for Generalized Zero-Shot Node Classification

Abstract:
Graphs in the real world keep evolving with the integration of new nodes, and it is often infeasible to manually label all the new nodes promptly. In this case, graph learning algorithms can come in handy and perform classification on these newly emerging nodes. Typically, if unseen classes exist (i.e., no training samples from these classes), one can perform zero-shot learning (ZSL) or generalized ZSL (GZSL). During testing, ZSL aims to classify samples within unseen classes, whereas GZSL aims to classify samples within both seen and unseen classes, which is even more challenging. In our previous work, we proposed a decomposed graph prototype network (DGPN) to decompose the graph convolution operation for handling the zero-shot node classification (ZNC) problem. However, DGPN is not well-suited for the generalized ZNC (GZNC) problem. To this end, in this article, we propose a novel graph generative model, multihop reconstruction graph autoencoder (MHR-GAE). Unlike DGPN, MHR-GAE utilizes a multihop encoder with class semantic descriptions (CSDs) (as condition signals) to reconstruct the information and generate nodes of unseen classes. Thus, it can handle both the ZNC and GZNC problems and obtain competitive performance. We evaluate our model on real-world datasets, and the experimental results demonstrate that MHR-GAE outperforms other baseline methods.

---

PaperID: 147,   

Authors:  Xue Qiao, Shuang Gu, Jiayuan Cheng, Chen Peng, Zhiwei Xiong, Hong Shen, Gan Jiang

Title: Fine-Grained Entity Recognition via Large Language Models

Abstract:
Fine-grained entity recognition (FGER) attracts increasing attention in information extraction and many other natural language understanding applications. However, it is a quite challenging problem for a specific domain due to the lack of specific-domain labeled data. To address this challenge, recent advancements in language modeling such as generative pretrained transformer (GPT) offer promising alternatives. Since large language models (LLMs) can be used for various tasks, such as text generation, summarization, and information extraction without labeled data, we incorporated them into the FGER field. Nonetheless, when too many verbose labels are fed to LLMs simultaneously, LLMs occasionally generate content that diverges from user input, contradicts previously generated context, or misaligns with established world knowledge, also called the “hallucination” phenomenon. In this article, we propose a new method called FGER-GPT to address these issues. Our approach leverages multiple inference chains and incorporates a hierarchical strategy for recognizing fine-grained entities, resulting in a significant performance boost. Importantly, neither coarse-grained nor fine-grained entity annotations are used in our proposed approach, which avoids the heavy labor consumption of labeling. Extensive experiments conducted on widely used datasets have demonstrated that the proposed FGER-GPT achieves competitive performance compared to state-of-the-art approaches in low-resource scenarios, highlighting its feasibility for real-world applications.

---

PaperID: 148,   

Authors:  Shenglun Chen, Xinzhu Ma, Hong Zhang, Haojie Li, Baoli Sun, Zhihui Wang

Title: Real-Time Depth Completion With Multimodal Feature Alignment

Abstract:
As a key problem in computer vision, depth completion aims to recover dense depth maps from sparse ones [generally derived from light detection and ranging (LiDAR)]. Most methods introduce synchronous RGB images and leverage multimodal fusion to integrate multimodal features from these modalities to describe the complete scene. However, their different natural characteristics lead to inconsistency in features, potentially impacting the effectiveness of multimodal feature fusion. To address this issue, we propose a feature alignment network (FANet) that introduces an alignment scheme to enhance the consistency between multimodal features. This scheme aligns the modality-invariant semantic context, which is invariant to changes in modality and represents the correlation between a pixel and its surroundings. Specifically, we first design an asymmetric context extraction (ACE) module to extract modality-invariant semantic contexts from multimodal features within limited GPU memory, and then pull them closer to improve consistency. Crucially, our alignment scheme is only applied during the training phase, and no additional computation cost is incurred in the inference phase. Moreover, we introduce a simple yet effective refinement module to refine estimated results via residual learning based on intermediate depth maps and sparse depth maps. Extensive experiments on KITTI and VOID datasets demonstrate that our method achieves competitive performance against typical real-time methods. In addition, we embed the proposed alignment scheme and refinement module into other methods to demonstrate their effectiveness.

---

PaperID: 149,   

Authors:  Jun Fu, Xianrui Ji, Dexiong Chen, Guosheng Hu, Shuang Li, Xiating Feng

Title: AdvMixUp: Adversarial MixUp Regularization for Deep Learning

Abstract:
Deep neural networks (DNNs) have shown significant progress in many application fields. However, overfitting remains a significant challenge in their development. While existing data-augmentation techniques such as MixUp have been successful in preventing overfitting, they often fail to generate hard mixed samples near the decision boundary, impeding model optimization. In this article, we present adversarial MixUp (AdvMixUp), a novel sample-dependent method for regularizing DNNs. AdvMixUp addresses this issue by incorporating adversarial training (AT) to create sample-dependent and feature-level interpolation masks, generating more challenging mixed samples. These virtual samples enable DNNs to learn more robust features, ultimately reducing overfitting. Empirical evaluations on CIFAR-10, CIFAR-100, Tiny-ImageNet, and ImageNet demonstrate that AdvMixUp outperforms existing MixUp variants.

---

PaperID: 150,   

Authors:  Jakub Kwiatkowski, Krzysztof Krawiec

Title: Staged Self-Supervised Learning for Raven Progressive Matrices

Abstract:
This study presents and investigates abstract compositional transformers (ACTs), a class of deep learning (DL) architectures based on the transformer blueprint, designed to handle abstract reasoning tasks that require completing spatial visual patterns. We combine ACTs with choice-making modules and apply them to Raven progressive matrices (RPMs), logical puzzles that require selecting the correct image from the available answers. We devise a number of ACT variants, train them in several modes and with additional augmentations, subject them to ablations, demonstrate their data scalability, and analyze their behavior and latent representations that emerged in the process. Using self-supervision allows us to successfully train ACTs on relatively small training sets, mitigate several biases identified in RPMs in past studies, and achieve SotA results on the two most popular RPM benchmarks.

---

PaperID: 151,   

Authors:  Yuheng Wang, Zhenping Lan, Yanguo Sun, Nan Wang, Jiansong Li, Xincheng Yang

Title: Few-Shot Learning Based on Multimodal Information Processing

Abstract:
Few-shot learning aims to develop models with strong generalization capabilities using a small number of training samples. However, most learning methods rely solely on the visual features of a few samples to represent entire categories, leading to poor category representativeness. In contrast, humans can utilize multimodal information to learn category features, thereby making them more representative. Hence, this article emulates the human multimodal learning mechanism by integrating visual features with textual information, thereby facilitating the model’s acquisition of more representative and robust category features. Specifically, this article introduces a novel multimodal fusion mechanism—the visual-semantic fusion selection mechanism (VSFSM)—which comprises a fusion selection module (FS-Module) and a category enhancement module (CE-Module). These two modules collaboratively enhance the model’s classification performance. The FS-Module aligns and fuses semantic information with visual features across both channel and spatial dimensions, performing feature selection and reconstruction. This process not only generates representative category features but also mitigates the impact of noise. The CE-Module guides the model to emphasize category-specific features in the query images, ultimately yielding representative visual-semantic category features while reducing the interference of noise in the query images. Additionally, to better facilitate few-shot learning, this article introduces a novel objective loss function for optimized training. Extensive comparative and ablation experiments conducted on multiple datasets further validate the effectiveness of the proposed method.

---

PaperID: 152,   

Authors:  Weicai Li, Tiejun Lv, Wei Ni, Jingbo Zhao, Ekram Hossain, H. Vincent Poor

Title: Route-and-Aggregate Decentralized Federated Learning Under Communication Errors

Abstract:
Decentralized federated learning (D-FL) allows clients to aggregate learning models locally, offering flexibility and scalability. Existing D-FL methods use gossip protocols, which are inefficient when not all nodes in the network are D-FL clients. This article puts forth a new D-FL strategy, termed route-and-aggregate (R&A) D-FL, where participating clients exchange models with their peers through established routes (as opposed to flooding) and adaptively normalize their aggregation coefficients to compensate for communication errors. The impact of routing and imperfect links on the convergence of R&A D-FL is analyzed, revealing that convergence is minimized when routes with the minimum end-to-end (E2E) packet error rates (PERs) are employed to deliver models. Our analysis is experimentally validated through three image classification tasks and two next-word prediction tasks, utilizing widely recognized datasets and models. R&A D-FL outperforms the flooding-based D-FL method in terms of training accuracy by 35% in our tested ten-client network, and shows strong synergy between D-FL and networking. In another test with ten D-FL clients, the training accuracy of R&A D-FL with communication errors approaches that of the ideal centralized federated learning (C-FL) without communication errors, as the number of routing nodes (i.e., nodes that do not participate in the training of D-FL) rises to 28.

---

PaperID: 153,   

Authors:  Yongri Piao, Zhi Wang, Tingwei Liu, Jihao Yin, Miao Zhang, Huchuan Lu

Title: Learning Discriminative Representation for Co-Salient Object Detection

Abstract:
Co-salient object detection (CoSOD) is the task of identifying and emphasizing the common salient objects in a collection of images. The current co-salient object detection frameworks often extract features and model interimage relations separately. Although these methods achieve promising performance in many scenes, separating the feature extraction and relation modeling falls short of obtaining discriminative features for co-salient objects, resulting in subperformance, especially in some complex and cluttered real-world scenes. In this article, we introduce a novel CoSOD framework to unify feature extraction and interimage relation modeling. We design an early token interaction module (ETIM) that bridges information flow between branches to simultaneously realize feature extraction and interimage information interaction. To further enhance our network’s capability to distinguish co-salient objects from other irrelevant foreground objects, we introduce a pixel-to-group contrastive (PGC) learning method. This approach aids in eliminating the need for additional interaction modules while preserving features’ discriminative power for co-salient objects. Our proposed CoSOD framework only includes a backbone embedded with ETIM, a decoder without interaction modules and a project head only used during the training phase. Extensive experiments on three challenging benchmarks, that is, CoCA, CoSOD3k, and Cosal2015, demonstrate that our proposed method can outperform current leading-edge models and achieve the new state-of-the-art. The source code is available at https://github.com/zhiwang98/LDRNet

---

PaperID: 154,   

Authors:  Zhiqiang Pang, Hong Wang, Qi Xie, Deyu Meng, Zongben Xu

Title: TRG-Net: An Interpretable and Controllable Rain Generator

Abstract:
Exploring and modeling the rain generation mechanism is critical for augmenting paired data to ease the training of rainy image processing models. Most of the conventional methods handle this task in an artificial physical rendering manner, through elaborately designing fundamental elements constituting rains. These kinds of methods, however, are over-dependent on human subjectivity, which limits their adaptability to real rains. In contrast, recent deep learning (DL) methods have achieved great success by training a neural network-based generator from pre-collected rainy image data. However, current methods usually design the generator in a “closed box” manner, increasing the learning difficulty and data requirements. To address these issues, this study proposes a novel DL-based rain generator, which fully takes the physical generation mechanism underlying rains into consideration and well encodes the learning of the fundamental rain factors (i.e., shape, orientation, length, width, and sparsity) explicitly into the deep network. Its significance lies in that the generator not only elaborately designs essential elements of the rain to simulate expected rains, like conventional artificial strategies, but also finely adapts to complicated and diverse practical rainy images, like DL methods. By rationally adopting the filter parameterization technique, the proposed rain generator is finely controllable with respect to rain factors and able to learn the distribution of these factors purely from data without the need for rain factor labels. Our unpaired generation experiments demonstrate that the rain generated by the proposed rain generator is not only of higher quality but also more effective for deraining and downstream tasks compared to current state-of-the-art rain generation methods. Besides, the paired data augmentation experiments, including both in-distribution and out-of-distribution (OOD), further validate the diversity of samples generated by our model for in-distribution deraining and OOD generalization tasks.

---

PaperID: 155,   

Authors:  Guozhi Liu, Weiwei Lin, Tiansheng Huang, Fang Shi, Wentai Wu, Li Shen

Title: AdaptiveFL: Communication-Adaptive Federated Learning Under Dynamic Bandwidth

Abstract:
Federated learning (FL) is a distributed machine learning paradigm that enables heterogeneous devices to train a model collaboratively. Recognizing communication as a bottleneck in FL, existing communication-efficient solutions, e.g., HeteroFL and LotteryFL, etc., utilize gradient sparsification to reduce communication costs. However, existing solutions fail to address the dynamic bandwidth issue in which the bandwidth of each client is constantly changing throughout the training process. In this article, we propose AdaptiveFL, a communication-adaptive FL framework, considering the dynamic constraints of bandwidth. The design of AdaptiveFL follows two key steps: 1) in each round, each device selects a best-fit sub-model for communication per currently available bandwidth; and 2) to guarantee the performance of each sub-model sent under dynamic bandwidth constraints, AdaptiveFL employs a local training method that enables each device to train a “tailorable” local model, which can be tailored to any sparsity with competitive accuracy. We compare AdaptiveFL with several communication-efficient SOTA methods and demonstrate that AdaptiveFL outperforms other baselines by a large margin.

---

PaperID: 156,   

Authors:  Chen Li, Zeyi Liu, Xiao He, Pengyu Han

Title: Factorization-Based Broad Learning System With Time-Dependent Structure

Abstract:
In response to the increasing complexity of tasks in artificial intelligence, broad learning systems (BLSs) have emerged as essential tools, especially given the limitations of deep neural networks, such as their extensive training and computational demands. This study addresses the computational inefficiencies and numerical instabilities inherent in traditional BLS when handling complex tasks in dynamic environments. To mitigate these challenges, we propose an enhanced version of BLS incorporating QR factorization (QRF), referred to as QRBLS, which is known for improving numerical stability. This framework replaces the traditional method of computing output weights, which typically relies on the Moore-Penrose pseudoinverse. The primary contribution of this article is the integration of QRF into the BLS architecture, thereby improving stability when processing large-scale datasets. QRBLS also features a dynamic updating mechanism that adjusts model parameters efficiently with new data, enabling continuous learning without the need for full-model re-evaluation. In addition, a time-dependent structure (TDS) enhances the model’s responsiveness to temporal data changes, increasing its utility in dynamic environments. Validation through numerical experiments demonstrated that QRBLS outperformed traditional BLS, exhibiting superior stability and adaptability in handling data anomalies and rapid updates. The integration of QRF and TDS significantly improves the adaptability and computational efficiency of BLS, providing a robust solution for large scale and dynamic AI applications. QRBLS effectively addresses challenges related to numerical instability and continuous learning, offering practical improvements in real-world settings.

---

PaperID: 157,   

Authors:  Xiang Hao, Chenxiang Ma, Qu Yang, Jibin Wu, Kay Chen Tan

Title: Toward Ultralow-Power Neuromorphic Speech Enhancement With Spiking-FullSubNet

Abstract:
Speech enhancement (SE) is critical for improving speech intelligibility and quality in various audio devices. In recent years, deep learning-based methods have significantly improved SE performance, but they often come with a high computational cost, which is prohibitive for a large number of edge devices, such as headsets and hearing aids. This work proposes an ultralow-power SE system based on the brain-inspired spiking neural network (SNN) called Spiking-FullSubNet. Spiking-FullSubNet follows a full-band and subband fusioned approach to effectively capture both global and local spectral information. To enhance the efficiency of computationally expensive subband modeling, we introduce a frequency partitioning method inspired by the sensitivity profile of the human peripheral auditory system. Furthermore, we introduce a novel spiking neuron model that can dynamically control the input information integration and forgetting, enhancing the multiscale temporal processing capability of SNN, which is critical for speech denoising. Experiments conducted on the recent Intel Neuromorphic Deep Noise Suppression (N-DNS) Challenge dataset show that the Spiking-FullSubNet surpasses state-of-the-art (SOTA) methods by large margins in terms of both speech quality and energy efficiency metrics. Notably, our system won the championship of the Intel N-DNS Challenge (algorithmic track), opening up a myriad of opportunities for ultralow-power SE at the edge. Our source code and model checkpoints are publicly available at github.com/haoxiangsnr/spiking-fullsubnet

---

PaperID: 158,   

Authors:  Xinyu Chen, Jian Wang, Jie Yang, Chao Zhang, Dacheng Tao

Title: (δ, ε)-K Segmentation for Characterizing Well-Clusterable Sets

Abstract:
Kleinberg (2002) introduced three axioms to formalize the behavior of clustering algorithms and presented that no clustering algorithm satisfies them. However, this result usually is inconsistent with the practical experience of clustering algorithms. In this article, we reformulate these axioms to fill the gap and verify the existence of clustering algorithms satisfying the modified axioms when the point set is well-clusterable. In particular, the concept of (\delta ,\varepsilon) –K segmentation is proposed to characterize the set that has the potential to be clustered well. Then, we verify the existence and the uniqueness of (\delta ,\varepsilon) –K segmentation of a set, respectively. Next, we demonstrate that the (\delta ,\varepsilon) –K segmentation is compatible with K–means, Min-Cut, DBSCAN, and several clustering internal evaluation (CIE) indexes. In addition, the ratio \delta / \varepsilon can be treated as the measure of not only characterizing well-clusterable sets but also of evaluating the performance of clustering results, respectively.

---

PaperID: 159,   

Authors:  Jinjing Shi, Ren-Xin Zhao, Wenxuan Wang, Shichao Zhang, Xuelong Li

Title: QSAN: A Near-Term Achievable Quantum Self-Attention Network

Abstract:
Self-attention mechanism (SAM) is good at capturing the intrinsic connection between features to dramatically boost the performance of machine learning models. Nevertheless, the capability of SAM is not equipped with many current quantum machine learning (QML) models, thus confining their expansion on massive high-dimensional quantum data. To address the above problems, a quantum SAM (QSAM) consisting of a quantum logic similarity (QLS)-based quantum bit self-attention score matrix (QBSASM) is introduced to augment the data representation of SAM exponentially. According to QSAM, the framework and quantum circuits of a one-step achievable quantum self-attention network (QSAN) are designed to consider measurement times compression fully. Moreover, a prototype of quantum coordinates is presented during the design process to describe the mathematical relationship between the output bits and the control bits to facilitate the programming. Ultimately, MNIST binary classification experiments on the PennyLane platform and comparisons with cutting-edge QML models demonstrate QSAN converges about 1.7× and 2.3× faster than hardware-efficient ansatz and quantum approximate optimization algorithm (QAOA) ansatz, respectively, with similar parameter configurations and 100% prediction accuracy, which indicates that it has a better learning capability. In the CIFAR-10 classification experiments, QSAN achieves high prediction accuracy at a small scale relative to classical machine learning models. Predictably, QSAN elevates the efficiency of QML models and lays the foundation for future quantum computers to perform machine learning on massive amounts of data while promoting the advancement of quantum computer vision and other fields.

---

PaperID: 160,   

Authors:  Haonan Xin, Danyang Wu, Jitao Lu, Rong Wang, Feiping Nie, Xuelong Li

Title: Multiview Clustering via Block Diagonal Graph Filtering

Abstract:
Graph-based multiview clustering methods have gained significant attention in recent years. In particular, incorporating graph filtering into these methods allows for the exploration and utilization of both feature and topological information, resulting in a commendable improvement in clustering accuracy. However, these methods still exhibit several limitations: 1) the graph filters are predetermined, which disconnects the link with subsequent clustering tasks and 2) the separability of the filtered features is poor, which may not be suitable for the clustering. To mitigate these aforementioned issues, we propose Multiview Clustering via Block Diagonal Graph Filtering (MvC-BDGF), which can learn cluster-friendly graph filters. Specifically, the block diagonal graph filter with localized characteristics, which could make the filtered features very discriminating, is innovatively designed. The MvC-BDGF model seamlessly integrates the learning of graph filters with the acquisition of consensus graphs, forming a unified framework. This integration allows the model to obtain optimal filters and simultaneously acquire corresponding clustering labels. To solve the optimization problem in the MvC-BDGF model, an iterative solver based on the coordinate descent method is devised. Finally, a large number of experiments on benchmark datasets fully demonstrate the effectiveness and superiority of the proposed model. The code is available at https://github.com/haonanxin/MvC-BDGF_code.

---

PaperID: 161,   

Authors:  Xijia Tang, Chao Xu, Hong Tao, Xiaoyu Ma, Chenping Hou

Title: Confidence-Based PU Learning With Instance-Dependent Label Noise

Abstract:
Positive and unlabeled (PU) learning, which trains binary classifiers using only PU data, has gained vast attentions in recent years. Traditional PU learning often assumes that all the positive samples are labeled accurately. Nevertheless, due to the reasons such as sample ambiguity and insufficient algorithms, label noise is almost unavoidable in this scenario. Current PU algorithms neglect the label noise issue in the positive set, which is often biased toward certain instances rather than being uniformly distributed in practical applications. We define this important but understudied problem as PU learning with instance-dependent label noise (PUIDN). To eliminate the adverse impact of IDN, we leverage confidence scores for each instance in the positive set, which establish the connection between samples and labels without any assumption on noise distribution. Then, we propose an unbiased estimator for classification risk considering both label and confidence information, which can be computed immediately from PUIDN data along with their confidence scores. Moreover, our classification framework integrates an optimization strategy of alternating iteration based on the correlation between different confidence information, thereby alleviating the additional requirement for training data. Theoretically, we derive a generalization error bound for our proposed method. Experimentally, the effectiveness of our approach is demonstrated through various types of numerical results.

---

PaperID: 162,   

Authors:  Lanlan Feng, Yipeng Liu, Ziming Liu, Ce Zhu

Title: Online Nonconvex Robust Tensor Principal Component Analysis

Abstract:
Robust tensor principal component analysis (RTPCA) based on tensor singular value decomposition (t-SVD) separates the low-rank component and the sparse component from the multiway data. For streaming data, online RTPCA (ORTPCA) processes tensor data sequentially, where the low-rank component is updated based on the latest estimation and the newly arrived sample. It enhances both computation and storage efficiency. However, in most of the existing ORTPCA methods, the relaxation from tensor multirank to the convex tensor nuclear norm (TNN) may have a certain modeling error, which leads to unavoidable tracking accuracy loss. In this article, a tensor Schatten-p norm ( 0\lt p\lt 1 ) is applied to provide a tighter approximation of the tensor rank. A Lemma is deduced to divide the Schatten-p norm into terms to be updated in an online way. Based on it, the corresponding online nonconvex RTPCA (ONRTPCA) method is proposed for efficient tensor subspace tracking. Moreover, we incorporate the dynamic forgetting window into ONRTPCA to adaptively track varying subspaces. In addition, this article also provides convergence analysis and complexity analysis. Experimental results on synthetic data and real-world video data show that our proposed method achieves superior subspace tracking accuracy in comparison with a series of state-of-the-art methods while maintaining a high convergence speed and low memory requirement.

---

PaperID: 163,   

Authors:  Eunho Koo, Tongseok Lim

Title: Node Classification in Networks via Simplicial Interactions

Abstract:
In the node classification task, it is natural to presume that densely connected nodes tend to exhibit similar attributes. Given this, it is crucial to first define what constitutes a dense connection and to develop a reliable mathematical tool for assessing node cohesiveness. In this article, we propose a probability-based objective function for semi-supervised node classification that takes advantage of higher order networks’ capabilities. The proposed function reflects the philosophy aligned with the intuition behind classifying within higher order networks, as it is designed to reduce the likelihood of nodes interconnected through higher order networks bearing different labels. In addition, we propose the stochastic block tensor model (SBTM) as a graph generation model designed specifically to address a significant limitation of the traditional stochastic block model (SBM), which does not adequately represent the distribution of higher order structures in real networks. We evaluate the objective function using networks generated by the SBTM, which include both balanced and imbalanced scenarios. Furthermore, we present an approach that integrates the objective function with graph neural network (GNN)-based semi-supervised node classification methodologies, aiming for additional performance gains. Our results demonstrate that in challenging classification scenarios—characterized by a low probability of homo-connections, a high probability of hetero-connections, and limited prior node information—models based on the higher order network outperform pairwise interaction-based models. Furthermore, the experimental results suggest that integrating our proposed objective function with existing GNN-based node classification approaches enhances the classification performance by efficiently learning higher order structures distributed in the network.

---

PaperID: 164,   

Authors:  Lanjihong Ma, Yao-Xiang Ding, Peng Zhao, Zhi-Hua Zhou

Title: Learning Objective Adaptation by Correlation-Based Model Reuse

Abstract:
In open-environment machine learning (open ML), the learning objectives can vary according to specific real-world requirements. Models tailored for initial objectives may not be appropriate for the varied objectives. Retraining models from scratch for every single objective can be computationally intensive. Therefore, it is desirable to reuse models trained on the original objectives to help learn under the varied objectives. To this end, it is essential to characterize the objective correlations to better reuse the models. Previous works only consider the relative importance between pairs of previous and varied objectives, also known as previous-varied objectives correlations, ignoring correlations among the original objectives themselves. In this article, we demonstrate the importance of cross-original objective correlations. We propose a novel approach that employs the optimal transport technique to model correlations across all previous and varied objectives and then facilitates model reuse by utilizing learned transportation discrepancies to incorporate model reusabilities. Our empirical results show that our approach significantly outperforms existing benchmarks and well captures the underlying objective structure, validating the importance of accurate objective correlation modeling for learning with varied objectives.

---

PaperID: 165,   

Authors:  Xuyang Zhong, Chen Liu

Title: Toward Mitigating Architecture Overfitting on Distilled Datasets

Abstract:
Dataset distillation (DD) methods have demonstrated remarkable performance for neural networks trained with very limited training data. However, a significant challenge arises in the form of architecture overfitting: the distilled training dataset synthesized by a specific network architecture (i.e., training network) generates poor performance when trained by other network architectures (i.e., test networks), especially when the test networks have a larger capacity than the training network. This article introduces a series of approaches to mitigate this issue. Among them, DropPath renders the large model to be an implicit ensemble of its subnetworks, and knowledge distillation (KD) ensures each subnetwork acts similar to the small but well-performing teacher network. These methods, characterized by their smoothing effects, significantly mitigate architecture overfitting. We conduct extensive experiments to demonstrate the effectiveness and generality of our methods. Particularly, across various scenarios involving different tasks and different sizes of distilled data, our approaches significantly mitigate architecture overfitting. Furthermore, our approaches achieve comparable or even superior performance when the test network is larger than the training network. Codes are available at https://github.com/CityU-MLO/mitigate_architecture_overfitting.

---

PaperID: 166,   

Authors:  José D. Huerta-Morales, Chenglong You, Omar S. Magaña-Loaiza, Shi-Hai Dong, Roberto de J. León-Montiel, Mario Alan Quiroz-Juárez

Title: Smart Machine Vision for Universal Spatial-Mode Reconstruction

Abstract:
Structured light beams, in particular, those carrying orbital angular momentum (OAM), have gained a lot of attention due to their potential for enlarging the transmission capabilities of communication systems. However, the use of OAM-carrying light in communications faces two major problems, namely distortions introduced during propagation in disordered media, such as the atmosphere or optical fibers, and the large divergence that high-order OAM modes experience. While the use of nonorthogonal modes may offer a way to circumvent the divergence of high-order OAM fields, artificial intelligence (AI) algorithms have shown promise for solving the mode-distortion issue. Unfortunately, current AI-based algorithms make use of large-amount data-handling protocols that generally lead to large processing time and high power consumption. Here, we show that a low-power, low-cost image sensor can act as an artificial neural network that simultaneously detects and reconstructs distorted OAM-carrying beams. We demonstrate the capabilities of our device by reconstructing (with a 95% efficiency) individual Vortex, Laguerre-Gaussian (LG), and Bessel modes, as well as hybrid (nonorthogonal) coherent superpositions of such modes. Our work provides a potentially useful basis for the development of low-power-consumption, light-based communication devices.

---

PaperID: 167,   

Authors:  Yuxun Qu, Yongqiang Tang, Chenyang Zhang, Xiangrui Cai, Xiaojie Yuan, Wensheng Zhang

Title: Dual-Space Contrastive Learning for Open-World Semi-Supervised Classification

Abstract:
Despite recent progress in semi-supervised learning (SSL), its scalability remains limited in realistic scenarios where unseen classes may appear in the unlabeled data. To address this challenge, open-world SSL (OWSSL) is proposed in recent years and attracts much attention. One core difficulty in OWSSL is to enhance the representative ability for unlabeled samples, especially for those in novel classes. More recently, several works introduce contrastive learning into OWSSL and achieve impressive performance. However, they mainly focus on conducting contrastive learning solely in either feature or prediction spaces, while ignoring the thorough exploration of information potentials in dual spaces. In this study, we propose a novel method to handle OWSSL tasks via dual-space contrastive learning (DSCL). DSCL contains two modules: intraspace contrastive learning and interspace contrastive learning. In the intraspace module, we bridge the two spaces with a learnable classifier and impose contrastive learning in the dual spaces, such that the category discriminative information could be effectively utilized to improve the representative ability. In interspace module, to further enhance the utilization of complementary information from dual spaces, we introduce neighborhood information from feature space to enhance predictive learning and meanwhile utilize the cluster structure from the prediction spaces to improve intraclass compactness of the features. Compared with state-of-the-art competitors, the proposed DSCL achieves superior performance on the popular benchmarks, i.e., CIFAR100, Imagenet100, CIFAR10, CUB-200, and Scar.

---

PaperID: 168,   

Authors:  Jun Wang, Chuang Sun, Asoke K. Nandi, Ruqiang Yan, Xuefeng Chen

Title: Brain-Inspired Meta-Learning for Few-Shot Bearing Fault Diagnosis

Abstract:
Deep learning has attracted much attention in bearing fault diagnosis because of its high precision and end-to-end modules. However, in real industrial scenarios, some complex mechanical structures and working environments hinder data collection and fault reproduction, which makes bearing fault diagnosis with few samples a practical but challenging issue. As a data-driven approach, the standard deep learning method cannot extract features from a few samples due to overfitting. Neuroscience research has shown that the learning mechanism of the biological brain is more adaptable to learning tasks with few samples. Motivated by this, we propose a brain-inspired meta-learning (BIML) strategy for diagnosing few-shot bearing faults. Specifically, we design a brain-like learning algorithm for spiking neural networks (SNNs) based on the biological nervous system’s learning mechanism and introduce a meta-learning strategy to apply it to the fault diagnosis task of bearing with few samples. Experimental results show that BIML is better than existing few-shot bearing fault diagnosis methods. Subsequently, we conduct a theoretical analysis of the effectiveness of BIML strategies and verify our analysis through experiments.

---

PaperID: 169,   

Authors:  Gang Wang, Yufei Chen

Title: Two-View Correspondence Learning With Local Consensus Transformer

Abstract:
Correspondence learning is a crucial component in multiview geometry and computer vision. The presence of heavy outliers (mismatches) consistently renders the matching problem to be highly challenging. In this article, we revisit the benefits of local consensus (LC) in traditional feature matching and introduce the concept of LC to design a trainable neural network capable of capturing the underlying correspondences. This network is named the LC transformer (LCT) and is specifically tailored for wide-baseline stereo applications. Our network architecture comprises three distinct operations. To establish the neighbor topology, we employ a dynamic graph-based embedding layer as the initial step. Subsequently, these local topologies serve as guidance for the multihead self-attention layer, enabling it to extract a more extensive contextual understanding through channel attention (CA). Following this, order-aware graph pooling is applied to extract the global context information from the embedded LC. Through the experimental analysis, the ablation study reveals that PointNet-like learning models can, indeed, benefit from the incorporation of LC. The proposed model achieves state-of-the-art performance in both challenging scenes, namely, the YFCC100M outdoor and SUN3D indoor environments, even in the presence of more than 90% outliers.

---

PaperID: 170,   

Authors:  Xiao Cui, Yulei Qin, Yuting Gao, Enwei Zhang, Zihan Xu, Tong Wu, Ke Li, Xing Sun, Wengang Zhou, Houqiang Li

Title: SinKD: Sinkhorn Distance Minimization for Knowledge Distillation

Abstract:
Knowledge distillation (KD) has been widely adopted to compress large language models (LLMs). Existing KD methods investigate various divergence measures including the Kullback-Leibler (KL), reverse KL (RKL), and Jensen-Shannon (JS) divergences. However, due to limitations inherent in their assumptions and definitions, these measures fail to deliver effective supervision when a distribution overlap exists between the teacher and the student. In this article, we show that the aforementioned KL, RKL, and JS divergences, respectively, suffer from issues of mode-averaging, mode-collapsing, and mode-underestimation, which deteriorates logits-based KD for diverse natural language processing (NLP) tasks. We propose the Sinkhorn KD (SinKD) that exploits the Sinkhorn distance to ensure a nuanced and precise assessment of the disparity between distributions of teacher and student models. Besides, thanks to the properties of the Sinkhorn metric, we get rid of sample-wise KD that restricts the perception of divergences inside each teacher-student sample pair. Instead, we propose a batch-wise reformulation to capture the geometric intricacies of distributions across samples in the high-dimensional space. A comprehensive evaluation of GLUE and SuperGLUE, in terms of comparability, validity, and generalizability, highlights our superiority over state-of-the-art (SOTA) methods on all kinds of LLMs with encoder-only, encoder-decoder, and decoder-only architectures. Codes and models are available at https://github.com/2018cx/SinKD.

---

PaperID: 171,   

Authors:  Yanbei Liu, Lu Yu, Shichuan Zhao, Xiao Wang, Lei Geng, Zhitao Xiao, Shuai Ma, Yanwei Pang

Title: Graph Neural Networks With Adaptive Confidence Discrimination

Abstract:
Graph neural networks (GNNs) have demonstrated remarkable success for semisupervised node classification. However, these GNNs are still limited to the conventionally semisupervised framework and cannot fully leverage the potential value of large numbers of unlabeled samples. The pseudolabeling method in semisupervised learning (SSL) is widely recognized because it can clearly leverage unlabeled samples. Nevertheless, the existing pseudolabeling methods usually utilize a fixed threshold for all classes and only use a portion of unlabeled samples (ones with high prediction confidence), which leads to class imbalance and low data utilization. To solve these problems, we propose GNNs with adaptive confidence discrimination (ACDGNN) to fully utilize unlabeled samples for facilitating semisupervised node classification. Specifically, an adaptive confidence discrimination module is designed to divide all unlabeled nodes into two subsets by comparing their confidence scores with the adaptive confidence threshold at each training epoch. Then, different constraint strategies for two subset nodes are employed. Unlabeled nodes with high confidence are used to iteratively expand the label set, while ones with low confidence learn discriminative features by applying contrastive learning. Validated by extensive experiments, the proposed ACDGNN delivers significant accuracy gains over the previous SOTAs: an average improvement of 2.0% on all datasets and 5.7% on the Flickr dataset in particular.

---

PaperID: 172,   

Authors:  Dayang Wang, Feng-Lei Fan, Bojian Hou, Hao Zhang, Zhen Jia, Boce Zhang, Rongjie Lai, Hengyong Yu, Fei Wang

Title: Manifoldron: Direct Space Partition via Manifold Discovery

Abstract:
A neural network (NN) with the widely-used ReLU activation has been shown to partition the sample space into many convex polytopes for prediction. However, the parametric way a NN and other machine learning models use to partition the space has imperfections, e.g., the compromised interpretability for complex models, the inflexibility in decision boundary construction due to the generic character of the model, and the risk of being trapped into shortcut solutions. In contrast, although the nonparameterized models can adorably avoid or downplay these issues, they are usually insufficiently powerful either due to over-simplification or the failure to accommodate the manifold structures of data. In this context, we first propose a new type of machine learning models referred to as Manifoldron that directly derives decision boundaries from data and partitions the space via manifold structure discovery. Then, we systematically analyze the key characteristics of the Manifoldron such as manifold characterization capability and its link to NNs. The experimental results on four synthetic examples, 20 public benchmark datasets, and one real-world application demonstrate that the proposed Manifoldron performs competitively compared to the mainstream machine learning models. We have shared our code in https://github.com/wdayang/Manifoldron for free download and evaluation.

---

PaperID: 173,   

Authors:  Zhenbo Huang, Jing Zhao, Shiliang Sun

Title: De-Pessimism Offline Reinforcement Learning via Value Compensation

Abstract:
Offline reinforcement learning (RL) has been widely used in practice due to its efficient data utilization, but it still faces the challenge of training vulnerability caused by policy deviation. Existing offline RL methods that add policy constraints or perform conservative Q-value estimation are pessimistic, making the learned policy suboptimal. In this article, we address the pessimism problem by focusing on accurate Q-value estimation. We propose the de-pessimism (DEP) operator to estimate Q values using the optimal Bellman operator or the compensation operator according to whether the actions are in the behavior support set. The compensation operator qualitatively determines the positive or negative nature of out-of-distribution (OOD) actions based on their performance compared with the behavior actions. It leverages differences in state values to compensate for the Q value of positive OOD actions, thereby alleviating pessimism. We theoretically demonstrate the convergence of DEP and its effectiveness in policy improvement. To further advance the practical application, we integrate DEP into the soft actor-critic (SAC) algorithm, yielding the value-compensated de-pessimism offline RL (DoRL-VC). Experimentally, DoRL-VC achieves state-of-the-art (SOTA) performance across mujoco locomotion, Maze 2-D, and challenging Adroit tasks, illustrating the efficacy of DEP in mitigating pessimism.

---

PaperID: 174,   

Authors:  Gui-Bin Bian, Xiang-Rong Tang, Ruichen Ma, Qiang Ye, Zhen Li, Ming-Yang Zhang, Han Ren, Yu-Peng Zhai

Title: Few-Human-Interaction Reinforcement Learning for Autonomous Transbronchial Intervention

Abstract:
The transbronchial interventional surgery presents challenges with winding and convoluted pathways, prone to compression and friction. Current autonomous planning struggles to reach deeper bronchial positions, and hard to consider multiple conflicting goals simultaneously. This article introduces an innovative planning scheme with preference weights to achieve smooth, frictionless, and collision-free autonomous transbronchial intervention with continuum robot (CR). A few-human-interaction twin-delayed deep deterministic policy gradient (FHITD3) generated from surgeon preference guidance is proposed, which determines the optimal strategy for the motion of CR. Preference knowledge is generated through interaction between human and few diversity samples. An abstract actuator space description is proposed for the posture and position representation of CR during movement within bronchus. A contact motion analysis strategy is proposed to calculate real-time attitude of CR in contact with bronchus. In addition, an oscillation suppression approach to address CR’s unsmooth distal end trajectory is proposed. Simulated experiments show that the CR autonomously completes intervention tasks with a smooth and stable trajectory, reducing distal end oscillation by over 45%. It achieves a target endpoint within the fourth level bronchus (approximately 5 mm diameter) with over 90% probability.

---

PaperID: 175,   

Authors:  Yaohang Wu, Renwei Dian, Shutao Li

Title: Multistage Spatial-Spectral Fusion Network for Spectral Super-Resolution

Abstract:
Spectral super-resolution (SSR) aims to restore a hyperspectral image (HSI) from a single RGB image, in which deep learning has shown impressive performance. However, the majority of the existing deep-learning-based SSR methods inadequately address the modeling of spatial-spectral features in HSI. That is to say, they only sufficiently capture either the spatial correlations or the spectral self-similarity, which results in a loss of discriminative spatial-spectral features and hence limits the fidelity of the reconstructed HSI. To solve this issue, we propose a novel SSR network dubbed multistage spatial-spectral fusion network (MSFN). From the perspective of network design, we build a multistage Unet-like architecture that differentially captures the multiscale features of HSI both spatialwisely and spectralwisely. It consists of two types of the self-attention mechanism, which enables the proposed network to achieve global modeling of HSI comprehensively. From the perspective of feature alignment, we innovatively design the spatial fusion module (SpatialFM) and spectral fusion module (SpectralFM), aiming to preserve the comprehensively captured spatial correlations and spectral self-similarity. In this manner, the multiscale features can be better fused and the accuracy of reconstructed HSI can be significantly enhanced. Quantitative and qualitative experiments on the two largest SSR datasets (i.e., NTIRE2022 and NTIRE2020) demonstrate that our MSFN outperforms the state-of-the-art SSR methods. The code implementation will be uploaded at https://github.com/Matsuri247/MSFN-for-Spectral-Super-Resolution.

---

PaperID: 176,   

Authors:  Zhiwen Cao, Xijiong Xie

Title: Partition-Level Tensor Learning-Based Multiview Unsupervised Feature Selection

Abstract:
Multiview unsupervised feature selection is an emerging direction in the machine learning community because of its ability to identify informative patterns and reduce the dimensionality of multiview data. Although numerous methods have been proposed and shown to be effective, they have some limitations: 1) most existing algorithms fail to improve the model performance along the view dimension; 2) they rarely incorporate more discriminative partition information; and 3) the negative effects of marginal samples are not considered. To solve these problems, we propose a novel method termed as partition-level tensor learning-based multiview unsupervised feature selection (PTFS). The proposed method optimizes a low-rank constrained tensor assembled by the inner product of base partition matrices. By doing so, PTFS simultaneously leverages the high-order view correlation and indirectly integrates discriminative partition information. Besides, a statistic-based adaptive self-paced strategy is introduced to ensure that confident samples are prioritized for training the model. Moreover, an effective alternating optimization method is designed to solve the resulting optimization problem. Extensive experiments on ten datasets demonstrate the effectiveness and efficiency of the proposed method compared to the state-of-the-art methods. The code is available at https://github.com/HdTgon/2023-TNNLS-PTFS.

---

PaperID: 177,   

Authors:  Hao Li, Wei Wang, Mengzhu Wang, Huibin Tan, Long Lan, Zhigang Luo, Xinwang Liu, Kenli Li

Title: STFormer: Spatial-Temporal-Aware Transformer for Video Instance Segmentation

Abstract:
Video instance segmentation (VIS) is a challenging task, requiring handling object classification, segmentation, and tracking in videos. Existing Transformer-based VIS approaches have shown remarkable success, combining encoded features and instance queries as decoder inputs. However, their decoder inputs are low-resolution due to computational cost, resulting in a loss of fine-grained information, sensitivity to background interference, and poor handling of small objects. Moreover, the queries are randomly initialized without location information, hindering convergence efficiency and accurate object instance localization. To address these issues, we propose a novel VIS approach, STFormer, with a spatial-temporal feature aggregation (STFA) module and spatial-temporal-aware Transformer (STT). Specifically, STFA obtains robust high-resolution masked features efficiently for the decoder, while STT’s location-guided instance query (LGIQ) improves initial instance queries. STFormer preserves more fine-grained information, improves convergence efficiency, and localizes object instance features accurately. Extensive experiments on YouTube-VIS 2019, YouTube-VIS 2021, and OVIS datasets show that STFormer outperforms mainstream VIS methods.

---

PaperID: 178,   

Authors:  Xiang Chen, Min Liu, Rongguang Wang, Renjiu Hu, Dongdong Liu, Gaolei Li, Yaonan Wang, Hang Zhang

Title: Spatially Covariant Image Registration With Text Prompts

Abstract:
Medical images are often characterized by their structured anatomical representations and spatially inhomogeneous contrasts. Leveraging anatomical priors in neural networks can greatly enhance their utility in resource-constrained clinical settings. Prior research has harnessed such information for image segmentation, yet progress in deformable image registration has been modest. Our work introduces textSCF, a novel method that integrates spatially covariant filters and textual anatomical prompts encoded by visual-language models, to fill this gap. This approach optimizes an implicit function that correlates text embeddings of anatomical regions to filter weights. textSCF not only boosts computational efficiency but can also retain or improve registration accuracy. By capturing the contextual interplay between anatomical regions, it offers impressive interregional transferability and the ability to preserve structural discontinuities during registration. textSCF’s performance has been rigorously tested on intersubject brain magnetic resonance imaging (MRI) and abdominal computerized tomography (CT) registration tasks, outperforming existing state-of-the-art models in the MICCAI Learn2Reg 2021 challenge and leading the leaderboard. In abdominal registrations, textSCF’s larger model variant improved the Dice score by 11.3% over the second-best model, while its smaller variant maintained similar accuracy but with an 89.13% reduction in network parameters and a 98.34% decrease in computational operations.

---

PaperID: 179,   

Authors:  Xinye Wang, Lei Duan, Lili Guan, Jiaxuan Xu, Chengxin He

Title: Learning Decision Boundaries for Multidimensional Anomaly Detection

Abstract:
In this article, we attempt to study a problem of identifying outliers from multiple dimensions, terming it as multidimensional anomaly detection. This task assumes that each sample corresponds to heterogeneous discriminative spaces, where each space characterizes distinct semantic information along one dimension. Consequently, the abnormalities exhibited by a sample will vary under these semantically different dimensions. In contrast to traditional anomaly detection, multidimensional anomaly detection offers a more holistic assessment of a sample’s abnormalities. However, the heterogeneity of discriminative spaces leads to incomparability of the outputs from different dimensions which is the major difficulty in designing multidimensional anomaly detection methods. This article introduces a novel model, maximum margin multidimensional anomaly detection (ALOE), specifically tailored for multidimensional anomaly detection. ALOE constructs a convex optimization problem with nonlinear constraints. The primary objective is to simultaneously learn multiple decision boundaries, utilizing the maximum margin principle and covariance regularization, while distinguishing between outliers and normal samples under multiple dimensions by capturing the correlation among multiple dimensions. To obtain the optimal decision boundary under each dimension, we devise an alternating optimization method for this convex optimization problem. To validate the effectiveness of ALOE, we conduct extensive experiments on 12 real-world datasets, comparing its performance against 34 anomaly detection methods. The experimental results demonstrate the superior performance of ALOE.

---

PaperID: 180,   

Authors:  Wenbiao Yan, Jihua Zhu, Yiyang Zhou, Jinqian Chen, Haozhe Cheng, Kun Yue, Qinghai Zheng

Title: Neighbor-Based Completion for Addressing Incomplete Multiview Clustering

Abstract:
Driven by the complementarity and consistency inherent in multiview data, multiview clustering (MVC) has garnered widespread attention in various domains. Real-world data often encounters the issue of missing information, leading to a surge of interest in the domain of incomplete MVC (IMVC). Despite existing approaches having made significant progress in addressing IMVC, two significant challenges persist: 1) many alignment-based methodologies tend to overlook the topological relationships among instances and 2) the view representations based on completion lack reconstructive properties, casting doubt on their alignment with the actual view representations. In response, we present a novel approach termed neighbor-based completion for addressing IMVC (NBIMVC), which capitalizes on the topological information among instances and the consistent information across views. Specifically, our method uses autoencoders to learn feature representations for each view and leverages nearest-neighbor relationships between unique and complete instances to complete missing features in missing views. Subsequently, we enforce hard negative alignment constraints on complete paired instances in the feature space. Finally, we ensure the consistency of views in the semantic space by employing cluster information and a shared clustering network, which facilitates the final multiview categories output and effectively resolves the IMVC problem. Extensive experimental evaluations validate the efficacy of our proposed method, showcasing comparable or superior performance to existing approaches.

---

PaperID: 181,   

Authors:  Kaili Xiang, Ruotong Ming, Siyu Chen, Frank L. Lewis

Title: Neuroadaptive Control With Enhanced Stability and Reliability

Abstract:
The performance of neural network (NN)-driven control systems hinges on the reliability and functionality of the NN unit in the controller. Maintaining the compact set condition for NN training signals (inputs) during operation is crucial for preserving the NN’s universal learning and approximation capabilities, yet this requirement is often overlooked in existing studies. This article introduces a constraint transformation-based design method that ensures excitation signals always originate from a fixed region, regardless of initial conditions. By meeting the compactness condition required by the universal approximation theorem, this approach safeguards the functionality of the NN-driven control unit. Additionally, a decaying damping rate is employed to enable the tracking error to asymptotically converge to zero, rather than being ultimately uniformly bounded (UUB). To further ensure robust operation even if the NN underperforms due to an insufficient number of neurons or violation of the compact set condition, a new control strategy is developed based on the worst case behavior of NNs. This “fail-secure” mechanism significantly enhances the reliability of the NN-based control scheme. The effectiveness and benefits of the proposed method are confirmed through numerical simulations, demonstrating its potential to substantially improve the robustness and performance of NN-driven control systems.

---

PaperID: 182,   

Authors:  Jiawen Zhu, Xin Chen, Haiwen Diao, Shuai Li, Jun-Yan He, Chenyang Li, Bin Luo, Dong Wang, Huchuan Lu

Title: Exploring Dynamic Transformer for Efficient Object Tracking

Abstract:
The speed-precision tradeoff is a critical problem in visual object tracking, as it typically requires low latency and is deployed on resource-constrained platforms. Existing solutions for efficient tracking primarily focus on lightweight backbones or modules, which, however, come at a sacrifice in precision. In this article, inspired by dynamic network routing, we propose DyTrack, a dynamic transformer framework for efficient tracking. Real-world tracking scenarios exhibit varying levels of complexity. We argue that a simple network is sufficient for easy video frames, while more computational resources should be assigned to difficult ones. DyTrack automatically learns to configure proper reasoning routes for different inputs, thereby improving the utilization of the available computational budget and achieving higher performance at the same running speed. We formulate instance-specific tracking as a sequential decision problem and incorporate terminating branches to intermediate layers of the model. Furthermore, we propose a feature recycling mechanism to maximize computational efficiency by reusing the outputs of predecessors. Additionally, a target-aware self-distillation strategy is designed to enhance the discriminating capabilities of early-stage predictions by mimicking the representation patterns of the deep model. Extensive experiments demonstrate that DyTrack achieves promising speed-precision tradeoffs with only a single model. For instance, DyTrack obtains 64.9% area under the curve (AUC) on LaSOT with a speed of 256 fps.

---

PaperID: 183,   

Authors:  Mingxiang Liao, Fang Wan, Zonghao Guo, Qixiang Ye

Title: Hierarchical AttentionShift for Pointly Supervised Instance Segmentation

Abstract:
Pointly supervised instance segmentation (PSIS) remains a challenging task when appearance variances across object parts cause semantic inconsistency. In this article, we propose a hierarchical AttentionShift approach, to solve the semantic inconsistency issue through exploiting the hierarchical nature of semantics and the flexibility of key-point representation. The estimation of hierarchical attention is defined upon key-point sets. The representative key points are iteratively estimated spatially and in the feature space to capture the fine-grained semantics and cover the full object extent. Hierarchical AttentionShift is performed at instance, part, and fine-grained levels, optimizing object semantics while promoting the conventional self-attention activation to hierarchical activation with local refinement. Experiments on PASCAL VOC 2012 Aug and MS-COCO 2017 benchmarks show that hierarchical AttentionShift improves the state-of-the-art (SOTA) method by 10.4% and 7.0% upon mean average precision (mAP)50, respectively. When applying hierarchical AttentionShift to the segment anything model (SAM), 9.4% AP improvement on the COCO test-dev is achieved. Hierarchical AttentionShift provides a fresh insight to regularize the self-attention mechanism for fine-grained vision tasks. The code is available at github.com/MingXiangL/AttentionShift.

---

PaperID: 184,   

Authors:  Joseph Kelley, Martin T. Hagan

Title: Adaptive Multistep Prediction With Sequence-to-Sequence (Seq2Seq) Models

Abstract:
This brief demonstrates for the first time that the sequence-to-sequence (Seq2Seq) model is an adaptive multistep predictor. The Seq2Seq model is fixed-weight adaptive, which means that the model can adapt to time-varying behaviors without having to update its weights and biases. Instead, the learning algorithm is embedded into the recurrent neural network (RNN) decoder. This brief examines the Seq2Seq model’s ability to adapt to time-varying behaviors using both simulated and experimental data, and it also identifies a mechanism within the model that enables the adaptation.

---

PaperID: 185,   

Authors:  Dan Li, Haibao Wang, Shihui Ying

Title: Multiview Representation Learning With One-to-Many Dynamic Relationships

Abstract:
Integrating information from multiple views to obtain potential representations with stronger expressive ability has received significant attention in practical applications. Most existing algorithms usually focus on learning either the consistent or complementary representation of views and, subsequently, integrate one-to-one corresponding sample representations between views. Although these approaches yield effective results, they do not fully exploit the information available from multiple views, limiting the potential for further performance improvement. In this article, we propose an unsupervised multiview representation learning method based on sample relationships, which enables the one-to-many fusion of intraview and interview information. Due to the heterogeneity of views, we need mainly face the two following challenges: 1) the discrepancy in the dimensions of data across different views and 2) the characterization and utilization of sample relationships across these views. To address these two issues, we adopt two modules: the dimension consistency relationship enhancement module and the multiview graph learning module. Thereinto, the relationship enhancement module addresses the discrepancy in data dimensions across different views and dynamically selects data dimensions for each sample that bolsters intraview relationships. The multiview graph learning module devises a novel multiview adjacency matrix to capture both intraview and interview sample relationships. To achieve one-to-many fusion and obtain multiview representations, we employ the graph autoencoder structure. Furthermore, we extend the proposed architecture to the supervised case. We conduct extensive experiments on various real-world multiview datasets, focusing on clustering and multilabel classification tasks, to evaluate the effectiveness of our method. The results demonstrate that our approach significantly improves performance compared to existing methods, highlighting the potential of leveraging sample relationships for multiview representation learning. Our code is released at https://github.com/lilidan-orm/one-to-many-multiview on GitHub.

---

PaperID: 186,   

Authors:  Xingyu Zhao, Yuexuan An, Lei Qi, Xin Geng

Title: Scalable Label Distribution Learning for Multi-Label Classification

Abstract:
Multi-label classification (MLC) refers to the problem of tagging a given instance with a set of relevant labels. Most existing MLC methods are based on the assumption that the correlation of two labels in each label pair is symmetric, which is violated in many real-world scenarios. Moreover, most existing methods design learning processes associated with the number of labels, which makes their computational complexity a bottleneck when scaling up to large-scale output space. To tackle these issues, we propose a novel method named scalable label distribution learning (SLDL) for MLC, which can describe different labels as distributions in a latent space, where the label correlation is asymmetric and the dimension is independent of the number of labels. Specifically, SLDL first converts labels into continuous distributions within a low-dimensional latent space and leverages the asymmetric metric to establish the correlation between different labels. Then, it learns the mapping from the feature space to the latent space, resulting in the computational complexity is no longer related to the number of labels. Finally, SLDL leverages a nearest neighbor-based strategy to decode the latent representations and obtain the final predictions. Extensive experiments illustrate that SLDL achieves very competitive classification performances with little computational consumption.

---

PaperID: 187,   

Authors:  Xue Zhang, Xiaohan Zhang, Jiangtao Wang, Jiacheng Ying, Zehua Sheng, Heng Yu, Chunguang Li, Hui-Liang Shen

Title: TFDet: Target-Aware Fusion for RGB-T Pedestrian Detection

Abstract:
Pedestrian detection plays a critical role in computer vision as it contributes to ensuring traffic safety. Existing methods that rely solely on RGB images suffer from performance degradation under low-light conditions due to the lack of useful information. To address this issue, recent multispectral detection approaches have combined thermal images to provide complementary information and have obtained enhanced performances. Nevertheless, few approaches focus on the negative effects of false positives (FPs) caused by noisy fused feature maps. Different from them, we comprehensively analyze the impacts of FPs on detection performance and find that enhancing feature contrast can significantly reduce these FPs. In this article, we propose a novel target-aware fusion strategy for multispectral pedestrian detection, named TFDet. The target-aware fusion strategy employs a fusion-refinement paradigm. In the fusion phase, we reveal the parallel- and cross-channel similarities in RGB and thermal features and learn an adaptive receptive field to collect useful information from both features. In the refinement phase, we use a segmentation branch to discriminate the pedestrian features from the background features. We propose a correlation-maximum loss function to enhance the contrast between the pedestrian features and background features. As a result, our fusion strategy highlights pedestrian-related features and suppresses unrelated ones, generating more discriminative fused features. TFDet achieves state-of-the-art performance on two multispectral pedestrian benchmarks, KAIST and LLVIP, with absolute gains of 0.65% and 4.1% over the previous best approaches, respectively. TFDet can easily extend to multiclass object detection scenarios. It outperforms the previous best approaches on two multispectral object detection benchmarks, FLIR and M3FD, with absolute gains of 2.2% and 1.9%, respectively. Importantly, TFDet has comparable inference efficiency to the previous approaches and has remarkably good detection performance even under low-light conditions, which is a significant advancement for ensuring road safety. The code will be made publicly available at https://github.com/XueZ-phd/TFDet.git.

---

PaperID: 188,   

Authors:  Botao Dong, Longyang Huang, Ning Pang, Hongtian Chen, Weidong Zhang

Title: Historical Decision-Making Regularized Maximum Entropy Reinforcement Learning

Abstract:
The challenge of the exploration-exploitation dilemma persists in off-policy reinforcement learning (RL) algorithms, impeding the improvement of policy performance and sample efficiency. To tackle this challenge, a novel historical decision-making regularized maximum entropy (HDMRME) RL algorithm is developed to strike the balance between exploration and exploitation. Built upon the maximum entropy RL framework, the historical decision-making regularization method is proposed to enhance the exploitation capability of RL policies. The theoretical analysis involves proving the convergence of HDMRME, investigating the tradeoff between exploration and exploitation of HDMRME, examining the disparity between the Q-function learned through HDMRME and the classic one, and analyzing the suboptimality of the trained policy. The performance of HDMRME is evaluated across various continuous-action control tasks from Mujoco and OpenAI Gym platforms. Comparative experiments demonstrate that HDMRME exhibits superior sample efficiency and achieves more competitive performance compared with other state-of-the-art RL algorithms.

---

PaperID: 189,   

Authors:  Chaoqin Huang, Haoyan Guan, Aofan Jiang, Ya Zhang, Michael W. Spratling, Xinchao Wang, Yanfeng Wang

Title: Few-Shot Anomaly Detection via Category-Agnostic Registration Learning

Abstract:
Most existing anomaly detection (AD) methods require a dedicated model for each category. Such a paradigm, despite its promising results, is computationally expensive and inefficient, thereby failing to meet the requirements for real-world applications. Inspired by how humans detect anomalies, by comparing a query image to known normal ones, this article proposes a novel few-shot AD (FSAD) framework. Using a training set of normal images from various categories, registration, aiming to align normal images of the same categories, is leveraged as the proxy task for self-supervised category-agnostic representation learning. At test time, an image and its corresponding support set, consisting of a few normal images from the same category, are supplied, and anomalies are identified by comparing the registered features of the test image to its corresponding support image features. Such a setup enables the model to generalize to novel test categories. It is, to our best knowledge, the first FSAD method that requires no model fine-tuning for novel categories: enabling a single model to be applied to all categories. Extensive experiments demonstrate the effectiveness of the proposed method. Particularly, it improves the current state-of-the-art (SOTA) for FSAD by 11.3% and 8.3% on the MVTec and MPDD benchmarks, respectively. The source code is available at https://github.com/Haoyan-Guan/CAReg.

---

PaperID: 190,   

Authors:  Guang Yang, Zheng Xu, Jing Huo, Shangdong Yang, Tianyu Ding, Xingguo Chen, Yang Gao

Title: State Abstraction via Deep Supervised Hash Learning

Abstract:
State abstraction is a widely used technique in reinforcement learning (RL) that compresses the state space to accelerate learning algorithms. However, designing an effective abstraction function in large-scale or high-dimensional state space problems remains a significant challenge. In this brief, we present a novel state abstraction method based on deep supervised hash learning (DSH) and provide a theoretical analysis of its near-optimal property. Furthermore, by leveraging the DSH-based representation as the optimization objective, we propose a direct and concise optimization method based on the target value. In addition, we construct an auxiliary learning task for state abstraction that can be combined with various RL algorithms. In particular, we apply the DSH-based state abstraction to both deep Q-learning (DQN) and soft actor-critic (SAC). Extensive experiments are conducted on Atari and several classic control benchmarks to evaluate the effectiveness of the DSH-based state abstraction method, showing that our method surpasses existing state abstraction algorithms in performance.

---

PaperID: 191,   

Authors:  Songze Li, Tonghua Su, Xu-Yao Zhang, Zhongjie Wang

Title: Continual Learning With Knowledge Distillation: A Survey

Abstract:
The foremost challenge in continual learning is to mitigate catastrophic forgetting, allowing a model to retain knowledge of previous tasks while learning new tasks. Knowledge distillation (KD), a form of regularization, has gained significant attention for its ability to maintain a model’s performance on previous tasks by mimicking the outputs of earlier models during the learning of new tasks, thus reducing forgetting. This article offers a comprehensive survey of continual learning methods employing KD within the realm of image classification. We provide a detailed analysis of how KD is utilized in continual learning methods, categorizing its application into three distinct paradigms. Besides, we classify these methods based on the type of knowledge source used and thoroughly examine how KD consolidates memory in continual learning from the perspective of loss functions. In addition, we have conducted extensive experiments on CIFAR-100, TinyImageNet, and ImageNet-100 across ten KD-integrated continual learning methods to analyze the role of KD in continual learning, and we have further discussed its effectiveness in other continual learning tasks. Our extensive experimental evidence demonstrates that KD plays a crucial role in mitigating forgetting in continual learning and substantiates that, when used with data replay, classification bias adversely affects the effectiveness of KD, whereas employing a separated softmax loss can significantly enhance its efficacy.

---

PaperID: 192,   

Authors:  Mingcheng Dai, Daniel W. C. Ho, Baoyong Zhang, Deming Yuan, Shengyuan Xu

Title: Distributed Online Convex Optimization With Statistical Privacy

Abstract:
We focus on the problem of distributed online constrained convex optimization with statistical privacy in multiagent systems. The participating agents aim to collaboratively minimize the cumulative system-wide cost while a passive adversary corrupts some of them. The passive adversary collects information from corrupted agents and attempts to estimate the private information of the uncorrupted ones. In this scenario, we adopt a correlated perturbation mechanism with globally balanced property to cover the local information of agents to enable privacy preservation. This work is the first attempt to integrate such a mechanism into the distributed online (sub)gradient descent algorithm, and then a new algorithm called privacy-preserving distributed online convex optimization (PP-DOCO) is designed. It is proved that the designed algorithm provides a statistical privacy guarantee for uncorrupted agents and achieves an expected regret in \mathcal O(\sqrt K) for convex cost functions, where K denotes the time horizon. Furthermore, an improved expected regret in \mathcal O(\log (K)) is derived for strongly convex cost functions. The obtained results are equivalent to the best regret scalings achieved by state-of-the-art algorithms. The privacy bound is established to describe the level of statistical privacy using the notion of Kullback–Leibler divergence (KLD). In addition, we observe that a tradeoff exists between our algorithm’s expected regret and statistical privacy. Finally, the effectiveness of our algorithm is validated by simulation results.

---

PaperID: 193,   

Authors:  Chaohua Shi, Mingrui Zhu, Nannan Wang, Xinbo Gao

Title: PStyle-3D: Example-Based 3-D-Aware Portrait Style Domain Adaptation

Abstract:
The creation of high-quality artistic portraits is a critical and desirable task in the field of computer vision. While recent3-D generative models have achieved impressive results in generating images with view consistency and intricate 3-D shapes, their application for generating artistic portraits is often more challenging than 2-D generative models due to the potentially destructive impact of 3-D structures on human faces. This article introduces a novel approach that leverages a meticulously designed domain feature extraction module to extract the specific feature information from both the source natural face domain and the target artistic portrait domain. These extracted features are seamlessly integrated into a 3-D representation, generating multiview consistent 3-D artistic portraits. To fuse the features of the source and target domains better, we propose a new module for domain adaptation. This module adds a path to the style path established by StyleGAN to introduce the artistic portrait domain information and regulate the target domain’s feature information in \mathcal S space. Our domain adaptation module is implemented in each StyleBlock of the 3-D representation generator to integrate the target domain information with the original facial information. Experimental results demonstrate that our approach generates high-quality 3-D artistic portraits that outperform existing approaches in preserving 3-D geometric information and multiview consistency.

---

PaperID: 194,   

Authors:  Ugochukwu Ejike Akpudo, Yongsheng Gao, Jun Zhou, Andrew Lewis

Title: TraNCE: Transformative Nonlinear Concept Explainer for CNNs

Abstract:
Convolutional neural networks (CNNs) have succeeded remarkably in various computer vision tasks. However, they are not intrinsically explainable. While feature-level understanding of CNNs reveals where the models looked, concept-based explainability methods provide insights into what the models saw. However, their assumption of linear reconstructability of image activations fails to capture the intricate relationships within these activations. Their fidelity-only approach to evaluating global explanations also presents a new concern. For the first time, we address these limitations with the novel transformative nonlinear concept explainer (TraNCE) for CNNs. Unlike linear reconstruction assumptions made by existing methods, TraNCE captures the intricate relationships within the activations. This study presents three original contributions to the CNN explainability literature: 1) an automatic concept discovery mechanism based on variational autoencoders (VAEs). This transformative concept discovery process enhances the identification of meaningful concepts from image activations; 2) a visualization module that leverages the Bessel function to create a smooth transition between prototypical image pixels, revealing not only what the CNN saw but also what the CNN avoided, thereby mitigating the challenges of concept duplication as documented in previous works; and 3) a new metric, the faith score, integrates both coherence and fidelity for comprehensive evaluation of explainer faithfulness and consistency. Based on the investigations on publicly available datasets, we prove that a valid decomposition of a high-dimensional image activation should follow a nonlinear reconstruction, contributing to the explainer’s efficiency. We also demonstrate quantitatively that, besides accuracy, consistency is crucial for the meaningfulness of concepts and human trust. The code is available at https://github.com/daslimo/TrANCE.

---

PaperID: 195,   

Authors:  Bowen Duan, Shiming Chen, Yufei Guo, Guo-Sen Xie, Weiping Ding, Yisong Wang

Title: Visual-Semantic Graph Matching Net for Zero-Shot Learning

Abstract:
Zero-shot learning (ZSL) aims to leverage additional semantic information to recognize unseen classes. To transfer knowledge from seen to unseen classes, most ZSL methods often learn a shared embedding space by simply aligning visual embeddings with semantic prototypes. However, methods trained under this paradigm often struggle to learn robust embedding space because they align the two modalities in an isolated manner among classes, which ignore the crucial class relationship during the alignment process. To address the aforementioned challenges, this article proposes a visual-semantic graph matching net (VSGMN), which leverages semantic relationships among classes to aid in visual-semantic embedding. VSGMN uses a graph build net (GBN) and a graph matching net (GMN) to achieve two-stage visual-semantic alignment. Specifically, GBN first uses an embedding-based approach to build visual and semantic graphs in the semantic space and align the embedding with its prototype for first-stage alignment. In addition, to supplement unseen class relationships in these graphs, GBN also builds the unseen class nodes based on semantic relationships. In the second stage, GMN continuously integrates neighbor and cross-graph information into the constructed graph nodes and aligns the node relationships between the two graphs under the class relationship constraint. Extensive experiments on three benchmark datasets demonstrate that VSGMN achieves superior performance in both conventional and generalized ZSL (GZSL) scenarios. The implementation of our VSGMN and experimental results are available at github: https://github.com/dbwfd/VSGMN.

---

PaperID: 196,   

Authors:  Zhihan Ning, Zhixing Jiang, David Zhang

Title: Exploiting Meta-Learned Confidences for Imbalanced Multilabel Learning

Abstract:
Multilabel learning deals with datasets where each sample is associated with multiple labels. It is commonly assumed that label correlations should be well exploited to build an effective multilabel classifier. Moreover, the class imbalance problem occurs in many multilabel datasets and should be tackled to reduce the classification bias. While many multilabel learning methods have been proposed, research on imbalanced multilabel learning (IMLL) is relatively deficient. To address these issues, we exploit the value of meta-learned confidences, i.e., the prediction confidences iteratively updated over the out-of-bag samples, for IMLL. First, such meta-confidences can be fused to the original feature space to learn high-order label correlations. Second, meta-confidences can be used to calibrate the prediction results to alleviate class imbalance. Motivated by these, we propose an ensemble learning method named meta-confidence ensemble (MCE) for IMLL. Specifically, MCE iteratively makes bootstrap replicates of the multilabel training set, leverages the out-of-bag samples to generate meta-confidences, and fuses them to the original feature space to learn label correlations. A sparse projection method is presented to avoid overfitting and improve the ensemble diversity. Finally, the prediction result of an unseen sample is determined by the calibrated plurality vote of MCE’s base classifiers. Extensive experiments demonstrated the effectiveness and superiority of MCE for IMLL. Codes have been made publicly available at https://github.com/ CUHKSZ-NING/MCEClassifier.

---

PaperID: 197,   

Authors:  Jiamin Liu, Heng Lian

Title: Kernel-Based Decentralized Policy Evaluation for Reinforcement Learning

Abstract:
We investigate the decentralized nonparametric policy evaluation problem within reinforcement learning (RL), focusing on scenarios where multiple agents collaborate to learn the state-value function using sampled state transitions and privately observed rewards. Our approach centers on a regression-based multistage iteration technique employing infinite-dimensional gradient descent (GD) within a reproducing kernel Hilbert space (RKHS). To make computation and communication more feasible, we employ Nyström approximation to project this space into a finite-dimensional one. We establish statistical error bounds to describe the convergence of value function estimation, marking the first instance of such analysis within a fully decentralized nonparametric framework. We compare the regression-based method to the kernel temporal difference (TD) method in some numerical studies.

---

PaperID: 198,   

Authors:  Chengyu Wang, Shan Zhao, Tianwei Yan, Shezheng Song, Wentao Ma, Kuien Liu, Meng Wang

Title: Hierarchical Label-Enhanced Contrastive Learning for Chinese NER

Abstract:
Recently, character–word lattice structures have achieved promising results for Chinese named entity recognition (NER), reducing word segmentation errors and increasing word boundary information for character sequences. However, constructing the lattice structure is complex and time-consuming, thus these lattice-based models usually suffer from low inference speed. Moreover, the quality of the lexicon affects the accuracy of the NER model. Since noise words can potentially confuse NER, limited coverage of the lexicon can cause lattice-based models to degenerate into partial character-based models. In this article, we propose a hierarchical label-enhanced contrastive learning (HLCL) method for Chinese NER. Instead of relying on the lattice structure, HLCL offers an alternative solution to robustly integrate entity boundary and type information with the help of both labels semantic and contrastive learning. HLCL is empowered by two techniques: 1) sentence-level contrastive learning (SCL) to model global mutual information between two different modalities (e.g., labels and sentences) and 2) token-level contrastive learning (TCL) to close the gap between representations of different characters (e.g., label-enhanced characters and original characters), resulting in local mutual information. With the well-designed contrastive learning scheme and the concise model during inference, HLCL can fully leverage the transferable label semantic and has a superb speed of inference. Experiments on four Chinese NER datasets show that HLCL obtains excellent efficiency as well as performance compared with existing lattice-based approaches.

---

PaperID: 199,   

Authors:  Yongjeong Oh, Jaeho Lee, Christopher G. Brinton, Yo-Seb Jeon

Title: Communication-Efficient Split Learning via Adaptive Feature-Wise Compression

Abstract:
This article proposes a novel communication-efficient split learning (SL) framework, named SplitFC, which reduces the communication overhead required for transmitting intermediate features and gradient vectors during the SL training process. The key idea of SplitFC is to leverage different dispersion degrees exhibited in the columns of the matrices. SplitFC incorporates two compression strategies: 1) adaptive feature-wise dropout and 2) adaptive feature-wise quantization. In the first strategy, the intermediate feature vectors are dropped with adaptive dropout probabilities determined based on the standard deviation of these vectors. Then, by the chain rule, the intermediate gradient vectors associated with the dropped feature vectors are also dropped. In the second strategy, the non-dropped intermediate feature and gradient vectors are quantized using adaptive quantization levels determined based on the ranges of the vectors. To minimize the quantization error, the optimal quantization levels of this strategy are derived in a closed-form expression. Simulation results on the MNIST, CIFAR-100, and CelebA datasets demonstrate that SplitFC outperforms state-of-the-art SL frameworks by significantly reducing communication overheads while maintaining high accuracy.

---

PaperID: 200,   

Authors:  Sajjad Kachuee, Mohammad Sharifkhani

Title: Latency Adjustable Transformer Encoder for Language Understanding

Abstract:
Adjusting the latency, power, and accuracy of natural language understanding models is a desirable objective of an efficient architecture. This article proposes an efficient Transformer architecture that adjusts the inference computational cost adaptively with a desired inference latency speedup. In the fine-tuning phase, the proposed method detects less important hidden sequence elements (word-vectors) and eliminates them in each encoder layer using a proposed attention context contribution (ACC) metric. After the fine-tuning phase, with the novel offline-tuning property, the inference latency of the model can be adjusted in a wide range of inference speedup selections without any further training. Extensive experiments reveal that most word-vectors in higher Transformer layers contribute less to subsequent layers, allowing their removal to improve inference latency. Experimental results on various language understanding, text generation, and instruction tuning tasks and benchmarks demonstrate the approach’s effectiveness across diverse datasets, with minimal impact on the input’s global context. The technique improves the time-to-first-token (TTFT) of Llama3 by up to 2.9× , with a minor performance drop. The suggested approach posits that in large language models (LLMs), although the complete network is necessary for training, it can be truncated during the fine-tuning phase.

---

PaperID: 201,   

Authors:  Xiaohui Wei, Haibo Liu, Puhong Duan, Shutao Li

Title: Dual-Structural Bipartite Graph Learning for Multiview Clustering

Abstract:
Bipartite graph (BiG) has been proven to be efficient in handling massive multiview data for clustering. However, how to regulate the structural information of view-specific anchors and view-shared BiG is still open and needs to be further studied. Hence, a novel dual-structural BiG learning (DsBiGL) method is proposed in the article. It transforms BiG learning into a joint optimization problem of IntrA-view and InteR-view subspace learning (IASL and IRSL) with the structural constraints, such as k-nearest neighbor (KNN) and low-rank. On one hand, IASL uses the KNN and view-specific low-rank constraints to enhance the discriminativeness of view-specific anchors. On the other hand, IRSL uses an adaptive weighting strategy to obtain view-shared BiG directly from multiview samples, where the KNN and view-shared low-rank constraints are adopted to encode local connectivity and cluster information between samples. Note that IASL and IRSL are integrated into a unified optimization model, which ensures the interactive enhancement of view-specific anchor representation and view-shared BiG learning. Finally, an algorithm based on iterative optimization is designed to solve the proposed DsBiGL model. Experimental results on various multiview datasets have demonstrated the superiority of DsBiGL in terms of clustering results when compared with other comparative methods.

---

PaperID: 202,   

Authors:  Yao Lyu, Xiangteng Zhang, Shengbo Eben Li, Jingliang Duan, Letian Tao, Qing Xu, Lei He, Keqiang Li

Title: Conformal Symplectic Optimization for Stable Reinforcement Learning

Abstract:
Training deep reinforcement learning (RL) agents necessitates overcoming the highly unstable nonconvex stochastic optimization inherent in the trial-and-error mechanism. To tackle this challenge, we propose a physics-inspired optimization algorithm called relativistic adaptive gradient descent (RAD), which enhances long-term training stability. By conceptualizing neural network (NN) training as the evolution of a conformal Hamiltonian system, we present a universal framework for transferring long-term stability from conformal symplectic integrators to iterative NN updating rules, where the choice of kinetic energy governs the dynamical properties of resulting optimization algorithms. By utilizing relativistic kinetic energy, RAD incorporates principles from special relativity and limits parameter updates below a finite speed, effectively mitigating abnormal gradient influences. In addition, RAD models NN optimization as the evolution of a multiparticle system where each trainable parameter acts as an independent particle with an individual adaptive learning rate. We prove RAD’s sublinear convergence under general nonconvex settings, where smaller gradient variance and larger batch sizes contribute to tighter convergence. Notably, RAD degrades to the well-known adaptive moment estimation (ADAM) algorithm when its speed coefficient is chosen as one and symplectic factor as a small positive value. Experimental results show RAD outperforming nine baseline optimizers with five RL algorithms across twelve environments, including standard benchmarks and challenging scenarios. Notably, RAD achieves up to a 155.1% performance improvement over ADAM in Atari games, showcasing its efficacy in stabilizing and accelerating RL training.

---

PaperID: 203,   

Authors:  Jianpeng Chen, Yawen Ling, Jie Xu, Yazhou Ren, Shudong Huang, Xiaorong Pu, Zhifeng Hao, Philip S. Yu, Lifang He

Title: Variational Graph Generator for Multiview Graph Clustering

Abstract:
Multiview graph clustering (MGC) methods are increasingly being studied due to the explosion of multiview data with graph structural information. The critical point of MGC is to better utilize view-specific and view-common information in features and graphs of multiple views. However, existing works have an inherent limitation that they are unable to concurrently utilize the consensus graph information across multiple graphs and the view-specific feature information. To address this issue, we propose a variational graph generator for MGC (VGMGC). Specifically, a novel variational graph generator is proposed to extract common information among multiple graphs. This generator infers a reliable variational consensus graph based on a priori assumption over multiple graphs. Then, a simple yet effective graph encoder in conjunction with the multiview clustering objective is presented to learn the desired graph embeddings for clustering, which embeds the inferred view-common graph and view-specific graphs together with features. Finally, theoretical results illustrate the rationality of the VGMGC by analyzing the uncertainty of the inferred consensus graph with the information bottleneck (IB) principle. Extensive experiments demonstrate the superior performance of our VGMGC over state-of-the-art methods (SOTAs). The source code is publicly available at: https://github.com/cjpcool/VGMGC.

---

PaperID: 204,   

Authors:  Xiaobo Shen, Yinfan Chen, Weiwei Liu, Yuhui Zheng, Quan-Sen Sun, Shirui Pan

Title: Graph Convolutional Multi-Label Hashing for Cross-Modal Retrieval

Abstract:
Cross-modal hashing encodes different modalities of multimodal data into low-dimensional Hamming space for fast cross-modal retrieval. In multi-label cross-modal retrieval, multimodal data are often annotated with multiple labels, and some labels, e.g., “ocean” and “cloud,” often co-occur. However, existing cross-modal hashing methods overlook label dependency that is crucial for improving performance. To fulfill this gap, this article proposes graph convolutional multi-label hashing (GCMLH) for effective multi-label cross-modal retrieval. Specifically, GCMLH first generates word embedding of each label and develops label encoder to learn highly correlated label embedding via graph convolutional network (GCN). In addition, GCMLH develops feature encoder for each modality, and feature fusion module to generate highly semantic feature via GCN. GCMLH uses teacher-student learning scheme to transfer knowledge from the teacher modules, i.e., label encoder and feature fusion module, to the student module, i.e., feature encoder, such that learned hash code can well exploit multi-label dependency and multimodal semantic structure. Extensive empirical results on several benchmarks demonstrate the superiority of the proposed method over existing state-of-the-arts.

---

PaperID: 205,   

Authors:  Chang Liu, Bohao Li, Mengnan Shi, Xiaozhong Chen, Qixiang Ye, Xiangyang Ji

Title: Explicit Margin Equilibrium for Few-Shot Object Detection

Abstract:
Under low data regimes, few-shot object detection (FSOD) transfers related knowledge from base classes with sufficient annotations to novel classes with limited samples in a two-step paradigm, including base training and balanced fine-tuning. In base training, the learned embedding space needs to be dispersed with large class margins to facilitate novel class accommodation and avoid feature aliasing while in balanced fine-tuning properly concentrating with small margins to represent novel classes precisely. Although obsession with the discrimination and representation dilemma has stimulated substantial progress, explorations for the equilibrium of class margins within the embedding space are still in full swing. In this study, we propose a class margin optimization scheme, termed explicit margin equilibrium (EME), by explicitly leveraging the quantified relationship between base and novel classes. EME first maximizes base-class margins to reserve adequate space to prepare for novel class adaptation. During fine-tuning, it quantifies the interclass semantic relationships by calculating the equilibrium coefficients based on the assumption that novel instances can be represented by linear combinations of base-class prototypes. EME finally reweights margin loss using equilibrium coefficients to adapt base knowledge for novel instance learning with the help of instance disturbance (ID) augmentation. As a plug-and-play module, EME can also be applied to few-shot classification. Consistent performance gains upon various baseline methods and benchmarks validate the generality and efficacy of EME. The code is available at github.com/Bohao-Lee/EME.

---

PaperID: 206,   

Authors:  Yu-Xuan Zhang, Zhengchun Zhou, Xingxing He, Avik Ranjan Adhikary, Bapi Dutta

Title: Data-Driven Knowledge Fusion for Deep Multi-Instance Learning

Abstract:
Multi-instance learning (MIL) is a widely applied technique in practical applications that involve complex data structures. MIL can be broadly categorized into two types: traditional methods and those based on deep learning. These approaches have yielded significant results, especially regarding their problem-solving strategies and experiment validation, providing valuable insights for researchers in the MIL field. However, considerable knowledge is often trapped within the algorithm, leading to subsequent MIL algorithms that rely solely on the model’s data fitting to predict unlabeled samples. This results in a significant loss of knowledge and impedes the development of more powerful models. In this article, we propose a novel data-driven knowledge fusion for deep MIL (DKMIL) algorithm. DKMIL adopts a completely different idea from existing deep MIL methods by analyzing the decision-making of key samples in the dataset (referred to as the data-driven) and using the knowledge fusion module designed to extract valuable information from these samples to assist the model’s learning. In other words, this module serves as a new interface between data and the model, providing strong scalability and enabling prior knowledge from existing algorithms to enhance the model’s learning ability. Furthermore, to adapt the downstream modules of the model to more knowledge-enriched features extracted from the data-driven knowledge fusion (DDKF) module, we propose a two-level attention (TLA) module that gradually learns shallow- and deep-level features of the samples to achieve more effective classification. We will prove the scalability of the knowledge fusion module and verify the efficiency of the proposed architecture by conducting experiments on 62 datasets across five categories.

---

PaperID: 207,   

Authors:  Jiayi Tang, Long Zhao, Xinwang Liu

Title: Incomplete Multiview Clustering Based on Consensus Information

Abstract:
In contrast to traditional single-view clustering methods, multiview clustering (MVC) approaches aim to extract, analyze, and integrate structural information from diverse perspectives, providing a more comprehensive understanding of internal data structures. However, with an increasing number of views, maintaining the integrity of view information becomes challenging, giving rise to incomplete MVC (IMVC) methods. While existing IMVC methods have shown notable performance on many incomplete multiview (IMV) databases, they still grapple with two key shortcomings: 1) they treat the information of each view as a whole, disregarding the differences among samples within each view; and 2) they rely on eigenvalue and eigenvector operations on the view matrix, limiting their scalability for large-scale samples and views. To overcome these limitations, we propose a novel multiview clustering with consistent information (IMVC-CI) of sample points. Our method explores the multiview information set of sample points to extract consensus structural information and subsequently restores unknown information in each view. Importantly, our approach operates independently on individual sample points, eliminating the need for eigenvalue and eigenvector operations on the view information matrix and facilitating parallel computation. This significantly enhances algorithmic efficiency and mitigates challenges associated with dimensionality. Experimental results on various public datasets demonstrate that our algorithm outperforms state-of-the-art IMVC methods in terms of clustering performance and computational efficiency. The code for our article has been uploaded to https://github.com/PhdJiayiTang/IMVC-CI.git.

---

PaperID: 208,   

Authors:  Yunqi Huang, Chang Liu, Bohao Li, Hai-Jun Huang, Ronghui Zhang, Wei Ke, Xiaojun Jing

Title: Frequency-Aware Divide-and-Conquer for Efficient Real Noise Removal

Abstract:
Deep-learning-based approaches have achieved remarkable progress for complex real scenario denoising, yet their accuracy-efficiency tradeoff is still understudied, particularly critical for mobile devices. As real noise is unevenly distributed relative to underlay signals in different frequency bands, we introduce a frequency-aware divide-and-conquer strategy to develop a frequency-aware denoising network (FADN). FADN is materialized by stacking frequency-aware denoising blocks (FADBs), in which a denoised image is progressively predicted by a series of frequency-aware noise dividing and conquering operations. For noise dividing, FADBs decompose the noisy and clean image pairs into low- and high-frequency representations via a wavelet transform (WT) followed by an invertible network and recover the final denoised image by integrating the denoised information from different frequency bands. For noise conquering, the separated low-frequency representation of the noisy image is kept as clean as possible by the supervision of the clean counterpart, while the high-frequency representation combining the estimated residual from the successive FADB is purified under the corresponding accompanied supervision for residual compensation. Since our FADN progressively and pertinently denoises from frequency bands, the accuracy-efficiency tradeoff can be controlled as a requirement by the number of FADBs. Experimental results on the SIDD, DND, and NAM datasets show that our FADN outperforms the state-of-the-art methods by improving the peak signal-to-noise ratio (PSNR) and decreasing the model parameters. The code is released at https://github.com/NekoDaiSiki/FADN.

---

PaperID: 209,   

Authors:  Yu Hu, Fei Qi, Yiu-Ming Cheung, Hongmin Cai

Title: Discriminating Tensor Spectral Clustering for High-Dimension-Low-Sample-Size Data

Abstract:
Tensor spectral clustering (TSC) is a recently proposed approach to robustly group data into underlying clusters. Unlike the traditional spectral clustering (SC), which merely uses pairwise similarities of data in an affinity matrix, TSC aims at exploring their multiwise similarities in an affinity tensor to achieve better performance. However, the performance of TSC highly relies on the design of multiwise similarities, and it remains unclear especially for high-dimension-low-sample-size (HDLSS) data. To this end, this article has proposed a discriminating TSC (DTSC) for HDLSS data. Specifically, DTSC uses the proposed discriminating affinity tensor that encodes the pair-to-pair similarities, which are particularly constructed by the anchor-based distance. HDLSS asymptotic analysis shows that the proposed affinity tensor can explicitly differentiate samples from different clusters when the feature dimension is large. This theoretical property allows DTSC to improve the clustering performance on HDLSS data. Experimental results on synthetic and benchmark datasets demonstrate the effectiveness and robustness of the proposed method in comparison to several baseline methods.

---

PaperID: 210,   

Authors:  Yueyang Pi, Yilin Wu, Yang Huang, Yongquan Shi, Shiping Wang

Title: Inhomogeneous Diffusion-Induced Network for Multiview Semi-Supervised Classification

Abstract:
The challenges posed by heterogeneous data in practical applications have made multiview semi-supervised classification a focus of attention for researchers. While several graph-based approaches have been suggested for this task, they tend to use homogeneous feature propagation, leading to even diffusion of node information to their neighbors. However, this diffusion strategy results in nodes acquiring information of equal proportion from dissimilar samples. In this article, we propose a solution to address these issues by introducing a graph diffusion-induced network for multiview semi-supervised classification. By formulating a discretized partial differential equation on a manifold, we derive a nonlinear and inhomogeneous diffusion equation to govern information propagation on the graph. Then, we investigate the impact of various nonlinear activation functions on random switching edge directions and their suppressive effects on information diffusion between different nodes. In addition, the cross-view consistency under the semi-supervised scenarios is defined and guaranteed for better information fusion. The comprehensive experimental results demonstrate the superiority of the proposed method compared with state-of-the-art approaches. The effectiveness of the proposed approach in handling diverse and heterogeneous data showcases its potential for advancing multiview semi-supervised classification techniques.

---

PaperID: 211,   

Authors:  Liping Deng, Wen-Sheng Chen, Binbin Pan, Mingqing Xiao

Title: Hyperparameter Recommendation Integrated With Convolutional Neural Network

Abstract:
Hyperparameter recommendation via meta-learning has shown great promise in various studies. The main challenge for meta-learning is how to develop an effective meta-learner (learning algorithm) that can capture the intrinsic relationship between dataset characteristics and the empirical performance of hyperparameters. Existing meta-learners are mostly based on traditional machine-learning models that only learn data representations with a single layer, which are incapable of learning complex features from the data and often cannot capture those properties deeply embedded in data. To address this issue, in this article, we propose hyperparameter recommendation approaches by integrating the learning model with convolutional neural networks (CNNs). Specifically, we first formulate the recommendation task as a regression problem, where dataset characteristics are treated as predictors and the historical performance of hyperparameters as responses. We establish a CNN-based learning model with feature selection capability to serve as the regressor. We then develop a convolutional denoising autoencoder (ConvDAE) that can leverage the spatial structure of the entire hyperparameter performance space and evaluate the performance of hyperparameters via denoising when the performance of partial hyperparameters is available under the multidimensional framework. To make our approach being flexible in applications, we establish a comprehensive two-branch CNN model that can utilize both dataset characteristics and partial evaluations to make effective recommendations. We conduct extensive experiments on 400 real classification problems and the well-known SVM. Our proposed approaches outperform existing meta-learning baselines as well as various search algorithms, demonstrating the high effectiveness in hyperparameter recommendations via deep learning.

---

PaperID: 212,   

Authors:  Bin-Bin Jia, Jun-Ying Liu, Min-Ling Zhang

Title: Instance-Specific Loss-Weighted Decoding for Decomposition-Based Multiclass Classification

Abstract:
Multiclass classification problems are often addressed by decomposing them into a set of binary classification tasks. A critical step in this approach is the effective aggregation of predictions from each decomposed binary classifier to yield the final multiclass prediction, a process known as decoding. Existing studies have ignored the varying generalization ability of each binary classifier across different samples during decoding, potentially leading to suboptimal performance. In this article, we propose an instance-specific loss-weighted (ILW) decoding strategy that gauges the generalization ability of each binary classifier for one specific sample based on its neighboring samples. This estimated generalization ability is then used to adjust the importance of the binary classifier in determining the sample’s final prediction. Experimental results validate the effectiveness of the ILW decoding strategy. Furthermore, we demonstrate that softmax regression can be reinterpreted as a one-versus-rest (OvR) decomposition-based multiclass classification algorithm, enabling the application of our decoding strategy to enhance its performance. Comparative studies clearly demonstrate the superiority of the improved softmax regression over its traditional counterpart.

---

PaperID: 213,   

Authors:  Like Xin, Wanqi Yang, Lei Wang, Ming Yang

Title: Unpaired Multiview Clustering via Reliable View Guidance

Abstract:
This article focuses on unpaired multiview clustering (UMC), a challenging problem, where paired observed samples are unavailable across multiple views. The goal is to perform effective joint clustering using the unpaired observed samples in all views. In incomplete multiview clustering (IMC), existing methods typically rely on sample pairing between views to capture their complementary. However, this is not applicable in the case of UMC. Hence, we aim to extract the consistent cluster structure across views. In UMC, two challenging issues arise: the uncertain cluster structure due to the lack of labels and the uncertain pairing relationship due to the absence of paired samples. We assume that the view with a good cluster structure is the reliable view, which acts as a supervisor to guide the clustering of the other views. With the guidance of reliable views, a more certain cluster structure of these views is obtained while achieving alignment between the reliable views and the other views. Then, we propose reliable view guided UMC with one reliable view (RG-UMC) and reliable view guided UMC with multiple reliable views (RGs-UMC). Specifically, we design alignment modules with one reliable view and multiple reliable views, respectively, to adaptively guide the optimization process. Also, we utilize the compactness module to enhance the relationship of samples within the same cluster. Meanwhile, an orthogonal constraint is applied to the latent representation to obtain discriminate features. Extensive experiments show that both RG-UMC and RGs-UMC outperform the best state-of-the-art method by an average of 24.14% and 29.42% in normalized mutual information (NMI), respectively.

---

PaperID: 214,   

Authors:  Zhicheng Cai, Xiaohan Ding, Qiu Shen, Xun Cao

Title: RefConv: Reparameterized Refocusing Convolution for Powerful ConvNets

Abstract:
We propose reparameterized refocusing convolution (RefConv) as a replacement for regular convolutional layers, which is a plug-and-play module to improve the performance without any inference costs. Specifically, given a pretrained model, RefConv applies a trainable Refocusing Transformation to the basis kernels inherited from the pretrained model to establish connections among the parameters. For example, a depthwise RefConv can relate the parameters of a specific channel of convolution kernel to the parameters of the other kernel, i.e., make them refocus on the other parts of the model they have never attended to, rather than focus on the input features only. From another perspective, RefConv augments the priors of existing model structures by utilizing the representations encoded in the pretrained parameters as the priors and refocusing on them to learn novel representations, thus further enhancing the representational capacity of the pretrained model. The experimental results validated that RefConv can improve multiple convolutional neural network (CNN)-based models by a clear margin on image classification (up to 1.47% higher top-1 accuracy on ImageNet), object detection, semantic segmentation, and adversarial attacks without introducing any extra inference costs or altering the original model structure. Further studies demonstrated that RefConv can strengthen the spatial skeletons of kernels, reduce the redundancy of channels, and smooth the loss landscape, which explains its effectiveness.

---

PaperID: 215,   

Authors:  Liang Zhao, Qiongjie Xie, Zhengtao Li, Songtao Wu, Yi Yang

Title: Dynamic Graph Guided Progressive Partial View-Aligned Clustering

Abstract:
In recent years, there has been a growing focus on multiview data, driven by its rich complementary and consistent information, which has the potential to significantly enhance the performance of downstream tasks. Although many multiview clustering (MVC) methods have achieved promising results by integrating the information of multiple views to learn the consistent representation or consistent graph, these methods typically require complete and entirely accurate correspondences between multiview data, which is challenging to fulfill in practice leading to the problem of partially view-aligned clustering (PVC). To tackle it, we propose a novel method, called dynamic graph guided progressive partial view-aligned clustering (DGPPVC) in this article. To the best of our knowledge, this could be the first work to employ graph convolutional network (GCN) to address the problem of PVC, which explores GCN with dynamic adjacency matrix to reduce unreliable alignments and locate the feature representation with consistent graph structure. In particular, DGPPVC develops an end-to-end framework that encompasses graph construction, feature representation learning, and alignment relationships learning, in which the three parts mutually influence and benefit each other. Moreover, DGPPVC adopts a novel alignment learning strategy that progresses from simplicity to complexity, enabling the step-by-step acquisition of unknown correspondences between different modalities. By giving priority to simple instance pairs, a variant of Jaccard similarities is designed to identify more reliable and complex alignments progressively. During the gradual learning process of alignment relationships, the graph structure matrix is continually and dynamically optimized, thus acquiring a greater variety of graph information between different views. Experiments on several real-world datasets show our promising performance compared with the state-of-the-art methods in partially view-aligned clustering.

---

PaperID: 216,   

Authors:  Jiao Li, Liangxiao Jiang, Wenjun Zhang

Title: Label Consistency-Based Ground Truth Inference for Crowdsourcing

Abstract:
In crowdsourcing scenarios, we can obtain each instance’s multiple noisy labels from different crowd workers and then infer its unknown ground truth via a ground truth inference method. However, to the best of our knowledge, the existing ground truth inference methods always attempt to aggregate multiple noisy labels into a single consensus label as the ground truth. In this article, we aim to explore a new strategy, i.e., label selection, which directly selects the label of the highest quality worker as the ground truth. To this end, we propose a label consistency-based ground truth inference (LCGTI) method. In LCGTI, we argue that high-quality workers should have a low bias with other workers in labeling the same instances and a low variance with themselves in labeling similar instances. To estimate the bias, we calculate the label consistency of different workers on the same instances. To estimate the variance, we calculate the label consistency of the same worker on similar instances. Finally, we combine these two components to calculate the labeling quality of each worker on the inferred instance and perform label selection instead of label aggregation to achieve inference. The experimental results on 34 simulated and two real-world datasets show that LCGTI significantly outperforms all the other state-of-the-art label aggregation-based ground truth inference methods.

---

PaperID: 217,   

Authors:  Yanyan Shao, Dongyan Guo, Ying Cui, Zhenhua Wang, Liyan Zhang, Jianhua Zhang

Title: Graph Attention Network for Context-Aware Visual Tracking

Abstract:
Siamese-network-based trackers convert the general object tracking as a similarity matching task between a template and a search region. Using convolutional feature cross correlation (Xcorr) for similarity matching, a large number of Siamese trackers are proposed and achieved great success. However, due to the predefined size of the target feature, these trackers suffer from either retaining much background information or losing important foreground information. Moreover, the global matching between the target and search region also largely neglects the part-level structural information and the contextual information of the target. To tackle the aforementioned obstacles, in this article, we propose a simple context-aware Siamese graph attention network, which establishes part-to-part correspondence between the Siamese branches with a complete bipartite graph. The object information from the template is propagated to the search region via a graph attention mechanism. With such a design, a target-aware template input is enabled to replace the prefixed template region, which can adaptively fit the size and aspect ratio variations in different objects. Based on it, we further construct a context-aware feature matching mechanism to embed both the target and the contextual information in the search region. Experiments on challenging benchmarks including GOT-10k, TrackingNet, LaSOT, VOT2020, and OTB-100 demonstrate that the proposed SiamGAT outperforms many state-of-the-art trackers and achieves leading performance. Code is available at: https://git.io/SiamGAT.

---

PaperID: 218,   

Authors:  Xiaolong Fan, Maoguo Gong, Yue Wu, Mingyang Zhang, Hao Li, Xiangming Jiang

Title: CCGIB: A Cross-Channel Graph Information Bottleneck Principle

Abstract:
The empirical studies of most existing graph neural networks (GNNs) broadly take the original node feature and adjacency relationship as single-channel input, ignoring the rich information of multiple graph channels. To circumvent this issue, the multichannel graph analysis framework has been developed to fuse graph information across channels. How to model and integrate shared (i.e., consistency) and channel-specific (i.e., complementarity) information is a key issue in multichannel graph analysis. In this article, we propose a cross-channel graph information bottleneck (CCGIB) principle to maximize the agreement for common representations and the disagreement for channel-specific representations. Under this principle, we formulate the consistency and complementarity information bottleneck (IB) objectives. To enable optimization, a viable approach involves deriving variational lower bound and variational upper bound (VarUB) of mutual information terms, subsequently focusing on optimizing these variational bounds to find the approximate solutions. However, obtaining the lower bounds of cross-channel mutual information objectives proves challenging through direct utilization of variational approximation, primarily due to the independence of the distributions. To address this challenge, we leverage the inherent property of joint distributions and subsequently derive variational bounds to effectively optimize these information objectives. Extensive experiments on graph benchmark datasets demonstrate the superior effectiveness of the proposed method.

---

PaperID: 219,   

Authors:  Fusheng Hao, Liu Liu, Fuxiang Wu, Qieshi Zhang, Jun Cheng

Title: Class-Irrelevant Feature Removal for Few-Shot Image Classification

Abstract:
Most existing few-shot image classification methods employ global pooling to aggregate class-relevant local features in a data-drive manner. Due to the difficulty and inaccuracy in locating class-relevant regions in complex scenarios, as well as the large semantic diversity of local features, the class-irrelevant information could reduce the robustness of the representations obtained by performing global pooling. Meanwhile, the scarcity of labeled images exacerbates the difficulties of data-hungry deep models in identifying class-relevant regions. These issues severely limit deep models’ few-shot learning ability. In this work, we propose to remove the class-irrelevant information by making local features class relevant, thus bypassing the big challenge of identifying which local features are class irrelevant. The resulting class-irrelevant feature removal (CIFR) method consists of three phases. First, we employ the masked image modeling strategy to build an understanding of images’ internal structures that generalizes well. Second, we design a semantic-complementary feature propagation module to make local features class relevant. Third, we introduce a weighted dense-connected similarity measure, based on which a loss function is raised to fine-tune the entire pipeline, with the aim of further enhancing the semantic consistency of the class-relevant local features. Visualization results show that CIFR achieves the removal of class-irrelevant information by making local features related to classes. Comparison results on four benchmark datasets indicate that CIFR yields very promising performance.

---

PaperID: 220,   

Authors:  Guangzhi Ma, Jie Lu, Zhen Fang, Feng Liu, Guangquan Zhang

Title: Multiview Classification Through Learning From Interval-Valued Data

Abstract:
The classification problem concerning crisp-valued data has been well resolved. However, interval-valued data, where all of the observations’ features are described by intervals, are also a common data type in real-world scenarios. For example, the data extracted by many measuring devices are not exact numbers but intervals. In this article, we focus on a highly challenging problem called learning from interval-valued data (LIND), where we aim to learn a classifier with high performance on interval-valued observations. First, we obtain the estimation error bound of the LIND problem based on the Rademacher complexity. Then, we give the theoretical analysis to show the strengths of multiview learning on classification problems, which inspires us to construct a new algorithm called multiview interval information extraction (Mv-IIE) approach for improving classification accuracy on interval-valued data. The experiment comparisons with several baselines on both synthetic and real-world datasets illustrate the superiority of the proposed framework in handling interval-valued data. Moreover, we describe an application of Mv-IIE that we can prevent data privacy leakage by transforming crisp-valued (raw) data into interval-valued data.

---

PaperID: 221,   

Authors:  Taiheng Liu, Guang-Zhong Cao, Zhaoshui He, Shengli Xie, Xiuqin Deng

Title: Multimodal Fusion Network for 3-D Lane Detection

Abstract:
3-D lane detection is a challenging task due to the diversity of lanes, occlusion, dazzle light, and so on. Traditional methods usually use highly specialized handcrafted features and carefully designed postprocessing to detect them. However, these methods are based on strong assumptions and single modal so that they are easily scalable and have poor performance. In this article, a multimodal fusion network (MFNet) is proposed through using multihead nonlocal attention and feature pyramid for 3-D lane detection. It includes three parts: multihead deformable transformation (MDT) module, multidirectional attention feature pyramid fusion (MA-FPF) module, and top-view lane prediction (TLP) ones. First, MDT is presented to learn and mine multimodal features from RGB images, depth maps, and point cloud data (PCD) for achieving optimal lane feature extraction. Then, MA-FPF is designed to fuse multiscale features for presenting the vanish of lane features as the network deepens. Finally, TLP is developed to estimate 3-D lanes and predict their position. Experimental results on the 3-D lane synthetic and ONCE-3DLanes datasets demonstrate that the performance of the proposed MFNet outperforms the state-of-the-art methods in both qualitative and quantitative analyses and visual comparisons.

---

PaperID: 222,   

Authors:  Yingsong Cheng, Xinya Wang, Yong Ma, Xiaoguang Mei, Minghui Wu, Jiayi Ma

Title: General Hyperspectral Image Super-Resolution via Meta-Transfer Learning

Abstract:
Recent advances in deep learning-based methods have led to significant progress in the hyperspectral super-resolution (SR). However, the scarcity and the high dimension of data have hindered further development since deep models require sufficient data to learn stable patterns. Moreover, the huge domain differences between hyperspectral image (HSI) datasets pose a significant challenge in generalizability. To address these problems, we present a general hyperspectral SR framework via meta-transfer learning (MTL). We randomly sample various spectral ranges for SR tasks during MTL, allowing the model to accumulate diverse task experiences. Additionally, we implement a task schedule to gradually expand the number of bands, bridging the significant domain differences between datasets. By leveraging multiple datasets, we are able to achieve better performance and greater generalizability, making it applicable under various circumstances. Meanwhile, as a general framework, our scheme can be applied to existing methods to obtain performance improvements. In addition, we design an advanced network architecture based on the multifusion features to further improve the performance. Experiments demonstrate that our method not only achieves superior performance in both qualitative and quantitative terms but also can adapt robustly to a new and difficult sample, where few epochs can yield quite considerable results.

---

PaperID: 223,   

Authors:  Bin Gu, Hilal AlQuabeh, William de Vazelhes, Zhouyuan Huo, Heng Huang

Title: Stagewise Training With Exponentially Growing Training Sets

Abstract:
In the world of big data, training large-scale machine learning problems has gained considerable attention. Numerous innovative optimization strategies have been presented in recent years to accelerate the large-scale training process. However, the possibility of further accelerating the training process of various optimization algorithms remains an unresolved subject. To begin addressing this difficult problem, we exploit the researched findings that when training data are independent and identically distributed, the learning problem on a smaller dataset is not significantly different from the original one. Upon that, we propose a stagewise training technique that grows the size of the training set exponentially while solving nonsmooth subproblem. We demonstrate that our stagewise training via exponentially growing the size of the training sets (STEGSs) are compatible with a large number of proximal gradient descent and gradient hard thresholding (GHT) techniques. Interestingly, we demonstrate that STEGS can greatly reduce overall complexity while maintaining statistical accuracy or even surpassing the intrinsic error introduced by GHT approaches. In addition, we analyze the effect of the training data growth rate on the overall complexity. The practical results of applying l_2,1 - and l_0 -norms to a variety of large-scale real-world datasets not only corroborate our theories but also demonstrate the benefits of our STEGS framework.

---

PaperID: 224,   

Authors:  Lei Yuan, Lihe Li, Ziqian Zhang, Fuxiang Zhang, Cong Guan, Yang Yu

Title: Multiagent Continual Coordination via Progressive Task Contextualization

Abstract:
Cooperative multiagent reinforcement learning (MARL) has attracted significant attention and has the potential for many real-world applications. Previous arts mainly focus on facilitating the coordination ability from different aspects (e.g., nonstationarity and credit assignment) in single-task or multitask scenarios, ignoring the stream of tasks that appear in a continual manner. This ignorance makes the continual coordination an unexplored territory, neither in problem formulation nor efficient algorithms designed. Toward tackling the mentioned issue, this article proposes an approach, multiagent continual coordination via progressive task contextualization (MACPro). The key point lies in obtaining a factorized policy, using shared feature extraction layers but separated independent task heads, each specializing in a specific class of tasks. The task heads can be progressively expanded based on the learned task contextualization. Moreover, to cater to the popular centralized training with decentralized execution (CTDE) paradigm in MARL, each agent learns to predict and adopt the most relevant policy head based on local information in a decentralized manner. We show in multiple multiagent benchmarks that existing continual learning methods fail, while MACPro is able to achieve close-to-optimal performance. More results also disclose the effectiveness of MACPro from multiple aspects, such as high generalization ability.

---

PaperID: 225,   

Authors:  Xiangkun He, Jianye Hao, Xu Chen, Jun Wang, Xuewu Ji, Chen Lv

Title: Robust Multiobjective Reinforcement Learning Considering Environmental Uncertainties

Abstract:
Numerous real-world decision or control problems involve multiple conflicting objectives whose relative importance (preference) is required to be weighed in different scenarios. While Pareto optimality is desired, environmental uncertainties (e.g., environmental changes or observational noises) may mislead the agent into performing suboptimal policies. In this article, we present a novel multiobjective optimization paradigm, robust multiobjective reinforcement learning (RMORL) considering environmental uncertainties, to train a single model that can approximate robust Pareto-optimal policies across the entire preference space. To enhance policy robustness against environmental changes, an environmental disturbance is modeled as an adversarial agent across the entire preference space via incorporating a zero-sum game into a multiobjective Markov decision process (MOMDP). Additionally, we devise an adversarial defense technique against observational perturbations, which ensures that policy variations, perturbed by adversarial attacks on state observations, remain within bounds under any specified preferences. The proposed technique is assessed in five multiobjective environments with continuous action spaces, showcasing its effectiveness through comparisons with competitive baselines, which encompass classical and state-of-the-art schemes.

---

PaperID: 226,   

Authors:  Jia-Qi Lin, Man-Sheng Chen, Xi-Ran Zhu, Chang-Dong Wang, Haizhang Zhang

Title: Dual Information Enhanced Multiview Attributed Graph Clustering

Abstract:
Multiview attributed graph clustering is an important approach to partition multiview data based on the attribute characteristics and adjacent matrices from different views. Some attempts have been made in using graph neural network (GNN), which have achieved promising clustering performance. Despite this, few of them pay attention to the inherent specific information embedded in multiple views. Meanwhile, they are incapable of recovering the latent high-level representation from the low-level ones, greatly limiting the downstream clustering performance. To fill these gaps, a novel dual information enhanced multiview attributed graph clustering (DIAGC) method is proposed in this article. Specifically, the proposed method introduces the specific information reconstruction (SIR) module to disentangle the explorations of the consensus and specific information from multiple views, which enables graph convolutional network (GCN) to capture the more essential low-level representations. Besides, the contrastive learning (CL) module maximizes the agreement between the latent high-level representation and low-level ones and enables the high-level representation to satisfy the desired clustering structure with the help of the self-supervised clustering (SC) module. Extensive experiments on several real-world benchmarks demonstrate the effectiveness of the proposed DIAGC method compared with the state-of-the-art baselines.

---

PaperID: 227,   

Authors:  Hongmin Liu, Yuefeng Cai, Bin Fan, Jinglin Xu

Title: ScalableTrack: Scalable One-Stream Tracking via Alternating Learning

Abstract:
Transformer-based one-stream trackers are widely used to extract features and interact information for visual object tracking. However, the current one-stream tracker has fixed computational dimensions between different stages, which limits the network’s ability to learn context clues and global representations, resulting in a decrease in the ability to distinguish between targets and backgrounds. To address this issue, a new scalable one-stream tracking framework, ScalableTrack, is proposed. It unifies feature extraction and information integration by intrastage mutual guidance, leveraging the scalability of target-oriented features to enhance object sensitivity and obtain discriminative global representations. In addition, we bridge interstage contextual cues by introducing an alternating learning strategy and solve the arrangement problem of the two modules. The alternating learning strategy uses alternate stacks of feature extraction and information interaction to focus on tracked objects and prevent catastrophic forgetting of target information between different stages. Experiments on eight challenging benchmarks (TrackingNet, GOT-10k, VOT2020, UAV123, LaSOT, LaSOText, OTB100, and TC128) show that ScalableTrack outperforms state-of-the-art (SOTA) methods with better generalization and global representation ability.

---

PaperID: 228,   

Authors:  Mingcai Chen, Yu Zhao, Bing He, Zongbo Han, Junzhou Huang, Bingzhe Wu, Jianhua Yao

Title: Learning With Noisy Labels Over Imbalanced Subpopulations

Abstract:
Learning with noisy labels (LNL) has attracted significant attention from the research community. Many recent LNL methods rely on the assumption that clean samples tend to have a “small loss.” However, this assumption often fails to generalize to some real-world cases with imbalanced subpopulations, that is, training subpopulations that vary in sample size or recognition difficulty. Therefore, recent LNL methods face the risk of misclassifying those “informative” samples (e.g., hard samples or samples in the tail subpopulations) into noisy samples, leading to poor generalization performance. To address this issue, we propose a novel LNL method to deal with noisy labels and imbalanced subpopulations simultaneously. It first leverages sample correlation to estimate samples’ clean probabilities for label correction and then utilizes corrected labels for distributionally robust optimization (DRO) to further improve the robustness. Specifically, in contrast to previous works using classification loss as the selection criterion, we introduce a feature-based metric that takes the sample correlation into account for estimating samples’ clean probabilities. Then, we refurbish the noisy labels using the estimated clean probabilities and the pseudo-labels from the model’s predictions. With refurbished labels, we use DRO to train the model to be robust to subpopulation imbalance. Extensive experiments on a wide range of benchmarks demonstrate that our technique can consistently improve state-of-the-art (SOTA) robust learning paradigms against noisy labels, especially when encountering imbalanced subpopulations. We provide our code in https://github.com/chenmc1996/LNL-IS.

---

PaperID: 229,   

Authors:  Qi Liu, Yanjie Li, Xiongtao Shi, Ke Lin, Yuecheng Liu, Yunjiang Lou

Title: Distributional Policy Gradient With Distributional Value Function

Abstract:
In this article, we propose a distributional policy-gradient method based on distributional reinforcement learning (RL) and policy gradient. Conventional RL algorithms typically estimate the expectation of return, given a state–action pair. Furthermore, distributional RL algorithms consider the return as a random variable and estimate the return distribution that can characterize the probability of different returns resulted by environmental uncertainties. Thus, the return distribution provides more valuable information than its expectation, leading to superior policies in general. Although distributional RL has been investigated widely in value-based RL methods, very few policy-gradient methods take advantage of distributional RL. To bridge this research gap, we propose a distributional policy-gradient method by introducing a distributional value function to the policy gradient (DVDPG). We estimate the distribution of policy gradient instead of the expectation estimated in conventional policy-gradient RL methods. Furthermore, we propose two policy-gradient value sampling mechanisms to do policy improvement. First, we propose a distribution-probability-sampling method that samples the policy-gradient value according to the quantile probability of return distribution. Second, a uniform sample mechanism is proposed. With our sample mechanisms, the proposed distributional policy-gradient method enhances the stochasticity of the policy gradient, improving the exploration efficiency and benefiting to avoid falling into local optimal solutions. In sparse-reward tasks, the distribution-probability-sampling method outperforms the uniform sample mechanism. In dense-reward tasks, the two sample mechanisms perform similarly. Moreover, we show that the conventional policy-gradient method is a special case of the proposed method. Experimental results on various sparse-reward and dense-reward OpenAI-gym tasks illustrate the efficiency of the proposed method, outperforming baselines in almost environments.

---

PaperID: 230,   

Authors:  Yuexuan An, Hui Xue, Xingyu Zhao, Ning Xu, Pengfei Fang, Xin Geng

Title: Leveraging Bilateral Correlations for Multi-Label Few-Shot Learning

Abstract:
Multi-label few-shot learning (ML-FSL) refers to the task of tagging previously unseen images with a set of relevant labels, giving a small number of training examples. Modeling the correlations between instances and labels, formulated in the existing methods, allows us to extract more available knowledge from limited examples. However, they simply explore the instance and label correlations with a uniform importance assumption without considering the discrepancy of importance in different instances or labels, making the utilization of instance and label correlations a bottleneck for ML-FSL. To tackle the issue, we propose a unified framework named bilateral correlation reconstruction (BCR) to enable the network to effectively mine underlying instance and label correlations with varying importance information from both instance-to-label and label-to-instance perspectives. Specifically, from the instance-to-label perspective, we refine prototypes per category by reweighting each image with its specific instance-importance degree extracted from the similarity between the instance and the corresponding category. From the label-to-instance perspective, we smooth labels for each image by recovering latent label-importance with considering the integrated topology of all samples in a task. Experimental results on multiple benchmarks validate that BCR could outperform existing ML-FSL methods by large margins.

---

PaperID: 231,   

Authors:  Meijun Fu, Xiaomin Wang, Jun Wang, Zhang Yi

Title: Prototype Bayesian Meta-Learning for Few-Shot Image Classification

Abstract:
Meta-learning aims to leverage prior knowledge from related tasks to enable a base learner to quickly adapt to new tasks with limited labeled samples. However, traditional meta-learning methods have limitations as they provide an optimal initialization for all new tasks, disregarding the inherent uncertainty induced by few-shot tasks and impeding task-specific self-adaptation initialization. In response to this challenge, this article proposes a novel probabilistic meta-learning approach called prototype Bayesian meta-learning (PBML). PBML focuses on meta-learning variational posteriors within a Bayesian framework, guided by prototype-conditioned prior information. Specifically, to capture model uncertainty, PBML treats both meta- and task-specific parameters as random variables and integrates their posterior estimates into hierarchical Bayesian modeling through variational inference (VI). During model inference, PBML employs Laplacian estimation to approximate the integral term over the likelihood loss, deriving a rigorous upper-bound for generalization errors. To enhance the model’s expressiveness and enable task-specific adaptive initialization, PBML proposes a data-driven approach to model the task-specific variational posteriors. This is achieved by designing a generative model structure that incorporates prototype-conditioned task-dependent priors into the random generation of task-specific variational posteriors. Additionally, by performing latent embedding optimization, PBML decouples the gradient-based meta-learning from the high-dimensional variational parameter space. Experimental results on benchmark datasets for few-shot image classification illustrate that PBML attains state-of-the-art or competitive performance when compared to other related works. Versatility studies demonstrate the adaptability and applicability of PBML in addressing diverse and challenging few-shot tasks. Furthermore, ablation studies validate the performance gains attributed to the inference and model components.

---

PaperID: 232,   

Authors:  Arash Vahabpour, Tianyi Wang, Qiujing Lu, Omead Pooladzandi, Vwani Roychowdhury

Title: Diverse Imitation Learning via Self-Organizing Generative Models

Abstract:
Imitation learning (IL) is a well-known problem in the field of Markov decision process (MDP), where one is given multiple demonstration trajectories generated by expert(s), and the goal is to replicate the hidden expert-policies so that when the MDP is run independently, it generates trajectories close to the demonstrated ones. IL is one of the most useful tools used in building versatile robots that can learn from examples. This task becomes particularly challenging when the expert exhibits a mixture of behavior modes. Prior work has introduced latent variables to model variations of the expert policy. However, our experiments show that the existing works do not exhibit appropriate imitation of individual modes. To tackle this problem, we first draw inspiration from the well-known classical technique of self-organizing maps (SOMs) and introduce an encoder-free generative model—referred to as the self-organizing generative (SOG) model—for learning multimodal data distributions from samples. We then apply SOG for behavior cloning (BC)—a framework that learns deterministic policies without considering the environment—to accurately distinguish and imitate different modes. Then, we integrate it with generative adversarial IL (GAIL)—a framework that learns policies while considering the environment—to make the learning robust toward compounding errors at unseen states. We show that our method significantly outperforms the state of the art across multiple experiments within the MuJoCo simulator, including locomotion and robotic manipulation tasks.

---

PaperID: 233,   

Authors:  Changhao Chen, Stefano Rosa, Chris Xiaoxuan Lu, Bing Wang, Niki Trigoni, Andrew Markham

Title: Learning Selective Sensor Fusion for State Estimation

Abstract:
Autonomous vehicles and mobile robotic systems are typically equipped with multiple sensors to provide redundancy. By integrating the observations from different sensors, these mobile agents are able to perceive the environment and estimate system states, e.g., locations and orientations. Although deep learning (DL) approaches for multimodal odometry estimation and localization have gained traction, they rarely focus on the issue of robust sensor fusion—a necessary consideration to deal with noisy or incomplete sensor observations in the real world. Moreover, current deep odometry models suffer from a lack of interpretability. To this extent, we propose SelectFusion, an end-to-end selective sensor fusion module that can be applied to useful pairs of sensor modalities, such as monocular images and inertial measurements, depth images, and light detection and ranging (LIDAR) point clouds. Our model is a uniform framework that is not restricted to specific modality or task. During prediction, the network is able to assess the reliability of the latent features from different sensor modalities and to estimate trajectory at both scale and global pose. In particular, we propose two fusion modules—a deterministic soft fusion and a stochastic hard fusion—and offer a comprehensive study of the new strategies compared with trivial direct fusion. We extensively evaluate all fusion strategies both on public datasets and on progressively degraded datasets that present synthetic occlusions, noisy and missing data, and time misalignment between sensors, and we investigate the effectiveness of the different fusion strategies in attending the most reliable features, which in itself provides insights into the operation of the various models.

---

PaperID: 234,   

Authors:  Andong Lu, Cun Qian, Chenglong Li, Jin Tang, Liang Wang

Title: Duality-Gated Mutual Condition Network for RGBT Tracking

Abstract:
Low-quality modalities contain not only a lot of noisy information but also some discriminative features in RGB-Thermal (RGBT) tracking. However, the potentials of low-quality modalities are not well explored in existing RGBT tracking algorithms. In this work, we propose a novel duality-gated mutual condition network to fully exploit the discriminative information of all modalities while suppressing the effects of data noise. In specific, we design a mutual condition module, which takes the discriminative information of a modality as the condition to guide feature learning of target appearance in another modality. Such a module can effectively enhance target representations of all modalities even in the presence of low-quality modalities. To improve the quality of conditions and further reduce data noise, we propose a duality-gated mechanism and integrate it into the mutual condition module. To deal with the tracking failure caused by sudden camera motion, which often occurs in RGBT tracking, we design a resampling strategy based on optical flow. It does not increase much computational cost since we perform optical flow calculation only when the model prediction is unreliable and then execute resampling when the sudden camera motion is detected. Extensive experiments on four RGBT tracking benchmark datasets show that our method performs favorably against the state-of-the-art tracking algorithms.

---

PaperID: 235,   

Authors:  Ying Zhao, Shuang Li, Rui Zhang, Chi Harold Liu, Weipeng Cao, Xizhao Wang, Song Tian

Title: Semantic Correlation Transfer for Heterogeneous Domain Adaptation

Abstract:
Heterogeneous domain adaptation (HDA) is expected to achieve effective knowledge transfer from a label-rich source domain to a heterogeneous target domain with scarce labeled data. Most prior HDA methods strive to align the cross-domain feature distributions by learning domain invariant representations without considering the intrinsic semantic correlations among categories, which inevitably results in the suboptimal adaptation performance across domains. Therefore, to address this issue, we propose a novel semantic correlation transfer (SCT) method for HDA, which not only matches the marginal and conditional distributions between domains to mitigate the large domain discrepancy, but also transfers the category correlation knowledge underlying the source domain to target by maximizing the pairwise class similarity across source and target. Technically, the domainwise and classwise centroids (prototypes) are first computed and aligned according to the feature embeddings. Then, based on the derived classwise prototypes, we leverage the cosine similarity of each two classes in both domains to transfer the supervised source semantic correlation knowledge among different categories to target effectively. As a result, the feature transferability and category discriminability can be simultaneously improved during the adaptation process. Comprehensive experiments and ablation studies on standard HDA tasks, such as text-to-image, image-to-image, and text-to-text, have demonstrated the superiority of our proposed SCT against several state-of-the-art HDA methods.

---

PaperID: 236,   

Authors:  Hongxiang Jiang, Xiaoyan Luo, Jihao Yin, Huazhu Fu, Fuxiang Wang

Title: Orthogonal Subspace Representation for Generative Adversarial Networks

Abstract:
Disentanglement learning aims to separate explanatory factors of variation so that different attributes of the data can be well characterized and isolated, which promotes efficient inference for downstream tasks. Mainstream disentanglement approaches based on generative adversarial networks (GANs) learn interpretable data representation. However, most typical GAN-based works lack the discussion of the latent subspace, causing insufficient consideration of the variation of independent factors. Although some recent research analyzes the latent space on pretrained GANs for image editing, they do not emphasize learning representation directly from the subspace perspective. Appropriate subspace properties could facilitate corresponding feature representation learning to satisfy the independent variation requirements of the obtained explanatory factors, which is crucial for better disentanglement. In this work, we propose a unified framework for ensuring disentanglement, which fully investigates latent subspace learning (SL) in GAN. The novel GAN-based architecture explores orthogonal subspace representation (OSR) on vanilla GAN, named OSRGAN. To guide a subspace with strong correlation, less redundancy, and robust distinguishability, our OSR includes three stages, self-latent-aware, orthogonal subspace-aware, and structure representation-aware, respectively. First, the self-latent-aware stage promotes the latent subspace strongly correlated with the data space to discover interpretable factors, but with poor independence of variation. Second, the following orthogonal subspace-aware stage adaptively learns some 1-D linear subspace spanned by a set of orthogonal bases in the latent space. There is less redundancy between them, expressing the corresponding independence. Third, the structure representation-aware stage aligns the projection on the orthogonal subspace and the latent variables. Accordingly, feature representation in each linear subspace can be distinguishable, enhancing the independent expression of interpretable factors. In addition, we design an alternating optimization step, achieving a tradeoff training of OSRGAN on different properties. Despite it strictly constrains orthogonality, the loss weight coefficient of distinguishability induced by orthogonality could be adjusted and balanced with correlation constraint. To elucidate, this tradeoff training prevents our OSRGAN from overemphasizing any property and damaging the expressiveness of the feature representation. It takes into account both interpretable factors and their independent variation characteristics. Meanwhile, alternating optimization could keep the cost and efficiency of forward inference unchanged and will not burden the computational complexity. In theory, we clarify the significance of OSR, which brings better independence of factors, along with interpretability as correlation could converge to a high range faster. Moreover, through the convergence behavior analysis, including the objective functions under different constraints and the evaluation curve with iterations, our model demonstrates enhanced stability and definitely converges toward a higher peak for disentanglement. To depict the performance in downstream tasks, we compared the state-of-the-art GAN-based and even VAE-based approaches on different datasets. Our OSRGAN achieves higher disentanglement scores on FactorVAE, SAP, MIG, and VP metrics. All the experimental results illustrate that our novel GAN-based framework has considerable advantages on disentanglement.

---

PaperID: 237,   

Authors:  Ruoyu Zhao, Mingrui Zhu, Nannan Wang, Xinbo Gao

Title: Few-Shot Face Stylization via GAN Prior Distillation

Abstract:
Face stylization has made notable progress in recent years. However, when training on limited data, the performance of existing approaches significantly declines. Although some studies have attempted to tackle this problem, they either failed to achieve the few-shot setting (less than 10) or can only get suboptimal results. In this article, we propose GAN Prior Distillation (GPD) to enable effective few-shot face stylization. GPD contains two models: a teacher network with GAN Prior and a student network that fulfills end-to-end translation. Specifically, we adapt the teacher network trained on large-scale data in the source domain to the target domain using a handful of samples, where it can learn the target domain’s knowledge. Then, we can achieve few-shot augmentation by generating source domain and target domain images simultaneously with the same latent codes. We propose an anchor-based knowledge distillation module that can fully use the difference between the training and the augmented data to distill the knowledge of the teacher network into the student network. The trained student network achieves excellent generalization performance with the absorption of additional knowledge. Qualitative and quantitative experiments demonstrate that our method achieves superior results than state-of-the-art approaches in a few-shot setting.

---

PaperID: 238,   

Authors:  Jingcai Guo, Han Wang, Yuanyuan Xu, Wenchao Xu, Yufeng Zhan, Yuxia Sun, Song Guo

Title: Multimodal Dual-Embedding Networks for Malware Open-Set Recognition

Abstract:
Malware open-set recognition (MOSR) is an emerging research domain that aims at jointly classifying malware samples from known families and detecting the ones from novel unknown families, respectively. Existing works mostly rely on a well-trained classifier considering the predicted probabilities of each known family with a threshold-based detection to achieve the MOSR. However, our observation reveals that the feature distributions of malware samples are extremely similar to each other even between known and unknown families. Thus, the obtained classifier may produce overly high probabilities of testing unknown samples toward known families and degrade the model performance. In this article, we propose the multi\modal dual-embedding networks, dubbed MDENet, to take advantage of comprehensive malware features from different modalities to enhance the diversity of malware feature space, which is more representative and discriminative for down-stream recognition. Concretely, we first generate a malware image for each observed sample based on their numeric features using our proposed numeric encoder with a re- designed multiscale CNN structure, which can better explore their statistical and spatial correlations. Besides, we propose to organize tokenized malware features into a sentence for each sample considering its behaviors and dynamics, and utilize language models as the textual encoder to transform it into a representable and computable textual vector. Such parallel multimodal encoders can fuse the above two components to enhance the feature diversity. Last, to further guarantee the open-set recognition (OSR), we dually embed the fused multimodal representation into one primary space and an associated sub-space, i.e., discriminative and exclusive spaces, with contrastive sampling and \rho -bounded enclosing sphere regularizations, which resort to classification and detection, respectively. Moreover, we also enrich our previously proposed large-scaled malware dataset MAL-100 with multimodal characteristics and contribute an improved version dubbed MAL-100+. Experimental results on the widely used malware dataset Mailing and the proposed MAL-100+ demonstrate the effectiveness of our method.

---

PaperID: 239,   

Authors:  Shuo Yu, Huafei Huang, Yanming Shen, Pengfei Wang, Qiang Zhang, Ke Sun, Honglong Chen

Title: Formulating and Representing Multiagent Systems With Hypergraphs

Abstract:
Graph-learning methods, especially graph neural networks (GNNs), have shown remarkable effectiveness in handling non-Euclidean data and have achieved great success in various scenarios. Existing GNNs are primarily based on message-passing schemes, that is, aggregating information from neighboring nodes. However, the diversity and complexity of complex systems from real-world circumstances are not sufficiently taken into account. In these cases, the individual should be treated as an agent, with the ability to perceive their surroundings and interact with other individuals, rather than just be viewed as nodes in existing graph approaches. Additionally, the pairwise interactions used in existing methods also lack the expressiveness for the higher-order complex relations among multiple agents, thus limiting the performance in various tasks. In this work, we propose a Multiagent Hypergraph Force-learning method dubbed MHGForce. First, we formalize the multiagent system (MAS) and illustrate its connection to graph learning. Then, we propose a generalized multiagent hypergraph-learning framework. In this framework, we integrate message-passing and force-based interactions to devise a pluggable method. The method empowers graph approaches to excel in downstream tasks while effectively maintaining structural information in the representations. Experimental results on the Cora, Citeseer, Cora-CA, Zoo, and NTU2012 datasets in node classification demonstrate the effectiveness and generality of our proposed method. We also discuss the characteristics of the MHGForce and explore its role through parametric analysis and visualization. Finally, we give a discussion, conclude our work, and propose future directions.

---

PaperID: 240,   

Authors:  Riccardo Rosati, Luca Romeo, Víctor Manuel Vargas, Pedro Antonio Gutiérrez, Emanuele Frontoni, César Hervás-Martínez

Title: Learning Ordinal-Hierarchical Constraints for Deep Learning Classifiers

Abstract:
Real-world classification problems may disclose different hierarchical levels where the categories are displayed in an ordinal structure. However, no specific deep learning (DL) models simultaneously learn hierarchical and ordinal constraints while improving generalization performance. To fill this gap, we propose the introduction of two novel ordinal–hierarchical DL methodologies, namely, the hierarchical cumulative link model (HCLM) and hierarchical–ordinal binary decomposition (HOBD), which are able to model the ordinal structure within different hierarchical levels of the labels. In particular, we decompose the hierarchical–ordinal problem into local and global graph paths that may encode an ordinal constraint for each hierarchical level. Thus, we frame this problem as simultaneously minimizing global and local losses. Furthermore, the ordinal constraints are set by two approaches [ordinal binary decomposition (OBD) and cumulative link model (CLM)] within each global and local function. The effectiveness of the proposed approach is measured on four real-use case datasets concerning industrial, biomedical, computer vision, and financial domains. The extracted results demonstrate a statistically significant improvement to state-of-the-art nominal, ordinal, and hierarchical approaches.

---

PaperID: 241,   

Authors:  Pin Zhang, Wenhan Dong, Ming Cai, Shengde Jia, Zipeng Wang

Title: MEOL: A Maximum-Entropy Framework for Options Learning

Abstract:
Options, the temporally extended courses of actions that can be taken at varying time scale, have provided a concrete, key framework for learning levels of temporal abstraction in hierarchical tasks. While methods of learning options end-to-end is well researched, how to explore good options and actions simultaneously is still challenging. We address this issue by maximizing reward augmented with entropies of both option and action selection policy in options learning. To this end, we reveal our novel optimization objective by reformulating options learning from perspective of probabilistic inference and propose a soft options iteration method to guarantee convergence to the optimum. In implementation, we propose an off-policy algorithm called the maximum-entropy options critic (MEOC) and evaluate it on series of continuous control benchmarks. Comparative results demonstrate that our method outperforms baselines in efficiency and final result on most benchmarks, and the performance exhibits superiority and robustness especially on complex tasks. Ablated studies further explain that entropy maximization on hierarchical exploration promotes learning performance through efficient options specialization and multimodality in action level.

---

PaperID: 242,   

Authors:  Xiaoyan Zhao, Min Yang, Qiang Qu, Ruifeng Xu

Title: Few-Shot Relation Extraction With Automatically Generated Prompts

Abstract:
Relation extraction (RE) tends to struggle when the supervised training data is few and difficult to be collected. In this article, we elicit relational and factual knowledge from large pretrained language models (PLMs) for few-shot RE (FSRE) with prompting techniques. Concretely, we automatically generate a diverse set of natural language templates and modulate PLM’s behavior through these prompts for FSRE. To mitigate the template bias which leads to unstableness of few-shot learning, we propose a simple yet effective template regularization network (TRN) to prevent deep networks from over-fitting uncertain templates and thus stabilize the FSRE models. TRN alleviates the template bias with three mechanisms: 1) an attention mechanism over mini-batch to weight each template; 2) a ranking regularization mechanism to regularize the attention weights and constrain the importance of uncertain templates; and 3) a template calibration module with two calibrating techniques to modify the uncertain templates in the lowest-ranked group. Experimental results on two benchmark datasets (i.e., FewRel and NYT) show that our model has robust superiority over strong competitors. For reproducibility, we will release our code and data upon the publication of this article.

---

PaperID: 243,   

Authors:  Mengzhu Wang, Junze Liu, Ge Luo, Shanshan Wang, Wei Wang, Long Lan, Ye Wang, Feiping Nie

Title: Smooth-Guided Implicit Data Augmentation for Domain Generalization

Abstract:
The training process of a domain generalization (DG) model involves utilizing one or more interrelated source domains to attain optimal performance on an unseen target domain. Existing DG methods often use auxiliary networks or require high computational costs to improve the model’s generalization ability by incorporating a diverse set of source domains. In contrast, this work proposes a method called Smooth-Guided Implicit Data Augmentation (SGIDA) that operates in the feature space to capture the diversity of source domains. To amplify the model’s generalization capacity, a distance metric learning (DML) loss function is incorporated. Additionally, rather than depending on deep features, the suggested approach employs logits produced from cross entropy (CE) losses with infinite augmentations. A theoretical analysis shows that logits are effective in estimating distances defined on original features, and the proposed approach is thoroughly analyzed to provide a better understanding of why logits are beneficial for DG. Moreover, to increase the diversity of the source domain, a sampling-based method called smooth is introduced to obtain semantic directions from interclass relations. The effectiveness of the proposed approach is demonstrated through extensive experiments on widely used DG, object detection, and remote sensing datasets, where it achieves significant improvements over existing state-of-the-art methods across various backbone networks.

---

PaperID: 244,   

Authors:  Fuli Wang, Karelia Pena-Pena, Wei Qian, Gonzalo R. Arce

Title: T-HyperGNNs: Hypergraph Neural Networks via Tensor Representations

Abstract:
Hypergraph neural networks (HyperGNNs) are a family of deep neural networks designed to perform inference on hypergraphs. HyperGNNs follow either a spectral or a spatial approach, in which a convolution or message-passing operation is conducted based on a hypergraph algebraic descriptor. While many HyperGNNs have been proposed and achieved state-of-the-art performance on broad applications, there have been limited attempts at exploring high-dimensional hypergraph descriptors (tensors) and joint node interactions carried by hyperedges. In this article, we depart from hypergraph matrix representations and present a new tensor-HyperGNN (T-HyperGNN) framework with cross-node interactions (CNIs). The T-HyperGNN framework consists of T-spectral convolution, T-spatial convolution, and T-message-passing HyperGNNs (T-MPHN). The T-spectral convolution HyperGNN is defined under the t-product algebra that closely connects to the spectral space. To improve computational efficiency for large hypergraphs, we localize the T-spectral convolution approach to formulate the T-spatial convolution and further devise a novel tensor-message-passing algorithm for practical implementation by studying a compressed adjacency tensor representation. Compared to the state-of-the-art approaches, our T-HyperGNNs preserve intrinsic high-order network structures without any hypergraph reduction and model the joint effects of nodes through a CNI layer. These advantages of our T-HyperGNNs are demonstrated in a wide range of real-world hypergraph datasets. The implementation code is available at https://github.com/wangfuli/T-HyperGNNs.git.

---

PaperID: 245,   

Authors:  Renwei Dian, Yuanye Liu, Shutao Li

Title: Spectral Super-Resolution via Deep Low-Rank Tensor Representation

Abstract:
Spectral super-resolution has attracted the attention of more researchers for obtaining hyperspectral images (HSIs) in a simpler and cheaper way. Although many convolutional neural network (CNN)-based approaches have yielded impressive results, most of them ignore the low-rank prior of HSIs resulting in huge computational and storage costs. In addition, the ability of CNN-based methods to capture the correlation of global information is limited by the receptive field. To surmount the problem, we design a novel low-rank tensor reconstruction network (LTRN) for spectral super-resolution. Specifically, we treat the features of HSIs as 3-D tensors with low-rank properties due to their spectral similarity and spatial sparsity. Then, we combine canonical-polyadic (CP) decomposition with neural networks to design an adaptive low-rank prior learning (ALPL) module that enables feature learning in a 1-D space. In this module, there are two core modules: the adaptive vector learning (AVL) module and the multidimensionwise multihead self-attention (MMSA) module. The AVL module is designed to compress an HSI into a 1-D space by using a vector to represent its information. The MMSA module is introduced to improve the ability to capture the long-range dependencies in the row, column, and spectral dimensions, respectively. Finally, our LTRN, mainly cascaded by several ALPL modules and feedforward networks (FFNs), achieves high-quality spectral super-resolution with fewer parameters. To test the effect of our method, we conduct experiments on two datasets: the CAVE dataset and the Harvard dataset. Experimental results show that our LTRN not only is as effective as state-of-the-art methods but also has fewer parameters. The code is available at https://github.com/renweidian/LTRN.

---

PaperID: 246,   

Authors:  Xiaowei Zhao, Dongming Wu, Feiping Nie, Weizhong Yu, Chen Zhao, Xuelong Li

Title: Nonlinear Locality-Preserving Projections With Dynamic Graph Learning

Abstract:
The affinity graph is regarded as a mathematical representation of the local manifold structure. The performance of locality-preserving projections (LPPs) and its variants is tied to the quality of the affinity graph. However, there are two drawbacks in current approaches. First, the pre-designed graph is inconsistent with the actual distribution of data. Second, the linear projection way would cause damage to the nonlinear manifold structure. In this article, we propose a nonlinear dimensionality reduction model, named deep locality-preserving projections (DLPPs), to solve these problems simultaneously. The model consists of two loss functions, each employing deep autoencoders (AEs) to extract discriminative features. In the first loss function, the affinity relationships among samples in the intermediate layer are determined adaptively according to the distances between samples. Since the features of samples are obtained by nonlinear mapping, the manifold structure can be kept in the low-dimensional space. Additionally, the learned affinity graph is able to avoid the influence of noisy and redundant features. In the second loss function, the affinity relationships among samples in the last layer (also called the reconstruction layer) are learned. This strategy enables denoised samples to have a good manifold structure. By integrating these two functions, our proposed model minimizes the mismatch of the manifold structure between samples in the denoising space and the low-dimensional space, while reducing sensitivity to the initial weights of the graph. Extensive experiments on toy and benchmark datasets have been conducted to verify the effectiveness of our proposed model.

---

PaperID: 247,   

Authors:  Enrico Picco, Alessandro Lupo, Serge Massar

Title: Deep Photonic Reservoir Computer for Speech Recognition

Abstract:
Speech recognition is a critical task in the field of artificial intelligence (AI) and has witnessed remarkable advancements thanks to large and complex neural networks, whose training process typically requires massive amounts of labeled data and computationally intensive operations. An alternative paradigm, reservoir computing (RC), is energy efficient and is well adapted to implementation in physical substrates, but exhibits limitations in performance when compared with more resource-intensive machine learning algorithms. In this work, we address this challenge by investigating different architectures of interconnected reservoirs, all falling under the umbrella of deep RC (DRC). We propose a photonic-based deep reservoir computer and evaluate its effectiveness on different speech recognition tasks. We show specific design choices that aim to simplify the practical implementation of a reservoir computer while simultaneously achieving high-speed processing of high-dimensional audio signals. Overall, with the present work, we hope to help the advancement of low-power and high-performance neuromorphic hardware.

---

PaperID: 248,   

Authors:  Zhenyu Zhou, Lei Luo, Tianrui Liu, Qing Liao, Xinwang Liu, En Zhu

Title: Category Alignment Mechanism for Few-Shot Image Classification

Abstract:
While humans can excel at image classification tasks by comparing a few images, existing metric-based few-shot classification methods are still not well adapted to novel tasks. Performance declines rapidly when encountering new patterns, as feature embeddings cannot effectively encode discriminative properties. Moreover, existing matching methods inadequately utilize support set samples, focusing only on comparing query samples to category prototypes without exploiting contrastive relationships across categories for discriminative features. In this work, we propose a method where query samples select their most category-representative features for matching, making feature embeddings adaptable and category-related. We introduce a category alignment mechanism (CAM) to align query image features with different categories. CAM ensures features chosen for matching are distinct and strongly correlated to intra- and inter-contrastive relationships within categories, making extracted features highly related to their respective categories. CAM is parameter-free, requires no extra training to adapt to new tasks, and adjusts features for matching when task categories change. We also implement a cross-validation-based feature selection technique for support samples, generating more discriminative category prototypes. We implement two versions of inductive and transductive inference and conduct extensive experiments on six datasets to demonstrate the effectiveness of our algorithm. The results indicate that our method consistently yields performance improvements on benchmark tasks and surpasses the current state-of-the-art methods.

---

PaperID: 249,   

Authors:  Yunjie Tian, Lingxi Xie, Jiemin Fang, Jianbin Jiao, Qixiang Ye, Qi Tian

Title: Exploring Complicated Search Spaces With Interleaving-Free Sampling

Abstract:
Conventional neural architecture search (NAS) algorithms typically work on search spaces with short-distance node connections. We argue that such designs, though safe and stable, are obstacles to exploring more effective network architectures. In this brief, we explore the search algorithm upon a complicated search space with long-distance connections and show that existing weight-sharing search algorithms fail due to the existence of interleaved connections (ICs). Based on the observation, we present a simple-yet-effective algorithm, termed interleaving-free neural architecture search (IF-NAS). We further design a periodic sampling strategy to construct subnetworks during the search procedure, avoiding the ICs to emerge in any of them. In the proposed search space, IF-NAS outperforms both random sampling and previous weight-sharing search algorithms by significant margins. It can also be well-generalized to the microcell-based spaces. This study emphasizes the importance of macrostructure and we look forward to further efforts in this direction. The code is available at github.com/sunsmarterjie/IFNAS.

---

PaperID: 250,   

Authors:  Huan Tian, Bo Liu, Tianqing Zhu, Wanlei Zhou, Philip S. Yu

Title: MultiFair: Model Fairness With Multiple Sensitive Attributes

Abstract:
While existing fairness interventions show promise in mitigating biased predictions, most studies concentrate on single-attribute protections. Although a few methods consider multiple attributes, they either require additional constraints or prediction heads, incurring high computational overhead or jeopardizing the stability of the training process. More critically, they consider per-attribute protection approaches, raising concerns about fairness gerrymandering where certain attribute combinations remain unfair. This work aims to construct a neutral domain containing fused information across all subgroups and attributes. It delivers fair predictions as the fused input contains neutralized information for all considered attributes. Specifically, we adopt mixup operations to generate samples with fused information. However, our experiments reveal that directly adopting the operations leads to degraded prediction results. The excessive mixup operations result in unrecognizable training data. To this end, we design three distinct mixup schemes that balance information fusion across attributes while retaining distinct visual features critical for training valid models. Extensive experiments with multiple datasets and up to eight sensitive attributes demonstrate that the proposed MultiFair method can deliver fairness protections for multiple attributes while maintaining valid prediction results.

---

PaperID: 251,   

Authors:  Peipei Yuan, Xinge You, Hong Chen, Yingjie Wang, Qinmu Peng, Bin Zou

Title: Sparse Additive Machine With the Correntropy-Induced Loss

Abstract:
Sparse additive machines (SAMs) have shown competitive performance on variable selection and classification in high-dimensional data due to their representation flexibility and interpretability. However, the existing methods often employ the unbounded or nonsmooth functions as the surrogates of 0–1 classification loss, which may encounter the degraded performance for data with outliers. To alleviate this problem, we propose a robust classification method, named SAM with the correntropy-induced loss (CSAM), by integrating the correntropy-induced loss (C-loss), the data-dependent hypothesis space, and the weighted \ell _q,1 -norm regularizer ( q\geq 1 ) into additive machines. In theory, the generalization error bound is estimated via a novel error decomposition and the concentration estimation techniques, which shows that the convergence rate \mathcal O(n^-1/4) can be achieved under proper parameter conditions. In addition, the theoretical guarantee on variable selection consistency is analyzed. Experimental evaluations on both synthetic and real-world datasets consistently validate the effectiveness and robustness of the proposed approach.

---

PaperID: 252,   

Authors:  Jianxin Lin, Yongqiang Tang, Junping Wang, Wensheng Zhang

Title: Constrained Maximum Cross-Domain Likelihood for Domain Generalization

Abstract:
As a recent noticeable topic, domain generalization aims to learn a generalizable model on multiple source domains, which is expected to perform well on unseen test domains. Great efforts have been made to learn domain-invariant features by aligning distributions across domains. However, existing works are often designed based on some relaxed conditions which are generally hard to satisfy and fail to realize the desired joint distribution alignment. In this article, we propose a novel domain generalization method, which originates from an intuitive idea that a domain-invariant classifier can be learned by minimizing the Kullback–Leibler (KL)-divergence between posterior distributions from different domains. To enhance the generalizability of the learned classifier, we formalize the optimization objective as an expectation computed on the ground-truth marginal distribution. Nevertheless, it also presents two obvious deficiencies, one of which is the side-effect of entropy increase in KL-divergence and the other is the unavailability of ground-truth marginal distributions. For the former, we introduce a term named maximum in-domain likelihood to maintain the discrimination of the learned domain-invariant representation space. For the latter, we approximate the ground-truth marginal distribution with source domains under a reasonable convex hull assumption. Finally, a constrained maximum cross-domain likelihood (CMCL) optimization problem is deduced, by solving which the joint distributions are naturally aligned. An alternating optimization strategy is carefully designed to approximately solve this optimization problem. Extensive experiments on four standard benchmark datasets, i.e., Digits-DG, PACS, Office-Home, and miniDomainNet, highlight the superior performance of our method.

---

PaperID: 253,   

Authors:  Erdal Akin

Title: Deep Reinforcement Learning-Based Multirestricted Dynamic-Request Transportation Framework

Abstract:
Unmanned aerial vehicles (UAVs) are used in many areas where their usage is increasing constantly. Their popularity, therefore, maintains its importance in the technology world. Parallel to the development of technology, human standards, and surroundings should also improve equally. This study is developed based on the possibility of timely delivery of urgent medical requests in emergency situations. Using UAVs for delivering urgent medical requests will be very effective due to their flexible maneuverability and low costs. However, off-the-shelf UAVs suffer from limited payload capacity and battery constraints. In addition, urgent requests may be requested at an uncertain time, and delivering in a short time may be crucial. To address this issue, we proposed a novel framework that considers the limitations of the UAVs and dynamically requested packages. These previously unknown packages have source–destination pairs and delivery time intervals. Furthermore, we utilize deep reinforcement learning (DRL) algorithms, deep Q-network (DQN), proximal policy optimization (PPO), and advantage actor–critic (A2C) to overcome this unknown environment and requests. The comprehensive experimental results demonstrate that the PPO algorithm has a faster and more stable training performance than the other DRL algorithms in two different environmental setups. Also, we implemented an extension version of a Brute-force (BF) algorithm, assuming that all requests and environments are known in advance. The PPO algorithm performs very close to the success rate of the BF algorithm.

---

PaperID: 254,   

Authors:  Jie Lin, Ting-Zhu Huang, Xi-Le Zhao, Teng-Yu Ji, Qibin Zhao

Title: Tensor Robust Kernel PCA for Multidimensional Data

Abstract:
Recently, the tensor nuclear norm (TNN)-based tensor robust principle component analysis (TRPCA) has achieved impressive performance in multidimensional data processing. The underlying assumption in TNN is the low-rankness of frontal slices of the tensor in the transformed domain (e.g., Fourier domain). However, the low-rankness assumption is usually violative for real-world multidimensional data (e.g., video and image) due to their intrinsically nonlinear structure. How to effectively and efficiently exploit the intrinsic structure of multidimensional data remains a challenge. In this article, we first suggest a kernelized TNN (KTNN) by leveraging the nonlinear kernel mapping in the transform domain, which faithfully captures the intrinsic structure (i.e., implicit low-rankness) of multidimensional data and is computed at a lower cost by introducing kernel trick. Armed with KTNN, we propose a tensor robust kernel PCA (TRKPCA) model for handling multidimensional data, which decomposes the observed tensor into an implicit low-rank component and a sparse component. To tackle the nonlinear and nonconvex model, we develop an efficient alternating direction method of multipliers (ADMM)-based algorithm. Extensive experiments on real-world applications collectively verify that TRKPCA achieves superiority over the state-of-the-art RPCA methods.

---

PaperID: 255,   

Authors:  Jiahao Qi, Zhiqiang Gong, Xingyue Liu, Chen Chen, Ping Zhong

Title: Masked Spatial-Spectral Autoencoders Are Excellent Hyperspectral Defenders

Abstract:
Deep learning (DL) methodology contributes a lot to the development of hyperspectral image (HSI) analysis community. However, it also makes HSI analysis systems vulnerable to adversarial attacks. To this end, we propose a masked spatial–spectral autoencoder (MSSA) in this article under self-supervised learning theory, for enhancing the robustness of HSI analysis systems. First, a masked sequence attention learning (MSAL) module is conducted to promote the inherent robustness of HSI analysis systems along spectral channel. Then, we develop a graph convolutional network (GCN) with learnable graph structure to establish global pixel-wise combinations. In this way, the attack effect would be dispersed by all the related pixels among each combination, and a better defense performance is achievable in spatial aspect. Finally, to improve the defense transferability and address the problem of limited labeled samples, MSSA employs spectra reconstruction as a pretext task and fits the datasets in a self-supervised manner. Comprehensive experiments over three benchmarks verify the effectiveness of MSSA in comparison with the state-of-the-art hyperspectral classification methods and representative adversarial defense strategies.

---

PaperID: 256,   

Authors:  Baoyao Yang, Pong C. Yuen, Yiqun Zhang, An Zeng

Title: Allosteric Feature Collaboration for Model-Heterogeneous Federated Learning

Abstract:
Although federated learning (FL) has achieved outstanding results in privacy-preserved distributed learning, the setting of model homogeneity among clients restricts its wide application in practice. This article investigates a more general case, namely, model-heterogeneous FL (M-hete FL), where client models are independently designed and can be structurally heterogeneous. M-hete FL faces new challenges in collaborative learning because the parameters of heterogeneous models could not be directly aggregated. In this article, we propose a novel allosteric feature collaboration (AlFeCo) method, which interchanges knowledge across clients and collaboratively updates heterogeneous models on the server. Specifically, an allosteric feature generator is developed to reveal task-relevant information from multiple client models. The revealed information is stored in the client-shared and client-specific codes. We exchange client-specific codes across clients to facilitate knowledge interchange and generate allosteric features that are dimensionally variable for model updates. To promote information communication between different clients, a dual-path (model–model and model–prediction) communication mechanism is designed to supervise the collaborative model updates using the allosteric features. Client models are fully communicated through the knowledge interchange between models and between models and predictions. We further provide theoretical evidence and convergence analysis to support the effectiveness of AlFeCo in M-hete FL. The experimental results show that the proposed AlFeCo method not only performs well on classical FL benchmarks but also is effective in model-heterogeneous federated antispoofing. Our codes are publicly available at https://github.com/ybaoyao/AlFeCo.

---

PaperID: 257,   

Authors:  Yupei Zhang, Yifei Wang, Yuxin Li, Yunan Xu, Shuangshuang Wei, Shuhui Liu, Xuequn Shang

Title: Federated Discriminative Representation Learning for Image Classification

Abstract:
Acquiring big-size datasets to raise the performance of deep models has become one of the most critical problems in representation learning (RL) techniques, which is the core potential of the emerging paradigm of federated learning (FL). However, most current FL models concentrate on seeking an identical model for isolated clients and thus fail to make full use of the data specificity between clients. To enhance the classification performance of each client, this study introduces the FDRL, a federated discriminative RL model, by partitioning the data features of each client into a global subspace and a local subspace. More specifically, FDRL learns the global representation for federated communication between those isolated clients, which is to capture common features from all protected datasets via model sharing, and local representations for personalization in each client, which is to preserve specific features of clients via model differentiating. Toward this goal, FDRL in each client trains a shared submodel for federated communication and, meanwhile, a not-shared submodel for locality preservation, in which the two models partition client-feature space by maximizing their differences, followed by a linear model fed with combined features for image classification. The proposed model is implemented with neural networks and optimized in an iterative manner between the server of computing the global model and the clients of learning the local classifiers. Thanks to the powerful capability of local feature preservation, FDRL leads to more discriminative data representations than the compared FL models. Experimental results on public datasets demonstrate that our FDRL benefits from the subspace partition and achieves better performance on federated image classification than the state-of-the-art FL models.

---

PaperID: 258,   

Authors:  Yilin Shao, Long Sun, Licheng Jiao, Xu Liu, Fang Liu, Lingling Li, Shuyuan Yang

Title: CoT: Contourlet Transformer for Hierarchical Semantic Segmentation

Abstract:
The Transformer–convolutional neural network (CNN) hybrid learning approach is gaining traction for balancing deep and shallow image features for hierarchical semantic segmentation. However, they are still confronted with a contradiction between comprehensive semantic understanding and meticulous detail extraction. To solve this problem, this article proposes a novel Transformer–CNN hybrid hierarchical network, dubbed contourlet transformer (CoT). In the CoT framework, the semantic representation process of the Transformer is unavoidably peppered with sparsely distributed points that, while not desired, demand finer detail. Therefore, we design a deep detail representation (DDR) structure to investigate their fine-grained features. First, through contourlet transform (CT), we distill the high-frequency directional components from the raw image, yielding localized features that accommodate the inductive bias of CNN. Second, a CNN deep sparse learning (DSL) module takes them as input to represent the underlying detailed features. This memory- and energy-efficient learning method can keep the same sparse pattern between input and output. Finally, the decoder hierarchically fuses the detailed features with the semantic features via an image reconstruction-like fashion. Experiments demonstrate that CoT achieves competitive performance on three benchmark datasets: PASCAL Context [57.21% mean intersection over union (mIoU)], ADE20K (54.16% mIoU), and Cityscapes (84.23% mIoU). Furthermore, we conducted robustness studies to validate its resistance against various sorts of corruption. Our code is available at: https://github.com/yilinshao/CoT-Contourlet-Transformer.

---

PaperID: 259,   

Authors:  Lihao Liu, Angelica I. Avilés-Rivero, Carola-Bibiane Schönlieb

Title: Contrastive Registration for Unsupervised Medical Image Segmentation

Abstract:
Medical image segmentation is an important task in medical imaging, as it serves as the first step for clinical diagnosis and treatment planning. While major success has been reported using deep learning supervised techniques, they assume a large and well-representative labeled set. This is a strong assumption in the medical domain where annotations are expensive, time-consuming, and inherent to human bias. To address this problem, unsupervised segmentation techniques have been proposed in the literature. Yet, none of the existing unsupervised segmentation techniques reach accuracies that come even near to the state-of-the-art of supervised segmentation methods. In this work, we present a novel optimization model framed in a new convolutional neural network (CNN)-based contrastive registration architecture for unsupervised medical image segmentation called CLMorph. The core idea of our approach is to exploit image-level registration and feature-level contrastive learning, to perform registration-based segmentation. First, we propose an architecture to capture the image-to-image transformation mapping via registration for unsupervised medical image segmentation. Second, we embed a contrastive learning mechanism in the registration architecture to enhance the discriminative capacity of the network at the feature level. We show that our proposed CLMorph technique mitigates the major drawbacks of existing unsupervised techniques. We demonstrate, through numerical and visual experiments, that our technique substantially outperforms the current state-of-the-art unsupervised segmentation methods on two major medical image datasets.

---

PaperID: 260,   

Authors:  Francisco Munguia-Galeano, Ah-Hwee Tan, Ze Ji

Title: Deep Reinforcement Learning With Explicit Context Representation

Abstract:
Though reinforcement learning (RL) has shown an outstanding capability for solving complex computational problems, most RL algorithms lack an explicit method that would allow learning from contextual information. On the other hand, humans often use context to identify patterns and relations among elements in the environment, along with how to avoid making wrong actions. However, what may seem like an obviously wrong decision from a human perspective could take hundreds of steps for an RL agent to learn to avoid. This article proposes a framework for discrete environments called Iota explicit context representation (IECR). The framework involves representing each state using contextual key frames (CKFs), which can then be used to extract a function that represents the affordances of the state; in addition, two loss functions are introduced with respect to the affordances of the state. The novelty of the IECR framework lies in its capacity to extract contextual information from the environment and learn from the CKFs’ representation. We validate the framework by developing four new algorithms that learn using context: Iota deep Q-network (IDQN), Iota double deep Q-network (IDDQN), Iota dueling deep Q-network (IDuDQN), and Iota dueling double deep Q-network (IDDDQN). Furthermore, we evaluate the framework and the new algorithms in five discrete environments. We show that all the algorithms, which use contextual information, converge in around 40000 training steps of the neural networks, significantly outperforming their state-of-the-art equivalents.

---

PaperID: 261,   

Authors:  Qiyuan Zhang, Shu Leng, Xiaoteng Ma, Qihan Liu, Xueqian Wang, Bin Liang, Yu Liu, Jun Yang

Title: CVaR-Constrained Policy Optimization for Safe Reinforcement Learning

Abstract:
Current constrained reinforcement learning (RL) methods guarantee constraint satisfaction only in expectation, which is inadequate for safety-critical decision problems. Since a constraint satisfied in expectation remains a high probability of exceeding the cost threshold, solving constrained RL problems with high probabilities of satisfaction is critical for RL safety. In this work, we consider the safety criterion as a constraint on the conditional value-at-risk (CVaR) of cumulative costs, and propose the CVaR-constrained policy optimization algorithm (CVaR-CPO) to maximize the expected return while ensuring agents pay attention to the upper tail of constraint costs. According to the bound on the CVaR-related performance between two policies, we first reformulate the CVaR-constrained problem in augmented state space using the state extension procedure and the trust-region method. CVaR-CPO then derives the optimal update policy by applying the Lagrangian method to the constrained optimization problem. In addition, CVaR-CPO utilizes the distribution of constraint costs to provide an efficient quantile-based estimation of the CVaR-related value function. We conduct experiments on constrained control tasks to show that the proposed method can produce behaviors that satisfy safety constraints, and achieve comparable performance to most safe RL (SRL) methods.

---

PaperID: 262,   

Authors:  Susanna Yu Gordleeva, Yuliya Tsybina, Mikhail Krivonosov, Ivan Yu. Tyukin, Victor B. Kazantsev, Alexey Zaikin, Alexander N. Gorban

Title: Situation-Based Neuromorphic Memory in Spiking Neuron-Astrocyte Network

Abstract:
Mammalian brains operate in very special surroundings: to survive they have to react quickly and effectively to the pool of stimuli patterns previously recognized as danger. Many learning tasks often encountered by living organisms involve a specific set-up centered around a relatively small set of patterns presented in a particular environment. For example, at a party, people recognize friends immediately, without deep analysis, just by seeing a fragment of their clothes. This set-up with reduced “ontology” is referred to as a “situation.” Situations are usually local in space and time. In this work, we propose that neuron-astrocyte networks provide a network topology that is effectively adapted to accommodate situation-based memory. In order to illustrate this, we numerically simulate and analyze a well-established model of a neuron-astrocyte network, which is subjected to stimuli conforming to the situation-driven environment. Three pools of stimuli patterns are considered: external patterns, patterns from the situation associative pool regularly presented to the network and learned by the network, and patterns already learned and remembered by astrocytes. Patterns from the external world are added to and removed from the associative pool. Then, we show that astrocytes are structurally necessary for an effective function in such a learning and testing set-up. To demonstrate this we present a novel neuromorphic computational model for short-term memory implemented by a two-net spiking neural-astrocytic network. Our results show that such a system tested on synthesized data with selective astrocyte-induced modulation of neuronal activity provides an enhancement of retrieval quality in comparison to standard spiking neural networks trained via Hebbian plasticity only. We argue that the proposed set-up may offer a new way to analyze, model, and understand neuromorphic artificial intelligence systems.

---

PaperID: 263,   

Authors:  Pengpeng Zeng, Haonan Zhang, Lianli Gao, Xiangpeng Li, Jin Qian, Heng Tao Shen

Title: Visual Commonsense-Aware Representation Network for Video Captioning

Abstract:
Generating consecutive descriptions for videos, that is, video captioning, requires taking full advantage of visual representation along with the generation process. Existing video captioning methods focus on an exploration of spatial–temporal representations and their relationships to produce inferences. However, such methods only exploit the superficial association contained in a video itself without considering the intrinsic visual commonsense knowledge that exists in a video dataset, which may hinder their capabilities of knowledge cognitive to reason accurate descriptions. To address this problem, we propose a simple, yet effective method, called visual commonsense-aware representation network (VCRN), for video captioning. Specifically, we construct a Video Dictionary, a plug-and-play component, obtained by clustering all video features from the total dataset into multiple clustered centers without additional annotation. Each center implicitly represents a visual commonsense concept in a video domain, which is utilized in our proposed visual concept selection (VCS) component to obtain a video-related concept feature. Next, a concept-integrated generation (CIG) component is proposed to enhance caption generation. Extensive experiments on three public video captioning benchmarks: MSVD, MSR-VTT, and VATEX, demonstrate that our method achieves state-of-the-art performance, indicating the effectiveness of our method. In addition, our method is integrated into the existing method of video question answering (VideoQA) and improves this performance, which further demonstrates the generalization capability of our method. The source code has been released at https://github.com/zchoi/VCRN.

---

PaperID: 264,   

Authors:  Lingbo Liu, Mengmeng Liu, Guanbin Li, Ziyi Wu, Junfan Lin, Liang Lin

Title: Road Network-Guided Fine-Grained Urban Traffic Flow Inference

Abstract:
Accurate inference of fine-grained traffic flow from coarse-grained one is an emerging yet crucial problem, which can help greatly reduce the number of the required traffic monitoring sensors for cost savings. In this work, we note that traffic flow has a high correlation with road network, which was either completely ignored or simply treated as an external factor in previous works. To facilitate this problem, we propose a novel road-aware traffic flow magnifier (RATFM) that explicitly exploits the prior knowledge of road networks to fully learn the road-aware spatial distribution of fine-grained traffic flow. Specifically, a multidirectional 1-D convolutional layer is first introduced to extract the semantic feature of the road network. Subsequently, we incorporate the road network feature and coarse-grained flow feature to regularize the short-range spatial distribution modeling of road-relative traffic flow. Furthermore, we take the road network feature as a query to capture the long-range spatial distribution of traffic flow with a transformer architecture. Benefiting from the road-aware inference mechanism, our method can generate high-quality fine-grained traffic flow maps. Extensive experiments on three real-world datasets show that the proposed RATFM outperforms state-of-the-art models under various scenarios. Our code and datasets are released at https://github.com/luimoli/RATFM.

---

PaperID: 265,   

Authors:  Feng-Lei Fan, Mengzhou Li, Fei Wang, Rongjie Lai, Ge Wang

Title: On Expressivity and Trainability of Quadratic Networks

Abstract:
Inspired by the diversity of biological neurons, quadratic artificial neurons can play an important role in deep learning models. The type of quadratic neurons of our interest replaces the inner-product operation in the conventional neuron with a quadratic function. Despite promising results so far achieved by networks of quadratic neurons, there are important issues not well addressed. Theoretically, the superior expressivity of a quadratic network over either a conventional network or a conventional network via quadratic activation is not fully elucidated, which makes the use of quadratic networks not well grounded. In practice, although a quadratic network can be trained via generic backpropagation, it can be subject to a higher risk of collapse than the conventional counterpart. To address these issues, we first apply the spline theory and a measure from algebraic geometry to give two theorems that demonstrate better model expressivity of a quadratic network than the conventional counterpart with or without quadratic activation. Then, we propose an effective training strategy referred to as referenced linear initialization (ReLinear) to stabilize the training process of a quadratic network, thereby unleashing the full potential in its associated machine learning tasks. Comprehensive experiments on popular datasets are performed to support our findings and confirm the performance of quadratic deep learning. We have shared our code in https://github.com/FengleiFan/ReLinear.

---

PaperID: 266,   

Authors:  Zheng Zhang, Qingrui Zhang, Bo Zhu, Xiaohan Wang, Tianjiang Hu

Title: EASpace: Enhanced Action Space for Policy Transfer

Abstract:
Formulating expert policies as macro actions promises to alleviate the long-horizon issue via structured exploration and efficient credit assignment. However, traditional option-based multipolicy transfer methods suffer from inefficient exploration of macro action’s length and insufficient exploitation of useful long-duration macro actions. In this article, a novel algorithm named enhanced action space (EASpace) is proposed, which formulates macro actions in an alternative form to accelerate the learning process using multiple available suboptimal expert policies. Specifically, EASpace formulates each expert policy into multiple macro actions with different execution times. All the macro actions are then integrated into the primitive action space directly. An intrinsic reward, which is proportional to the execution time of macro actions, is introduced to encourage the exploitation of useful macro actions. The corresponding learning rule that is similar to intraoption Q-learning is employed to improve the data efficiency. Theoretical analysis is presented to show the convergence of the proposed learning rule. The efficiency of EASpace is illustrated by a grid-based game and a multiagent pursuit problem. The proposed algorithm is also implemented in physical systems to validate its effectiveness.

---

PaperID: 267,   

Authors:  Shiye Lei, Fengxiang He, Haowen Chen, Dacheng Tao

Title: Attentive Learning Facilitates Generalization of Neural Networks

Abstract:
This article studies the generalization of neural networks (NNs) by examining how a network changes when trained on a training sample with or without out-of-distribution (OoD) examples. If the network’s predictions are less influenced by fitting OoD examples, then the network learns attentively from the clean training set. A new notion, dataset-distraction stability, is proposed to measure the influence. Extensive CIFAR-10/100 experiments on the different VGG, ResNet, WideResNet, ViT architectures, and optimizers show a negative correlation between the dataset-distraction stability and generalizability. With the distraction stability, we decompose the learning process on the training set \mathcal S into multiple learning processes on the subsets of \mathcal S drawn from simpler distributions, i.e., distributions of smaller intrinsic dimensions (IDs), and furthermore, a tighter generalization bound is derived. Through attentive learning, miraculous generalization in deep learning can be explained and novel algorithms can also be designed.

---

PaperID: 268,   

Authors:  Jie Lian, Lizhi Wang, He Sun, Hua Huang

Title: GT-HAD: Gated Transformer for Hyperspectral Anomaly Detection

Abstract:
Hyperspectral anomaly detection (HAD) aims to distinguish between the background and anomalies in a scene, which has been widely adopted in various applications. Deep neural network (DNN)-based methods have emerged as the predominant solution, wherein the standard paradigm is to discern the background and anomalies based on the error of self-supervised hyperspectral image (HSI) reconstruction. However, current DNN-based methods cannot guarantee correspondence between the background, anomalies, and reconstruction error, which limits the performance of HAD. In this article, we propose a novel gated transformer network for HAD (GT-HAD). Our key observation is that the spatial–spectral similarity in HSI can effectively distinguish between the background and anomalies, which aligns with the fundamental definition of HAD. Consequently, we develop GT-HAD to exploit the spatial–spectral similarity during HSI reconstruction. GT-HAD consists of two distinct branches that model the features of the background and anomalies, respectively, with content similarity as constraints. Furthermore, we introduce an adaptive gating unit to regulate the activation states of these two branches based on a content-matching method (CMM). Extensive experimental results demonstrate the superior performance of GT-HAD. The original code is publicly available at https://github.com/jeline0110/ GT-HAD, along with a comprehensive benchmark of state-of-the-art HAD methods.

---

PaperID: 269,   

Authors:  Yulu Fu, Yuting Li, Qiong Huang, Jinrong Cui, Jie Wen

Title: Anchor Graph Network for Incomplete Multiview Clustering

Abstract:
Incomplete multiview clustering (IMVC) has received extensive attention in recent years. However, existing works still have several shortcomings: 1) some works ignore the correlation of sample pairs in the global structural distribution; 2) many methods are computational expensive, thus cannot be applicable to the large-scale incomplete data clustering tasks; and 3) some methods ignore the refinement of the bipartite graph structure. To address the above issues, we propose a novel anchor graph network for IMVC, which includes a generative model and a similarity metric network. Concretely, the method uses a generative model to construct bipartite graphs, which can mine latent global structure distributions of sample pairs. Later, we use graph convolution network (GCN) with the constructed bipartite graphs to learn the structural embeddings. Notably, the introduction of bipartite graphs can greatly reduce the computational complexity and thus enable our model to handle large-scale data. Unlike previous works based on bipartite graph, our method employs bipartite graphs to guide the learning process in GCNs. In addition, an innovative adaptive learning strategy that can construct robust bipartite graphs is incorporated into our method. Extensive experiments demonstrate that our method achieves the comparable or superior performance compared with the state-of-the-art methods.

---

PaperID: 270,   

Authors:  Lei Zhai, Yitong Li, Zhixi Feng, Shuyuan Yang, Hao Tan

Title: Learning Cross-Domain Features With Dual-Path Signal Transformer

Abstract:
The past decade has witnessed the rapid development of deep neural networks (DNNs) for automatic modulation classification (AMC). However, most of the available works learn signal features from only a single domain via DNNs, which is not reliable enough to work in uncertain and complex electromagnetic environments. In this brief, a new cross-domain signal transformer (CDSiT) is proposed for AMC, to explore the latent association between different domains of signals. By constructing a signal fusion bottleneck (SFB), CDSiT can implicitly fuse and classify signal features with complementary structures in different domains. Extensive experiments are performed on RadioML2016.10A and RadioML2018.01A, and the results show that CDSiT outperforms its counterparts, particularly for some modulation modes that are difficult to classify before. Through ablation experiences, we also verify the effectiveness of each module in CDSiT.

---

PaperID: 271,   

Authors:  Yu Jiang, Cong Hua, Yifan Feng, Yue Gao

Title: Hierarchical Set-to-Set Representation for 3-D Cross-Modal Retrieval

Abstract:
Three-dimensional in-domain retrieval has recently achieved significant success, but 3-D cross-modal retrieval still faces problems and challenges. Existing methods only rely on a simple global feature (GF), which overlooks the local information of complex 3-D objects and the connections between similar local features across complex multimodal instances. To tackle this issue, we propose a hierarchical set-to-set representation (HSR) and a corresponding hierarchical similarity that incorporates global-to-global and local-to-local similarity metrics. Specifically, we employ feature extractors for each modality to learn both GFs and local feature sets. We then project these features into their respective common space and use bilinear pooling to generate compact-set features that maintain the invariant for set-to-set similarity measurement. To facilitate effective hierarchical similarity measurement, we design an operation to combine the GF and the compact-set feature to generate the hierarchical representation for 3-D cross-modal retrieval, which preserves hierarchical similarity measurement. To optimize the framework, we adopt the joint loss functions, including cross-modal center loss (CMCL), mean square loss, and cross-entropy loss, to reduce the cross-modal discrepancy for each instance and minimize the distances between the instances in the same category. Experimental results demonstrate that our method outperforms the state-of-the-art methods on the 3-D cross-modal retrieval task on both ModelNet10 and ModelNet40 datasets.

---

PaperID: 272,   

Authors:  Ruiwen Yuan, Yongqiang Tang, Yajing Wu, Wensheng Zhang

Title: Clustering Enhanced Multiplex Graph Contrastive Representation Learning

Abstract:
Multiplex graph representation learning has attracted considerable attention due to its powerful capacity to depict multiple relation types between nodes. Previous methods generally learn representations of each relation-based subgraph and then aggregate them into final representations. Despite the enormous success, they commonly encounter two challenges: 1) the latent community structure is overlooked and 2) consistent and complementary information across relation types remains largely unexplored. To address these issues, we propose a clustering-enhanced multiplex graph contrastive representation learning model (CEMR). In CEMR, by formulating each relation type as a view, we propose a multiview graph clustering framework to discover the potential community structure, which promotes representations to incorporate global semantic correlations. Moreover, under the proposed multiview clustering framework, we develop cross-view contrastive learning and cross-view cosupervision modules to explore consistent and complementary information in different views, respectively. Specifically, the cross-view contrastive learning module equipped with a novel negative pairs selecting mechanism enables the view-specific representations to extract common knowledge across views. The cross-view cosupervision module exploits the high-confidence complementary information in one view to guide low-confidence clustering in other views by contrastive learning. Comprehensive experiments on four datasets confirm the superiority of our CEMR when compared to the state-of-the-art rivals.

---

PaperID: 273,   

Authors:  Wenqian Dong, Yihan Yang, Jiahui Qu, Yunsong Li, Yufei Yang, Xiuping Jia

Title: Feature Pyramid Fusion Network for Hyperspectral Pansharpening

Abstract:
Hyperspectral (HS) pansharpening aims at fusing an observed HS image with a panchromatic (PAN) image, to produce an image with the high spectral resolution of the former and the high spatial resolution of the latter. Most of the existing convolutional neural networks (CNNs)-based pansharpening methods reconstruct the desired high-resolution image from the encoded low-resolution (LR) representation. However, the encoded LR representation captures semantic information of the image and is inadequate in reconstructing fine details. How to effectively extract high-resolution and LR representations for high-resolution image reconstruction is the main objective of this article. In this article, we propose a feature pyramid fusion network (FPFNet) for pansharpening, which permits the network to extract multiresolution representations from PAN and HS images in two branches. The PAN branch starts from the high-resolution stream that maintains the spatial resolution of the PAN image and gradually adds LR streams in parallel. The structure of the HS branch remains highly consistent with that of the PAN branch, but starts with the LR stream and gradually adds high-resolution streams. The representations with corresponding resolutions of PAN and HS branches are fused and gradually upsampled in a coarse to fine manner to reconstruct the high-resolution HS image. Experimental results on three datasets demonstrate the significant superiority of the proposed FPFNet over the state-of-the-art methods in terms of both qualitative and quantitative comparisons.

---

PaperID: 274,   

Authors:  Like Xin, Wanqi Yang, Lei Wang, Ming Yang

Title: Selective Contrastive Learning for Unpaired Multi-View Clustering

Abstract:
In this article, we investigate a novel but insufficiently studied issue, unpaired multi-view clustering (UMC), where no paired observed samples exist in multi-view data, and the goal is to leverage the unpaired observed samples in all views for effective joint clustering. Existing methods in incomplete multi-view clustering usually utilize the sample pairing relationship between views to connect the views for joint clustering, but unfortunately, it is invalid for the UMC case. Therefore, we strive to mine a consistent cluster structure between views and propose an effective method, namely selective contrastive learning for UMC (scl-UMC), which needs to solve the following two challenging issues: 1) uncertain clustering structure under no supervision information and 2) uncertain pairing relationship between the clusters of views. Specifically, for the first one, we design an inner-view (IV) selective contrastive learning module to enhance the clustering structures and alleviate the uncertainty, which selects confident samples near the cluster centroids to perform contrastive learning in each view. For the second one, we design a cross-view (CV) selective contrastive learning module to first iteratively match the clusters between views and then tighten the matched clusters. Also, we utilize mutual information to further enhance the correlation of the matched clusters between views. Extensive experiments show the efficiency of our methods for UMC, compared with the state-of-the-art methods.

---

PaperID: 275,   

Authors:  Bohao Li, Chang Liu, Mengnan Shi, Xiaozhong Chen, Xiangyang Ji, Qixiang Ye

Title: Proposal Distribution Calibration for Few-Shot Object Detection

Abstract:
Adapting object detectors learned with sufficient supervision to novel classes under low data regimes is charming yet challenging. In few-shot object detection (FSOD), the two-step training paradigm is widely adopted to mitigate the severe sample imbalance, i.e., holistic pre-training on base classes, then partial fine-tuning in a balanced setting with all classes. Since unlabeled instances are suppressed as backgrounds in the base training phase, the learned region proposal network (RPN) is prone to produce biased proposals for novel instances, resulting in dramatic performance degradation. Unfortunately, the extreme data scarcity aggravates the proposal distribution bias, hindering the region of interest (RoI) head from evolving toward novel classes. In this brief, we introduce a simple yet effective proposal distribution calibration (PDC) approach to neatly enhance the localization and classification abilities of the RoI head by recycling its localization ability endowed in base training and enriching high-quality positive samples for semantic fine-tuning. Specifically, we sample proposals based on the base proposal statistics to calibrate the distribution bias and impose additional localization and classification losses upon the sampled proposals for fast expanding the base detector to novel classes. Experiments on the commonly used Pascal VOC and MS COCO datasets with explicit state-of-the-art performances justify the efficacy of our PDC for FSOD. Code is available at github.com/Bohao-Lee/PDC.

---

PaperID: 276,   

Authors:  Yu Duan, Feiping Nie, Huimin Chen, Zhanxuan Hu, Rong Wang, Xuelong Li

Title: Hyperbolic Hierarchical Representation Learning for Generalized Category Discovery

Abstract:
This study addresses the problem of generalized category discovery (GCD), an advanced and challenging semi-supervised learning scenario that deals with unlabeled data from both known and novel categories. Although recent research has effectively engaged with this issue, these studies typically map features into Euclidean space, which fails to maintain the latent semantic hierarchy of the training samples effectively. This limitation restricts the exploration of more detailed and rich information and degrades the performance in discovering new categories. The emerging field of hyperbolic representation learning suggests that hyperbolic geometry could be advantageous for extracting semantic information to tackle this problem. Motivated by this, we proposed hyperbolic hierarchical representation learning for GCD (HypGCD). Specifically, HypGCD enhances representations in hyperbolic space, building upon the Euclidean space representation from two perspectives: instance-class level and instance-instance level. At the instance-class level, HypGCD endeavors to construct well-defined clusters, with each sample forming a robust hierarchical cluster structure. Concurrently, at the instance-instance level, HypGCD anticipates that a subset of samples will display a tree-like structure in local space, which aligns more closely with real-world scenarios. Finally, HypGCD optimizes the Euclidean and hyperbolic space collectively to obtain refined features. Additionally, we show that HypGCD is exceptionally effective, achieving state-of-the-art (SOTA) results on several datasets. The code is available at https://github.com/DuannYu/HypGCD

---

PaperID: 277,   

Authors:  Kayol Soares Mayer, Jonathan Aguiar Soares, Ariadne A. Cruz, Dalton Soares Arantes

Title: Adaptive Learning Rate Methods for Complex-Valued Neural Networks

Abstract:
Artificial neural networks (ANNs) have become a popular tool in digital signal processing (DSP). Among the widespread ANN architectures, complex-valued neural networks (CVNNs) have been extensively studied in image processing and telecommunications. Unlike their real-valued counterparts, CVNNs can handle signals directly in the complex domain. Due to this capability, CVNNs usually exhibit higher accuracy and improved convergence compared to real-valued neural networks (RVNNs). Despite their improved performance in several applications, CVNNs still lag behind RVNNs in terms of learning techniques and heuristics. In this context, we propose adaptive learning rate approaches for CVNNs, extending the well-known adaptive gradient (AdaGrad), root-mean-square propagation (RMSProp), AdaMax, AMSGrad, softplus AMSGrad (SAMSGrad), Nesterov-accelerated adaptive moment estimation (Nadam), and DiffGrad to the complex domain. Computational complexities of the proposed optimizers are analyzed for CVNN architectures. Results are compared in terms of mean-squared-error convergence.

---

PaperID: 278,   

Authors:  Yuchen Xiao, Lei Yuan, Lihe Li, Ziqian Zhang, Yi-Chen Li, Yang Yu

Title: Generalizable Offline Multiobjective Reinforcement Learning via Preference-Conditioned Diffuser

Abstract:
Multiobjective reinforcement learning (MORL) addresses sequential decision-making problems with multiple objectives by learning policies optimized for diverse pReferences. While traditional methods necessitate costly online interaction with the environment, recent approaches leverage static datasets containing precollected trajectories, making offline MORL the preferred choice for real-world applications. However, existing offline MORL techniques suffer from limited expressiveness and poor generalization on out-of-distribution (OOD) preferences. To overcome these limitations, we propose diffusion-based MORL (DiffMORL), a generalizable diffusion-based planning frame work for MORL. Leveraging the strong expressiveness and generation capability of diffusion models, DiffMORL further boosts its generalization through offline data mixup, which mitigates the memorization phenomenon and facilitates feature learning by data augmentation. By training on the augmented data, DiffMORL is able to condition on a given preference, whether in-distribution or OOD, to plan the desired trajectory and extract the corresponding action. Evaluations conducted on the datasets for MORL (D4MORL) benchmark demonstrate that DiffMORL achieves state-of-the-art results across nearly all tasks. Notably, it surpasses the best baseline on 14 out of 18 metrics for OOD generalization, underscoring its remarkable generalization ability in offline MORL scenarios.

---

PaperID: 279,   

Authors:  Martin Krutsky, Gustav Sír

Title: Geometric Deep Learning for the Rubik's Cube Group

Abstract:
The Rubik’s cube, a widely recognized combinatorial puzzle with an astronomically vast state space, has been the subject of various research experiments with neural networks used as heuristic estimators to navigate the state-space exploration. However, prior efforts have overlooked the intriguing symmetries inherent to this domain. Drawing on geometric deep learning principles, this article introduces a novel neural architecture that explicitly leverages these symmetries, grounded in a rigorous group-theoretical analysis. The design of the proposed symmetry-invariant model is then validated empirically through an innovative universal procedure for detecting model symmetry invariance. Finally, experimental results demonstrate that the symmetry-aware neural architecture exhibits enhanced generalization and problem-solving efficacy compared with the state of the art.

---

PaperID: 280,   

Authors:  Zechao Li, Hao Tang, Zhimao Peng, Guo-Jun Qi, Jinhui Tang

Title: Knowledge-Guided Semantic Transfer Network for Few-Shot Image Recognition

Abstract:
Deep learning-based models have been shown to outperform human beings in many computer vision tasks with massive available labeled training data in learning. However, humans have an amazing ability to easily recognize images of novel categories by browsing only a few examples of these categories. In this case, few-shot learning comes into being to make machines learn from extremely limited labeled examples. One possible reason why human beings can well learn novel concepts quickly and efficiently is that they have sufficient visual and semantic prior knowledge. Toward this end, this work proposes a novel knowledge-guided semantic transfer network (KSTNet) for few-shot image recognition from a supplementary perspective by introducing auxiliary prior knowledge. The proposed network jointly incorporates vision inferring, knowledge transferring, and classifier learning into one unified framework for optimal compatibility. A category-guided visual learning module is developed in which a visual classifier is learned based on the feature extractor along with the cosine similarity and contrastive loss optimization. To fully explore prior knowledge of category correlations, a knowledge transfer network is then developed to propagate knowledge information among all categories to learn the semantic-visual mapping, thus inferring a knowledge-based classifier for novel categories from base categories. Finally, we design an adaptive fusion scheme to infer the desired classifiers by effectively integrating the above knowledge and visual information. Extensive experiments are conducted on two widely used Mini-ImageNet and Tiered-ImageNet benchmarks to validate the effectiveness of KSTNet. Compared with the state of the art, the results show that the proposed method achieves favorable performance with minimal bells and whistles, especially in the case of one-shot learning.

---

PaperID: 281,   

Authors:  Shifei Ding, Wei Du, Ling Ding, Jian Zhang, Lili Guo, Bo An

Title: Multiagent Reinforcement Learning With Graphical Mutual Information Maximization

Abstract:
Communication learning is an important research direction in the multiagent reinforcement learning (MARL) domain. Graph neural networks (GNNs) can aggregate the information of neighbor nodes for representation learning. In recent years, several MARL methods leverage GNN to model information interactions between agents to coordinate actions and complete cooperative tasks. However, simply aggregating the information of neighboring agents through GNNs may not extract enough useful information, and the topological relationship information is ignored. To tackle this difficulty, we investigate how to efficiently extract and utilize the rich information of neighbor agents as much as possible in the graph structure, so as to obtain high-quality expressive feature representation to complete the cooperation task. To this end, we present a novel GNN-based MARL method with graphical mutual information (MI) maximization to maximize the correlation between input feature information of neighbor agents and output high-level hidden feature representations. The proposed method extends the traditional idea of MI optimization from graph domain to multiagent system, in which the MI is measured from two aspects: agent features information and agent topological relationships. The proposed method is agnostic to specific MARL methods and can be flexibly integrated with various value function decomposition methods. Considerable experiments on various benchmarks demonstrate that the performance of our proposed method is superior to the existing MARL methods.

---

PaperID: 282,   

Authors:  Yujiao Hu, Yuan Yao, Jinchao Chen, Zhihao Wang, Qingmin Jia, Yan Pan

Title: Solving Scalable Multiagent Routing Problems With Reinforcement Learning

Abstract:
Multiagent routing problems, arising from practical applications, such as logistics, transportation, and emergency response, face challenges due to the exponential growth of the search space with increasing problem scales. This article proposes RouteMaker to address the often-overlooked multiagent routing problems involving dedicated multiple depots. RouteMaker leverages role-interaction-based graph neural network (RIGNN) to realize effective locations assignments and integrates an advanced planner to plan travel path for each agent. RouteMaker is trained on small-scale problems and can produce comparable or superior approximate optimal solutions compared with the best heuristic baselines. Notably, the learned RouteMaker generalizes seamlessly to large-scale problems and real-world problems without the need for fine-tuning, delivering significantly higher quality solutions in relatively less time. For scenarios involving 40 agents and 1000 locations, RouteMaker achieves over 600× speed improvement and more than 88% cost reduction, compared with the representative classical heuristic solver (ORTools).

---

PaperID: 283,   

Authors:  Haodong Zhang, Tao Ren, Yifan Wang, Fanchun Meng, Wei Ju, Ying Tian

Title: Cluster-Aware Few-Shot Molecular Property Prediction With Factor Disentanglement

Abstract:
Molecular property prediction plays a crucial role in drug discovery, but is always challenged by the limited number of effective labels. Compared with existing methods, we argue that the auxiliary properties of the molecule and the heterogeneous structure of different property prediction tasks have always been ignored. In this article, we propose a novel framework termed Meta-DREAM for few-shot molecular property prediction, which tailors to learning the transferable knowledge within different clusters of tasks. Specifically, we first construct a heterogeneous molecule relation graph (HMRG) with molecule–property and molecule–molecule relations to utilize many-to-many correlations between properties and molecules. The meta-learning episode can, then, be reformulated as a subgraph of HMRG. Next, we propose a disentangled graph encoder to explicitly discriminate the underlying factors of the task. In addition, we introduce a soft clustering module to group each factorized task representation into appropriate clusters and preserve knowledge generalization within a cluster and customization among clusters. In this way, each disentangled factor serves as a cluster-aware parameter gate for the task-specific meta-learner. Extensive experiments on five commonly used molecular datasets show that Meta-DREAM consistently outperforms existing state-of-the-art methods and verifies the effectiveness of each module.

---

PaperID: 284,   

Authors:  Xiaoyan Kui, Haonan Yan, Qinsong Li, Min Zhang, Liming Chen, Beiji Zou

Title: ChebMixer: Efficient Graph Representation Learning With MLP Mixer

Abstract:
Graph neural networks (GNNs) have achieved remarkable success in learning graph representations, especially graph Transformers, which have recently shown superior performance on various graph mining tasks. However, the graph Transformer generally treats nodes as tokens, which results in quadratic complexity regarding the number of nodes during self-attention computation. The graph multilayer perceptron (MLP) mixer addresses this challenge using the efficient MLP Mixer technique from computer vision. However, the time-consuming process of extracting graph tokens limits its performance. In this article, we present a novel architecture named ChebMixer, a newly proposed graph MLP Mixer that uses fast Chebyshev polynomials-based spectral filtering to extract a sequence of tokens. First, we produce multiscale representations of graph nodes via fast Chebyshev polynomial-based spectral filtering. Next, we consider each node’s multiscale representations as a sequence of tokens and refine the node representation with an effective MLP Mixer. Finally, we aggregate the multiscale representations of nodes through Chebyshev interpolation. Owing to the powerful representation capabilities and fast computational properties of the MLP Mixer, we can quickly extract more informative node representations to improve the performance of downstream tasks. The experimental results prove our significant improvements in various scenarios, ranging from homogeneous and heterophilic graph node classification to medical image segmentation. Compared with NAGphormer, the average performance improved by 1.45% on homogeneous graphs and 4.15% on heterophilic graphs. And the average performance improved by 1.39% on medical image segmentation tasks compared with VM-UNet. We will release the source code after this article is accepted.

---

PaperID: 285,   

Authors:  Yuanchao Su, Lianru Gao, Antonio Plaza, Xu Sun, Mengying Jiang, Guang Yang

Title: SRViT: Self-Supervised Relation-Aware Vision Transformer for Hyperspectral Unmixing

Abstract:
Vision transformer (ViT) has recently been a popular topic in the foundation model field, taking advantage of its strong scalability and outstanding representation capabilities. As a deep model, ViT introduces a new architecture for achieving hyperspectral image (HSI) unmixing. However, traditional ViTs overlook pixel-level spatial continuity by partitioning the input image into nonoverlapping fixed-size patches. This approach disrupts local structural relationships and hinders the model’s ability to capture fine-grained spatial dependencies, resulting in suboptimal feature representation for dense prediction tasks in unmixing. To address these challenges, this article proposes the development of a self-supervised relation-aware ViT (SRViT). SRViT incorporates a self-embedded module comprising encoders, a pixel-level position encoder (PLPE), a self-supervised contrastive mechanism (SCM), and a decoder. The self-embedded module and PLPE preserve local correlations in HSI across different views, facilitating cross-view learning through SCM to ensure generalization. In addition, the decoder incorporates Kronecker-factored approximate curvature (K-FAC) to capture the local geometric structure of spectral information. Ultimately, SRViT learns endmembers and fractional abundance as the unmixing result. The effectiveness and competitiveness of SRViT have been systematically validated through comparative experiments, demonstrating its superior performance. The source code is available at the following link: https://github.com/yuanchaosu/TNNLS-SRViT

---

PaperID: 286,   

Authors:  Lin Zhao, Xiao Chen, Eric Z. Chen, Yikang Liu, Terrence Chen, Shanhui Sun

Title: Retrieval-Augmented Few-Shot Medical Image Segmentation With Foundation Models

Abstract:
Medical image segmentation is crucial for clinical decision-making, but the scarcity of annotated data presents significant challenges. Few-shot segmentation (FSS) methods show promise but often require training on the target domain and struggle to generalize across different modalities. Similarly, adapting foundation models such as the segment anything model (SAM) for medical imaging has limitations, including the need for fine-tuning and domain-specific adaptation. To address these issues, we propose a novel method that adapts DINOv2 and SAM 2 for retrieval-augmented few-shot medical image segmentation. Our approach uses DINOv2’s feature as query to retrieve similar samples from limited annotated data, which are then encoded as memories and stored in memory bank. With the memory attention mechanism of SAM 2, the model leverages these memories as conditions to generate accurate segmentation of the target image. We evaluated our framework on three medical image segmentation tasks, demonstrating superior performance and generalizability across various modalities without the need for any retraining or fine-tuning. Overall, this method offers a practical and effective solution for few-shot medical image segmentation and holds significant potential as a valuable annotation tool in clinical applications.

---

PaperID: 287,   

Authors:  Wanyi Ning, Jingyu Wang, Qi Qi, Haifeng Sun, Daixuan Cheng, Cong Liu, Lei Zhang, Zirui Zhuang, Jianxin Liao

Title: Federated Fine-Tuning on Heterogeneous LoRAs With Error-Compensated Aggregation

Abstract:
Federated learning (FL) has recently been applied to the parameter-efficient fine-tuning (PEFT) of large language models (LLMs). While promising, client resource heterogeneity has imposed the challenge of the “bucket effect” to FL, where model configuration must cater to the client with the fewest resources. To tackle this issue, heterogeneous low-rank adaptation (LoRA) has recently emerged in FL, which enables clients to do local fine-tuning with different LoRA ranks. However, existing works in this area typically adopt zero-padding, stacking, or singular value decomposition (SVD) for LoRA aggregation, which often incur precision loss or significant overhead, limiting their practicality. In this article, we propose ECLoRA, a novel method for federated fine-tuning with heterogeneous LoRA settings across clients. ECLoRA employs randomized SVD (RSVD) to dramatically reduce aggregation overhead while introducing an error compensation (EC) mechanism that incorporates the decomposition error from previous rounds to improve aggregation precision. Extensive experiments on four widely used foundation models across six public tasks demonstrate the effectiveness of ECLoRA. Specifically, ECLoRA is: (1) accurate, significantly improving the final model performance; (2) fast, accelerating convergence with an average speedup of 1.54× to 3.01× ; and (3) practical, reducing aggregation time by approximately 40× compared to classical SVD.

---

PaperID: 288,   

Authors:  Ye Wang, Jingjing Wang, Ruijie Zhu, Hang Fu, Jianrui Chen, C. L. Philip Chen

Title: Facilitating Multiagent Coordination Relying on Graph Information Representation

Abstract:
The popular multiagent reinforcement learning (MARL) methods primarily focus on exploring the capability of value functions to facilitate multiagent coordination. These MARL methods, following the centralized training with decentralized execution (CTDE) paradigm, tend to design ingenious network architectures while overlooking the impact of coordination through expanding local observation information. To tackle this deficiency, we model the multiagent systems (MASs) as a graph and use a graph neural network (GNN) to extract rich information between one agent and the others efficiently. Moreover, we propose a multigraph-neural-network information representation (MGIR) method that uses the power of GNN in local observation information extraction, enabling the acquisition of higher quality information. Specifically, multiple GNNs are used during centralized training to characterize the MAS from different perspectives and extract representations of latent variables. During distributed execution, these latent variables are leveraged to expand local observation information. Extensive comparative experiments substantiate that our proposed MGIR demonstrates superior coordination performance when compared with baseline methods. In addition, it can be flexibly integrated into various value function decomposition methods of MARL.

---

PaperID: 289,   

Authors:  Changxin Zhang, Xinglong Zhang, Yixing Lan, Hao Gao, Xin Xu

Title: Game-Theoretic Constrained Policy Optimization for Safe Reinforcement Learning

Abstract:
Safe reinforcement learning (RL) aims to optimize the task performance with safety guarantees. One common modeling scheme to study safe RL problems is the constrained Markov decision process (CMDP). However, current safe RL methods within the CMDP framework face challenges in tradeoffs among various objectives and gradient conflicts of policy updating. To cope with these challenges, this article presents a novel safe RL approach called game-theoretic constrained policy optimization (GCPO). The proposed approach formulates the CMDP problem as a general-sum Markov game with multiple players, where a task player seeks to maximize the reward objective, while constraint players aim to minimize constraint objectives until they are fulfilled. By doing so, GCPO adopts the learning mode with multiple subpolicies, each aligned with a distinct objective, that collectively constitute the overall behavior of the agent. The learning convergence of the GCPO can be ensured with the contraction mapping to the Nash equilibrium. Furthermore, a novel dominant timescale update rule is presented for multiplayer policy learning to guarantee constraint satisfaction. The learning convergence and constraint satisfaction of GCPO are theoretically analyzed. Consequently, GCPO eliminates the necessity of tuning tradeoff parameters and mitigates gradient conflicts during multiobjective policy updating. Experimental results show that GCPO outperforms state-of-the-art safe RL algorithms in a quadrotor trajectory tracking task and various high-dimensional robot locomotion benchmarks. Moreover, GCPO exhibits robustness to diverse scales of task rewards and constraint costs without the need for intricate tradeoffs.

---

PaperID: 290,   

Authors:  Jie Chen, Hua Mao, Yuanbiao Gou, Xi Peng

Title: Hierarchical Sparse Representation Clustering for High-Dimensional Data Streams

Abstract:
Data stream clustering reveals patterns within continuously arriving, potentially unbounded data sequences. Numerous data stream algorithms have been proposed to cluster data streams. The existing data stream clustering algorithms still face significant challenges when addressing high-dimensional data streams. First, it is intractable to measure the similarities among high-dimensional data objects via Euclidean distances when constructing and merging microclusters. Second, these algorithms are highly sensitive to the noise contained in high-dimensional data streams. In this article, we propose a hierarchical sparse representation clustering (HSRC) framework for clustering high-dimensional data streams. HSRC first employs a sparse representation-based technique to learn an affinity matrix for data objects in individual landmark windows with a fixed size, where the number of neighboring data objects is automatically selected. The sparse representation-based technique ensures that highly correlated data samples within clusters are grouped together. Then, HSRC applies a spectral clustering technique to the affinity matrix to generate microclusters. These microclusters are subsequently merged into macroclusters based on their sparse similarity degrees (SSDs). In addition, HSRC introduces sparsity residual values (SRVs) to adaptively select representative data objects from the current landmark window. These representatives serve as dictionary samples for the next landmark window. Finally, HSRC refines each macrocluster through fine-tuning. In particular, HSRC enables the detection of outliers in high-dimensional data streams via the associated SRVs. The experimental results obtained on several benchmark datasets demonstrate the effectiveness and robustness of the proposed HSRC framework.

---

PaperID: 291,   

Authors:  Lei Deng, Xiao-Yang Liu, Danny H. K. Tsang

Title: Real-Time Network Latency Estimation With Pretrained Generative Models

Abstract:
Network latency estimation is critical for network performance monitoring and management. However, with the escalating demand for real-time performance monitoring and rapid network adjustments in contemporary networks, existing latency estimation methodologies fall short of meeting the need for instantaneous estimation. In this article, we propose a pretrained generative model-based scheme (PGM) for real-time network latency estimation. PGM operates in two stages. First, we employ a pretrained generative model to relax the low-rank constraint typically associated with latency matrix completion (MC). The pretrained generative model well learns the low-rank characteristics of latency matrices in the pretraining stage and can map a condensed latent representation to the matrix space. Second, instead of directly optimizing the matrix, we turn to optimizing the latent representation. Leveraging the low-rank structure achieved by the pretrained generative model simplifies our optimization process, enabling real-time estimation. We also provide a theoretical recovery guarantee to reveal the error bound of PGM. Experimental results on real-world datasets show that the proposed scheme can achieve accurate latency estimation within 50 ms while maintaining the relative squared error (RSE) of estimation at no more than 0.11 (as evidenced using the PlanetLab dataset).

---

PaperID: 292,   

Authors:  Wen Zhang, Jiangpeng Zhao, Lean Yu, Song Wang

Title: GMM Enhanced Anchor-Based Spectral Clustering for Large-Scale Data

Abstract:
Anchor-based methods are proposed to make use of anchors to produce an affinity matrix of objects to improve the scalability of traditional spectral clustering (SC). Nevertheless, the membership heterogeneity of objects inside a cluster, which would bring about low quality of anchors and hurt the clustering accuracy, is commonly neglected by existing anchor-based algorithms. To address this problem, this article proposes a novel approach to adopt the Gaussian mixture model (GMM) to enhance anchor-based SC for large-scale data in a two-stage divide-and-conquer manner. In the first stage, GMM with expectation maximization (EM) algorithm is employed to divide the objects into two categories as prior-consistent objects and prior-uncertain objects in considering the membership heterogeneity of objects. In the second stage, anchor-based SC is conducted on the prior-uncertain objects by sampling the anchors from the Gaussian components derived from the first stage. Then, the produced clusters in the second stage are aligned with those Gaussian components by maximizing the membership of objects with respect to clusters. The computation complexity of the proposed GMM-SC approach is much smaller than that of the anchor-based SC. The experiments on large-scale datasets also validate the superiority of the proposed GMM-SC approach over state-of-the-art techniques.

---

PaperID: 293,   

Authors:  Simin Li, Ruixiao Xu, Jingqiao Xiu, Yuwei Zheng, Pu Feng, Yuqing Ma, Bo An, Yaodong Yang, Xianglong Liu

Title: Robust Multi-Agent Reinforcement Learning by Mutual Information Regularization

Abstract:
In cooperative multi-agent reinforcement learning (MARL), ensuring robustness against cooperative agents making unpredictable or worst-case adversarial actions is crucial for real-world deployment. In multi-agent settings, each agent may be perturbed or unperturbed, leading to an exponential increase in potential threat scenarios as the number of agents grows. Existing robust MARL methods either enumerate, or approximate all possible threat scenarios, leading to intense computation and insufficient robustness. In contrast, humans develop robust behaviors by maintaining a general level of caution rather than preparing for every possible threat. Inspired by human decision making, we frame robust MARL as a control-as-inference problem, and optimize worst-case robustness across all threat scenarios implicitly optimized through off-policy evaluation. Specifically, we introduce mutual information regularization as robust regularization (MIR3), which maximizes a lower bound on robustness during routine training, serving as a kind of caution for MARL without adversarial inputs. Further insights show that MIR3 acts as an information bottleneck, preventing agents from over-reacting to others and aligning policies with robust action priors. In the presence of worst-case adversaries, our MIR3 significantly surpasses baseline methods in robustness and training efficiency, and maintaining cooperative performance in StarCraft II, quadrotor swarm control, and robot swarm control. When deploying the robot swarm control algorithm in the real world, our method also outperforms the best baseline by 14.29% in reward. See code and demo videos at https://github.com/DIG-Beihang/MIR3

---

PaperID: 294,   

Authors:  Wentao Li, Lingwei Wei, Witold Pedrycz, Weiping Ding, Chao Zhang, Tao Zhan, Shuyin Xia

Title: Granular-Ball Regeneration Clustering With Principle of Justifiable Granularity

Abstract:
Classical clustering algorithms such as k-means face limitations in handling clusters with heterogeneous shapes, densities, and sizes, while exhibiting sensitivity to initial centroid selection. To overcome these challenges, this article proposes a novel clustering framework based on regenerated granular ball (RGGB) with the principle of justifiable granularity. Unlike existing granular-ball (GB) techniques that overemphasize purity criteria at the expense of uncontrolled ball sizes, RGGB dynamically adjusts granularity levels through iterative regeneration, achieving an optimal balance between detailed data representation and computational efficiency. This adaptability enhances stability in capturing data similarities while mitigating sensitivity to initialization. To validate the method, we integrate RGGB with a novel k-nearest neighbor (KNN) classifier using regenerated GBs to evaluate classification performance and demonstrate practical applications. Experiments on diverse public and realistic datasets demonstrate that the RGGB-based KNN algorithm consistently outperforms existing techniques, including traditional KNN and other methods, making a promising advancement in clustering and classification tasks.

---

PaperID: 295,   

Authors:  Zhen Peng, Yunfan Wang, Qika Lin, Bin Shi, Chen Chen, Bo Dong, Chao Shen

Title: End-to-End Abnormal Subgraph Detection via Subgraph-Level Contrastive Learning

Abstract:
Abnormal subgraph (AS) detection plays a significant role in ensuring the security of many high-impact domains. Unlike node anomaly detection, identifying subgraph anomalies is extremely challenging due to the exponentially large subgraph space caused by various combinations of nodes and edges. Moreover, in the absence of supervisory signals, how to quantify the abnormality of subgraphs poses another pressing challenge. Traditional methods typically rely on handcrafted subgraph anomaly measures, making it hard to handle potential unknown anomalies with limited prior knowledge. Recent deep learning-based techniques are predominantly designed to discover individual node anomalies, which could be suboptimal for AS detection due to the inconsideration of collaborative behaviors between nodes in the subgraph. In fact, existing studies have put very little effort into this task, and even dedicated performance evaluation metrics are not yet available. To address the above challenges and promote related research, in this article, we propose a end-to-end unsupervised subgraph anomaly detection framework (EndSubG), which jointly models subgraph partition and AS detection as a whole instead of treating them as two separate stages. Specifically, EndSubG uncovers potential AS boundaries that violate the Homophily assumption by modeling the edge existence probability, then achieves anomaly-aware graph embedding and subgraph partition based on the refined topology. By forming a coarsened subgraph network, EndSubG picks out subgraph anomalies by learning the “subgraph-vicinity” matching patterns. Additionally, we design an evaluation metric weighted normalized mutual information centered on AS (AS-WNMI) specifically for subgraph anomaly detection, which is a variant of vanilla NMI and quantifies detection performance from both subgraph partition and anomaly recognition. The experimental results on synthetic and real-world datasets corroborate the superiority of end-to-end unsupervised subgraph anomaly detection framework (EndSubG) in terms of area under the curve (AUC), average precision (AP), and AS-WNMI. We also provide an intuitive analysis of the detected subgraphs through visualization for better understanding.

---

PaperID: 296,   

Authors:  Hong Liu, Xiuxiu Qiu, Yiming Shi, Miao Xu, Zelin Zang, Zhen Lei

Title: Deep Multimanifold Transformation-Based Multivariate Time Series Fault Detection

Abstract:
Unsupervised fault detection in multivariate time series (MTS) plays a vital role in ensuring the stable operation of complex systems. Traditional methods often assume that normal data follow a single Gaussian distribution and identify anomalies as deviations from this distribution. However, this simplified assumption fails to capture the diversity and structural complexity of real-world time series, which can lead to misjudgments and reduced detection performance in practical applications. To address this issue, we propose a new method that combines a neighborhood-driven data augmentation strategy with a multimanifold representation learning framework. By incorporating information from local neighborhoods, the augmentation module can simulate contextual variations of normal data, enhancing the model’s adaptability to distributional changes. In addition, we design a structure-aware feature learning approach that encourages natural clustering of similar patterns in the feature space while maintaining sufficient distinction between different operational states. Extensive experiments on several public benchmark datasets demonstrate that our method achieves superior performance in terms of both accuracy and robustness, showing strong potential for generalization and real-world deployment.

---

PaperID: 297,   

Authors:  Heng Zhou, Zhenxi Zhang, Chengyang Li, Chunna Tian, Yongqiang Xie, Zhongbo Li, Xiao-Jun Wu

Title: Deformation-Resilient Multigranularity Learning for Unaligned RGB-T Semantic Segmentation

Abstract:
RGB–Thermal semantic segmentation (SS) aims to combine visual light and thermal images to determine the semantic category for each pixel and create an object mask. While existing methods typically rely on well-aligned RGB–T image pairs, real-world RGB–T pairs are often unaligned, and pixel-by-pixel alignment is both challenging and time-consuming. To address this critical issue, we introduce a new unaligned RGB–T SS benchmark and propose the deformation-resilient multigranularity learning (DML) method. DML explores the spatial consistency and modal complementarity of RGB–T and mitigates the interference of warped modalities by aligning multimodal features in a coarse-to-fine multigranularity strategy. Specifically, DML constructs a deformation-aware complementary feature enhancer (DCFE), which consists of deformation-aware feature alignment (DFA) and complementary feature aggregation (CFA) modules. DFA enhances the spatial alignment of RGB–T by estimating the deformation field of warped features. Then, CFA aggregates complementary contexts of modal differences across multiple scales to produce deformation-resilient and robust RGB–T feature representations. Finally, we design the multigranularity mask refinement engine (MMFE), which combines class-agnostic saliency prediction (CSP) and class-aware edge generation (CEG) auxiliary tasks to provide useful boundary and positional cues for SS decoders. The MMFE enhances semantic alignment and interclass separability, yielding object masks with sharp boundaries. Quantitative and qualitative experiments on aligned and unaligned datasets validate the effectiveness of our proposed DML, consistently outperforming existing methods designed for aligned RGB–T data. The new unaligned RGB–T SS benchmark and code are available at https://github.com/VisionVerse/Unaligned-RGBT-Semantic-Segmentation

---

PaperID: 298,   

Authors:  Haoen Huang, Zhigang Zeng, Jun Wang

Title: A Trust-Region Projection Neural Network for Nonlinear Programming

Abstract:
The trust-region method and projection neural networks are two branches of optimization approaches with different operational principles and characteristics. In this article, a trust-region projection neural network (TRPNN) is proposed by integrating the trust-region method and projection neural networks. TRPNN is a discrete-time neurodynamic optimization model that inherits the exploration–exploitation capability of the trust-region method and the local search capability of projection neural networks. TRPNN is theoretically proven to be convergent to a Karush–Kuhn–Tuchker (KKT) point of nonlinear programming problems. The efficacy of TRPNNs leveraged in a collaborative neurodynamic framework is numerically demonstrated for global optimization in the presence of nonconvexity in objective functions or constraints.

---

PaperID: 299,   

Authors:  Hengrong Ju, Yang Lu, Weiping Ding, Wei Zhang, Xibei Yang

Title: Multigranularity Information Fused Contrastive Learning With Multiview Clustering

Abstract:
Contrastive multiview clustering (MVC) has emerged as a mainstream approach in MVC due to its superior representation learning capabilities. Traditional contrastive multiview learning methods extract both low- and high-level information from raw data. However, only high-level information is utilized for clustering. Since both types of information are essential for effective clustering, this limitation hampers performance. Moreover, effectively quantifying the importance of different views remains a critical challenge in contrastive MVC. Additionally, the absence of structural information during clustering further weakens clustering performance. To address these issues, this article proposes a multigranularity (MG) information fused contrastive learning with MVC (MGCMVC). Inspired by the concept of MG, low- and high-level features are reconstructed into fine- and coarse-granularity features. First, an MG adaptive weighting sample-level contrastive learning mechanism is introduced to fuse MG features to enhance clustering performance and mitigate clustering performance degradation caused by variations in view quality. Second, a structure-oriented cluster-level contrastive learning approach is designed to preserve structural information and enforce cross-view clustering consistency. Extensive and comprehensive experiments on ten widely used datasets demonstrate that MGCMVC achieves the state-of-the-art performance. The source code is available at https://github.com/Luyangabc/MGCMVC

---

PaperID: 300,   

Authors:  Wuque Cai, Hongze Sun, Qianqian Liao, Jiayi He, Duo Chen, Dezhong Yao, Daqing Guo

Title: Robust Spatiotemporal Prototype Learning for Spiking Neural Networks

Abstract:
Spiking neural networks (SNNs) leverage their spike-driven nature to achieve high energy efficiency, positioning them as a promising alternative to traditional artificial neural networks (ANNs). The spiking decoder, a crucial component for output, significantly affects the performance of SNNs. However, current rate coding schemes for decoding of SNNs often lack robustness and do not have a training framework suitable for robust learning, while alternatives to rate coding generally produce worse overall performance. To address these challenges, we propose spatiotemporal prototype (STP) learning for SNNs, which uses multiple learnable binarized prototypes for distance-based decoding. In addition, we introduce a cotraining framework that jointly optimizes prototypes and model parameters, enabling mutual adaptation of the two components. STP learning clusters feature centers through supervised learning to ensure effective aggregation around the prototypes, while maintaining enough spacing between prototypes to handle noise and interference. This dual capability results in superior stability and robustness. On eight benchmark datasets with diverse challenges, the STP-SNN model achieves performance comparable to or superior to state-of-the-art methods. Notably, STP learning demonstrates exceptional robustness and stability in multitask experiments. Overall, these findings reveal that STP learning is an effective means of improving the performance and robustness of SNNs.

---

PaperID: 301,   

Authors:  Hongdou Yao, Pengfei Han, Jun Chen, Zheng Wang, Yansheng Qiu, Xiao Wang, Yimin wang, Xiaoyu Chai, Chenglong Cao, Wei Jin

Title: MonOri: Orientation-Guided PnP for Monocular 3-D Object Detection

Abstract:
Monocular 3-D object detection is a challenging task in the field of autonomous driving and has made great progress. However, current monocular image methods tend to incorporate additional information such as pseudolabels to improve algorithm performance while overlooking the geometric relationship between the object’s keypoints, resulting in low performance for occluded object detection. To address this issue, we find that introducing the orientation information of objects in the 3-D detection pipeline can help improve the detection performance of occluded objects. An orientation-guided perspective-n-point (PnP) for monocular 3-D object detection method named MonOri is presented in this article, which uses object’s orientation to guide keypoints’ optimization. Considering the existence of different deformation objects in the scene, we design the feature aggregation detection module (FADM), which consists of the feature focus fusion module (FFFM) and CondConv detection module (CCDM). First, FFFM can highlight signals from irregularly occluded objects, effectively modeling features of elongated and small-sized objects. This module enhances the model’s ability to recognize elongated and small-sized objects in complex scenes. Then, the CCDM is designed to improve the network’s ability to estimate object keypoints’ location regression under occlusion conditions and minimize the network computational overhead. Finally, considering that the unoccluded portions of occluded objects are closely related to the orientation of the objects, an orientation-guided keypoints’ selection module (OGKSM) is proposed to enhance the accuracy of objected optimization for keypoint positions and spatial location inference of the object. Experimental results indicate that the MonOri method achieves competitive results; it is also demonstrated that the orientation information is introduced in the PnP algorithm to estimate the object’s spatial position that can mitigate the impact of occlusion on object detection, thus improving the recognition rate of occluded objects. Our code is available at https://github.com/DL-YHD/MonOri

---

PaperID: 302,   

Authors:  Qing Tian, Jiangsen Yu, Yi Zhao, Wen Li, Zhen Lei

Title: Evidential Deep Learning for Open-Set Active Domain Adaptation

Abstract:
Open-set domain adaptation (OSDA) seeks to transfer knowledge from a labeled source domain to an unlabeled target domain containing novel classes. Traditional OSDA methods rarely account for the uncertainty in predictions and typically require additional training overhead. Evidential deep learning (EDL) transforms the model’s predictions from point estimates to distributions over the probability simplex by replacing the standard softmax output of classification neural networks with Dirichlet distributions. Considering the presence of out-of-distribution novel classes in OSDA and the additional overhead of existing methods, we propose EDL for open-set active domain adaptation (EOSADA). Leveraging EDL, we construct an open-set classifier and employ a two-round selection strategy guided by the data uncertainty of target domain samples and semantic similarity scores with known classes. This strategy balances the selection of samples from known and novel classes while identifying informative samples, thereby maximizing the performance of the model in OSDA scenarios without modifying the model structure and utilizing a limited annotation budget. Extensive experiments demonstrate the superiority of our approach.

---

PaperID: 303,   

Authors:  Ge Zhang, Zhenyu Yang, Jia Wu, Pengfei Jiao, Jian Yang

Title: Enhancing Graph Neural Networks for Out-of-Distribution Graph Detection

Abstract:
Graph neural networks (GNNs) have shown promise in graph classification tasks, but they struggle to identify out-of-distribution (OOD) graphs often encountered in real-world scenarios, posing a significant obstacle for their open-world deployment. Due to the unpredictable nature of the various distributions to which OOD graphs adhere, the challenge of OOD graph detection lies in enabling models to capture distribution differences between in-distribution (ID) and OOD graphs. Current methods often introduce a subset of OOD patterns, such as synthetic OOD graphs, to facilitate learning the discrimination between ID and OOD graphs. However, these OOD patterns may not sufficiently encapsulate the entire range of OOD graphs, leading to inadequate learning of the distribution differences between ID and OOD graphs. In this article, we propose a novel OOD graph detection algorithm, ODGNN. The ODGNN does not expose GNNs to any OOD patterns during model training, thus reducing bias toward specific types of OOD graph samples and enhancing OOD graph detection. The algorithm differentiates graphs by evaluating whether the input graphs conform to established ID graph class-conditioned distributions. Specifically, during model training, the ODGNN integrates a Gaussian encoder into GNNs to characterize ID graph classes using distinct class-conditioned distributions. During inference, OOD graphs are mapped to a representation space distant from ID graphs due to their divergence from any known class-conditioned distribution. Extensive experiments conducted on real-world datasets validate the effectiveness of the ODGNN in enhancing OOD detection performance across various GNN-based graph classification models. The ODGNN also demonstrates superior performance compared to state-of-the-art OOD graph detection competitors.

---

PaperID: 304,   

Authors:  Da Ding, Youquan Wang, Haicheng Tao, Jia Wu, Jie Cao

Title: A Dual-Discriminator Generative Adversarial Network for Anomaly Detection

Abstract:
Multivariate time series anomaly detection has shown potential in various fields, such as finance, aerospace, and security. The fuzzy definition of data anomalies, the complexity of data patterns, and the scarcity of abnormal data samples pose significant challenges to anomaly detection. Researchers have extensively employed autoencoders (AEs) and generative adversarial networks (GANs) in studying time series anomaly detection methods. However, relying on reconstruction error, the AE-based anomaly detection algorithm needs more effective regularization methods, rendering it susceptible to the problem of overfitting. Meanwhile, GAN-based anomaly detection algorithms require high-quality training data, significantly impacting their practical deployment. We propose a novel GAN based on a dual-discriminator structure to address these issues. The model first processes the data with the generator to obtain the reconstruction error and then calculates pseudo-labels to divide the data into two categories. One data category is input into the first discriminator, where a minor loss between the data and its reconstructed counterpart is better. The other data category is input into the second discriminator, where a larger loss between the data and its reconstructed counterpart is better. Through this process, the model can effectively constrain the generator, retaining information on normal data during data reconstruction while discarding information on abnormal data. After conducting experiments on multiple benchmark datasets, the proposed GAN based on a dual-discriminator structure achieved good results in anomaly detection, outperforming several advanced methods. Additionally, the model also performed well in practical transformer data.

---

PaperID: 305,   

Authors:  Runze Yang, Longbing Cao, Jianxun Li, Jie Yang

Title: Variational Hierarchical N-BEATS Model for Long-Term Time-Series Forecasting

Abstract:
Long-term time-series forecasting (LTSF) is gaining increasing attention due to its significant challenges and real-world applications. However, existing studies underexplore the role of hierarchical timestamp information in LTSF. We find this information crucial, as neglecting it may lead to the loss of broader perspectives necessary for understanding hierarchical effects, such as weekly and yearly patterns. Therefore, we propose VH-NBEATS, an interpretable variational hierarchical model that extends the N-BEATS architecture to address the challenges outlined above. VH-NBEATS consists of two blocks: the hierarchical timestamp block and the harmonic seasonal block, which are designed to capture hierarchical seasonal and trending effects. To tackle the high variability often observed in time series, VH-NBEATS incorporates a variational autoencoder (VAE), significantly enhancing the standard deterministic approach. The experimental results are evaluated on seven real-world datasets, demonstrating state-of-the-art (SOTA) performance for LTSF. We also prove that the hierarchical timestamp block can enable plug-and-play with any methods, such as PatchTST, Informer, and DLinear, for better performance.

---

PaperID: 306,   

Authors:  Qian Zhao, Gang Li, Bin He, Runjie Shen

Title: Deep Learning for Low-Light Vision: A Comprehensive Survey

Abstract:
Visual recognition in low-light environments is a challenging problem since degraded images are the stacking of multiple degradations (noise, low light and blur, etc.). It has received extensive attention from academia and industry in the era of deep learning. Existing surveys focus on low-light image enhancement (LLIE) methods and normal-light visual recognition methods, while few comprehensive surveys of low-light-related vision tasks. This article provides a comprehensive survey of the latest advancements in low-light vision, including methods, datasets, and evaluation metrics, in two aspects: visual quality-driven and recognition quality-driven. On the visual quality-driven aspect, we survey a large number of very recent LLIE methods. On the recognition quality-driven aspect, we survey low-light object detection techniques in the deep learning era using more intuitive categorization method. Furthermore, a quantitative benchmarking of different methods is conducted on several widely adopted low-light vision-related datasets. Finally, we discuss the challenges that exist in low-light vision and future directions worth exploring. We provide a public website that will continue to track developments in this promising field.

---

PaperID: 307,   

Authors:  Jianan Li, Huan Chen, Wangcai Zhao, Rui Chen, Tingfa Xu

Title: Mixed-Granularity Implicit Representation for Continuous Hyperspectral Compressive Reconstruction

Abstract:
Hyperspectral images (HSIs) are crucial across numerous fields but are hindered by the long acquisition times associated with traditional spectrometers. The coded aperture snapshot spectral imaging (CASSI) system mitigates this issue through a compression technique that accelerates the acquisition process. However, reconstructing HSIs from compressed data presents challenges due to fixed spatial and spectral resolution constraints. This study introduces a novel method using implicit neural representation (INR) for continuous HSI reconstruction. We propose the mixed-granularity implicit representation (MGIR) framework, which includes a hierarchical spectral–spatial implicit encoder (HSSIE) for efficient multiscale implicit feature extraction. This is complemented by a mixed-granularity local feature aggregator (MGLFA) that adaptively integrates local features across scales, combined with a decoder that merges coordinate information for precise reconstruction. By leveraging INRs, the MGIR framework enables reconstruction at any desired spatial–spectral resolution, significantly enhancing the flexibility and adaptability of the CASSI system. Extensive experimental evaluations confirm that our model produces reconstructed images at arbitrary resolutions and matches the state-of-the-art methods across varying spectral–spatial compression ratios (CRs). The code will be released at https://github.com/chh11/MGIR.

---

PaperID: 308,   

Authors:  Xing Wei, Zhaoxin Ji, Fan Yang, Chong Zhao, Bin Wen, Yang Lu

Title: Unsupervised Domain Adaptation via Bidirectional Transmission Generator Self-Training

Abstract:
Unsupervised domain adaptation (UDA) aims to transfer knowledge from the labeled source domain to the fully unlabeled target domain, thus improving the classification performance of the target domain. Recently, self-training methods have shown their effectiveness on UDA. It iteratively trains target data using the generated target pseudo-labels. However, the feature space for generating pseudo-labels contains a large amount of source information, which traps the model in the source domain, making it challenging for the generator to learn discriminative features of the target domain. In this article, we propose a self-training domain adaptation (DA) model with bidirectional transmission generators (BDTGs). Specifically, we design a bidirectional transmission structure for generators, using exponential moving average (EMA) as the bridge between two generators. The structure has two advantages: 1) by transmitting weight parameters to each other during the training process, it promotes the shift of the feature space, thereby alleviating the difficulty of the model in adapting target domain features and 2) the transmission disturbs the classification boundary and is able to expose unreliable target samples near the boundary. We design a cosine similarity-based filter to identify such samples, to reduce the influence of noisy pseudo-labels with incorrect semantic information on the model. Extensive experiments conducted on five benchmark UDA datasets show that our approach has superior classification performance.

---

PaperID: 309,   

Authors:  Zhewei Zhang, Yujun Cheng, Junyu Shen, Xuejing Li, Shengjin Wang

Title: Probably Approximately Correct Bayes Meta-Learning With Parameterized-Bounded Guarantees

Abstract:
In meta-learning, the learner extracts knowledge from the observed tasks and quickly adapts to unseen future tasks. We provide a novel and rigorous-analyzed probably approximately correct Bayes (PAC-Bayes) meta-learning method with parameterized bounds, which learns a posterior distribution from given priors and the data samples. The proposed method is designed to improve generalization stabilities with tighter bound guarantees. We prove that the proposed PAC-Bayes bound of the meta-learner is tighter than previous work under a given condition in a rigorous theoretical way. An explicit theoretical analysis of the generalization errors is also given based on the proposed meta-learning method. Using the proposed bound in our work, we deduce an optimal objective function of the meta-learner that should be minimized during the meta-training process. We validate our theoretical hypothesis by conducting synthetic and real-world environments for meta-learning. Both rigorous proofs and experimental results reveal that our method yields state-of-the-art performances under a variety of meta-learning tasks in terms of accuracy and uncertainty robustness.

---

PaperID: 310,   

Authors:  Jing-Cheng Pang, Tian Xu, Shengyi Jiang, Yu-Ren Liu, Yang Yu

Title: Reinforcement Learning With Sparse-Executing Action via Sparsity Regularization

Abstract:
Reinforcement learning (RL) has demonstrated impressive performance in decision-making tasks like embodied control, autonomous driving, and financial trading. In many decision-making tasks, the agents often encounter the problem of executing actions under limited budgets. However, classic RL methods typically overlook the challenges posed by such sparse-executing actions. They operate under the assumption that all actions can be taken for an unlimited number of times, both in the formulation of the problem and in the development of effective algorithms. To tackle the issue of limited action execution in RL, this article first formalizes the problem as a sparse action Markov decision process (SA-MDP), in which specific actions in the action space can only be executed for a limited time. Then, we propose a policy optimization algorithm, Action Sparsity REgularization (ASRE), which adaptively handles each action with a distinct preference. ASRE operates through two steps. First, ASRE evaluates action sparsity by constrained action sampling. Following this, ASRE incorporates the sparsity evaluation into policy learning by way of an action distribution regularization. We provide theoretical identification that validates the convergence of ASRE to a regularized optimal value function. Experiments on tasks with known sparse-executing actions, where classical RL algorithms struggle to train policy efficiently, show that ASRE effectively constrains the action sampling and outperforms baselines. Moreover, we present that ASRE can generally improve the performance in Atari games, demonstrating its broad applicability.

---

PaperID: 311,   

Authors:  Yabin Zhang, Jiehong Lin, Ruihuang Li, Kui Jia, Lei Zhang

Title: Point-DAE: Denoising Autoencoders for Self-Supervised Point Cloud Learning

Abstract:
Masked autoencoder (MAE) has demonstrated its effectiveness in self-supervised point cloud learning. Considering that masking is a kind of corruption, in this work we explore a more general denoising autoencoder for point cloud learning (Point-DAE) by investigating more types of corruptions beyond masking. Specifically, we degrade the point cloud with certain corruptions as input, and learn an encoder-decoder model to reconstruct the original point cloud from its corrupted version. Three corruption families (i.e., density/masking, noise, and affine transformation) and a total of 14 corruption types are investigated with traditional non-Transformer encoders. Besides the popular masking corruption, we identify another effective corruption family, i.e., affine transformation. The affine transformation disturbs all points globally, which is complementary to the masking corruption where some local regions are dropped. We also validate the effectiveness of affine transformation corruption with the Transformer backbones, where we decompose the reconstruction of the complete point cloud into the reconstructions of detailed local patches and rough global shape, alleviating the position leakage problem in the reconstruction. Extensive experiments on tasks of object classification, few-shot learning, robustness testing, part segmentation, and 3-D object detection validate the effectiveness of the proposed method. The codes are available at https://github.com/YBZh/Point-DAE

---

PaperID: 312,   

Authors:  Yiming Yang, Weipeng Hu, Qiaolin He, Haifeng Hu

Title: Dynamic Modality-Camera-Invariant Clustering for Unsupervised Visible-Infrared Person Re-Identification

Abstract:
Unsupervised learning visible–infrared person re-identification (USL-VI-ReID) offers a more flexible and cost-effective alternative compared to supervised methods. This field has gained increasing attention due to its promising potential. Existing methods simply cluster modality-specific samples and employ strong association techniques to achieve instance-to-cluster or cluster-to-cluster cross-modality associations. However, they ignore cross-camera differences, leading to noticeable issues with excessive splitting of identities. Consequently, this undermines the accuracy and reliability of cross-modal associations. To address these issues, we propose a novel dynamic modality–camera-invariant clustering (DMIC) framework for USL-VI-ReID. Specifically, our DMIC naturally integrates modality–camera-invariant expansion (MIE), dynamic neighborhood clustering (DNC), and hybrid modality contrastive learning (HMCL) into a unified framework, which eliminates both the cross-modality and cross-camera discrepancies in clustering. MIE fuses intermodal and intercamera distance coding to bridge the gaps between modalities and cameras at the clustering level. DNC employs two dynamic search strategies to refine the network’s optimization objective, transitioning from improving discriminability to enhancing cross-modal and cross-camera generalizability. Moreover, HMCL is designed to optimize instance- and cluster-level distributions. Memories for intramodality and intermodality training are updated using randomly selected samples, facilitating real-time exploration of modality-invariant representations. Extensive experiments have demonstrated that our DMIC addresses the limitations present in current clustering approaches and achieves competitive performance, which significantly reduces the performance gap with supervised methods.

---

PaperID: 313,   

Authors:  Wuhao Wang, Zhiyong Chen, Lepeng Zhang

Title: Multistate Temporal Difference Target for Model-Free Reinforcement Learning

Abstract:
Temporal difference (TD) learning is a fundamental technique in reinforcement learning that updates value function estimates for states or state-action pairs using a TD target. This target represents an improved estimate of the true value by incorporating both immediate rewards and the estimated value of subsequent states. We propose an enhanced multistate TD (MSTD) target that utilizes multiple subsequent states for a more accurate value function estimation compared to traditional TD learning, which relies on a single subsequent state. Building on this new MSTD concept, we develop actor-critic algorithms that include the management of replay buffers in two modes and integrate with deep deterministic policy optimization (DDPG) and soft actor-critic (SAC). Numerical experiment results demonstrate that algorithms employing the MSTD target improve learning performance compared to traditional methods. In addition, we analyze the convergence of Q-learning with MSTD.

---

PaperID: 314,   

Authors:  Zixin Zhang, Fan Qi, Changsheng Xu

Title: Knowledge-Guided Label Distribution Calibration for Federated Affective Computing

Abstract:
The federated learning (FL) paradigm can significantly solve the rising public concern about data privacy in affective computing. However, conventional FL methods perform poorly due to the uniqueness of the task, as the personalized emotion data vary from client to client. To resolve the privacy-utility paradox, this work proposes a framework that largely improves federated affective computing (FAC) via calibrating the global feature space and communicating privacy-agnostic auxiliary information. The framework consists of two components: first, an emotion hemisphere (EH) representation structure is proposed, which utilizes emotional prior knowledge to unify the emotion global feature space of different clients. Second, the server uses the normalized parameter importance matrix to guide the model aggregation. It retains crucial parameters for individual local models, thereby alleviating the slow convergence problem in the global model caused by the skewed label distribution. The proposed framework yields significant performance gains, and extensive experiments on three emotion datasets demonstrate the effectiveness and the practicality of our approach.

---

PaperID: 315,   

Authors:  Hui Chen, Hengyu Liu, Zhangkai Wu, Xuhui Fan, Longbing Cao

Title: FedSI: Federated Subnetwork Inference for Efficient Uncertainty Quantification

Abstract:
While deep neural networks (DNNs)-based personalized federated learning (PFL) is demanding for addressing data heterogeneity and shows promising performance, existing methods for federated learning (FL) suffer from efficient systematic uncertainty quantification. The Bayesian DNNs-based PFL is usually questioned of either oversimplified model structures or high computational and memory costs. In this article, we introduce FedSI, a novel Bayesian DNNs-based subnetwork inference (SI) PFL framework. FedSI is simple and scalable by leveraging Bayesian methods to incorporate systematic uncertainties effectively. It implements a client-specific SI mechanism, selects network parameters with large variance to be inferred through posterior distributions, and fixes the rest as deterministic ones. FedSI achieves fast and scalable inference while preserving the systematic uncertainties to the fullest extent. Extensive experiments on four different benchmark datasets demonstrate that FedSI outperforms existing Bayesian and non-Bayesian FL baselines in heterogeneous FL scenarios.

---

PaperID: 316,   

Authors:  Wenjue He, Zheng Zhang, Xiaofeng Zhu

Title: Dual-Correlation-Guided Anchor Learning for Scalable Incomplete Multi-View Clustering

Abstract:
Efficiently learning informative yet compact representations from heterogeneous data remains challenging in incomplete multi-view clustering (IMC). The prevalent resource-efficient IMC models excel in constructing small-size anchors for fast similarity learning and data partition. However, existing anchor-based methods still suffer from shared deficiencies: 1) unstable and less informative anchor generation by random anchor selection or clueless learning and 2) imbalanced coherence and versatility capabilities of the learned anchors among different views. To mitigate these issues, we propose a novel dual-correlation-guided anchor learning (DCGA) method for scalable IMC, which learns informative anchor spaces to simultaneously incorporate both intra-view and inter-view correlations. Specifically, the intra-view anchor space is constructed and stabilized by compressing the view-specific data under the guidance of the conceived anchors as a bottleneck (A3B) strategy, with a strict theoretic analysis. Importantly, we, for the first time, build an unsupervised anchor learning scheme for incomplete multi-view data under the guidance of the bottleneck of information flow with the well-defined IB principle. As such, our model can simultaneously eliminate information redundancy and preserve the versatile knowledge derived from each view. Moreover, to endow the coherence of the learned anchors, an informative anchor constraint (IAC) is imposed to align the anchor spaces across different views. Extensive experiments on seven datasets against 11 state-of-the-art IMC methods validate the effectiveness and efficiency of our method. Code is available at https://github.com/DarrenZZhang/TNNLS25-DCGA

---

PaperID: 317,   

Authors:  Liang-Bo Ning, Zuo-Wei Zhang, Weiping Ding, Dian Shao, Yining Zhu

Title: Multilevel Distribution Alignment for Multisource Universal Domain Adaptation

Abstract:
The multisource universal domain adaptation (MSUDA) relaxes the constraints between the source and target domains, enabling the transfer of knowledge between domains without any restrictions on the number of source domains and the existence of unknown (private) categories. However, identifying the unknown samples in the target domain is extremely challenging since there are no available samples with the same label in source domains. Another immense challenge lies in extracting domain-invariant features for knowledge transfer since there are distribution discrepancies between each source and target domain. In this article, we propose the multirepresentation DA network (MRDAN) to classify the unlabeled targets by harnessing multiple source domains with nonidentical label sets. First, we propose a threshold-free conflict-based predictions with uncertainty (CPU) module, which comprehensively mines the complementary knowledge from different source domains to identify both known and unknown samples simultaneously. To accurately extract the domain-invariant features for recognizing known and unknown samples, a multilevel distribution alignment (MLDA) strategy is introduced to decrease the distribution discrepancy between multiple domains with nonidentical category spaces progressively. Finally, comprehensive experiments conducted on three commonly used datasets demonstrate the effectiveness of the proposed MRDAN in recognizing both known and unknown samples.

---

PaperID: 318,   

Authors:  Na Li, Chunyi Zhou, Yansong Gao, Hui Chen, Zhi Zhang, Boyu Kuang, Anmin Fu

Title: Machine Unlearning: Taxonomy, Metrics, Applications, Challenges, and Prospects

Abstract:
Personal digital data is a critical asset, and governments worldwide have enforced laws and regulations to protect data privacy. Data users have been endowed with the “right to be forgotten” (RTBF) of their data. In the course of machine learning (ML), the forgotten right requires a model provider to delete user data and its subsequent impact on ML models upon user requests. Machine unlearning (MU) emerges to address this, which has garnered ever-increasing attention from both industry and academia. Specifically, MU allows model providers to eliminate the influence of unlearned data without retraining the model from scratch, ensuring the model behaves as if it never encountered this data. While the area has developed rapidly, there is a lack of comprehensive surveys to capture the latest advancements. Recognizing this shortage, we conduct an extensive exploration to map the landscape of MU including the (fine-grained) taxonomy of unlearning algorithms under centralized and distributed settings, debate on approximate unlearning, verification and evaluation metrics, and challenges and solutions across various applications. We also focus on the motivations, challenges, and specific methods for deploying unlearning in large language models (LLMs), as well as the potential attacks targeting unlearning processes. The survey concludes by outlining potential directions for future research, hoping to serve as a beacon for interested scholars.

---

PaperID: 319,   

Authors:  Debo Cheng, Jiuyong Li, Lin Liu, Ziqi Xu, Weijia Zhang, Jixue Liu, Thuc Duy Le

Title: Disentangled Representation Learning for Causal Inference With Instruments

Abstract:
Latent confounders are a fundamental challenge for inferring causal effects from observational data. The instrumental variable (IV) approach is a practical way to address this challenge. Existing IV-based estimators need a known IV or other strong assumptions, such as the existence of two or more IVs in the system, which limits the application of the IV approach. In this article, we consider a relaxed requirement, which assumes there is an IV proxy in the system without knowing which variable is the proxy. We propose a variational autoencoder (VAE)-based disentangled representation learning method to learn an IV representation from a dataset with latent confounders and then utilize the IV representation to obtain an unbiased estimation of the causal effect from the data. Extensive experiments on synthetic and real-world data have demonstrated that the proposed algorithm outperforms the existing IV-based estimators and VAE-based estimators.

---

PaperID: 320,   

Authors:  Sajad Darabi, Piotr Bigaj, Dawid Majchrowski, Artur Kasymov, Pawel M. Morkisz, Alex Fit-Florea

Title: A Framework for Large-Scale Synthetic Graph Dataset Generation

Abstract:
Recently, there has been increasing interest in developing and deploying deep graph learning algorithms for various tasks, such as fraud detection and recommender systems. However, there is a limited number of publicly available graph-structured datasets, most of which are small compared with production-sized applications or limited in their application domain. In this work, we tackle this shortcoming by proposing a synthetic graph generation tool that enables scaling datasets to production-size graphs with trillions of edges and billions of nodes. The proposed method comprises a series of parametric models that can either be randomly initialized or fit to proprietary datasets. These models can then be released to researchers to study graph methods on the synthetic data, facilitating prototype development and novel applications. We demonstrate the generalizability of the framework across various datasets, mimicking their structural and feature distributions, as well as the ability to scale them to varying sizes, demonstrating their usefulness for benchmarking and model development. Code can be found on GitHub.

---

PaperID: 321,   

Authors:  Zhiang Liu, Yang Liu, Yongchun Fang

Title: Diffusion Model-Based Path Follower for a Salamander-Like Robot

Abstract:
Salamander-like robots, renowned for their versatile locomotion, present unique challenges in the development of effective path-following controllers due to their distinctive movement patterns and complex body structures. Conventional path-following controllers, while effective for various bionic robots, struggle with the intricate modeling for salamander-like robots and often require laborious manual tuning. Conversely, learning-based methods offer promising alternatives but face issues such as reliance on environmental interactions, short-sighted prediction, and irrational design of state space and reward function. To overcome these limitations, this article proposes a diffusion model-based hierarchical control framework that treats path tracking as a sequence generation problem. The diffusion model’s capability to model joint distributions of state, action, and reward sequences enables it to outperform other learning-based approaches in efficient data utilization, stable training, and long-horizon dependency modeling. Our framework integrates a high-level policy driven by guided diffusion with a low-level controller for parsing commands into executable movements via inverse kinematics, reducing the action space and improving learning efficiency. In addition, we design a more reasonable state space and reward function tailored to the path-following task, addressing shortcomings in prior learning-based controllers. Furthermore, we optimize the diffusion model (DM) by developing lightweight network architectures and incorporating advanced attention mechanisms, to ensure its practical deployment on physical robots with limited computational resources, without compromising performance. Extensive simulations and real-world experiments demonstrate the framework’s effectiveness, efficiency, and robustness in diverse path-following tasks for salamander-like robots, marking a significant advancement in the control of biomimetic robots.

---

PaperID: 322,   

Authors:  Bosheng Qin, Juncheng Li, Siliang Tang, Yueting Zhuang

Title: DBA: Efficient Transformer With Dynamic Bilinear Low-Rank Attention

Abstract:
Many studies have aimed to improve Transformer model efficiency using low-rank-based methods that compress sequence length with predetermined or learned compression matrices. However, these methods fix compression coefficients for tokens in the same position during inference, ignoring sequence-specific variations. They also overlook the impact of hidden state dimensions on efficiency gains. To address these limitations, we propose dynamic bilinear low-rank attention (DBA), an efficient and effective attention mechanism that compresses sequence length using input-sensitive dynamic compression matrices. DBA achieves linear time and space complexity by jointly optimizing sequence length and hidden state dimension while maintaining state-of-the-art performance. Specifically, we demonstrate through experiments and the properties of low-rank matrices that sequence length can be compressed with compression coefficients dynamically determined by the input sequence. In addition, we illustrate that the hidden state dimension can be approximated by extending the Johnson-Lindenstrauss lemma, thereby introducing only a small amount of error. DBA optimizes the attention mechanism through bilinear forms that consider both the sequence length and hidden state dimension. Moreover, the theoretical analysis substantiates that DBA excels at capturing high-order relationships in cross-attention problems. Experimental results across different tasks with varied sequence length conditions demonstrate that DBA achieves state-of-the-art performance compared to several robust baselines. DBA also maintains higher processing speed and lower memory usage, highlighting its efficiency and effectiveness across diverse applications.

---

PaperID: 323,   

Authors:  Guanghui Zhu, Zhennan Zhu, Hongyang Chen, Chunfeng Yuan, Yihua Huang

Title: HAGNN: Hybrid Aggregation for Heterogeneous Graph Neural Networks

Abstract:
Heterogeneous graph neural networks (GNNs) have been successful in handling heterogeneous graphs. In existing heterogeneous GNNs, meta-path plays an essential role. However, recent work pointed out that a simple homogeneous graph model without a meta-path can also achieve comparable results, which calls into question the necessity of a meta-path. In this article, we first present the intrinsic difference between meta-path-based and meta-path-free models, i.e., how to select neighbors for node aggregation. Then, we propose a novel framework to utilize the rich type of semantic information in heterogeneous graphs comprehensively, namely, hybrid aggregation for heterogeneous GNNs (HAGNNs). The core of HAGNN is to leverage the meta-path neighbors and the directly connected neighbors simultaneously for node aggregations. HAGNN divides the overall aggregation process into two phases: meta-path-based intratype aggregation and meta-path-free intertype aggregation. During the intratype aggregation phase, we propose a new data structure called a fused meta-path graph and perform structural semantic aware aggregation on it. Finally, we combine the embeddings generated by each phase. Compared with existing heterogeneous GNN models, HAGNN can take full advantage of the heterogeneity in heterogeneous graphs. Extensive experimental results on node classification, node clustering, and link prediction tasks show that HAGNN outperforms the existing modes, demonstrating the effectiveness and efficiency of HAGNN.

---

PaperID: 324,   

Authors:  Xu Chen, Yahong Han, Changlin Li, Xiaojun Chang, Yifan Sun, Yi Yang

Title: A Static-Dynamic Composition Framework for Efficient Action Recognition

Abstract:
The dynamic inference, which adaptively allocates computational budgets for different samples, is a prevalent approach for achieving efficient action recognition. Current studies primarily focus on a data-efficient regime that reduces spatial or temporal redundancy, or their combination, by selecting partial video data, such as clips, frames, or patches. However, these approaches often utilize fixed and computationally expensive networks. From a different perspective, this article introduces a novel model-efficient regime that addresses network redundancy by dynamically selecting a partial network in real time. Specifically, we acknowledge that different channels of the neural network inherently contain redundant semantics either spatially or temporally. Therefore, by decreasing the width of the network, we can enhance efficiency while compromising the feature capacity. To strike a balance between efficiency and capacity, we propose the static-dynamic composition (SDCOM) framework, which comprises a static network with a fixed width and a dynamic network with a flexible width. In this framework, the static network extracts the primary feature with essential semantics from the input frame and simultaneously evaluates the gap toward achieving a comprehensive feature representation. Based on these evaluation results, the dynamic network activates a minimal width to extract a supplementary feature that fills the identified gap. We optimize the dynamic feature extraction through the employment of the slimmable network mechanism and a novel meta-learning scheme introduced in this article. Empirical analysis reveals that by combining the primary feature with an extremely lightweight supplementary feature, we can accurately recognize a large majority of frames (76%~92%). As a result, our proposed SDCOM significantly enhances recognition efficiency. For instance, on ActivityNet, FCVID, and Mini-Kinetics datasets, SDCOM saves 90% of the baseline’s floating point operations (FLOPs) while achieving comparable or superior accuracy when compared with state-of-the-art methods.

---

PaperID: 325,   

Authors:  Pingping Pan, Yunjian Zhang, You Li, Yishan Ye, Wei He, Yutao Zhu, Renzhong Guo

Title: Interpretable Optimization-Inspired Deep Network for Off-Grid Frequency Estimation

Abstract:
The accuracy of on-grid frequency estimation methods suffers from the quantization error of discrete grids. In this article, a deep unfolded network for off-grid frequency estimation is proposed, dubbed OGFreq. In the OGFreq, there exist two kinds of variables. One is the batch-oriented dictionary for frequency-domain transform, and the other one is the instance-specific on-grid frequency and off-grid bias. As the dictionary is required to be universally applicable among all observed signals, network layers are designed and network weights are updated to approximate the transform bases in a data-driven way. Besides, instance-specific on-grid frequencies and off-grid biases are solved by unfolding the iterative soft-threshold algorithm (ISTA). In addition, the instance-specific hyperparameters for sparsity in ISTA are obtained by an encoder-decoder soft-threshold (EDS) module with the attention mechanism. In this way, the dictionary, on-grid frequency, and off-grid bias are learned in a unified data-driven framework. Numerical experiments show that the OGFreq obtains 4% lower false negative rate (FNR) when the SNR is 20 dB. Moreover, the computational complexity is one order of magnitude lower than the iteration-based off-grid frequency estimation methods. Finally, the robustness of the OGFreq is discussed when extended to the impulse noise and damped signals.

---

PaperID: 326,   

Authors:  Xinhang Wan, Jiyuan Liu, Hao Yu, Qian Qu, Ao Li, Xinwang Liu, Ke Liang, Zhibin Dong, En Zhu

Title: Contrastive Continual Multiview Clustering With Filtered Structural Fusion

Abstract:
Multiview clustering thrives in applications where views are collected in advance by extracting consistent and complementary information among views. However, it overlooks scenarios where data views are collected sequentially, i.e., real-time data. Due to privacy issues or memory burden, previous views are not available with time in these situations. Some methods are proposed to handle it but are trapped in a stability-plasticity dilemma. In specific, these methods undergo a catastrophic forgetting of prior knowledge when a new view is attained. Such a catastrophic forgetting problem (CFP) would cause the consistent and complementary information hard to get and affect the clustering performance. To tackle this, we propose a novel method termed contrastive continual multiview clustering with filtered structural fusion (CCMVC-FSF). Precisely, considering that data correlations play a vital role in clustering and prior knowledge ought to guide the clustering process of a new view, we develop a data buffer to store filtered structural information and utilize it to guide the generation of a robust partition matrix via contrastive learning. Additionally, to address the high complexity involved in acquiring and storing structural information, we propose a sampling strategy called clustering then sample. Furthermore, we theoretically connect CCMVC-FSF with semisupervised learning and knowledge distillation. Extensive experiments exhibit the excellence of the proposed method. Our code is publicly available at https://github.com/wanxinhang/CCMVC-FSF/.

---

PaperID: 327,   

Authors:  Heng Tian, Ziqiu Chi, Zhe Wang, Wei Guo, Mengping Yang, Xinlei Xu

Title: Transductive Parameter-Free Propagation Framework for Few-Shot Distribution Rectification

Abstract:
Few-shot learning (FSL) is challenging due to the scarce labeled novel-class data. Researchers have to train the embedding function with auxiliary base-class data to obtain the novel-class embeddings. However, the domain gap makes the novel-class embedding unsatisfactory, as the novel class and the base class are disjoint. Recent studies prove that embedding rectification shows great potential, introduces miscellaneous variants, and achieves similar performances. Nonetheless, while each method demonstrates unique strengths, they often address distinct challenges in isolation, limiting their applicability in more complex or diverse scenarios. In this article, we take a closer look at these methods and hypothesize that a general embedding rectification framework is more essential to the model’s performance. To verify our observation, we propose: 1) a distribution propagation (DisP) layer distinguishes the inter-class margin and increases intra-class aggregation, performing the task-level rectification; and 2) a prototype propagation (ProtoP) layer moves the prototype toward the ideal class center, applying the prototype-query level rectification. Our framework aims to maximize the actual data distribution. Although pseudo-labeling proves effective in achieving this goal, a significant challenge is ensuring the reliable retention of only high-confidence predictions. To overcome this, we introduce a distribution-based pseudo-labeling method pseudo-query upgrade (PseQUp) that provides more reliable pseudo-labeling samples without relying on confidence scores. We evaluate the proposed method in both transfer learning and meta-learning scenarios. Empirical experiments show the applicable and plug-and-play ability of the proposed methods.

---

PaperID: 328,   

Authors:  Wenqi Yang, Chang Tang, Xinwang Liu, Guanghui Yue, Yuanyuan Liu, Changqing Zhang, En Zhu

Title: Smooth Multiple Kernel k-Means via Underlying Graph Filtering

Abstract:
Clustering has attracted more and more attention as one of the most fundamental techniques in the field of unsupervised learning. To deal with nonlinear problems, clustering methods have been extended to the kernel version. As a traditional kernel clustering algorithm, multiple kernel k-means (MKKM) aims to learn clustering results from a consensus kernel obtained by combining a set of predefined kernels optimally. However, we observe that the existing MKKM algorithm and its variants insufficiently consider the noise that existed in kernel space and the underlying structure of kernelized data points. To this end, we propose a novel smooth MKKM via underlying graph filtering (SMKKM-UGF) to learn the smooth representations of kernelized data points through their nearby nodes in the underlying graph. In particular, different from the common graph filter, we jointly update the graph filter while learning the smooth kernel, so that the graph filter can be guaranteed to adapt to the updating kernel space constantly. Besides, an iterative algorithm with proven convergence is designed to solve the resultant optimization problem. Extensive experiments have been performed on numerous benchmark datasets, whose results prove the superiority of the proposed SMKKM-UGF compared to the other state-of-the-art clustering methods. The demo code of this work is publicly available at https://github.com/wqyang23/SMKKM-UGF.git.

---

PaperID: 329,   

Authors:  Yuhao Zhou, Yuxin Tian, Mingjia Shi, Yuanxi Li, Yanan Sun, Qing Ye, Jiancheng Lv

Title: E-3SFC: Communication-Efficient Federated Learning With Double-Way Features Synthesizing

Abstract:
The exponential growth in model sizes has significantly increased the communication burden in federated learning (FL). Existing methods to alleviate this burden by transmitting compressed gradients often face high compression errors, which slow down the model’s convergence. To simultaneously achieve high compression effectiveness and lower compression errors, we study the gradient compression problem from a novel perspective. Specifically, we propose a systematical algorithm termed extended single-step synthetic features compressing (E-3SFC), which consists of three subcomponents, i.e., the single-step synthetic features compressor (3SFC), a double-way compression (DWC) algorithm, and a communication budget scheduler (BS). First, we regard the process of gradient computation of a model as decompressing gradients from corresponding inputs, while the inverse process is considered as compressing the gradients. Based on this, we introduce a novel gradient compression method termed 3SFC, which utilizes the model itself as a decompressor, leveraging training priors such as model weights and objective functions. The 3SFC compresses raw gradients into tiny synthetic features in a single-step simulation, incorporating error feedback (EF) to minimize overall compression errors. To further reduce communication overhead, 3SFC is extended to E-3SFC, allowing DWC and dynamic communication budget scheduling. Our theoretical analysis under both strongly convex and nonconvex conditions demonstrates that 3SFC achieves linear and sublinear convergence rates with aggregation noise. Extensive experiments across six datasets and six models reveal that 3SFC outperforms the state-of-the-art methods by up to 13.4% while reducing communication costs by 111.6 times. These findings suggest that 3SFC can significantly enhance communication efficiency in FL without compromising model performance.

---

PaperID: 330,   

Authors:  Bin Sun, Zuxiang Long, Ziyu Ma, Shutao Li

Title: Cascade Fusion and Correlation Enhancement for Knowledge Distillation

Abstract:
Knowledge distillation (KD) improves the performance of a compact student network by transferring learned knowledge from a cumbersome teacher network. In the existing approaches, the multiscale feature knowledge is transferred via densely connected paths, which increases the optimization difficulty. Moreover, correlations among the labels are neglected despite their capability to enhance the intraclass similarity of samples. To solve these issues, we propose cascade fusion and correlation enhancement for KD (CC-KD). The multiscale feature knowledge is transferred via much simpler paths, which are constructed by fusing features of different scales with cross-scale attention (CSA) in a cascade manner, thereby reducing the optimization difficulty. On the other hand, the relational knowledge of teacher logits is further enhanced by correlations of the corresponding labels, so that the student can produce more similar logits for the samples in the same category. Extensive experimental results on five public datasets (i.e., CIFAR100/10, ImageNet, RAF-DB, and FERPlus) indicate superior performance of the proposed method over several state-of-the-arts (SOTAs). More specifically, our method obtains an accuracy of 71.70% on ImageNet and achieves a new record of 90.20% on RAF-DB with fewer calculations and parameters.

---

PaperID: 331,   

Authors:  Vittorio Mazzia, Alessandro Pedrani, Andrea Caciolai, Kay Rottmann, Davide Bernardi

Title: A Survey on Knowledge Editing of Neural Networks

Abstract:
Deep neural networks are becoming increasingly pervasive in academia and industry, matching and surpassing human performance in a wide variety of fields and related tasks. However, just as humans, even the largest artificial neural networks (ANNs) make mistakes, and once-correct predictions can become invalid as the world progresses in time. Augmenting datasets with samples that account for mistakes or up-to-date information has become a common workaround in practical applications. However, the well-known phenomenon of catastrophic forgetting poses a challenge in achieving precise changes in the implicitly memorized knowledge of neural network parameters, often requiring a full model retraining to achieve desired behaviors. That is expensive, unreliable, and incompatible with the current trend of large self-supervised pretraining, making it necessary to find more efficient and effective methods for adapting neural network models to changing data. To address this need, knowledge editing (KE) is emerging as a novel area of research that aims to enable reliable, data-efficient, and fast changes to a pretrained target model, without affecting model behaviors on previously learned tasks. In this survey, we provide a brief review of this recent artificial intelligence field of research. We first introduce the problem of editing neural networks, formalize it in a common framework and differentiate it from more notorious branches of research such as continuous learning. Next, we provide a review of the most relevant KE approaches and datasets proposed so far, grouping works under four different families: regularization techniques, meta-learning, direct model editing, and architectural strategies. Finally, we outline some intersections with other fields of research and potential directions for future works.

---

PaperID: 332,   

Authors:  Brian B. Moser, Arundhati S. Shanbhag, Federico Raue, Stanislav Frolov, Sebastian Palacio, Andreas Dengel

Title: Diffusion Models, Image Super-Resolution, and Everything: A Survey

Abstract:
Diffusion models (DMs) have disrupted the image super-resolution (SR) field and further closed the gap between image quality and human perceptual preferences. They are easy to train and can produce very high-quality samples that exceed the realism of those produced by previous generative methods. Despite their promising results, they also come with new challenges that need further research: high computational demands, comparability, lack of explainability, color shifts, and more. Unfortunately, entry into this field is overwhelming because of the abundance of publications. To address this, we provide a unified recount of the theoretical foundations underlying DMs applied to image SR and offer a detailed analysis that underscores the unique characteristics and methodologies within this domain, distinct from broader existing reviews in the field. This article articulates a cohesive understanding of DM principles and explores current research avenues, including alternative input domains, conditioning techniques, guidance mechanisms, corruption spaces, and zero-shot learning approaches. By offering a detailed examination of the evolution and current trends in image SR through the lens of DMs, this article sheds light on the existing challenges and charts potential future directions, aiming to inspire further innovation in this rapidly advancing area.

---

PaperID: 333,   

Authors:  Pengyu Zhang, Hao Ju, Weihua He, Yaoyuan Wang, Ziyang Zhang, Shengming Li, Dong Wang, Huchuan Lu, Xu Jia

Title: Event-Assisted Recurrent Network for Arbitrary-Temporal-Scale Blurry Image Unfolding

Abstract:
Recovering a sequence of latent sharp frames from a motion-blurred image is a challenging task. The bio-inspired event camera, which produces an event stream with high temporal resolution, has been exploited to promote the recovery performance. However, recovering sharp sequences with arbitrary temporal scales has been ignored for a long time. Existing works can only recover a fixed number of latent frames from a blurry image once they are trained. In this work, we propose an event-assisted blurry image unfolding framework that can work across arbitrary temporal scales. A bi-directional recurrent network is employed to encode events corresponding to each latent frame, which gathers information over all events in the exposure time. Features of both the blurry image and events are fused together and fed to a bi-directional latent sequence decoder (BiLSD) to produce a sequence of latent sharp frames. Extensive experiments show that the proposed method not only performs favorably against state-of-the-art methods in recovering a fixed number of frames from a blurry image but can be well generalized to arbitrary-temporal-scale blurry image unfolding.

---

PaperID: 334,   

Authors:  Yiming Wu, Chenduo Ying, Ning Zheng, Wen-An Zhang, Shanying Zhu

Title: Whole-Process Privacy-Preserving and Sybil-Resilient Consensus for Multiagent Networks

Abstract:
This article is concerned with the co-design of privacy-preserving and resilient consensus protocol for a class of multiagent networks (MANs), where the information exchanges over communication networks among the agents suffer from eavesdropping and Sybil attacks. First, we introduce a new attack model in which an adversarial agent could launch a Sybil attack, generating a large number of spurious entities in the network, thereby gaining disproportionate influence. In this communication framework, a whole-process privacy-preserving mechanism is designed that is capable of protecting both initial and current states of agents. Then, instead of existing methods requiring identifying and mitigating Sybil nodes, a degree-based mean-subsequence-reduced (D-MSR) resilient strategy is implemented, showcasing its significant properties: 1) ensuring the effectiveness of aforementioned designed privacy protection strategy; 2) allowing the network to contain Sybil nodes without elimination; and 3) reaching consensus among the normal agents. Finally, several numerical simulations are provided to validate the effectiveness of the proposed results.

---

PaperID: 335,   

Authors:  Mingyuan Li, Tong Jia, Hao Wang, Bowen Ma, Hui Lu, Shuyang Lin, Da Cai, Dongyue Chen

Title: AO-DETR: Anti-Overlapping DETR for X-Ray Prohibited Items Detection

Abstract:
Prohibited item detection in X-ray images is one of the most essential and highly effective methods widely employed in various security inspection scenarios. Considering the significant overlapping phenomenon in X-ray prohibited item images, we propose an anti-overlapping detection transformer (AO-DETR) based on one of the state-of-the-art (SOTA) general object detectors, DETR with improved denoising anchor boxes (DINO). Specifically, to address the feature coupling issue caused by overlapping phenomena, we introduce the category-specific one-to-one assignment (CSA) strategy to constrain category-specific object queries in predicting prohibited items of fixed categories, which can enhance their ability to extract features specific to prohibited items of a particular category from the overlapping foreground-background features. To address the edge blurring problem caused by overlapping phenomena, we propose the look forward densely (LFD) scheme, which improves the localization accuracy of reference boxes in mid-to-high-level decoder layers and enhances the ability to locate blurry edges of the final layer. Similar to DINO, our AO-DETR provides two different versions with distinct backbones, tailored to meet diverse application requirements. Extensive experiments on the PIXray, OPIXray, and HIXray datasets demonstrate that the proposed method surpasses the SOTA object detectors, indicating its potential applications in the field of prohibited item detection. The source code will be available at: https://github.com/Limingyuan001/AO-DETR.

---

PaperID: 336,   

Authors:  Yuyan Ruan, Dawei Yang, Ziqi Tang, An-ran Ran, Jiguang Wang, Carol Y. Cheung, Hao Chen

Title: Reference-Based OCT Angiogram Super-Resolution With Learnable Texture Generation

Abstract:
Optical coherence tomography angiography (OCTA) can visualize retinal microvasculature and is important to qualitatively and quantitatively identify potential biomarkers for different retinal diseases. However, the resolution of optical coherence tomography (OCT) angiograms inevitably decreases when increasing the field-of-view (FOV) given a fixed acquisition time. To address this issue, we propose a novel reference-based super-resolution (RefSR) framework to preserve the resolution of the OCT angiograms while increasing the scanning area. Specifically, textures from the normal RefSR pipeline are used to train a learnable texture generator (LTG), which is designed to generate textures according to the input. The key difference between the proposed method and traditional RefSR models is that the textures used during inference are generated by the LTG instead of being searched from a single reference (Ref) image. Since the LTG is optimized throughout the whole training process, the available texture space is significantly enlarged and no longer limited to a single Ref image, but extends to all textures contained in the training samples. Moreover, our proposed LTGNet does not require an Ref image at the inference phase, thereby becoming invulnerable to the selection of the Ref image. Both experimental and visual results show that LTGNet has competitive performance and robustness over state-of-the-art methods, indicating good reliability and promise in real-life deployment. The source code is available at https://github.com/RYY0722/LTGNet.

---

PaperID: 337,   

Authors:  Xiaoliang Hu, Pengcheng Guo, Yadong Li, Guangyu Li, Zhen Cui, Jian Yang

Title: TVDO: Tchebycheff Value-Decomposition Optimization for Multiagent Reinforcement Learning

Abstract:
In cooperative multiagent reinforcement learning (MARL), centralized training with decentralized execution (CTDE) has recently attracted more attention due to the physical demand. However, the most dilemma therein is the inconsistency between jointly-trained policies and individually executed actions. In this article, we propose a factorized Tchebycheff value-decomposition optimization (TVDO) method to overcome the trouble of inconsistency. In particular, a nonlinear Tchebycheff aggregation function is formulated to realize the global optimum by tightly constraining the upper bound of individual action-value bias, which is inspired by the Tchebycheff method of multiobjective optimization (MOO). We theoretically prove that, under no extra limitations, the factorized value decomposition with Tchebycheff aggregation satisfies the sufficiency and necessity of individual-global-max (IGM), which guarantees the consistency between the global and individual optimal action-value function. Empirically, in the climb and penalty game, we verify that TVDO precisely expresses the global-to-individual value decomposition with a guarantee of policy consistency. Meanwhile, we evaluate TVDO in the StarCraft multiagent challenge (SMAC) benchmark, and extensive experiments demonstrate that TVDO achieves a significant performance superiority over some SOTA MARL baselines.

---

PaperID: 338,   

Authors:  Liangming Chen, Long Jin, Mingsheng Shang

Title: Efficient Loss Landscape Reshaping for Convolutional Neural Networks

Abstract:
Theoretical and empirical evidence highlights a positive correlation between the flatness of loss landscapes around minima and generalization. However, most current approaches that seek to find flat minima either incur high computational costs or struggle to balance generalization, training stability, and convergence. This work proposes reshaping the loss landscape to induce the optimizer toward flat regions, an approach that has negligible computational costs and does not compromise training stability, convergence, or efficiency. We focus on nonlinear, loss-dependent reshaping functions underpinned by theoretical insights to reshape the loss landscape. To design these functions, we first identify where and how these functions should be applied. With the aid of recently developed tools in stochastic optimization, theoretical analysis shows that steepening the low-loss landscape improves the rate of sharp minimum escape while flattening the high- and ultralow-loss landscapes enhances training stability and optimization performance, respectively. Simulations and experiments reveal that the subtly designed reshaping functions not only induce optimizers to find flat minima and improve generalization performance but also stabilize training, promote optimization, and keep efficiency. Our approach is evaluated on image classification, adversarial robustness, and natural language processing (NLP) tasks and achieves significant improvement in generalization performance with negligible computational cost. We believe that the new perspective introduced in this work will broadly impact the field of deep neural network training. The code is available at https://github.com/LongJin-lab/LLR.

---

PaperID: 339,   

Authors:  Wenyan Pan, Wentao Ma, Tongqing Zhou, Shan Zhao, Lichuan Gu, Guolong Shi, Zhihua Xia

Title: Dual-Decoupling With Frequency-Spatial Domains for Image Manipulation Localization

Abstract:
Leveraging trace-rich features within embedded spaces has been established as effective in image manipulation localization (IML). Nevertheless, the feature of manipulated traces frequently comprises substantial redundant information only loosely related to IML tasks. This complexity has hindered existing methods in fully comprehending the essence of trace features. In light of this challenge, we introduce a novel decoupling representation learning network (DRN) tailored for IML. The DRN excels at decoupling intricate multidomain information and transforming it into representations directly pertinent to IML objectives. This is achieved through a meticulously designed frequency decoupling representation learning module (FDM) and spatial decoupling representation learning module (SDM). Specifically, the FDM operates by acquiring distinct low and high-frequency components to effectively decouple redundant information. The decoupled high-frequency components are then harnessed as intricate trace complements, enhancing the overall aggregation process. In addition, the redundant information is expertly separated into authentic and manipulated representations through the use of channel activation maps in SDM. Through extensive experimentation on three public benchmarks including CASIA, NIST, and Coverage, our method consistently demonstrates superior performance and enhanced robustness compared with existing state-of-the-art methods.

---

PaperID: 340,   

Authors:  Yuting Wang, Rong Wang, Feiping Nie, Xuelong Li

Title: Fast Multiview Semi-Supervised Classification With Optimal Bipartite Graph

Abstract:
As data collection becomes increasingly facile and descriptions of data grow more diverse, exploring heterogeneous multiview data is becoming essential. Extracting valuable insights from vast multiview datasets is profoundly meaningful which can leverage the diversity of multiple features to improve classification accuracy. As is well-known, semi-supervised learning (SSL) utilizes limited set of labeled samples to train models when addressing label scarcity. However, although the existing multiview semi-supervised algorithms can accomplish classification task, they often struggle with high complexity problem and lack interpretability, more transparent, and low-complexity approaches are worth studying. Besides, the interplay between graph structure and multiview consistency makes a deeper understanding of underlying data patterns but challenges persist in optimizing graph and ensuring scalability. In this article, we propose a fast multiview semi-supervised algorithm based on anchor graph (BGFMS), which improves the classification performance. It could significantly reduce the computational complexity by converting the label prediction of the original data into the forecast for few anchor points and avoids the additional processing procedure. Extensive experimental results on synthetic dataset and different real datasets validate the effectiveness and efficiency of our algorithm.

---

PaperID: 341,   

Authors:  Zhenchen Li, Xu Yang, Yanchao Zhang, Shaofeng Zeng, Jingbin Yuan, Jiazheng Liu, Zhiyong Liu, Hua Han

Title: Deep Graph Reinforcement Learning for Solving Multicut Problem

Abstract:
The multicut problem, also known as correlation clustering, is a classic combinatorial optimization problem that aims to optimize graph partitioning given only node (dis)similarities on edges. It serves as an elegant generalization for several graph partitioning problems and has found successful applications in various areas such as data mining and computer vision. However, the multicut problem with an exponentially large number of cycle constraints proves to be NP-hard, and existing solvers either suffer from exponential complexity or often give unsatisfactory solutions due to inflexible heuristics driven by hand-designed mechanisms. In this article, we propose a deep graph reinforcement learning method to solve the multicut problem within a combinatorial decision framework involving sequential edge contractions. The customized subgraph neural network adapts to the dynamically edge-contracted graph environment by extracting bilevel connected features from both contracted and original graphs. Our method can learn to infer feasible multicut solutions end-to-end toward optimization of the multicut objective in a data-driven manner. More specifically, by exploring the decision space adaptively, it implicitly gains heuristic knowledge from topological patterns of instances and thereby generates more targeted heuristics overcoming the short-sightedness inherent in the hand-designed ones. During testing, the learned heuristics iteratively contract graphs to construct high-quality solutions within polynomial time. Extensive experiments on synthetic and real-world multicut instances show the superiority of our method over existing combinatorial solvers, while also maintaining a certain level of out-of-distribution generalization ability.

---

PaperID: 342,   

Authors:  Can Xu, Le Hui, Jin Xie, Jian Yang

Title: Weakly Supervised Object Localization With Progressive Activation Diffusion

Abstract:
Weakly supervised object localization (WSOL) aims to locate objects with only image-level labels. Previous works mainly follow the framework of class activation map (CAM), which discovers the objects by estimating the contribution of each pixel position to the category prediction. However, most of them overlook the pixel-level spatial and semantic contextual correlation, resulting in: 1) limited activation ranges that only highlight the most discriminative parts rather than the entire object and 2) low activation values for some foreground parts, especially regions near the boundary between foreground and background. To alleviate this issue, we propose an activation diffusion network (ADNet) to progressively refine both the range and value of activations on the localization map. Specifically, a context propagation module is first developed to learn the top-down spatial dependency between adjacent feature maps, which helps back-propagate the activation from the discriminative part to its surroundings for more complete objects. Then, a diffusion probability distillation module (DPDM) is proposed, which transfers the pixel-level semantic correlation emerging in the image generation process to the localization map generation in a teacher-student learning manner. This helps boost the value of the activated foreground region and stimulates the value of neighboring inactivated foreground positions to sharpen the object boundary. Experiments on various datasets and backbones demonstrate the superiority of our ADNet over state-of-the-art (SOTA) methods in object localization and segmentation, yielding 82.2% and 62.2% Top-1 Loc on Caltech-UCSD Birds-200-2011 (CUB) and ImageNet Large-ScaleVisual Recognition Challenge (ILSVRC) datasets and 76.6% pixel average precision (PxAP) on OpenImages dataset. Qualitative results also show that we can achieve a more complete and consistent activation covering the whole object.

---

PaperID: 343,   

Authors:  En-Hui Yang, Shayan Mohajer Hamidi, Linfeng Ye, Renhao Tan, Beverly Yang

Title: Conditional Mutual Information Constrained Deep Learning for Classification

Abstract:
The concepts of conditional mutual information (CMI) and normalized CMI (NCMI) are introduced to measure the concentration and separation performance of a classification deep neural network (DNN) in the output probability distribution space of the DNN, where CMI and the ratio between CMI and NCMI represent the intraclass concentration and interclass separation of the DNN, respectively. By using NCMI to evaluate popular DNNs pretrained over CIFAR-100 and ImageNet in the literature, it is shown that their validation accuracies are more or less inversely proportional to their NCMI values. Based on this observation, the standard deep learning (DL) framework is further modified to minimize the standard cross entropy (CE) function subject to an NCMI constraint, yielding CMI constrained DL (CMIC-DL). A novel alternating learning algorithm is proposed to solve such a constrained optimization problem. Extensive experimental results show that DNNs trained within CMIC-DL outperform the state-of-the-art models trained within the standard DL and other loss functions in the literature in terms of both accuracy and robustness against adversarial attacks. In addition, visualizing the evolution of the learning process through the lens of CMI and NCMI is also advocated.

---

PaperID: 344,   

Authors:  Dan Zhang, Tong Zhang, C. L. Philip Chen, Tao Zhang

Title: Broad Learning System Based on Fractional Feature Optimization

Abstract:
Broad learning system (BLS) have demonstrated excellent performance in terms of both speed and accuracy in tasks such as image classification. In BLS, the feature nodes predominantly utilize linear features, and sparse representation is mainly employed in the feature optimization component. The robustness of these features to different data needs to be improved. Although there are many improved algorithms for BLS in feature optimization, there is no improvement based on fractional calculus at present. This article proposes BLS-FC, a novel data classification and regression method that can seamlessly combine BLS and fractional calculation. Fractional calculus describes the properties of data between integer orders and has memory properties. Fractional Fourier transform (Frft) also has time domain and frequency domain information. First, Frft is added to the broad learning feature node extraction to enrich the node features, which is called BLS-Frft. Second, fractional calculus is integrated into the BLS-Frft sparse representation feature optimization, and the feature representation capability is enhanced by fractional differential memory. This part is called BLS-FS. Finally, in order to solve the problem of unstable features of random fractional order subspaces, a fractional order multiscale feature interaction based on BLS-Frft is proposed, which is called BLS-MF. Experimental results across various classification and regression datasets demonstrate the superior performance of the proposed method.

---

PaperID: 345,   

Authors:  Jiaxu Leng, Jia Wang, Mengjingcheng Mo, Ji Gan, Wen Lu, Xinbo Gao

Title: Difficulty-Guided Variant Degradation Learning for Blind Image Super-Resolution

Abstract:
Recent blind super-resolution (BSR) methods are explored to handle unknown degradations and achieve impressive performance. However, the prevailing assumption in most BSR methods is the spatial invariance of degradation kernels across the entire image, which leads to significant performance declines when faced with spatially variant degradations caused by object motion or defocusing. Additionally, these methods do not account for the human visual system's tendency to focus differently on areas of varying perceptual difficulty, as they uniformly process each pixel during reconstruction. To cope with these issues, we propose a difficulty-guided variant degradation learning network for BSR, named difficulty-guided degradation learning (DDL)-BSR, which explores the relationship between reconstruction difficulty and degradation estimation. Accordingly, the proposed DDL-BSR consists of three customized networks: reconstruction difficulty prediction (RDP), space-variant degradation estimation (SDE), and degradation and difficulty-informed reconstruction (DDR). Specifically, RDP learns the reconstruction difficulty with the proposed reconstruction-distance supervision. Then, SDE is designed to estimate space-variant degradation kernels according to the difficulty map. Finally, both degradation kernels and reconstruction difficulty are fed into DDR, which takes into account such two prior knowledge information to guide super-resolution (SR). Experimental analysis on various synthetic datasets demonstrates that DDL-BSR invariably surpasses state-of-the-art (SOTA) methods, producing SR images with enhanced realism and texture quality. Code is available at https://github.com/JiaWang0704/DDL-BSR.

---

PaperID: 346,   

Authors:  Zizhou Wang, Yan Wang, Yangqin Feng, Jiawei Du, Yong Liu, Rick Siow Mong Goh, Liangli Zhen

Title: Continuous Disentangled Joint Space Learning for Domain Generalization

Abstract:
Domain generalization (DG) aims to learn a model on one or multiple observed source domains that can generalize to unseen target test domains. Previous approaches have focused on extracting domain-invariant information from multiple source domains, but domain-specific information is also closely tied to semantics in individual domains and is not well-suited for generalization to the target domain. In this article, we propose a novel DG method called continuous disentangled joint space learning (CJSL), which leverages both domain-invariant and domain-specific information for more effective DG. The key idea behind CJSL is to formulate and learn a continuous joint space (CJS) for domain-specific representations from source domains through iterative feature disentanglement. This learned CJS can then be used to simulate domain-specific representations for test samples from a mixture of multiple domains via Monte Carlo sampling during the inference stage. Unlike existing approaches, which exploit domain-invariant feature vectors only or aim to learn a universal domain-specific feature extractor, we simulate domain-specific representations via sampling the latent vectors in the learned CJS for the test sample to fully use the power of multiple domain-specific classifiers for robust prediction. Empirical results demonstrate that CJSL outperforms 19 state-of-the-art (SOTA) methods on seven benchmarks, indicating the effectiveness of our proposed method.

---

PaperID: 347,   

Authors:  Yan Xiao, Yaochu Jin, Bin Wang, Yan Zhang, Kuangrong Hao, Haizhou Li

Title: Zero-Shot Relation Classification Through Inference on Category Attributes

Abstract:
The goal of relationship classification (RC) is to predict the semantic relationship between two entities in a given sentence. With the advent of deep learning and pretrained language models, RC research has progressed by leaps and bounds. However, the current studies are focused mainly on predicting semantic relationships from a predefined set. How to recognize unseen relationships remains a challenge, which is also known as the zero-shot RC (ZSRC) task. Some ZSRC-related methods directly map relationship categories to numerical indices, constraining the model’s ability to autonomously infer and understand these relationships, while others rely heavily on manual definitions. To address these issues and inspired by the way of reasoning in which humans perform RC tasks, we propose a new framework to handle the ZSRC task through inference on category attributes (ICAs). The main idea of ICA is to detect the semantic relationship between promises, which are RC sentences, and hypotheses, which are relational sentences of entities created by templates. Specifically, instead of manual design, we introduce two hypothesis templates derived from the label words (LWs) and descriptions (LDs) associated with each relationship. These templates are used to automatically convert the RC data into the textual entailment (TE) format. Furthermore, they are fine-tuned with a pretrained TE model, facilitating the acquisition of relational knowledge and enabling the generalization of semantic reasoning rules learned from seen classes to unseen classes. Moreover, to implement multirelationship semantic inference for all unseen classes, we propose an entailment difference mechanism to enhance the reasoning capability of the model. Besides the current ZSRC test setting, we also examine our method in an even more challenging setting to deal with data scarcity in real-world applications. The outstanding performance of ICA on the FewRel and Wiki-ZSL datasets demonstrates its effectiveness in the ZSRC task.

---

PaperID: 348,   

Authors:  Zhao Kang, Xuanting Xie, Bingheng Li, Erlin Pan

Title: CDC: A Simple Framework for Complex Data Clustering

Abstract:
In today’s digital era driven by data, the amount and complexity of the collected data, such as multiview, non-Euclidean, and multirelational, are growing exponentially or even faster. Clustering, which unsupervisedly extracts valid knowledge from data, is extremely useful in practice. However, existing methods are independently developed to handle one particular challenge at the expense of the others. In this work, we propose a simple but effective framework for complex data clustering (CDC) that can efficiently process different types of data with linear complexity. We first use graph filtering (GF) to fuse geometric structure and attribute information. We then reduce complexity with high-quality anchors that are adaptively learned via a novel similarity-preserving (SP) regularizer. We illustrate the cluster-ability of our proposed method theoretically and experimentally. In particular, we deploy CDC to graph data of size 111 M.

---

PaperID: 349,   

Authors:  Kehui Ding, Jun Shu, Deyu Meng, Zongben Xu

Title: Improve Noise Tolerance of Robust Loss via Noise-Awareness

Abstract:
Robust loss minimization is an important strategy for handling robust learning issue on noisy labels. Current approaches for designing robust losses involve the introduction of noise-robust factors, i.e., hyperparameters, to control the trade-off between noise robustness and learnability. However, finding suitable hyperparameters for different datasets with noisy labels is a challenging and time-consuming task. Moreover, existing robust loss methods usually assume that all training samples share common hyperparameters, which are independent of instances. This limits the ability of these methods to distinguish the individual noise properties of different samples and overlooks the varying contributions of diverse training samples in helping models understand underlying patterns. To address above issues, we propose to assemble robust loss with instance-dependent hyperparameters to improve their noise tolerance with theoretical guarantee. To achieve setting such instance-dependent hyperparameters for robust loss, we propose a meta-learning method which is capable of adaptively learning a hyperparameter prediction function, called noise-aware-robust-loss-adjuster (NARL-Adjuster). Through mutual amelioration between hyperparameter prediction function and classifier parameters in our method, both of them can be simultaneously finely ameliorated and coordinated to attain solutions with good generalization capability. Four SOTA robust loss functions are attempted to be integrated with our algorithm, and comprehensive experiments substantiate the general availability and effectiveness of the proposed method in both its noise tolerance and performance. Meanwhile, the explicit parameterized structure makes the meta-learned prediction function ready to be transferrable and plug-and-play to unseen datasets with noisy labels. Specifically, we transfer our meta-learned NARL-Adjuster to unseen tasks, including several real noisy datasets, and achieve better performance compared with conventional hyperparameter tuning strategy, even with carefully tuned hyperparameters.

---

PaperID: 350,   

Authors:  Fangfang Li, Quanxue Gao, Qianqian Wang, Ming Yang, Cheng Deng

Title: Tensorized Soft Label Learning Based on Orthogonal NMF

Abstract:
Recently, a strong interest has been in multiview high-dimensional data collected through cross-domain or various feature extraction mechanisms. Nonnegative matrix factorization (NMF) is an effective method for clustering these high-dimensional data with clear physical significance. However, existing multiview clustering based on NMF only measures the difference between the elements of the coefficient matrix without considering the spatial structure relationship between the elements. And they often require postprocessing to achieve clustering, making the algorithms unstable. To address this issue, we propose minimizing the Schatten p-norm of the tensor, which consists of a coefficient matrix of different views. This approach considers each element’s spatial structure in the coefficient matrices, crucial for effectively capturing complementary information presented in different views. Furthermore, we apply orthogonal constraints to the cluster index matrix to make it sparse and provide a strong interpretation of the clustering. This allows us to obtain the cluster label directly without any postprocessing. To distinguish the importance of different views, we utilize adaptive weights to assign varying weights to each view. We introduce an unsupervised optimization scheme to solve and analyze the computational complexity of the model. Through comprehensive evaluations of six benchmark datasets and comparisons with several multiview clustering algorithms, we empirically demonstrate the superiority of our proposed method.

---

PaperID: 351,   

Authors:  Ruida Xi, Nianchang Huang, Changzhou Lai, Qiang Zhang, Jungong Han

Title: FMCNet+: Feature-Level Modality Compensation for Visible-Infrared Person Re-Identification

Abstract:
For visible-infrared person re-identification (VI-ReID), current models that compensate modality-specific information strive to generate missing modality images from existing ones to bridge the cross-modality discrepancies. Despite that, those generated images often suffer from low qualities due to the significant modality gap and include interfering information, e.g., inconsistent colors, thus severely degrading the subsequent VI-ReID performance. Alternatively, we propose a feature-level modality compensation network, i.e., FMCNet+, for VI-ReID in this article as an improved version of our previous work (FMCNet). The core of FMCNet+ is to compensate for the missing modality-specific information at the feature level, rather than at the image level, enabling our model to generate more person-related and discriminative modality-specific features for VI-ReID. Concretely, FMCNet+ aims to progressively generate missing modality-specific features by fully exploring the relationships among single-modality features, modality-shared features, and modality-specific features, instead of directly generating them through a generative adversarial way as in the previous FMCNet. To this end, three modules, i.e., single-modality feature decomposition (SFD), modality characteristic dictionary learning (MCDL), and missing modality-specific feature compensation (MMFC), are incorporated in FMCNet+. Experimental results demonstrate the superiority of our proposed FMCNet+ over existing ones, especially for those that compensate for modality-specific information at the image level. Our intriguing findings highlight the necessity of feature-level modality compensation in VI-ReID. Our code and pre-trained models will be released on https://github.com/jssyzsfzy/FMCNet_series.

---

PaperID: 352,   

Authors:  Yunhao Gao, Mengmeng Zhang, Wei Li, Ran Tao

Title: Distribution-Independent Domain Generalization for Multisource Remote Sensing Classification

Abstract:
The availability of multisource remote sensing data provides the possibility for comprehensive observation. Convolutional neural networks (CNNs) naturally integrate multisource feature extractors and classifiers into an end-to-end multilayer design. However, CNN assumes data are independent and identically distributed. In practice, it is not always possible to access the labels or even data of the testing scenes. Therefore, the CNN-based methods have exposed its limitation on generalization ability. To solve the issue, a feature-distribution-independent network (FDINet) is designed for multisource remote sensing cross-domain classification without feature alignment and decoupling operations. On one hand, an elegantly designed baseline is used for extracting multisource cross-domain features. The baseline extracts the common line and texture features through shallow weight-sharing networks. More importantly, the modality prediction probability is used to measure the similarity between the source domains and the target domains, thereby improving cross-domain collaboration capabilities. On the other hand, the sharpness-aware feature discriminating (SAFD) strategy is developed for model optimization. Specifically, the generalization ability is improved by minimizing the sharpness of local optima. To avoid the decrease in feature discrimination caused by the gradient conflict between sharpness and overall loss, the discrimination constraints are designed to balance feature discrimination and generalization ability. Comprehensive experiments are conducted on two datasets, which demonstrate that the proposed FDINet outperforms other competitors in terms of quantitative and qualitative analyses.

---

PaperID: 353,   

Authors:  Yubin Xiao, Di Wang, Boyang Li, Huanhuan Chen, Wei Pang, Xuan Wu, Hao Li, Dong Xu, Yanchun Liang, You Zhou

Title: Reinforcement Learning-Based Nonautoregressive Solver for Traveling Salesman Problems

Abstract:
The traveling salesman problem (TSP) is a well-known combinatorial optimization problem (COP) with broad real-world applications. Recently, neural networks (NNs) have gained popularity in this research area because as shown in the literature, they provide strong heuristic solutions to TSPs. Compared to autoregressive neural approaches, nonautoregressive (NAR) networks exploit the inference parallelism to elevate inference speed but suffer from comparatively low solution quality. In this article, we propose a novel NAR model named NAR4TSP, which incorporates a specially designed architecture and an enhanced reinforcement learning (RL) strategy. To the best of our knowledge, NAR4TSP is the first TSP solver that successfully combines RL and NAR networks. The key lies in the incorporation of NAR network output decoding into the training process. NAR4TSP efficiently represents TSP-encoded information as rewards and seamlessly integrates it into RL strategies, while maintaining consistent TSP sequence constraints during both training and testing phases. Experimental results on both synthetic and real-world TSPs demonstrate that NAR4TSP outperforms five state-of-the-art (SOTA) models in terms of solution quality, inference speed, and generalization to unseen scenarios.

---

PaperID: 354,   

Authors:  Jinyang Liu, Shutao Li, Lishan Tan, Renwei Dian

Title: Denoiser Learning for Infrared and Visible Image Fusion

Abstract:
Infrared image (IR) and visible image (VI) fusion creates fusion images that contain richer information and gain improved visual effects. Existing methods generally use the operators of manual design, such as intensity and gradient operators, to mine the image information. However, it is hard for them to achieve a complete and accurate description of information, which limits the image fusion performance. To this end, a novel information measurement method is proposed to achieve IR and VI fusion. Its core idea is to guide a generator in achieving image fusion by learning the denoisers. Specifically, by using denoisers to restore fusion images with different noise interference to source images, a mutual competition relationship is formed between denoisers, which helps the generator thoroughly explore the data specificity of the source images and guide it to achieve more accurate feature representation. In addition, a semantic adaptive measurement loss function is proposed to constrain the generator, which fuses semantic information adaptively by considering the semantic information density of different source images. The results of quantitative and qualitative experiments have shown that the proposed method can achieve a higher quality information fusion and has a faster fusion speed on three public datasets when compared with advanced methods.

---

PaperID: 355,   

Authors:  Fan Zhang, Huiying Liu, Qing Cai, Chun-Mei Feng, Binglu Wang, Shanshan Wang, Junyu Dong, David Zhang

Title: Federated Cross-Incremental Self-Supervised Learning for Medical Image Segmentation

Abstract:
Federated cross learning has shown impressive performance in medical image segmentation. However, it encounters the catastrophic forgetting issue caused by data heterogeneity across different clients and is particularly pronounced when simultaneously facing pixelwise label deficiency problem. In this article, we propose a novel federated cross-incremental self-supervised learning method, coined FedCSL, which not only can enable any client in the federation incrementally yet effectively learn from others without inducing knowledge forgetting or requiring massive labeled samples, but also preserve maximum data privacy. Specifically, to overcome the catastrophic forgetting issue, a novel cross-incremental collaborative distillation (CCD) mechanism is proposed, which distills explicit knowledge learned from previous clients to subsequent clients based on secure multiparty computation (MPC). Besides, an effective retrospect mechanism is designed to rearrange the training sequence of clients per round, further releasing the power of CCD by enforcing interclient knowledge propagation. In addition, to alleviate the need of large-scale densely annotated pretraining medical datasets, we also propose a two-stage training framework, in which federated cross-incremental self-supervised pretraining paradigm first extracts robust yet general image-level patterns across multi-institutional data silos via a novel round-robin distributed masked image modeling (MIM) pipeline; then, the resulting visual concepts, e.g., semantics, are transferred to the federated cross-incremental supervised fine-tuning paradigm, favoring various cross-silo medical image segmentation tasks. The experimental results on public datasets demonstrate the effectiveness of the proposed method as well as the consistently superior performance of our method over most state-of-the-art methods quantitatively and qualitatively.

---

PaperID: 356,   

Authors:  Lin Ma, Liang Hu, Yonghao Li, Weiping Ding, Wanfu Gao

Title: MI-MCF: A Mutual Information-Based Multilabel Causal Feature Selection

Abstract:
Multilabel causal feature selection has attracted extensive attention in recent years. Current multilabel causal feature selection algorithms typically employ existing Markov Blanket (MB) search methods for the initial construction of the MB, followed by further optimization. These methods generally treat labels and features as equally weighted nodes during the MB construction process. However, the search for spouse sets often involves extensive conditional independence (CI) tests, which are time-consuming. Furthermore, they fail to consider the distinct contributions of labels and features to the target nodes. Information theory is often used to evaluate the contributions of nodes. Inspired by this, we carry out a theoretical investigation into the causal relationships within multilabel datasets and propose the mutual information-based multilabel causal feature selection (MI-MCF) method. First, MI-MCF employs MI and conditional MI (CMI) instead of CI test when constructing the MB of labels without incurring significant time overhead. Then, MI-MCF uses MI to compare the contributions of features and labels to the target nodes. This helps identify which nodes should be retained when recovering features hindered by strong label correlation. Finally, MI-MCF eliminates spurious nodes through a symmetry check. Experiments on real-world datasets demonstrate that MI-MCF can autonomously determine the optimal number of selected features and consistently outperform compared methods. The code is available at https://github.com/malinjlu/MI-MCF.

---

PaperID: 357,   

Authors:  Guokai Hao, Yuanzheng Li, Yang Li, Lin Jiang, Zhigang Zeng

Title: Lyapunov-Based Safe Reinforcement Learning for Microgrid Energy Management

Abstract:
The rapid development of renewable energy sources (RESs) has led to their increased integration into microgrids (MGs), emphasizing the need for safe and efficient energy management in MG operations. We investigate the methods of MG energy management, primarily categorized into model-based and model-free approaches. Due to a lack of incremental knowledge, model-based methods need to be reengineered for new scenarios during the optimization process, leading to reduced computational efficiency. In contrast, model-free methods can obtain incremental knowledge via trial-and-error in the training phase, and output energy management scheme rapidly. However, ensuring the safety of the scheme during the training phases poses significant challenges. To address these challenges, we propose a safe reinforcement learning (SRL) framework. The proposed SRL framework initially includes a safety assessment optimization model (SAOM) to evaluate scheme constraints and refine unsafe schemes for ensuring MG safety. Subsequently, based on SAOM, the MG energy management issue is formulated as an assess-based constrained Markov decision process (A-CMDP), enabling the SRL can be adopted in this issue. After that, we adopt a Lyapunov-based safety policy optimization for agent policy learning to ensure that policy updates are confined within a safe boundary, theoretically ensuring the safety of the MG throughout the learning process. Numerical studies highlight the superior performance of our proposed method. Specifically, the SRL framework effectively learns energy management policy, ensures MG safety, and demonstrates outstanding outcomes in the economic operation of MG.

---

PaperID: 358,   

Authors:  Yang Liu, Xinshuo Wang, Xinbo Gao, Jungong Han, Ling Shao

Title: Concept-Aware Graph Convolutional Network for Compositional Zero-Shot Learning

Abstract:
Compositional zero-shot learning (CZSL) aims to identify unobservable compositional concepts with prior knowledge of known primitives (attributes and objects). Due to distribution differences between seen and unseen components, existing methods for CZSL often ignore intrinsic variations between primitives and suffer from domain bias problems. To address this challenge, we proposed a concept-aware graph convolutional network (GCN) that utilizes cross-attentions to extract features unique to attributes and objects from paired concept-sharing inputs. The proposed model utilizes the cosine similarity between visual features and synthetic embeddings to estimate the feasibility score for each unseen composition. This score is then employed as a weight in the graph adjacency matrix. Additionally, the proposed model incorporates the Earth mover’s distance (EMD) to further limit the concept of learning interest in disentanglers. Experimental results on three challenging dataset benchmarks, including UT-Zappos 50K, C-GQA, and MIT-States, demonstrate that the proposed model outperforms prior work in both closed- and open-world CZSL (OW-CZSL).

---

PaperID: 359,   

Authors:  Jintang Bian, Yixiang Lin, Xiaohua Xie, Chang-Dong Wang, Lingxiao Yang, Jian-Huang Lai, Feiping Nie

Title: Multilevel Contrastive Multiview Clustering With Dual Self-Supervised Learning

Abstract:
Multiview clustering (MVC) aims to integrate multiple related but different views of data to achieve more accurate clustering performance. Contrastive learning has found many applications in MVC due to its successful performance in unsupervised visual representation learning. However, existing MVC methods based on contrastive learning overlook the potential of high similarity nearest neighbors as positive pairs. In addition, these methods do not capture the multilevel (i.e., cluster, instance, and prototype levels) representational structure that naturally exists in multiview datasets. These limitations could further hinder the structural compactness of learned multiview representations. To address these issues, we propose a novel end-to-end deep MVC method called multilevel contrastive MVC (MCMC) with dual self-supervised learning (DSL). Specifically, we first treat the nearest neighbors of an object from the latent subspace as the positive pairs for multiview contrastive loss, which improves the compactness of the representation at the instance level. Second, we perform multilevel contrastive learning (MCL) on clusters, instances, and prototypes to capture the multilevel representational structure underlying the multiview data in the latent space. In addition, we learn consistent cluster assignments for MVC by adopting a DSL method to associate different level structural representations. The evaluation experiment showed that MCMC can achieve intracluster compactness, intercluster separability, and higher accuracy (ACC) in clustering performance. Our code is available at https://github.com/bianjt-morning/MCMC.

---

PaperID: 360,   

Authors:  Yi Liu, Song Guo, Jie Zhang, Yufeng Zhan, Qihua Zhou, Yingchun Wang

Title: Feature Correlation-Guided Knowledge Transfer for Federated Self-Supervised Learning

Abstract:
Extensive attention has been paid to the application of self-supervised learning (SSL) approaches on federated learning (FL) to tackle the label scarcity problem. Previous works on federated SSL (FedSSL) generally fall into two categories: parameter-based model aggregation or data-based feature sharing to achieve knowledge transfer among multiple unlabeled clients. Despite the progress, they inevitably rely on some assumptions, such as homogeneous models or the existence of an additional public dataset, which hinder the universality of the training frameworks for more general scenarios (e.g., unlabeled clients with heterogeneous models). Therefore, in this article, we propose a novel and general method named federated self-supervised learning with feature-correlation-based aggregation (FedFoA) to tackle the above limitations. By exchanging feature correlation instead of model parameters or feature mappings, our approach reduces the discrepancies of local representations learning processes, thus promoting collaboration between heterogeneous clients. A factorization-based method is designed to extract the cross-feature relation matrix from local representations, which serves as a knowledge medium for the aggregation phase. We demonstrate that FedFoA is a heterogeneity-supportive and privacy-preserving training framework and can be easily compatible with state-of-the-art FedSSL methods. Extensive empirical experiments demonstrate our proposed approach outperforms the state-of-the-art methods by a significant margin.

---

PaperID: 361,   

Authors:  Tongcun Liu, Xukai Bao, Jiaxin Zhang, Kai Fang, Hailin Feng

Title: Enhancing Session-Based Recommendation With Multi-Interest Hyperbolic Representation Networks

Abstract:
Session-based recommendation (SBR) aims to predict the next item a user might click within an ongoing session, without relying on user profiles or historical data. Modern approaches typically use graph networks to learn item embeddings in Euclidean space via graph convolution operations. However, they often struggle to capture the diversity of user interactions within short, hierarchically structured sessions, which is essential for accurate predictions in SBR. To tackle these challenges, we propose a multi-interest hyperbolic representation network (MIHRN) to enhance the performance of SBR by adeptly modeling both intricate high-order spatial structures and sequence relationships among items in hyperbolic geometry space. Specifically, we use a hyperbolic hypergraph neural network to exploit the high-order spatial relationships and local clustering structures inherent within sessions. Subsequently, a multiaspect interest representation module is designed to articulate the diversity of user interests. Extensive experiments on three real-world datasets demonstrate that the proposed method achieves performance improvements of 23.81%, 14.81%, and 36.84%, respectively, under the P@10 metric.

---

PaperID: 362,   

Authors:  Hao Dong, Gaëtan Frusque, Yue Zhao, Eleni N. Chatzi, Olga Fink

Title: NNG-Mix: Improving Semi-Supervised Anomaly Detection With Pseudo-Anomaly Generation

Abstract:
Anomaly detection (AD) is essential in identifying rare and often critical events in complex systems, finding applications in fields such as network intrusion detection, financial fraud detection, and fault detection in infrastructure and industrial systems. While AD is typically treated as an unsupervised learning task due to the high cost of label annotation, it is more practical to assume access to a small set of labeled anomaly samples from domain experts, as is the case for semi-supervised AD. Semi-supervised and supervised approaches can leverage such labeled data, resulting in improved performance. In this article, rather than proposing a new semi-supervised or supervised approach for AD, we introduce a novel algorithm for generating additional pseudo-anomalies on the basis of the limited labeled anomalies and a large volume of unlabeled data. This serves as an augmentation to facilitate the detection of new anomalies. Our proposed algorithm, named nearest neighbor Gaussian mix-up (NNG-Mix), efficiently integrates information from both labeled and unlabeled data to generate pseudo-anomalies. We compare the performance of this novel algorithm with commonly applied augmentation techniques, such as Mixup and Cutout. We evaluate NNG-Mix by training various existing semi-supervised and supervised AD algorithms on the original training data along with the generated pseudo-anomalies. Through extensive experiments on 57 benchmark datasets in ADBench, reflecting different data types, we demonstrate that NNG-Mix outperforms other data augmentation methods. It yields significant performance improvements compared to the baselines trained exclusively on the original training data. Notably, NNG-Mix yields up to 16.4%, 8.8%, and 8.0% improvements on Classical, CV, and NLP datasets in ADBench. Our source code is available at https://github.com/donghao51/NNG-Mix.

---

PaperID: 363,   

Authors:  Ziyi Zhang, Mingxuan Ouyang, Wanyu Lin, Hao Lan, Lei Yang

Title: Debiasing Graph Representation Learning Based on Information Bottleneck

Abstract:
Graph representation learning has shown superior performance in numerous real-world applications, such as finance and social networks. Nevertheless, most existing works might make discriminatory predictions due to insufficient attention to fairness in their decision-making processes. This oversight has prompted a growing focus on fair representation learning. Among recent explorations on fair representation learning, prior works based on the adversarial learning usually induce unstable or counterproductive performance. To achieve fairness in a stable manner, we present the design and implementation of graph representation learning based on fairness information bottleneck (GRAFair), a new framework based on a variational graph autoencoder (VGAE). The crux of GRAFair is the conditional fairness bottleneck (CFB), where the objective is to capture the trade-off between the utility of representations and sensitive information of interest. By applying variational approximation, we can make the optimization objective tractable. Particularly, GRAFair can be trained to produce informative representations of tasks while containing little sensitive information without adversarial training. Experiments on various real-world datasets demonstrate the effectiveness of our proposed method in terms of fairness, utility, robustness, and stability.

---

PaperID: 364,   

Authors:  Benjamin Devillers, Léopold Maytié, Rufin VanRullen

Title: Semi-Supervised Multimodal Representation Learning Through a Global Workspace

Abstract:
Recent deep learning models can efficiently combine inputs from different modalities (e.g., images and text) and learn to align their latent representations or to translate signals from one domain to another (as in image captioning or text-to-image generation). However, current approaches mainly rely on brute-force supervised training over large multimodal datasets. In contrast, humans (and other animals) can learn useful multimodal representations from only sparse experience with matched cross-modal data. Here, we evaluate the capabilities of a neural network architecture inspired by the cognitive notion of a “global workspace” (GW): a shared representation for two (or more) input modalities. Each modality is processed by a specialized system (pretrained on unimodal data and subsequently frozen). The corresponding latent representations are then encoded to and decoded from a single shared workspace. Importantly, this architecture is amenable to self-supervised training via cycle-consistency: encoding-decoding sequences should approximate the identity function. For various pairings of vision-language modalities and across two datasets of varying complexity, we show that such an architecture can be trained to align and translate between two modalities with very little need for matched data (from four to seven times less than a fully supervised approach). The GW representation can be used advantageously for downstream classification and cross-modal retrieval tasks and for robust transfer learning. Ablation studies reveal that both the shared workspace and the self-supervised cycle-consistency training are critical to the system’s performance.

---

PaperID: 365,   

Authors:  Xiangjie Kong, Wenyi Zhang, Hui Wang, Mingliang Hou, Xin Chen, Xiaoran Yan, Sajal K. Das

Title: Federated Graph Anomaly Detection via Contrastive Self-Supervised Learning

Abstract:
Attribute graph anomaly detection aims to identify nodes that significantly deviate from the majority of normal nodes, and has received increasing attention due to the ubiquity and complexity of graph-structured data in various real-world scenarios. However, current mainstream anomaly detection methods are primarily designed for centralized settings, which may pose privacy leakage risks in certain sensitive situations. Although federated graph learning offers a promising solution by enabling collaborative model training in distributed systems while preserving data privacy, a practical challenge arises as each client typically possesses a limited amount of graph data. Consequently, naively applying federated graph learning directly to anomaly detection tasks in distributed environments may lead to suboptimal performance results. We propose a federated graph anomaly detection framework via contrastive self-supervised learning (CSSL) [federated CSSL anomaly detection framework (FedCAD)] to address these challenges. FedCAD updates anomaly node information between clients via federated learning (FL) interactions. First, FedCAD uses pseudo-label discovery to determine the anomaly node of the client preliminarily. Second, FedCAD employs a local anomaly neighbor embedding aggregation strategy. This strategy enables the current client to aggregate the neighbor embeddings of anomaly nodes from other clients, thereby amplifying the distinction between anomaly nodes and their neighbor nodes. Doing so effectively sharpens the contrast between positive and negative instance pairs within contrastive learning, thus enhancing the efficacy and precision of anomaly detection through such a learning paradigm. Finally, the efficiency of FedCAD is demonstrated by experimental results on four real graph datasets.

---

PaperID: 366,   

Authors:  Chao Li, Jia Ning, Han Hu, Kun He

Title: Adaptive Channel Allocation for Robust Differentiable Architecture Search

Abstract:
Differentiable architecture search (DARTS) has attracted much attention due to its simplicity and significant improvement in efficiency. However, the excessive accumulation of the skip connection, when training epochs become large, makes it suffer from weak stability and low robustness, thus limiting its practical applications. Many works have attempted to restrict the accumulation of skip connections by indicators or manual design. These methods, however, are susceptible to human priors and hyperparameters. In this work, we suggest a more subtle and direct approach that no longer explicitly searches for skip connections in the search stage, based on the paradox that skip connections were proposed to guarantee the performance of very deep networks, but the networks in the search stage of DARTS are actually very shallow. Instead, by introducing channel importance ranking and channel allocation strategy, the skip connections are implicitly searched and automatically refilled unimportant channels in the evaluation stage. Our method, dubbed adaptive channel allocation (ACA) strategy, is a general-purpose approach for DARTS, which universally works in DARTS variants without introducing human priors, indicators, or hyperparameters. Extensive experiments on various datasets and DARTS variants verify that the ACA strategy is the most effective one among the existing methods in improving robustness and dealing with the collapse issue when training epochs become large.

---

PaperID: 367,   

Authors:  Yifan Wang, Xiao Luo, Chong Chen, Xian-Sheng Hua, Ming Zhang, Wei Ju

Title: DisenSemi: Semi-Supervised Graph Classification via Disentangled Representation Learning

Abstract:
Graph classification is a critical task in numerous multimedia applications, where graphs are employed to represent diverse types of multimedia data, including images, videos, and social networks. Nevertheless, in the real world, labeled graph data are always limited or scarce. To address this issue, we focus on the semi-supervised graph classification task, which involves both supervised and unsupervised models learning from labeled and unlabeled data. In contrast to recent approaches that transfer the entire knowledge from the unsupervised model to the supervised one, we argue that an effective transfer should only retain the relevant semantics that align well with the supervised task. We introduce a novel framework termed DisenSemi in this article, which learns disentangled representation for semi-supervised graph classification. Specifically, a disentangled graph encoder is proposed to generate factorwise graph representations for both supervised and unsupervised models. Then, we train two models via supervised objective and mutual information (MI)-based constraints, respectively. To ensure the meaningful transfer of knowledge from the unsupervised encoder to the supervised one, we further define an MI-based disentangled consistency regularization between two models and identify the corresponding rationale that aligns well with the current graph classification task. Experiments conducted on various publicly available datasets demonstrate the effectiveness of our DisenSemi.

---

PaperID: 368,   

Authors:  Tianhao Zhao, Xiaoyang Guo, Yutian Lin, Bo Du

Title: MixIR: Mixing Input and Representations for Contrastive Learning

Abstract:
Recently, contrastive learning has shown significant progress in learning visual representations from unlabeled data. The core idea is training the backbone to be invariant to different augmentations of an instance. While most methods only maximize the feature similarity between two augmented data, we further generate more challenging training samples and force the model to keep predicting aggregated representation on these hard samples. In this article, we propose MixIR, a mixture-based approach upon the traditional Siamese network. On the one hand, we input two augmented images of an instance to the backbone and obtain the aggregated representation by performing an elementwise maximum of two features. On the other hand, we take the mixture of these augmented images as input and expect the model prediction to be close to the aggregated representation. In this way, the model could access more variant data samples of an instance and keep predicting invariant representations for them. Thus, the learned model is more discriminative compared with previous contrastive learning methods. Extensive experiments on large-scale datasets show that MixIR steadily improves the baseline and achieves competitive results with state-of-the-art methods. Our code is available at https://github.com/happytianhao/MixIR.

---

PaperID: 369,   

Authors:  Jintao Huang, Chuangquan Chen, Chi-Man Vong, Yiu-Ming Cheung

Title: Broad Multitask Learning System With Group Sparse Regularization

Abstract:
The broad learning system (BLS) featuring lightweight, incremental extension, and strong generalization capabilities has been successful in its applications. Despite these advantages, BLS struggles in multitask learning (MTL) scenarios with its limited ability to simultaneously unravel multiple complex tasks where existing BLS models cannot adequately capture and leverage essential information across tasks, decreasing their effectiveness and efficacy in MTL scenarios. To address these limitations, we proposed an innovative MTL framework explicitly designed for BLS, named group sparse regularization for broad multitask learning system using related task-wise (BMtLS-RG). This framework combines a task-related BLS learning mechanism with a group sparse optimization strategy, significantly boosting BLS’s ability to generalize in MTL environments. The task-related learning component harnesses task correlations to enable shared learning and optimize parameters efficiently. Meanwhile, the group sparse optimization approach helps minimize the effects of irrelevant or noisy data, thus enhancing the robustness and stability of BLS in navigating complex learning scenarios. To address the varied requirements of MTL challenges, we presented two additional variants of BMtLS-RG: BMtLS-RG with sharing parameters of feature mapped nodes (BMtLS-RGf), which integrates a shared feature mapping layer, and BMtLS-RGf and enhanced nodes (BMtLS-RGfe), which further includes an enhanced node layer atop the shared feature mapping structure. These adaptations provide customized solutions tailored to the diverse landscape of MTL problems. We compared BMtLS-RG with state-of-the-art (SOTA) MTL and BLS algorithms through comprehensive experimental evaluation across multiple practical MTL and UCI datasets. BMtLS-RG outperformed SOTA methods in 97.81% of classification tasks and achieved optimal performance in 96.00% of regression tasks, demonstrating its superior accuracy and robustness. Furthermore, BMtLS-RG exhibited satisfactory training efficiency, outperforming existing MTL algorithms by 8.04–42.85 times.

---

PaperID: 370,   

Authors:  Ganchao Tan, Zengyu Wan, Yang Wang, Yang Cao, Zheng-Jun Zha

Title: Tackling Event-Based Lip-Reading by Exploring Multigrained Spatiotemporal Clues

Abstract:
Automatic lip-reading (ALR) is the task of recognizing words based on visual information obtained from the speaker’s lip movements. In this study, we introduce event cameras, a novel type of sensing device, for ALR. Event cameras offer both technical and application advantages over conventional cameras for ALR due to their higher temporal resolution, less redundant visual information, and lower power consumption. To recognize words from the event data, we propose a novel multigrained spatiotemporal features learning framework, which is capable of perceiving fine-grained spatiotemporal features from microsecond time-resolved event data. Specifically, we first convert the event data into event frames of multiple temporal resolutions to avoid losing too much visual information at the event representation stage. Then, they are fed into a multibranch subnetwork where the branch operating on low-rate frames can perceive spatially complete but temporally coarse features, while the branch operating on high frame rate can perceive spatially coarse but temporally fine features. Thus, fine-grained spatial and temporal features can be simultaneously learned by integrating the features perceived by different branches. Furthermore, to model the temporal relationships in the event stream, we design a temporal aggregation subnetwork to aggregate the features perceived by the multibranch subnetwork. In addition, we collect two event-based lip-reading datasets (DVS-Lip and DVS-LRW100) for the study of the event-based lip-reading task. Experimental results demonstrate the superiority of the proposed model over the state-of-the-art event-based action recognition models and video-based lip-reading models.

---

PaperID: 371,   

Authors:  Xiaolong Tang, Shuo Ye, Yufeng Shi, Tianheng Hu, Qinmu Peng, Xinge You

Title: Filter Pruning Based on Information Capacity and Independence

Abstract:
Filter pruning has gained widespread adoption for the purpose of compressing and speeding up convolutional neural networks (CNNs). However, the existing approaches are still far from practical applications due to biased filter selection and heavy computation cost. This article introduces a new filter pruning method that selects filters in an interpretable, multiperspective, and lightweight manner. Specifically, we evaluate the contributions of filters from both individual and overall perspectives. For the amount of information contained in each filter, a new metric called information capacity is proposed. Inspired by the information theory, we utilize the interpretable entropy to measure the information capacity and develop a feature-guided approximation process. For correlations among filters, another metric called information independence is designed. Since the aforementioned metrics are evaluated in a simple but effective way, we can identify and prune the least important filters with less computation cost. We conduct comprehensive experiments on benchmark datasets employing various widely used CNN architectures to evaluate the performance of our method. For instance, on ILSVRC-2012, our method outperforms state-of-the-art methods by reducing floating-point operations (FLOPs) by 77.4% and parameters by 69.3% for ResNet-50 with only a minor decrease in an accuracy of 2.64%.

---

PaperID: 372,   

Authors:  Yujun Cheng, Zhewei Zhang, Xuejing Li, Shengjin Wang

Title: Contrastive Unsupervised Representation Learning With Optimize-Selected Training Samples

Abstract:
Contrastive unsupervised representation learning (CURL) is a technique that seeks to learn feature sets from unlabeled data. It has found widespread and successful application in unsupervised feature learning, with the design of positive and negative pairs serving as the type of data samples. While CURL has seen empirical successes in recent years, there is still room for improvement in terms of the pair data generation process. This includes tasks such as combining and re-filtering samples, or implementing transformations among positive/negative pairs. We refer to this as the sample selection process. In this article, we introduce an optimized pair-data sample selection method for CURL. This method efficiently ensures that the two types of sampled data (similar pair and dissimilar pair) do not belong to the same class. We provide a theoretical analysis to demonstrate why our proposed method enhances learning performance by analyzing its error probability. Furthermore, we extend our proof into PAC-Bayes generalization to illustrate how our method tightens the bounds provided in previous literature. Our numerical experiments on text/image datasets show that our method achieves competitive accuracy with good generalization bounds.

---

PaperID: 373,   

Authors:  Haoran Li, Zhenwen Ren, Yulan Guo, Jiali You, Xiaojian You

Title: LSVC: A Lifelong Learning Approach for Stream-View Clustering

Abstract:
Multiview clustering (MVC) can achieve more accurate results by utilizing complementary information from multiple perspectives, compared to traditional single-view methods. However, current multiview techniques require all views to be available upfront, making them inadequate for dealing with prevalent data sources that arrive as streams, such as stem cell analysis and multicamera surveillance. To address this problem, in this article, we propose a method called lifelong stream-view clustering (LSVC), which comprises an embedding anchor knowledge library and three key components, enabling the capability to perform asynchronous clustering on stream views. These three components are specifically: 1) the knowledge extraction module that extracts the abstract knowledge of the newcome view over time and updates the shared knowledge library; 2) the knowledge transfer module that aligns the newcome view with the historical knowledge library, enabling the transfer of structure information to the knowledge library; and 3) the knowledge rule module that constraints the knowledge library to enjoy a fair amount of anchors for each cluster, improving the discrimination of knowledge. The experimental results show that LSVC outperforms traditional single-view clustering (SVC) and MVC methods as it gradually improves with the accumulation of stream views and tends to be stable over time.

---

PaperID: 374,   

Authors:  Chen Yang, Zhixi Feng, Shuyuan Yang, Qiukai Pan

Title: Open-ICL: Open-Set Modulation Classification via Incremental Contrastive Learning

Abstract:
Open-set modulation classification (OMC) of signals is a challenging task for handling “unknown” modulation types that are not included in the training dataset. This article proposes an incremental contrastive learning method for OMC, called Open-ICL, to accurately identify unknown modulation types of signals. First, a dual-path 1-D network (DONet) with a classification path (CLP) and a contrast path (COP) is designed to learn discriminative signal features cooperatively. In the COP, the deep features of the input signal are compared with the semantic feature centers (SFCs) of known classes calculated from the network, to infer its signal novelty. An unknown signal bank (USB) is defined to store unknown signals, and a novel moving intersection algorithm (MIA) is proposed to dynamically select reliable unknown signals for the USB. The “unknown” instances, together with SFCs, are continuously optimized and updated, facilitating the process of incremental learning. Furthermore, a dynamic adaptive threshold (DAT) strategy is proposed to enable Open-ICL to adaptively learn changing signal distributions. Extensive experiments are performed on two benchmark datasets, and the results demonstrate the effectiveness of Open-ICL for OMC.

---

PaperID: 375,   

Authors:  Jing Wang, Zhiqiang Kou, Yuheng Jia, Jianhui Lv, Xin Geng

Title: Label Distribution Learning by Exploiting Fuzzy Label Correlation

Abstract:
Researchers have proposed to exploit label correlation to alleviate the exponential-size output space of label distribution learning (LDL). In particular, some have designed LDL methods to consider local label correlation. These methods roughly partition the training set into clusters and then exploit local label correlation on each one. Each sample belongs to one cluster and therefore has only one local label correlation. However, in real-world scenarios, the training samples may have fuzziness and belong to multiple clusters with blended local label correlations, which challenge these works. To solve this problem, we propose in LDL fuzzy label correlation (FLC)—each sample blends, with fuzzy membership, multiple local label correlations. First, we propose two types of FLCs, i.e., fuzzy membership-induced label correlation (FC) and joint fuzzy clustering and label correlation (FCC). Then, we put forward LDL-FC and LDL-FCC to exploit these two FLCs, respectively. Finally, we conduct extensive experiments to justify that LDL-FC and LDL-FCC statistically outperform state-of-the-art LDL methods.

---

PaperID: 376,   

Authors:  Yingchun Wang, Song Guo, Jingcai Guo, Yuanhong Zhang, Weizhan Zhang, Qinghua Zheng, Jie Zhang

Title: Data Quality-Aware Mixed-Precision Quantization via Hybrid Reinforcement Learning

Abstract:
Mixed-precision quantization mostly predetermines the model bit-width settings before actual training due to the non-differential bit-width sampling process, obtaining suboptimal performance. Worse still, the conventional static quality-consistent training setting, i.e., all data is assumed to be of the same quality across training and inference, overlooks data quality changes in real-world applications which may lead to poor robustness of the quantized models. In this article, we propose a novel data quality-aware mixed-precision quantization framework, dubbed DQMQ, to dynamically adapt quantization bit-widths to different data qualities. The adaption is based on a bit-width decision policy that can be learned jointly with the quantization training. Concretely, DQMQ is modeled as a hybrid reinforcement learning (RL) task that combines model-based policy optimization with supervised quantization training. By relaxing the discrete bit-width sampling to a continuous probability distribution that is encoded with few learnable parameters, DQMQ is differentiable and can be directly optimized end-to-end with a hybrid optimization target considering both task performance and quantization benefits. Trained on mixed-quality image datasets, DQMQ can implicitly select the most proper bit-width for each layer when facing uneven input qualities. Extensive experiments on various benchmark datasets and networks demonstrate the superiority of DQMQ against existing fixed/mixed-precision quantization methods.

---

PaperID: 377,   

Authors:  Zhiling Fu, Zhe Wang, Chengwei Yu, Xinlei Xu, Dongdong Li

Title: Double Confidence Calibration Focused Distillation for Task-Incremental Learning

Abstract:
Task-incremental learning methods that adopt knowledge distillation face two significant challenges: confidence bias and knowledge loss. These challenges make it difficult to effectively balance the stability and plasticity of the network in the incremental learning process. In this article, we propose double confidence calibration focused distillation (DCCFD) to address these challenges. We introduce intratask and intertask confidence calibration (ECC) modules that can mitigate network overconfidence during incremental learning and reduce the degree of feature representation bias. We also propose a focused distillation (FD) module that can alleviate the problem of knowledge loss during the task increment process, improving model stability without reducing plasticity. Experimental results on the CIFAR-100, TinyImageNet, and CORE-50 datasets demonstrate the effectiveness of our method, with performance that matches or exceeds the state of the art. Furthermore, our method can be used as a plug-and-play module to consistently improve class-incremental learning methods.

---

PaperID: 378,   

Authors:  Yuan Gao, Qian Zhao, Laurence T. Yang, Jing Yang, Lei Ren

Title: Tensor-Representation-Based Multiview Attributed Graph Clustering With Smooth Structure

Abstract:
Over the past few years, multiview attributed graph clustering has achieved promising performance via various data augmentation strategies. However, we observe that the aggregation of node information in multilayer graph autoencoder (GAE) is prone to deviation, especially when edges or node attributes are randomly perturbed. To this end, we innovatively propose a tensor-representation-based multiview attributed graph clustering framework with smooth structure (MV_AGC) to avoid the bias caused by random view construction. Specifically, we first design a novel tensor-product-based high-order graph attention network (GAT) with structural constraints to realize efficient attribute fusion and semantic consistency encoding. By imposing attribute augmentation mechanisms and smooth constraints (SCs) on the proposed high-order graph attention autoencoder simultaneously, MV_AGC effectively eliminates the instability of reconstructed graph structures and learns a more compact node representation during training. In addition, we also theoretically analyze the stronger generality and expressiveness of the proposed tensor-product-based attention mechanism over the classical GAT and establish an intuitive connection between them. Furthermore, to address the performance degradation caused by clustering distribution updating, we further develop a simple yet effective clustering objective function-guided self-optimizing module for the final clustering performance improvement. Experimental results on the six benchmark datasets have demonstrated that our proposed method can achieve state-of-the-art clustering performance.

---

PaperID: 379,   

Authors:  Lanfan Jiang, Zilin Huang, Yu Chen, Wenxing Zhu

Title: Iterative-Weighted Thresholding Method for Group-Sparsity-Constrained Optimization With Applications

Abstract:
Taking advantage of the natural grouping structure inside data, group sparse optimization can effectively improve the efficiency and stability of high-dimensional data analysis, and it has wide applications in a variety of fields such as machine learning, signal processing, and bioinformatics. Although there has been a lot of progress, it is still a challenge to construct a group sparse-inducing function with good properties and to identify significant groups. This article aims to address the group-sparsity-constrained minimization problem. We convert the problem to an equivalent weighted \ell _p,q -norm ( p\gt 0 , 0\lt q\leq 1 ) constrained optimization model, instead of its relaxation or approximation problem. Then, by applying the proximal gradient method, a solution method with theoretical convergence analysis is developed. Moreover, based on the properties proved in the Lagrangian dual framework, the homotopy technique is employed to cope with the parameter tuning task and to ensure that the output of the proposed homotopy algorithm is an L-stationary point of the original problem. The proposed weighted framework, with the central idea of identifying important groups, is compatible with a wide range of support set identification strategies, which can better meet the needs of different applications and improve the robustness of the model in practice. Both simulated and real data experiments demonstrate the superiority of the proposed method in terms of group feature selection accuracy and computational efficiency. Extensive experimental results in application areas such as compressed sensing, image recognition, and classifier design show that our method has great potential in a wide range of applications. Our codes will be available at https://github.com/jianglanfan/ HIWT-GSC.

---

PaperID: 380,   

Authors:  Jing Lai, Junlin Xiong, Yu Kang

Title: Value Iteration for Stochastic LQR With Convergence Guarantees

Abstract:
This brief studies the discounted stochastic linear quadratic regulator (LQR) problem for systems suffering from additive noise of unknown mean. A completely model-free (MF) value iteration (VI) algorithm is developed to learn the optimal control policy using off-line system trajectories. The generated control policies are proven to converge to a small neighborhood of the optimal ones with high probability. In addition, an MF algorithm is proposed to learn a feasible discount factor. The proposed MF algorithms are illustrated through several examples.

---

PaperID: 381,   

Authors:  Haoxuanye Ji, Le Wang, Sanping Zhou, Wei Tang, Nanning Zheng, Gang Hua

Title: Meta Pairwise Relationship Distillation for Unsupervised Person Re-Identification

Abstract:
Unsupervised person re-identification (Re-ID) is challenging due to the lack of ground-truth labels. Most existing methods rely on pseudo labels estimated via iterative clustering and thus are highly susceptible to performance penalties incurred by the inaccurate estimated number of clusters. Alternatively, we utilize the sample pairs with pairwise pseudo labels to guide the feature learning to avoid the dilemma of determining cluster numbers. In this article, we propose a meta pairwise relationship distillation (MPRD) method that incorporates a graph convolutional network (GCN) to provide high-fidelity pairwise relationships to supervise the model training. A small amount of metadata with very-confidence pairwise relationships and the unlabeled pairs with the provided pseudo pairwise relationships participate in the GCN training. Besides, we introduce a hard sample deduction (HSD) module to timely mine the sample pairs with error-prone pairwise pseudo labels to mitigate the misled optimization by noisy labels. Furthermore, since the features of each positive pair represent the same person, we design a positive pair alignment (PPA) module to reduce the redundant information in each feature, which is achieved by minimizing the difference between each positive pair’s feature distributions. Extensive experiments on the Market-1501, DukeMTMC-reID, and MSMT17 datasets show that our method outperforms the state-of-the-art unsupervised methods.

---

PaperID: 382,   

Authors:  Xiaoliu Luo, Zhao Duan, Anyong Qin, Zhuotao Tian, Ting Xie, Taiping Zhang, Yuan Yan Tang

Title: Layer-Wise Mutual Information Meta-Learning Network for Few-Shot Segmentation

Abstract:
The goal of few-shot segmentation (FSS) is to segment unlabeled images belonging to previously unseen classes using only a limited number of labeled images. The main objective is to transfer label information effectively from support images to query images. In this study, we introduce a novel meta-learning framework called layer-wise mutual information (LayerMI), which enhances the propagation of label information by maximizing the mutual information (MI) between support and query features at each layer. Our approach involves the utilization of a LayerMI Block based on information-theoretic co-clustering. This block performs online co-clustering on the joint probability distribution obtained from each layer, generating a target-specific attention map. The LayerMI Block can be seamlessly integrated into the meta-learning framework and applied to all convolutional neural network (CNN) layers without altering the training objectives. Notably, the LayerMI Block not only maximizes MI between support and query features but also facilitates internal clustering within the image. Extensive experiments demonstrate that LayerMI significantly enhances the performance of baseline and achieves competitive performance compared to state-of-the-art methods on three challenging benchmarks: PASCAL- 5^i , COCO- 20^i , and FSS-1000.

---

PaperID: 383,   

Authors:  Shuqi Xu, Hao Zhang, Zhuping Wang

Title: Learning to Perform Trajectory Generation From Low-Quality Demonstrations

Abstract:
Human-robot skill transfer is an important means for robots to learn skills and has received more and more attention and research in recent years. Typically, to ensure effective skill transfer, a skill is demonstrated several times by a human, from which a robot learns the features contained in the demonstrations and reproduces the skill in a new environment. However, it is necessary to consider the cases such as errors in human demonstrations and sensor issues, resulting in imperfect demonstrations, unrelated data, information loss, and variations in the lengths and amplitudes of the demonstrations. Therefore, this brief proposes a new trajectory alignment and filtering method for extracting relatively useful information from multiple demonstrations. This method can be used in conjunction with most probabilistic movement learning methods (this brief uses probabilistic movement primitives (ProMPs) as an example) for learning from demonstrations (LfDs), so that the robot can eventually learn and generate trajectories for completing skills from multiple demonstrations of varying quality. The effectiveness of the proposed method is verified by simulation results.

---

PaperID: 384,   

Authors:  Licheng Jiao, Mengru Ma, Pei He, Xueli Geng, Xu Liu, Fang Liu, Wenping Ma, Shuyuan Yang, Biao Hou, Xu Tang

Title: Brain-Inspired Learning, Perception, and Cognition: A Comprehensive Review

Abstract:
The progress of brain cognition and learning mechanisms has provided new inspiration for the next generation of artificial intelligence (AI) and provided the biological basis for the establishment of new models and methods. Brain science can effectively improve the intelligence of existing models and systems. Compared with other reviews, this article provides a comprehensive review of brain-inspired deep learning algorithms for learning, perception, and cognition from microscopic, mesoscopic, macroscopic, and super-macroscopic perspectives. First, this article introduces the brain cognition mechanism. Then, it summarizes the existing studies on brain-inspired learning and modeling from the perspectives of neural structure, cognitive module, learning mechanism, and behavioral characteristics. Next, this article introduces the potential learning directions of brain-inspired learning from four aspects: perception, cognition, understanding, and decision-making. Finally, the top-ten open problems that brain-inspired learning, perception, and cognition currently face are summarized, and the next generation of AI technology has been prospected. This work intends to provide a quick overview of the research on brain-inspired AI algorithms and to motivate future research by illuminating the latest developments in brain science.

---

PaperID: 385,   

Authors:  Xiong Yang, Ding Wang

Title: Reinforcement Learning for Robust Dynamic Event-Driven Constrained Control

Abstract:
We consider a robust dynamic event-driven control (EDC) problem of nonlinear systems having both unmatched perturbations and unknown styles of constraints. Specifically, the constraints imposed on the nonlinear systems’ input could be symmetric or asymmetric. Initially, to tackle such constraints, we construct a novel nonquadratic cost function for the constrained auxiliary system. Then, we propose a dynamic event-triggering mechanism relied on the time-based variable and the system states simultaneously for cutting down the computational load. Meanwhile, we show that the robust dynamic EDC of original nonlinear-constrained systems could be acquired by solving the event-driven optimal control problem of the constrained auxiliary system. After that, we develop the corresponding event-driven Hamilton-Jacobi–Bellman equation, and then solve it through a unique critic neural network (CNN) in the reinforcement learning framework. To relax the persistence of excitation condition in tuning CNN’s weights, we incorporate experience replay into the gradient descent method. With the aid of Lyapunov’s approach, we prove that the closed-loop auxiliary system and the weight estimation error are uniformly ultimately bounded stable. Finally, two examples, including a nonlinear plant and the pendulum system, are utilized to validate the theoretical claims.

---

PaperID: 386,   

Authors:  Yue Liu, Sihang Zhou, Xihong Yang, Xinwang Liu, Wenxuan Tu, Liang Li, Xin Xu, Fuchun Sun

Title: Improved Dual Correlation Reduction Network With Affinity Recovery

Abstract:
Deep graph clustering, which aims to reveal the underlying graph structure and divide the nodes into different clusters without human annotations, is a fundamental yet challenging task. However, we observe that the existing methods suffer from the representation collapse problem and tend to encode samples with different classes into the same latent embedding. Consequently, the discriminative capability of nodes is limited, resulting in suboptimal clustering performance. To address this problem, we propose a novel deep graph clustering algorithm termed improved dual correlation reduction network (IDCRN) through improving the discriminative capability of samples. Specifically, by approximating the cross-view feature correlation matrix to an identity matrix, we reduce the redundancy between different dimensions of features, thus improving the discriminative capability of the latent space explicitly. Meanwhile, the cross-view sample correlation matrix is forced to approximate the designed clustering-refined adjacency matrix to guide the learned latent representation to recover the affinity matrix even across views, thus enhancing the discriminative capability of features implicitly. Moreover, we avoid the collapsed representation caused by the oversmoothing issue in graph convolutional networks (GCNs) through an introduced propagation regularization term, enabling IDCRN to capture the long-range information with the shallow network structure. Extensive experimental results on six benchmarks have demonstrated the effectiveness and efficiency of IDCRN compared with the existing state-of-the-art deep graph clustering algorithms. The code of IDCRN is released at IDCRN. Besides, we share a collection of deep graph clustering, including papers, codes, and datasets at ADGC.

---

PaperID: 387,   

Authors:  Jiang Lu, Changming Xiao, Changshui Zhang

Title: Meta-Modulation: A General Learning Framework for Cross-Task Adaptation

Abstract:
Building learning systems possessing adaptive flexibility to different tasks is critical and challenging. In this article, we propose a novel and general meta-learning framework, called meta-modulation (MeMo), to foster the adaptation capability of a base learner across different tasks where only a few training data are available per task. For one independent task, MeMo proceeds like a “feedback regulation system,” which achieves an adaptive modulation on the so-called definitive embeddings of query data to maximize the corresponding task objective. Specifically, we devise a type of efficient feedback information, definitive embedding feedback (DEF), to mathematize and quantify the unsuitability between the few training data and the base learner as well as the promising adjustment direction to reduce this unsuitability. The DEFs are encoded into high-level representation and temporarily stored as task-specific modulator templates by a modulation encoder. For coming query data, we develop an attention mechanism acting upon these modulator templates and combine both task/data-level modulation to generate the final data-specific meta-modulator. This meta-modulator is then used to modulate the query’s embedding for correct decision-making. Our framework is scalable for various base learner models like multi-layer perceptron (MLP), long short-term memory (LSTM), convolutional neural network (CNN), and transformer, and applicable to different learning problems like language modeling and image recognition. Experimental results on a 2-D point synthetic dataset and various benchmarks in language and vision domains demonstrate the effectiveness and competitiveness of our framework.

---

PaperID: 388,   

Authors:  Yunpeng Xiao, Jinsong Yang, Wanjing Zhao, Qian Li, Yucai Pang

Title: Cross-Domain Social Rumor-Propagation Model Based on Transfer Learning

Abstract:
Rumors in different topic domains have different text characteristics but similar emotional tendencies. To resolve the scarce-data problem in some rumor-topic domains, this study proposes a cross-domain rumor-propagation model, which is based on transfer learning. First, given the diversity and complexity of the rumor-propagation landscape, this study introduces a novel method, User-Retweet–Rumor2vec (URR2vec), which leverages the power of representation learning to uncover latent features within rumor topics. It also displays the forwarding relationship between users and rumors, user node information, and rumor-topic information in low-dimensional space. To capture the impact of human emotional cognition during rumor spreading, we also introduce a deep-learning model based on the natural language texts of rumor topics, which analyzes the sentiment in the text and uncovers the emotional correlations among users. Furthermore, a rumor-propagation prediction model based on the text-sentiment analysis-graph convolutional network (TSA-GCN) is proposed and pre-trained on existing rumor-topic data to ensure its prediction accuracy. Finally, considering the data sparsity at a rumor-topic outbreak, the trained propagation model is transferred to the rumor topic for prediction. Meanwhile, the rumor topic in different domains has different edges and conditional distribution, similar emotional characteristics, and network structure among the rumor topics. After fine-tuning the parameter and adding a domain adaptation layer in TSA-GCN, a domain adaptation model based on parameter and graph-structure migration is obtained.

---

PaperID: 389,   

Authors:  Gengyu Lyu, Zhen Yang, Xiang Deng, Songhe Feng

Title: L-VSM: Label-Driven View-Specific Fusion for Multiview Multilabel Classification

Abstract:
In the task of multiview multilabel (MVML) classification, each instance is represented by several heterogeneous features and associated with multiple semantic labels. Existing MVML methods mainly focus on leveraging the shared subspace to comprehensively explore multiview consensus information across different views, while it is still an open problem whether such shared subspace representation is effective to characterize all relevant labels when formulating a desired MVML model. In this article, we propose a novel label-driven view-specific fusion MVML method named L-VSM, which bypasses seeking for a shared subspace representation and instead directly encodes the feature representation of each individual view to contribute to the final multilabel classifier induction. Specifically, we first design a label-driven feature graph construction strategy and construct all instances under various feature representations into the corresponding feature graphs. Then, these view-specific feature graphs are integrated into a unified graph by linking the different feature representations within each instance. Afterward, we adopt a graph attention mechanism to aggregate and update all feature nodes on the unified graph to generate structural representations for each instance, where both intraview correlations and interview alignments are jointly encoded to discover the underlying consensuses and complementarities across different views. Moreover, to explore the widespread label correlations in multilabel learning (MLL), the transformer architecture is introduced to construct a dynamic semantic-aware label graph and accordingly generate structural semantic representations for each specific class. Finally, we derive an instance-label affinity score for each instance by averaging the affinity scores of its different feature representations with the multilabel soft margin loss. Extensive experiments on various MVML applications have verified that our proposed L-VSM has achieved superior performance against state-of-the-art methods. The codes are available at https://gengyulyu.github.io/homepage/assets/codes/LVSM.zip.

---

PaperID: 390,   

Authors:  Qi Chen, Chao Li, Jia Ning, Stephen Lin, Kun He

Title: GMConv: Modulating Effective Receptive Fields for Convolutional Kernels

Abstract:
In convolutional neural networks (CNNs), the convolutions are conventionally performed using a square kernel with a fixed N × N receptive field (RF). However, what matters most to the network is the effective receptive field (ERF), which indicates the extent to which input pixels contribute to an output pixel. Inspired by the property that ERFs typically exhibit a Gaussian distribution, we propose a Gaussian Mask convolutional kernel (GMConv). Specifically, GMConv utilizes the Gaussian function to generate a concentric symmetry mask that is placed over the kernel to refine the RF. We analyze the RFs of CNN kernels in different CNN layers and evaluate our approach through extensive experiments on image classification and object detection tasks. Over several tasks and standard base models, our approach compares favorably against the standard convolution. For instance, using GMConv for AlexNet and ResNet-50, the top-1 accuracy on ImageNet classification is boosted by 0.98% and 0.85%, respectively.

---

PaperID: 391,   

Authors:  Sinan Tan, Kuankuan Sima, Dunzheng Wang, Mengmeng Ge, Di Guo, Huaping Liu

Title: Self-Supervised 3-D Semantic Representation Learning for Vision-and-Language Navigation

Abstract:
In vision-and-language navigation (VLN) tasks, most current methods primarily utilize RGB images, overlooking the rich 3-D semantic data inherent to environments. To rectify this, we introduce a novel VLN framework that integrates 3-D semantic information into the navigation process. Our approach features a self-supervised training scheme that incorporates voxel-level 3-D semantic reconstruction to create a detailed 3-D semantic representation. A key component of this framework is a pretext task focused on region queries, which determines the presence of objects in specific 3-D areas. Following this, we devise an long short-term memory (LSTM)-based navigation model that is trained using our 3-D semantic representations. To maximize the utility of these 3-D semantic representations, we implement a cross-modal distillation strategy. This strategy encourages the RGB model’s outputs to emulate those from the 3-D semantic feature network, enabling the concurrent training of both branches to merge RGB and 3-D semantic data effectively. Comprehensive evaluations on both the R2R and R4R datasets reveal that our method significantly enhances performance in VLN tasks.

---

PaperID: 392,   

Authors:  Peng Xing, Hao Tang, Jinhui Tang, Zechao Li

Title: ADPS: Asymmetric Distillation Postsegmentation for Image Anomaly Detection

Abstract:
Knowledge distillation-based anomaly detection (KDAD) methods rely on the teacher–student paradigm to detect and segment anomalous regions by contrasting the unique features extracted by both networks. However, existing KDAD methods suffer from two main limitations: 1) the student network can effortlessly replicate the teacher network’s representations and 2) the features of the teacher network serve solely as a “reference standard” and are not fully leveraged. Toward this end, we depart from the established paradigm and instead propose an innovative approach called asymmetric distillation postsegmentation (ADPS). Our ADPS employs an asymmetric distillation paradigm that takes distinct forms of the same image as the input of the teacher–student networks, driving the student network to learn discriminating representations for anomalous regions. Meanwhile, a customized Weight Mask Block (WMB) is proposed to generate a coarse anomaly localization mask that transfers the distilled knowledge acquired from the asymmetric paradigm to the teacher network. Equipped with WMB, the proposed postsegmentation module (PSM) can effectively detect and segment abnormal regions with fine structures and clear boundaries. Experimental results demonstrate that the proposed ADPS outperforms the state-of-the-art methods in detecting and segmenting anomalies. Surprisingly, ADPS significantly improves average precision (AP) metric by \mathbf 9% and \mathbf 20% on the MVTec anomaly detection (AD) and KolektorSDD2 datasets, respectively.

---

PaperID: 393,   

Authors:  Man-Sheng Chen, Xi-Ran Zhu, Jia-Qi Lin, Chang-Dong Wang

Title: Contrastive Multiview Attribute Graph Clustering With Adaptive Encoders

Abstract:
Multiview attribute graph clustering aims to cluster nodes into disjoint categories by taking advantage of the multiview topological structures and the node attribute values. However, the existing works fail to explicitly discover the inherent relationships in multiview topological graph matrices while considering different properties between the graphs. Besides, they cannot well handle the sparse structure of some graphs in the learning procedure of graph embeddings. Therefore, in this article, we propose a novel contrastive multiview attribute graph clustering (CMAGC) with adaptive encoders method. Within this framework, the adaptive encoders concerning different properties of distinct topological graphs are chosen to integrate multiview attribute graph information by checking whether there exists high-order neighbor information or not. Meanwhile, the number of layers of the GCN encoders is selected according to the prior knowledge related to the characteristics of different topological graphs. In particular, the feature-level and cluster-level contrastive learning are conducted on the multiview soft assignment representations, where the union of the first-order neighbors from the corresponding graph pairs is regarded as the positive pairs for data augmentation and the sparse neighbor information problem in some graphs can be well dealt with. To the best of our knowledge, it is the first time to explicitly deal with the inherent relationships from the interview and intraview perspectives. Extensive experiments are conducted on several datasets to verify the superiority of the proposed CMAGC method compared with the state-of-the-art methods.

---

PaperID: 394,   

Authors:  Chun-Mei Feng, Zhanyuan Yang, Huazhu Fu, Yong Xu, Jian Yang, Ling Shao

Title: DONet: Dual-Octave Network for Fast MR Image Reconstruction

Abstract:
Magnetic resonance (MR) image acquisition is an inherently prolonged process, whose acceleration has long been the subject of research. This is commonly achieved by obtaining multiple undersampled images, simultaneously, through parallel imaging. In this article, we propose the dual-octave network (DONet), which is capable of learning multiscale spatial-frequency features from both the real and imaginary components of MR data, for parallel fast MR image reconstruction. More specifically, our DONet consists of a series of dual-octave convolutions (Dual-OctConvs), which are connected in a dense manner for better reuse of features. In each Dual-OctConv, the input feature maps and convolutional kernels are first split into two components (i.e., real and imaginary) and then divided into four groups according to their spatial frequencies. Then, our Dual-OctConv conducts intragroup information updating and intergroup information exchange to aggregate the contextual information across different groups. Our framework provides three appealing benefits: 1) it encourages information interaction and fusion between the real and imaginary components at various spatial frequencies to achieve richer representational capacity; 2) the dense connections between the real and imaginary groups in each Dual-OctConv make the propagation of features more efficient by feature reuse; and 3) DONet enlarges the receptive field by learning multiple spatial-frequency features of both the real and imaginary components. Extensive experiments on two popular datasets (i.e., clinical knee and fastMRI), under different undersampling patterns and acceleration factors, demonstrate the superiority of our model in accelerated parallel MR image reconstruction.

---

PaperID: 395,   

Authors:  Yutao Hu, Xiaolong Jiang, Xuhui Liu, Xiaoyan Luo, Yao Hu, Xianbin Cao, Baochang Zhang, Jun Zhang

Title: Hierarchical Self-Distilled Feature Learning for Fine-Grained Visual Categorization

Abstract:
Fine-grained visual categorization (FGVC) relies on hierarchical features extracted by deep convolutional neural networks (CNNs) to recognize closely alike objects. Particularly, shallow layer features containing rich spatial details are vital for specifying subtle differences between objects but are usually inadequately optimized due to gradient vanishing during backpropagation. In this article, hierarchical self-distillation (HSD) is introduced to generate well-optimized CNNs features for accurate fine-grained categorization. HSD inherits from the widely applied deep supervision and implements multiple intermediate losses for reinforced gradients. Besides that, we observe that the hard (one-hot) labels adopted for intermediate supervision hurt the performance of FGVC by enforcing overstrict supervision. As a solution, HSD seeks self-distillation where soft predictions generated by deeper layers of the network are hierarchically exploited to supervise shallow parts. Moreover, self-information entropy loss (SIELoss) is designed in HSD to adaptively soften intermediate predictions and facilitate better convergence. In addition, the gradient detached fusion (GDF) module is incorporated to produce an ensemble result with multiscale features via effective feature fusion. Extensive experiments on four challenging fine-grained datasets show that, with neglectable parameter increase, the proposed HSD framework and the GDF module both bring significant performance gains over different backbones, which also achieves state-of-the-art classification performance.

---

PaperID: 396,   

Authors:  Shijie Hao, Yuan Zhou, Yanrong Guo, Richang Hong, Jun Cheng, Meng Wang

Title: Real-Time Semantic Segmentation via Spatial-Detail Guided Context Propagation

Abstract:
Nowadays, vision-based computing tasks play an important role in various real-world applications. However, many vision computing tasks, e.g., semantic segmentation, are usually computationally expensive, posing a challenge to the computing systems that are resource-constrained but require fast response speed. Therefore, it is valuable to develop accurate and real-time vision processing models that only require limited computational resources. To this end, we propose the spatial-detail guided context propagation network (SGCPNet) for achieving real-time semantic segmentation. In SGCPNet, we propose the strategy of spatial-detail guided context propagation. It uses the spatial details of shallow layers to guide the propagation of the low-resolution global contexts, in which the lost spatial information can be effectively reconstructed. In this way, the need for maintaining high-resolution features along the network is freed, therefore largely improving the model efficiency. On the other hand, due to the effective reconstruction of spatial details, the segmentation accuracy can be still preserved. In the experiments, we validate the effectiveness and efficiency of the proposed SGCPNet model. On the Cityscapes dataset, for example, our SGCPNet achieves 69.5% mIoU segmentation accuracy, while its speed reaches 178.5 FPS on 768 × 1536 images on a GeForce GTX 1080 Ti GPU card. In addition, SGCPNet is very lightweight and only contains 0.61 M parameters. The code will be released at https://github.com/zhouyuan888888/SGCPNet.

---

PaperID: 397,   

Authors:  Shafin Rahman, Salman Khan, Nick Barnes

Title: Polarity Loss: Improving Visual-Semantic Alignment for Zero-Shot Detection

Abstract:
Conventional object detection models require large amounts of training data. In comparison, humans can recognize previously unseen objects by merely knowing their semantic description. To mimic similar behavior, zero-shot object detection (ZSD) aims to recognize and localize “unseen” object instances by using only their semantic information. The model is first trained to learn the relationships between visual and semantic domains for seen objects, later transferring the acquired knowledge to totally unseen objects. This setting gives rise to the need for correct alignment between visual and semantic concepts so that the unseen objects can be identified using only their semantic attributes. In this article, we propose a novel loss function called “polarity loss” that promotes correct visual-semantic alignment for an improved ZSD. On the one hand, it refines the noisy semantic embeddings via metric learning on a “semantic vocabulary” of related concepts to establish a better synergy between visual and semantic domains. On the other hand, it explicitly maximizes the gap between positive and negative predictions to achieve better discrimination between seen, unseen, and background objects. Our approach is inspired by embodiment theories in cognitive science that claim human semantic understanding to be grounded in past experiences (seen objects), related linguistic concepts (word vocabulary), and visual perception (seen/unseen object images). We conduct extensive evaluations on the Microsoft Common Objects in Context (MS-COCO) and Pascal Visual Object Classes (VOC) datasets, showing significant improvements over state of the art. Our code and evaluation protocols available at: https://github.com/salman-h-khan/PL-ZSD_Release.

---

PaperID: 398,   

Authors:  Lu Zhang, Zhiyong Liu, Xiangyu Zhu, Zhan Song, Xu Yang, Zhen Lei, Hong Qiao

Title: Weakly Aligned Feature Fusion for Multimodal Object Detection

Abstract:
To achieve accurate and robust object detection in the real-world scenario, various forms of images are incorporated, such as color, thermal, and depth. However, multimodal data often suffer from the position shift problem, i.e., the image pair is not strictly aligned, making one object has different positions in different modalities. For the deep learning method, this problem makes it difficult to fuse multimodal features and puzzles the convolutional neural network (CNN) training. In this article, we propose a general multimodal detector named aligned region CNN (AR-CNN) to tackle the position shift problem. First, a region feature (RF) alignment module with adjacent similarity constraint is designed to consistently predict the position shift between two modalities and adaptively align the cross-modal RFs. Second, we propose a novel region of interest (RoI) jitter strategy to improve the robustness to unexpected shift patterns. Third, we present a new multimodal feature fusion method that selects the more reliable feature and suppresses the less useful one via feature reweighting. In addition, by locating bounding boxes in both modalities and building their relationships, we provide novel multimodal labeling named KAIST-Paired. Extensive experiments on 2-D and 3-D object detection, RGB-T, and RGB-D datasets demonstrate the effectiveness and robustness of our method.

---

PaperID: 399,   

Authors:  Rui Liu, Pengwei Xing, Zichao Deng, Anran Li, Cuntai Guan, Han Yu

Title: Federated Graph Neural Networks: Overview, Techniques, and Challenges

Abstract:
Graph neural networks (GNNs) have attracted extensive research attention in recent years due to their capability to progress with graph data and have been widely used in practical applications. As societies become increasingly concerned with the need for data privacy protection, GNNs face the need to adapt to this new normal. Besides, as clients in federated learning (FL) may have relationships, more powerful tools are required to utilize such implicit information to boost performance. This has led to the rapid development of the emerging research field of federated GNNs (FedGNNs). This promising interdisciplinary field is highly challenging for interested researchers to grasp. The lack of an insightful survey on this topic further exacerbates the entry difficulty. In this article, we bridge this gap by offering a comprehensive survey of this emerging field. We propose a 2-D taxonomy of the FedGNN literature: 1) the main taxonomy provides a clear perspective on the integration of GNNs and FL by analyzing how GNNs enhance FL training as well as how FL assists GNN training and 2) the auxiliary taxonomy provides a view on how FedGNNs deal with heterogeneity across FL clients. Through discussions of key ideas, challenges, and limitations of existing works, we envision future research directions that can help build more robust, explainable, efficient, fair, inductive, and comprehensive FedGNNs.

---

PaperID: 400,   

Authors:  Van Tien Pham, Yassine Zniyed, Thanh Phuong Nguyen

Title: Enhanced Network Compression Through Tensor Decompositions and Pruning

Abstract:
Network compression techniques that combine tensor decompositions and pruning have shown promise in leveraging the advantages of both strategies. In this work, we propose enhanced Network cOmpRession through TensOr decompositions and pruNing (NORTON), a novel method for network compression. NORTON introduces the concept of filter decomposition, enabling a more detailed decomposition of the network while preserving the weight’s multidimensional properties. Our method incorporates a novel structured pruning approach, effectively integrating the decomposed model. Through extensive experiments on various architectures, benchmark datasets, and representative vision tasks, we demonstrate the usefulness of our method. NORTON achieves superior results compared to state-of-the-art (SOTA) techniques in terms of complexity and accuracy. Our code is also available for research purposes.

---

PaperID: 401,   

Authors:  Qian Li, Zhichao Wang, Haiyang Xia, Gang Li, Yanan Cao, Lina Yao, Guandong Xu

Title: HOT-GAN: Hilbert Optimal Transport for Generative Adversarial Network

Abstract:
Generative adversarial network (GAN) has achieved remarkable success in generating high-quality synthetic data by learning the underlying distributions of target data. Recent efforts have been devoted to utilizing optimal transport (OT) to tackle the gradient vanishing and instability issues in GAN. They use the Wasserstein distance as a metric to measure the discrepancy between the generator distribution and the real data distribution. However, most optimal transport GANs define loss functions in Euclidean space, which limits their capability in handling high-order statistics that are of much interest in a variety of practical applications. In this article, we propose a computational framework to alleviate this issue from both theoretical and practical perspectives. Particularly, we generalize the optimal transport-based GAN from Euclidean space to the reproducing kernel Hilbert space (RKHS) and propose Hilbert Optimal Transport GAN (HOT-GAN). First, we design HOT-GAN with a Hilbert embedding that allows the discriminator to tackle more informative and high-order statistics in RKHS. Second, we prove that HOT-GAN has a closed-form kernel reformulation in RKHS that can achieve a tractable objective under the GAN framework. Third, HOT-GAN’s objective enjoys the theoretical guarantee of differentiability with respect to generator parameters, which is beneficial to learn powerful generators via adversarial kernel learning. Extensive experiments are conducted, showing that our proposed HOT-GAN consistently outperforms the representative GAN works.

---

PaperID: 402,   

Authors:  Pengwei Liang, Junjun Jiang, Xianming Liu, Jiayi Ma

Title: Image Deblurring by Exploring In-Depth Properties of Transformer

Abstract:
Image deblurring continues to achieve impressive performance with the development of generative models. Nonetheless, there still remains a displeasing problem if one wants to improve perceptual quality and quantitative scores of recovered image at the same time. In this study, drawing inspiration from the research of transformer properties, we introduce the pretrained transformers to address this problem. In particular, we leverage deep features extracted from a pretrained vision transformer (ViT) to encourage recovered images to be sharp without sacrificing the performance measured by the quantitative metrics. The pretrained transformer can capture the global topological relations (i.e., self-similarity) of image, and we observe that the captured topological relationships about the sharp image will change when blur occurs. By comparing the transformer features between recovered image and target one, the pretrained transformer provides high-resolution blur-sensitive semantic information, which is critical in measuring the sharpness of the deblurred image. On the basis of the advantages, we present two types of novel perceptual losses to guide image deblurring. One regards the features as vectors and computes the discrepancy between representations extracted from recovered image and target one in Euclidean space. The other type considers the features extracted from an image as a distribution and compares the distribution discrepancy between recovered image and target one. We demonstrate the effectiveness of transformer properties in improving the perceptual quality while not sacrificing the quantitative scores peak signal-to-noise ratio (PSNR) over the most competitive models, such as Uformer, Restormer, and NAFNet, on defocus deblurring and motion deblurring tasks. The code is available at https://github. com/erfect2020/TransformerPerceptualLoss.

---

PaperID: 403,   

Authors:  Mingfeng Fan, Yaoxin Wu, Zhiguang Cao, Wen Song, Guillaume Sartoretti, Huan Liu, Guohua Wu

Title: Conditional Neural Heuristic for Multiobjective Vehicle Routing Problems

Abstract:
Existing neural heuristics for multiobjective vehicle routing problems (MOVRPs) are primarily conditioned on instance context, which failed to appropriately exploit preference and problem size, thus holding back the performance. To thoroughly unleash the potential, we propose a novel conditional neural heuristic (CNH) that fully leverages the instance context, preference, and size with an encoder–decoder structured policy network. Particularly, in our CNH, we design a dual-attention-based encoder to relate preferences and instance contexts, so as to better capture their joint effect on approximating the exact Pareto front (PF). We also design a size-aware decoder based on the sinusoidal encoding to explicitly incorporate the problem size into the embedding, so that a single trained model could better solve instances of various scales. Besides, we customize the REINFORCE algorithm to train the neural heuristic by leveraging stochastic preferences (SPs), which further enhances the training performance. Extensive experimental results on random and benchmark instances reveal that our CNH could achieve favorable approximation to the whole PF with higher hypervolume (HV) and lower optimality gap (Gap) than those of the existing neural and conventional heuristics. More importantly, a single trained model of our CNH can outperform other neural heuristics that are exclusively trained on each size. In addition, the effectiveness of the key designs is also verified through ablation studies.

---

PaperID: 404,   

Authors:  Feiping Nie, Han Liu, Rong Wang, Xuelong Li

Title: Parameter-Free Multiview K-Means Clustering With Coordinate Descent Method

Abstract:
Recently, more and more real-world datasets have been composed of heterogeneous but related features from diverse views. Multiview clustering provides a promising attempt at a solution for partitioning such data according to heterogeneous information. However, most existing methods suffer from hyper-parameter tuning trouble and high computational cost. Besides, there is still an opportunity for improvement in clustering performance. To this end, a novel multiview framework, called parameter-free multiview k -means clustering with coordinate descent method (PFMVKM), is presented to address the above problems. Specifically, PFMVKM is completely parameter-free and learns the weights via a self-weighted scheme, which can avoid the intractable process of hyper-parameters tuning. Moreover, our model is capable of directly calculating the cluster indicator matrix, with no need to learn the cluster centroid matrix and the indicator matrix simultaneously as previous multiview methods have to do. What’s more, we propose an efficient optimization algorithm utilizing the idea of coordinate descent, which can not only reduce the computational complexity but also improve the clustering performance. Extensive experiments on various types of real datasets illustrate that the proposed method outperforms existing state-of-the-art competitors and conforms well with the actual situation.

---

PaperID: 405,   

Authors:  Rui-Jie Zhu, Malu Zhang, Qihang Zhao, Haoyu Deng, Yule Duan, Liang-Jian Deng

Title: TCJA-SNN: Temporal-Channel Joint Attention for Spiking Neural Networks

Abstract:
Spiking neural networks (SNNs) are attracting widespread interest due to their biological plausibility, energy efficiency, and powerful spatiotemporal information representation ability. Given the critical role of attention mechanisms in enhancing neural network performance, the integration of SNNs and attention mechanisms exhibits tremendous potential to deliver energy-efficient and high-performance computing paradigms. In this article, we present a novel temporal-channel joint attention mechanism for SNNs, referred to as TCJA-SNN. The proposed TCJA-SNN framework can effectively assess the significance of spike sequence from both spatial and temporal dimensions. More specifically, our essential technical contribution lies on: 1) we employ the squeeze operation to compress the spike stream into an average matrix. Then, we leverage two local attention mechanisms based on efficient 1-D convolutions to facilitate comprehensive feature extraction at the temporal and channel levels independently and 2) we introduce the cross-convolutional fusion (CCF) layer as a novel approach to model the interdependencies between the temporal and channel scopes. This layer effectively breaks the independence of these two dimensions and enables the interaction between features. Experimental results demonstrate that the proposed TCJA-SNN outperforms the state-of-the-art (SOTA) on all standard static and neuromorphic datasets, including Fashion-MNIST, CIFAR10, CIFAR100, CIFAR10-DVS, N-Caltech 101, and DVS128 Gesture. Furthermore, we effectively apply the TCJA-SNN framework to image generation tasks by leveraging a variation autoencoder. To the best of our knowledge, this study is the first instance where the SNN-attention mechanism has been employed for high-level classification and low-level generation tasks. Our implementation codes are available at https://github.com/ridgerchu/TCJA.

---

PaperID: 406,   

Authors:  Zhiwei Chen, Siwei Wang, Liujuan Cao, Yunhang Shen, Rongrong Ji

Title: Adaptive Zone Learning for Weakly Supervised Object Localization

Abstract:
Weakly supervised object localization (WSOL) stands as a pivotal endeavor within the realm of computer vision, entailing the location of objects utilizing merely image-level labels. Contemporary approaches in WSOL have leveraged FPMs, yielding commendable outcomes. However, these existing FPM-based techniques are predominantly confined to rudimentary strategies of either augmenting the foreground or diminishing the background presence. We argue for the exploration and exploitation of the intricate interplay between the object’s foreground and its background to achieve efficient object localization. In this manuscript, we introduce an innovative framework, termed adaptive zone learning (AZL), which operates on a coarse-to-fine basis to refine FPMs through a triad of adaptive zone mechanisms. First, an adversarial learning mechanism (ALM) is employed, orchestrating an interplay between the foreground and background regions. This mechanism accentuates coarse-grained object regions in a mutually adversarial manner. Subsequently, an oriented learning mechanism (OLM) is unveiled, which harnesses local insights from both foreground and background in a fine-grained manner. This mechanism is instrumental in delineating object regions with greater granularity, thereby generating better FPMs. Furthermore, we propose a reinforced learning mechanism (RLM) as the compensatory mechanism for adversarial design, by which the undesirable foreground maps are refined again. Extensive experiments on CUB-200-2011 and ILSVRC datasets demonstrate that AZL achieves significant and consistent performance improvements over other state-of-the-art WSOL methods.

---

PaperID: 407,   

Authors:  Xiao-Yang Liu, Xiaodong Wang, Bo Yuan, Jiashu Han

Title: Spectral Tensor Layers for Communication-Free Distributed Deep Learning

Abstract:
In this article, we propose a novel spectral tensor layer for communication-free distributed deep learning. The overall framework is as follows: first, we represent the data in tensor form (instead of vector form) and replace the matrix product in conventional neural networks with the tensor product, which in effect imposes certain transformed-induced structure on the original weight matrices, e.g., a block-circulant structure; then, we apply a linear transform along a certain dimension to split the original dataset into multiple spectral subdatasets; as a result, the proposed spectral tensor network consists of parallel branches where each branch is a conventional neural network trained on a spectral subdataset with ZERO communication cost. The parallel branches are directly ensembled (i.e., the weighted sum of their outputs) to generate an overall network with substantially stronger generalization capability than that of each branch. Moreover, the proposed method enjoys a byproduct of decentralization gain in terms of memory and computation, compared with traditional networks. It is a natural yet elegant solution for heterogeneous data in federated learning (FL), where data at different nodes have different resolutions. Finally, we evaluate the proposed spectral tensor networks on the MNIST, CIFAR-10, ImageNet-1K, and ImageNet-21K datasets, respectively, to verify that they simultaneously achieve communication-free distributed learning, distributed storage reduction, parallel computation speedup, and learning with multiresolution data.

---

PaperID: 408,   

Authors:  Miaoyu Li, Ying Fu, Tao Zhang, Guanghui Wen

Title: Supervise-Assisted Self-Supervised Deep-Learning Method for Hyperspectral Image Restoration

Abstract:
Hyperspectral image (HSI) restoration is a challenging research area, covering a variety of inverse problems. Previous works have shown the great success of deep learning in HSI restoration. However, facing the problem of distribution gaps between training HSIs and target HSI, those data-driven methods falter in delivering satisfactory outcomes for the target HSIs. In addition, the degradation process of HSIs is usually disturbed by noise, which is not well taken into account in existing restoration methods. The existence of noise further exacerbates the dissimilarities within the data, rendering it challenging to attain desirable results without an appropriate learning approach. To track these issues, in this article, we propose a supervise-assisted self-supervised deep-learning method to restore noisy degraded HSIs. Initially, we facilitate the restoration network to acquire a generalized prior through supervised learning from extensive training datasets. Then, the self-supervised learning stage is employed and utilizes the specific prior of the target HSI. Particularly, to restore clean HSIs during the self-supervised learning stage from noisy degraded HSIs, we introduce a noise-adaptive loss function that leverages inner statistics of noisy degraded HSIs for restoration. The proposed noise-adaptive loss consists of Stein’s unbiased risk estimator (SURE) and total variation (TV) regularizer and fine-tunes the network with the presence of noise. We demonstrate through experiments on different HSI tasks, including denoising, compressive sensing, super-resolution, and inpainting, that our method outperforms state-of-the-art methods on benchmarks under quantitative metrics and visual quality. The code is available at https://github.com/ying-fu/SSDL-HSI.

---

PaperID: 409,   

Authors:  Xiangyin Kong, Yimeng He, Zhihuan Song, Tong Liu, Zhiqiang Ge

Title: Deep Probabilistic Principal Component Analysis for Process Monitoring

Abstract:
Probabilistic latent variable models (PLVMs), such as probabilistic principal component analysis (PPCA), are widely employed in process monitoring and fault detection of industrial processes. This article proposes a novel deep PPCA (DePPCA) model, which has the advantages of both probabilistic modeling and deep learning. The construction of DePPCA includes a greedy layer-wise pretraining phase and a unified end-to-end fine-tuning phase. The former establishes a hierarchical deep structure based on cascading multiple layers of the PPCA module to extract high-level features. The latter builds an end-to-end connection between the raw inputs and the final outputs to further improve the representation of the model to high-level features. After constructing the model structure of DePPCA, we first present the detailed training processes of the pretraining and fine-tuning stages, then clarify the theoretical merits of the proposed model from the perspective of variational inference. For process monitoring purposes, we develop two statistics based on the established DePPCA. The monitoring performance of these two statistics can remain superior even if the features extracted by DePPCA are significantly compressed to univariate. This makes the feature extraction process and online monitoring procedure of DePPCA quite fast. In other words, the proposed DePPCA can achieve accurate and efficient process monitoring by only extracting one feature for each sample. Finally, the effectiveness of DePPCA is evaluated on the Tennessee Eastman (TE) process and the multiphase flow (MPF) facility.

---

PaperID: 410,   

Authors:  Boyan Li, Luziwei Leng, Shuaijie Shen, Kaixuan Zhang, Jianguo Zhang, Jianxing Liao, Ran Cheng

Title: Efficient Deep Spiking Multilayer Perceptrons With Multiplication-Free Inference

Abstract:
Advancements in adapting deep convolution architectures for spiking neural networks (SNNs) have significantly enhanced image classification performance and reduced computational burdens. However, the inability of multiplication-free inference (MFI) to align with attention and transformer mechanisms, which are critical to superior performance on high-resolution vision tasks, imposes limitations on these gains. To address this, our research explores a new pathway, drawing inspiration from the progress made in multilayer perceptrons (MLPs). We propose an innovative spiking MLP architecture that uses batch normalization (BN) to retain MFI compatibility and introduce a spiking patch encoding (SPE) layer to enhance local feature extraction capabilities. As a result, we establish an efficient multistage spiking MLP network that blends effectively global receptive fields with local feature extraction for comprehensive spike-based computation. Without relying on pretraining or sophisticated SNN training techniques, our network secures a top-one accuracy of 66.39% on the ImageNet-1K dataset, surpassing the directly trained spiking ResNet-34 by 2.67%. Furthermore, we curtail computational costs, model parameters, and simulation steps. An expanded version of our network compares with the performance of the spiking VGG-16 network with a 71.64% top-one accuracy, all while operating with a model capacity 2.1 times smaller. Our findings highlight the potential of our deep SNN architecture in effectively integrating global and local learning abilities. Interestingly, the trained receptive field in our network mirrors the activity patterns of cortical cells.

---

PaperID: 411,   

Authors:  Xiaojie Yin, Bing Cao, Qinghua Hu, Qilong Wang

Title: RD-OpenMax: Rethinking OpenMax for Robust Realistic Open-Set Recognition

Abstract:
Open-set recognition (OSR) toward a practical open-world setting has attracted increasing research attention in recent years. However, existing OSR settings are either too idealized or focus on specific scenes such as long-tailed distribution and few-shot samples, which fail to capture the complexity of real-world scenarios. In this article, we propose a realistic OSR (ROSR) setting that covers a diverse range of challenging and real-world scenarios, including fine-grained cases with strong semantic correlation and a large number of species, few-shot samples, long-tailed sample distribution, dynamic inputs (e.g., images, spatio-temporal, and multimodal signals) and cross-domain adaptation. In particular, we rethink the simple and basic OpenMax for the ROSR setting and introduce a novel method, regularized discriminative OpenMax (RD-OpenMax), to handle the challenges in the ROSR setting. RD-OpenMax improves upon the basic OpenMax approach by introducing a covariance attention-based covariance pooling (CACP) module as a global aggregation step before the deep architecture’s classifier. This module explores rich statistical information on features and provides discriminative distance scores for OpenMax. To address the instability of extreme value theory (EVT) estimation due to insufficient training samples under few-shot and long-tailed scenarios, we propose a regularized EVT (REVT) method based on Monte Carlo sampling to recalibrate the distribution of distance scores. As such, our RD-OpenMax performs a REVT model of distance scores generated by discriminative CACP representations to distinguish known classes and recognize unknown ones effectively and robustly. Extensive experiments are conducted on more than ten visual benchmarks across several scenarios, and the empirical comparisons show that the ROSR setting challenges existing state-of-the-art OSR approaches. Moreover, our RD-OpenMax clearly outperforms its counterparts under the ROSR setting while performing favorably against state-of-the-arts under the traditional OSR setting.

---

PaperID: 412,   

Authors:  Lu Zeng, Xuan Chen, Xiaoshuang Shi, Heng Tao Shen

Title: Feature Noise Boosts DNN Generalization Under Label Noise

Abstract:
The presence of label noise in the training data has a profound impact on the generalization of deep neural networks (DNNs). In this study, we introduce and theoretically demonstrate a simple feature noise (FN) method, which directly adds noise to the features of training data and can enhance the generalization of DNNs under label noise. Specifically, we conduct theoretical analyses to reveal that label noise leads to weakened DNN generalization by loosening the generalization bound, and FN results in better DNN generalization by imposing an upper bound on the mutual information between the model weights and the features, which constrains the generalization bound. Furthermore, we conduct a qualitative analysis to discuss the ideal type of FN that obtains good label noise generalization. Finally, extensive experimental results on several popular datasets demonstrate that the FN method can significantly enhance the label noise generalization of state-of-the-art methods. The source codes of the FN method are available on https://github.com/zlzenglu/FN.

---

PaperID: 413,   

Authors:  Yidan Ma, Xinjie Shen, Danyang Wu, Jianfu Cao, Feiping Nie

Title: Cross-View Approximation on Grassmann Manifold for Multiview Clustering

Abstract:
In existing multiview clustering research, the comprehensive learning from multiview graph and feature spaces simultaneously remains insufficient when achieving a consistent clustering structure. In addition, a postprocessing step is often required. In light of these considerations, a cross-view approximation on Grassman manifold (CAGM) model is proposed to address inconsistencies within multiview adjacency matrices, feature matrices, and cross-view combinations from the two sources. The model uses a ratio-formed objective function, enabling parameter-free bidirectional fusion. Furthermore, the CAGM model incorporates a paired encoding mechanism to generate low-dimensional and orthogonal cross-view embeddings. Through the approximation of two measurable subspaces on the Grassmann manifold, the direct acquisition of the indicator matrix is realized. Furthermore, an effective optimization algorithm corresponding to the CAGM model is derived. Comprehensive experiments on four real-world datasets are conducted to substantiate the effectiveness of our proposed method.

---

PaperID: 414,   

Authors:  Kun Xie, Can Liu, Xin Wang, Xiaocan Li, Gaogang Xie, Jigang Wen, Kenli Li

Title: Neural Network Compression Based on Tensor Ring Decomposition

Abstract:
Deep neural networks (DNNs) have made great breakthroughs and seen applications in many domains. However, the incomparable accuracy of DNNs is achieved with the cost of considerable memory consumption and high computational complexity, which restricts their deployment on conventional desktops and portable devices. To address this issue, low-rank factorization, which decomposes the neural network parameters into smaller sized matrices or tensors, has emerged as a promising technique for network compression. In this article, we propose leveraging the emerging tensor ring (TR) factorization to compress the neural network. We investigate the impact of both parameter tensor reshaping and TR decomposition (TRD) on the total number of compressed parameters. To achieve the maximal parameter compression, we propose an algorithm based on prime factorization that simultaneously identifies the optimal tensor reshaping and TRD. In addition, we discover that different execution orders of the core tensors result in varying computational complexities. To identify the optimal execution order, we construct a novel tree structure. Based on this structure, we propose a top-to-bottom splitting algorithm to schedule the execution of core tensors, thereby minimizing computational complexity. We have performed extensive experiments using three kinds of neural networks with three different datasets. The experimental results demonstrate that, compared with the three state-of-the-art algorithms for low-rank factorization, our algorithm can achieve better performance with much lower memory consumption and lower computational complexity.

---

PaperID: 415,   

Authors:  Qiuyu Song, Xingxing Jiang, Jie Liu, Juanjuan Shi, Zhongkui Zhu

Title: Contrast-Assisted Domain-Specificity-Removal Network for Semi-Supervised Generalization Fault Diagnosis

Abstract:
Unknown domain shift caused by the unavailability of target domain during training phase degrades the performance of intelligent fault diagnosis models in practical applications. Domain generalization (DG)-based methods have recently emerged to alleviate the influence of domain shift and improve the generalization ability of models toward invisible working conditions. However, most existing studies are conducted on multiple fully labeled source domains. Meanwhile, domain-specific information related to the variations of working conditions is often neglected during model training. Therefore, in order to realize reliable generalization fault diagnosis based on partially labeled source domains, this article proposes a contrast-assisted domain-specificity-removal network (CDSRN) to extract transferable features from domain-specificity-removal perspective. Concretely, a domain-specific feature removal branch is designed to disentangle domain-invariant features and domain-specific features, thus excavating generalized information only in domain-invariance dimension. Simultaneously, proxy-contrastive representation enhancement module is embedded to facilitate the fault class-discriminative and domain-discriminative feature learning, thereby assisting the model in further improvement of generalization capability. Experimental studies confirm the effectiveness and competitiveness of the proposed CDSRN in semi-supervised generalization fault diagnosis.

---

PaperID: 416,   

Authors:  Hongyi Pan, Emadeldeen Hamdan, Xin Zhu, Salih Atici, Ahmet Enis Çetin

Title: Multichannel Orthogonal Transform-Based Perceptron Layers for Efficient ResNets

Abstract:
In this article, we propose a set of transform-based neural network layers as an alternative to the 3\,\, × 3 Conv2D layers in convolutional neural networks (CNNs). The proposed layers can be implemented based on orthogonal transforms, such as the discrete cosine transform (DCT), Hadamard transform (HT), and biorthogonal block wavelet transform (BWT). Furthermore, by taking advantage of the convolution theorems, convolutional filtering operations are performed in the transform domain using elementwise multiplications. Trainable soft-thresholding layers, that remove noise in the transform domain, bring nonlinearity to the transform domain layers. Compared with the Conv2D layer, which is spatial-agnostic and channel-specific, the proposed layers are location-specific and channel-specific. Moreover, these proposed layers reduce the number of parameters and multiplications significantly while improving the accuracy results of regular ResNets on the ImageNet-1K classification task. Furthermore, they can be inserted with a batch normalization (BN) layer before the global average pooling layer in the conventional ResNets as an additional layer to improve classification accuracy.

---

PaperID: 417,   

Authors:  Yingqian Wang, Zhengyu Liang, Longguang Wang, Jungang Yang, Wei An, Yulan Guo

Title: Real-World Light Field Image Super-Resolution Via Degradation Modulation

Abstract:
Recent years have witnessed the great advances of deep neural networks (DNNs) in light field (LF) image super-resolution (SR). However, existing DNN-based LF image SR methods are developed on a single fixed degradation (e.g., bicubic downsampling), and thus cannot be applied to super-resolve real LF images with diverse degradation. In this article, we propose a simple yet effective method for real-world LF image SR. In our method, a practical LF degradation model is developed to formulate the degradation process of real LF images. Then, a convolutional neural network is designed to incorporate the degradation prior into the SR process. By training on LF images using our formulated degradation, our network can learn to modulate different degradation while incorporating both spatial and angular information in LF images. Extensive experiments on both synthetically degraded and real-world LF images demonstrate the effectiveness of our method. Compared with existing state-of-the-art single and LF image SR methods, our method achieves superior SR performance under a wide range of degradation, and generalizes better to real LF images. Codes and models are available at https://yingqianwang.github.io/LF-DMnet/.

---

PaperID: 418,   

Authors:  Feiteng Mu, Wenjie Li

Title: Enhancing Narrative Commonsense Reasoning With Multilevel Causal Knowledge

Abstract:
Narratives is an account of the unfolding of events, along with explanations of how and why these processes and events came to be. To understand narratives, causality has been proven to be especially useful. Causality manifests itself primarily at both the event and sentence levels, offering essential insights into understanding narratives. However, previous works utilize either sentence-level or event-level causalities. In this article, we devise a two-stage approach to fully exploit both levels of causal relationships. In the first stage, by devising posttraining tasks, we inject sentence-level causalities into pretrained language models (PLMs). The causal-enhanced PLMs, which carry sentence-level causalities, can be transferred to downstream tasks. In the second stage, we utilize event causalities to further refine narrative commonsense reasoning. But, the event sparsity problem brings about the difficulty to learn event representations and capture useful causal semantics. To alleviate this problem, we break down events into multiple word components, enabling the retrieval of word–word relations between these components. And it is possible to alleviate the event sparsity problem since word–word relations capture the interplays between event components. Based on the event-level causalities and the word-level relations, we construct the hierarchical knowledge graph (KG) as knowledge ground. A KG-based reasoning process is then employed for narrative commonsense reasoning. Experimental results affirm the effectiveness of our framework.

---

PaperID: 419,   

Authors:  Luntong Li, Yuanheng Zhu

Title: Boosting On-Policy Actor-Critic With Shallow Updates in Critic

Abstract:
Deep reinforcement learning (DRL) benefits from the representation power of deep neural networks (NNs), to approximate the value function and policy in the learning process. Batch reinforcement learning (BRL) benefits from stable training and data efficiency with fixed representation and enjoys solid theoretical analysis. This work proposes least-squares deep policy gradient (LSDPG), a hybrid approach that combines least-squares reinforcement learning (RL) with online DRL to achieve the best of both worlds. LSDPG leverages a shared network to share useful features between policy (actor) and value function (critic). LSDPG learns policy, value function, and representation separately. First, LSDPG views deep NNs of the critic as a linear combination of representation weighted by the weights of the last layer and performs policy evaluation with regularized least-squares temporal difference (LSTD) methods. Second, arbitrary policy gradient algorithms can be applied to improve the policy. Third, an auxiliary task is used to periodically distill the features from the critic into the representation. Unlike most DRL methods, where the critic algorithms are often used in a nonstationary situation, i.e., the policy to be evaluated is changing, the critic in LSDPG is working on a stationary case in each iteration of the critic update. We prove that, under some conditions, the critic converges to the regularized TD fixpoint of current policy, and the actor converges to the local optimal policy. The experimental results on challenging Procgen benchmark illustrate the improvement of sample efficiency of LSDPG over proximal policy optimization and phasic policy gradient (PPG).

---

PaperID: 420,   

Authors:  Gaurav Kumar Nayak, Ruchit Rawal, Inder Khatri, Anirban Chakraborty

Title: Robust Few-Shot Learning Without Using Any Adversarial Samples

Abstract:
The high cost of acquiring and annotating samples has made the “few-shot” learning problem of prime importance. Existing works mainly focus on improving performance on clean data and overlook robustness concerns on the data perturbed with adversarial noise. Recently, a few efforts have been made to combine the few-shot problem with the robustness objective using sophisticated meta-learning techniques. These methods rely on the generation of adversarial samples in every episode of training, which further adds to the computational burden. To avoid such time-consuming and complicated procedures, we propose a simple but effective alternative that does not require any adversarial samples. Inspired by the cognitive decision-making process in humans, we enforce high-level feature matching between the base class data and their corresponding low-frequency samples in the pretraining stage via self distillation. The model is then fine-tuned on the samples of novel classes where we additionally improve the discriminability of low-frequency query set features via cosine similarity. On a one-shot setting of the CIFAR-FS dataset, our method yields a massive improvement of 60.55% and 62.05% in adversarial accuracy on the projected gradient descent (PGD) and state-of-the-art auto attack, respectively, with a minor drop in clean accuracy compared to the baseline. Moreover, our method only takes 1.69× of the standard training time while being \approx 5× faster than thestate-of-the-art adversarial meta-learning methods. The code is available at https://github.com/vcl-iisc/robust-few-shot-learning.

---

PaperID: 421,   

Authors:  Zijun Cui, Tian Gao, Kartik Talamadupula, Qiang Ji

Title: Knowledge-Augmented Deep Learning and its Applications: A Survey

Abstract:
Deep learning models, though having achieved great success in many different fields over the past years, are usually data-hungry, fail to perform well on unseen samples, and lack interpretability. Different kinds of prior knowledge often exists in the target domain, and their use can alleviate the deficiencies with deep learning. To better mimic the behavior of human brains, different advanced methods have been proposed to identify domain knowledge and integrate it into deep models for data-efficient, generalizable, and interpretable deep learning, which we refer to as knowledge-augmented deep learning (KADL). In this survey, we define the concept of KADL and introduce its three major tasks, i.e., knowledge identification, knowledge representation, and knowledge integration. Different from existing surveys that are focused on a specific type of knowledge, we provide a broad and complete taxonomy of domain knowledge and its representations. Based on our taxonomy, we provide a systematic review of existing techniques, different from existing works that survey integration approaches agnostic to the taxonomy of knowledge. This survey subsumes existing works and offers a bird’s-eye view of research in the general area of KADL. The thorough and critical reviews of numerous papers help not only understand current progress but also identify future directions for the research on KADL.

---

PaperID: 422,   

Authors:  Yifan Hu, Lei Deng, Yujie Wu, Man Yao, Guoqi Li

Title: Advancing Spiking Neural Networks Toward Deep Residual Learning

Abstract:
Despite the rapid progress of neuromorphic computing, inadequate capacity and insufficient representation power of spiking neural networks (SNNs) severely restrict their application scope in practice. Residual learning and shortcuts have been evidenced as an important approach for training deep neural networks, but rarely did previous work assessed their applicability to the specifics of SNNs. In this article, we first identify that this negligence leads to impeded information flow and the accompanying degradation problem in a spiking version of vanilla ResNet. To address this issue, we propose a novel SNN-oriented residual architecture termed MS-ResNet, which establishes membrane-based shortcut pathways, and further proves that the gradient norm equality can be achieved in MS-ResNet by introducing block dynamical isometry theory, which ensures the network can be well-behaved in a depth-insensitive way. Thus, we are able to significantly extend the depth of directly trained SNNs, e.g., up to 482 layers on CIFAR-10 and 104 layers on ImageNet, without observing any slight degradation problem. To validate the effectiveness of MS-ResNet, experiments on both frame-based and neuromorphic datasets are conducted. MS-ResNet104 achieves a superior result of 76.02% accuracy on ImageNet, which is the highest to the best of our knowledge in the domain of directly trained SNNs. Great energy efficiency is also observed, with an average of only one spike per neuron needed to classify an input sample. We believe our powerful and scalable models will provide strong support for further exploration of SNNs.

---

PaperID: 423,   

Authors:  Yixuan Zhou, Xing Xu, Jingkuan Song, Fumin Shen, Heng Tao Shen

Title: MSFlow: Multiscale Flow-Based Framework for Unsupervised Anomaly Detection

Abstract:
Unsupervised anomaly detection (UAD) attracts a lot of research interest and drives widespread applications, where only anomaly-free samples are available for training. Some UAD applications intend to locate the anomalous regions further even without any anomaly information. Although the absence of anomalous samples and annotations deteriorates the UAD performance, an inconspicuous, yet powerful statistics model, the normalizing flows, is appropriate for anomaly detection (AD) and localization in an unsupervised fashion. The flow-based probabilistic models, only trained on anomaly-free data, can efficiently distinguish unpredictable anomalies by assigning them much lower likelihoods than normal data. Nevertheless, the size variation of unpredictable anomalies introduces another inconvenience to the flow-based methods for high-precision AD and localization. To generalize the anomaly size variation, we propose a novel multiscale flow-based framework (MSFlow) composed of asymmetrical parallel flows followed by a fusion flow to exchange multiscale perceptions. Moreover, different multiscale aggregation strategies are adopted for image-wise AD and pixel-wise anomaly localization according to the discrepancy between them. The proposed MSFlow is evaluated on three AD datasets, significantly outperforming existing methods. Notably, on the challenging MVTec AD benchmark, our MSFlow achieves a new state-of-the-art (SOTA) with a detection AUORC score of up to 99.7%, localization AUCROC score of 98.8% and PRO score of 97.1%.

---

PaperID: 424,   

Authors:  Yan Luo, Yongkang Wong, Mohan S. Kankanhalli, Qi Zhao

Title: Learning to Predict Gradients for Semi-Supervised Continual Learning

Abstract:
A key challenge for machine intelligence is to learn new visual concepts without forgetting the previously acquired knowledge. Continual learning (CL) is aimed toward addressing this challenge. However, there still exists a gap between CL and human learning. In particular, humans are able to continually learn from the samples associated with known or unknown labels in their daily lives, whereas existing CL and semi-supervised CL (SSCL) methods assume that the training samples are associated with known labels. Specifically, we are interested in two questions: 1) how to utilize unrelated unlabeled data for the SSCL task and 2) how unlabeled data affect learning and catastrophic forgetting in the CL task. To explore these issues, we formulate a new SSCL method, which can be generically applied to existing CL models. Furthermore, we propose a novel gradient learner to learn from labeled data to predict gradients on unlabeled data. In this way, the unlabeled data can fit into the supervised CL framework. We extensively evaluate the proposed method on mainstream CL methods, adversarial CL (ACL), and semi-supervised learning (SSL) tasks. The proposed method achieves state-of-the-art performance on classification accuracy and backward transfer (BWT) in the CL setting while achieving the desired performance on classification accuracy in the SSL setting. This implies that the unlabeled images can enhance the generalizability of CL models on the predictive ability of unseen data and significantly alleviate catastrophic forgetting. The code is available at https://github.com/luoyan407/grad_prediction.git.

---

PaperID: 425,   

Authors:  Yu Luo, Tianying Ji, Fuchun Sun, Huaping Liu, Jianwei Zhang, Mingxuan Jing, Wenbing Huang

Title: Goal-Conditioned Hierarchical Reinforcement Learning With High-Level Model Approximation

Abstract:
Hierarchical reinforcement learning (HRL) exhibits remarkable potential in addressing large-scale and long-horizon complex tasks. However, a fundamental challenge, which arises from the inherently entangled nature of hierarchical policies, has not been understood well, consequently compromising the training stability and exploration efficiency of HRL. In this article, we propose a novel HRL algorithm, high-level model approximation (HLMA), presenting both theoretical foundations and practical implementations. In HLMA, a Planner constructs an innovative high-level dynamic model to predict the k -step transition of the Controller in a subtask. This allows for the estimation of the evolving performance of the Controller. At low level, we leverage the initial state of each subtask, transforming absolute states into relative deviations by a designed operator as Controller input. This approach facilitates the reuse of subtask domain knowledge, enhancing data efficiency. With this designed structure, we establish the local convergence of each component within HLMA and subsequently derive regret bounds to ensure global convergence. Abundant experiments conducted on complex locomotion and navigation tasks demonstrate that HLMA surpasses other state-of-the-art single-level RL and HRL algorithms in terms of sample efficiency and asymptotic performance. In addition, thorough ablation studies validate the effectiveness of each component of HLMA.

---

PaperID: 426,   

Authors:  Jinfu Fan, Linqing Huang, Chaoyu Gong, Yang You, Min Gan, Zhongjie Wang

Title: KMT-PLL: K-Means Cross-Attention Transformer for Partial Label Learning

Abstract:
Partial label learning (PLL) studies the problem of learning instance classification with a set of candidate labels and only one is correct. While recent works have demonstrated that the Vision Transformer (ViT) has achieved good results when training from clean data, its applications to PLL remain limited and challenging. To address this issue, we rethink the relationship between instances and object queries to propose K-means cross-attention transformer for PLL (KMT-PLL), which can continuously learn cluster centers and be used for downstream disambiguation tasks. More specifically, K-means cross-attention as a clustering process can effectively learn the cluster centers to represent label classes. The purpose of this operation is to make the similarity between instances and labels measurable, which can effectively detect noise labels. Furthermore, we propose a new corrected cross entropy formulation, which can assign weights to candidate labels according to the instance-to-label relevance to guide the training of the instance classifier. As the training goes on, the ground-truth label is progressively identified, and the refined labels and cluster centers in turn help to improve the classifier. Simulation results demonstrate the advantage of the KMT-PLL and its suitability for PLL.

---

PaperID: 427,   

Authors:  Giorgio Morales, John W. Sheppard

Title: Dual Accuracy-Quality-Driven Neural Network for Prediction Interval Generation

Abstract:
Accurate uncertainty quantification is necessary to enhance the reliability of deep learning (DL) models in real-world applications. In the case of regression tasks, prediction intervals (PIs) should be provided along with the deterministic predictions of DL models. Such PIs are useful or “high-quality (HQ)” as long as they are sufficiently narrow and capture most of the probability density. In this article, we present a method to learn PIs for regression-based neural networks (NNs) automatically in addition to the conventional target predictions. In particular, we train two companion NNs: one that uses one output, the target estimate, and another that uses two outputs, the upper and lower bounds of the corresponding PI. Our main contribution is the design of a novel loss function for the PI-generation network that takes into account the output of the target-estimation network and has two optimization objectives: minimizing the mean PI width and ensuring the PI integrity using constraints that maximize the PI probability coverage implicitly. Furthermore, we introduce a self-adaptive coefficient that balances both objectives within the loss function, which alleviates the task of fine-tuning. Experiments using a synthetic dataset, eight benchmark datasets, and a real-world crop yield prediction dataset showed that our method was able to maintain a nominal probability coverage and produce significantly narrower PIs without detriment to its target estimation accuracy when compared to those PIs generated by three state-of-the-art neural-network-based methods. In other words, our method was shown to produce higher quality PIs.

---

PaperID: 428,   

Authors:  Jing Xu, Xinglin Pan, Jingquan Wang, Wenjie Pei, Qing Liao, Zenglin Xu

Title: CORE: CORrelation-Guided Feature Enhancement for Few-Shot Image Classification

Abstract:
Few-shot classification aims to adapt classifiers trained on base classes to novel classes with a few shots. However, the limited amount of training data is often inadequate to represent the intraclass variations in novel classes. This can result in biased estimation of the feature distribution, which in turn results in inaccurate decision boundaries, especially when the support data are outliers. To address this issue, we propose a feature enhancement method called CORrelation-guided feature Enrichment that generates improved features for novel classes using weak supervision from the base classes. The proposed CORrelation-guided feature Enhancement (CORE) method utilizes an autoencoder (AE) architecture but incorporates classification information into its latent space. This design allows the CORE to generate more discriminative features while discarding irrelevant content information. After being trained on base classes, CORE’s generative ability can be transferred to novel classes that are similar to those in the base classes. By using these generative features, we can reduce the estimation bias of the class distribution, which makes few-shot learning (FSL) less sensitive to the selection of support data. Our method is generic and flexible and can be used with any feature extractor and classifier. It can be easily integrated into existing FSL approaches. Experiments with different backbones and classifiers show that our proposed method consistently outperforms existing methods on various widely used benchmarks.

---

PaperID: 429,   

Authors:  Shawon Dey, Hao Xu

Title: Distributed Adaptive Flocking Control for Large-Scale Multiagent Systems

Abstract:
This article presents a novel distributed flocking control method for large-scale multiagent systems (LS-MASs) operating in uncertain environments. When dealing with a massive number of flocking agents in uncertain environments, existing flocking methods encounter the problem of communication complexity and “Curse of dimensionality” caused by the exponential growth of agent interactions while solving PDE-based optimal flocking control for large-scale systems. The mean field game (MFG) method addresses this issue by transforming interactions among all agents into the interaction of each individual agent with average effects represented by a probability density function (pdf) of other agents. However, relying solely on a pdf term to consider other agents’ states can result in inefficient flocking performance due to the absence of a proficient coordination mechanism encompassing all agents involved in flocking. To overcome these difficulties and achieve the desired flocking performance for LS-MASs, the agents are decomposed into a finite number of subgroups. Each subgroup comprises a leader and followers, and a hybrid game theory is developed to manage both inter- and intragroup interactions. The method incorporates a cooperative game that links leaders from different groups to formulate distributed flocking control, a Stackelberg game that teams up leaders and followers within the same group to extend collective flocking behavior, and an MFG for followers to address the challenges of LS-MASs. Furthermore, to achieve distributed adaptive flocking using the hybrid game structure, we propose a hierarchical actor–critic-mass-based reinforcement learning technique. This approach incorporates a multiactor–critic method for leaders and an actor–critic-mass algorithm for followers, enabling adaptive flocking control in a distributed manner for large-scale agents. Finally, numerical simulation including comparison study and Lyapunov analysis demonstrates the effectiveness of the developed method.

---

PaperID: 430,   

Authors:  Ziyuan Yang, Wenjun Xia, Zexin Lu, Yingyu Chen, Xiaoxiao Li, Yi Zhang

Title: Hypernetwork-Based Physics-Driven Personalized Federated Learning for CT Imaging

Abstract:
In clinical practice, computed tomography (CT) is an important noninvasive inspection technology to provide patients’ anatomical information. However, its potential radiation risk is an unavoidable problem that raises people’s concerns. Recently, deep learning (DL)-based methods have achieved promising results in CT reconstruction, but these methods usually require the centralized collection of large amounts of data for training from specific scanning protocols, which leads to serious domain shift and privacy concerns. To relieve these problems, in this article, we propose a hypernetwork-based physics-driven personalized federated learning method (HyperFed) for CT imaging. The basic assumption of the proposed HyperFed is that the optimization problem for each domain can be divided into two subproblems: local data adaption and global CT imaging problems, which are implemented by an institution-specific physics-driven hypernetwork and a global-sharing imaging network, respectively. Learning stable and effective invariant features from different data distributions is the main purpose of global-sharing imaging network. Inspired by the physical process of CT imaging, we carefully design physics-driven hypernetwork for each domain to obtain hyperparameters from specific physical scanning protocol to condition the global-sharing imaging network, so that we can achieve personalized local CT reconstruction. Experiments show that HyperFed achieves competitive performance in comparison with several other state-of-the-art methods. It is believed as a promising direction to improve CT imaging quality and personalize the needs of different institutions or scanners without data sharing. Related codes have been released at https://github.com/Zi-YuanYang/HyperFed.

---

PaperID: 431,   

Authors:  Honglei Zhang, Xin Zhou, Zhiqi Shen, Yidong Li

Title: PrivFR: Privacy-Enhanced Federated Recommendation With Shared Hash Embedding

Abstract:
Federated recommender systems (FRSs), with their improved privacy-preserving advantages to jointly train recommendation models from numerous devices while keeping user data distributed, have been widely explored in modern recommender systems (RSs). However, conventional FRSs require transmitting the entire model between the server and clients, which brings a huge carbon footprint for cost-conscious cross-device learning tasks. While several efforts have been dedicated to improving the efficiency of FRSs, it’s suboptimal to treat the whole model as the objective of compact design. Besides, current research fails to handle the out-of-vocabulary (OOV) issue in real-world FRSs, where the items only occasionally appear in the testing phase but were not observed during the training process, which is another practical challenge and has not been well studied yet. To this end, we propose a privacy-enhanced federated recommendation framework with shared hash embedding, PrivFR, in cross-device settings, which is an efficient representation mechanism specialized for the embedding parameters without compromising the model capability. Specifically, it represents items in a resource-efficient way by delicately utilizing shared hash embedding and multiple hash functions. As such, it just maintains a small shared pool of hash embedding in local clients, rather than fitting all embedding vectors for each item, which can exactly achieve the dual advantages of conserving resources and handling the OOV issue. What’s more, we prove that this mechanism can protect the data privacy of local clients from a theoretical perspective. Extensive experiments show that our method not only effectively reduces storage and communication overheads, but also outperforms state-of-the-art FRSs.

---

PaperID: 432,   

Authors:  Xiongtao Zhang, Ji Wang, Weidong Bao, Yaohong Zhang, Xiaomin Zhu, Hao Peng, Xiang Zhao

Title: Improving Generalization and Personalization in Model-Heterogeneous Federated Learning

Abstract:
Conventional federated learning (FL) assumes the homogeneity of models, necessitating clients to expose their model parameters to enhance the performance of the server model. However, this assumption cannot reflect real-world scenarios. Sharing models and parameters raises security concerns for users, and solely focusing on the server-side model neglects clients’ personalization requirements, potentially impeding expected performance improvements of users. On the other hand, prioritizing personalization may compromise the generalization of the server model, thereby hindering extensive knowledge migration. To address these challenges, we put forth an important problem: How can FL ensure both generalization and personalization when clients’ models are heterogeneous? In this work, we introduce FedTED, which leverages a twin-branch structure and data-free knowledge distillation (DFKD) to address the challenges posed by model heterogeneity and diverse objectives in FL. The employed techniques in FedTED yield significant improvements in both personalization and generalization, while effectively coordinating the updating process of clients’ heterogeneous models and successfully reconstructing a satisfactory global model. Our empirical evaluation demonstrates that FedTED outperforms many representative algorithms, particularly in scenarios where clients’ models are heterogeneous, achieving a remarkable 19.37% enhancement in generalization performance and up to 9.76% improvement in personalization performance.

---

PaperID: 433,   

Authors:  Yunan Li, Tianyu Qi, Zhuoqi Ma, Dou Quan, Qiguang Miao

Title: Seeking a Hierarchical Prototype for Multimodal Gesture Recognition

Abstract:
Gesture recognition has drawn considerable attention from many researchers owing to its wide range of applications. Although significant progress has been made in this field, previous works always focus on how to distinguish between different gesture classes, ignoring the influence of inner-class divergence caused by gesture-irrelevant factors. Meanwhile, for multimodal gesture recognition, feature or score fusion in the final stage is a general choice to combine the information of different modalities. Consequently, the gesture-relevant features in different modalities may be redundant, whereas the complementarity of modalities is not exploited sufficiently. To handle these problems, we propose a hierarchical gesture prototype framework to highlight gesture-relevant features such as poses and motions in this article. This framework consists of a sample-level prototype and a modal-level prototype. The sample-level gesture prototype is established with the structure of a memory bank, which avoids the distraction of gesture-irrelevant factors in each sample, such as the illumination, background, and the performers’ appearances. Then the modal-level prototype is obtained via a generative adversarial network (GAN)-based subnetwork, in which the modal-invariant features are extracted and pulled together. Meanwhile, the modal-specific attribute features are used to synthesize the feature of other modalities, and the circulation of modality information helps to leverage their complementarity. Extensive experiments on three widely used gesture datasets demonstrate that our method is effective to highlight gesture-relevant features and can outperform the state-of-the-art methods.

---

PaperID: 434,   

Authors:  Wentao Fan, Chao Zhang, Huaxiong Li, Xiuyi Jia, Guoyin Wang

Title: Three-Stage Semisupervised Cross-Modal Hashing With Pairwise Relations Exploitation

Abstract:
Hashing methods have sparked a great revolution in cross-modal retrieval due to the low cost of storage and computation. Benefiting from the sufficient semantic information of labeled data, supervised hashing methods have shown better performance compared with unsupervised ones. Nevertheless, it is expensive and labor intensive to annotate the training samples, which restricts the feasibility of supervised methods in real applications. To deal with this limitation, a novel semisupervised hashing method, i.e., three-stage semisupervised hashing (TS3H) is proposed in this article, where both labeled and unlabeled data are seamlessly handled. Different from other semisupervised approaches that learn the pseudolabels, hash codes, and hash functions simultaneously, the new approach is decomposed into three stages as the name implies, in which all of the stages are conducted individually to make the optimization cost-effective and precise. Specifically, the classifiers of different modalities are learned via the provided supervised information to predict the labels of unlabeled data at first. Then, hash code learning is achieved with a simple but efficient scheme by unifying the provided and the newly predicted labels. To capture the discriminative information and preserve the semantic similarities, we leverage pairwise relations to supervise both classifier learning and hash code learning. Finally, the modality-specific hash functions are obtained by transforming the training samples to the generated hash codes. The new approach is compared with the state-of-the-art shallow and deep cross-modal hashing (DCMH) methods on several widely used benchmark databases, and the experiment results verify its efficiency and superiority.

---

PaperID: 435,   

Authors:  Liang Li, Tongyu Lu, Yaoqi Sun, Yuhan Gao, Chenggang Yan, Zhenghui Hu, Qingming Huang

Title: Progressive Decision Boundary Shifting for Unsupervised Domain Adaptation

Abstract:
Unsupervised domain adaptation (UDA) is attracting more attention from researchers for boosting the task-specific generalization on target domain. It focuses on addressing the domain shift between the labeled source domain and the unlabeled target domain. Recent biclassifier-based UDA models perform category-level alignment to reduce domain shift, and meanwhile, self-training is used for improving the discriminability of target instances. However, the error accumulation problem of instances with high semantic uncertainty may cause discriminability degradation and category-level misalignment. To solve this issue, we design the progressive decision boundary shifting algorithm, where stable category information of target instances is explored for learning a discriminability structure on target domain. Specifically, we first model the semantic uncertainty of instances by progressively shifting decision boundaries of category. Then, we introduce the uncertainty decoupling in a contrastive manner, where the discriminative information is learned from the source domain for instance with low semantic uncertainty. Furthermore, we minimize the predictive entropy of instances with high semantic uncertainty to reduce their prediction confidence. Extensive experiments on three popular datasets show that our model outperforms the current state-of-the-art (SOTA) UDA methods.

---

PaperID: 436,   

Authors:  Siyuan Li, Zicheng Liu, Zelin Zang, Di Wu, Zhiyuan Chen, Stan Z. Li

Title: GenURL: A General Framework for Unsupervised Representation Learning

Abstract:
Unsupervised representation learning (URL) that learns compact embeddings of high-dimensional data without supervision has achieved remarkable progress recently. However, the development of URLs for different requirements is independent, which limits the generalization of the algorithms, especially prohibitive as the number of tasks grows. For example, dimension reduction (DR) methods, t-SNE and UMAP, optimize pairwise data relationships by preserving the global geometric structure, while self-supervised learning, SimCLR and BYOL, focuses on mining the local statistics of instances under specific augmentations. To address this dilemma, we summarize and propose a unified similarity-based URL framework, GenURL, which can adapt to various URL tasks smoothly. In this article, we regard URL tasks as different implicit constraints on the data geometric structure that help to seek optimal low-dimensional representations that boil down to data structural modeling (DSM) and low-dimensional transformation (LDT). Specifically, DSM provides a structure-based submodule to describe the global structures, and LDT learns compact low-dimensional embeddings with given pretext tasks. Moreover, an objective function, general Kullback–Leibler (GKL) divergence, is proposed to connect DSM and LDT naturally. Comprehensive experiments demonstrate that GenURL achieves consistent state-of-the-art performance in self-supervised visual learning, unsupervised knowledge distillation (KD), graph embeddings (GEs), and DR.

---

PaperID: 437,   

Authors:  Jiaqi Gao, Zhizhong Huang, Yiming Lei, Hongming Shan, James Z. Wang, Fei-Yue Wang, Junping Zhang

Title: Deep Rank-Consistent Pyramid Model for Enhanced Crowd Counting

Abstract:
Most conventional crowd counting methods utilize a fully-supervised learning framework to establish a mapping between scene images and crowd density maps. They usually rely on a large quantity of costly and time-intensive pixel-level annotations for training supervision. One way to mitigate the intensive labeling effort and improve counting accuracy is to leverage large amounts of unlabeled images. This is attributed to the inherent self-structural information and rank consistency within a single image, offering additional qualitative relation supervision during training. Contrary to earlier methods that utilized the rank relations at the original image level, we explore such rank-consistency relation within the latent feature spaces. This approach enables the incorporation of numerous pyramid partial orders, strengthening the model representation capability. A notable advantage is that it can also increase the utilization ratio of unlabeled samples. Specifically, we propose a Deep Rank-consist Ent pyrAmid Model (DREAM), which makes full use of rank consistency across coarse-to-fine pyramid features in latent spaces for enhanced crowd counting with massive unlabeled images. In addition, we have collected a new unlabeled crowd counting dataset, FUDAN-UCC, comprising 4000 images for training purposes. Extensive experiments on four benchmark datasets, namely UCF-QNRF, ShanghaiTech PartA and PartB, and UCF-CC-50, show the effectiveness of our method compared with previous semi-supervised methods. The codes are available at https://github.com/bridgeqiqi/DREAM.

---

PaperID: 438,   

Authors:  Yecheng Guo, Liang Bai, Xian Yang, Jiye Liang

Title: Improving Image Contrastive Clustering Through Self-Learning Pairwise Constraints

Abstract:
In this article, a new unsupervised contrastive clustering (CC) model is introduced, namely, image CC with self-learning pairwise constraints (ICC-SPC). This model is designed to integrate pairwise constraints into the CC process, enhancing the latent representation learning and improving clustering results for image data. The incorporation of pairwise constraints helps reduce the impact of false negatives and false positives in contrastive learning, while maintaining robust cluster discrimination. However, obtaining prior pairwise constraints from unlabeled data directly is quite challenging in unsupervised scenarios. To address this issue, ICC-SPC designs a pairwise constraints learning module. This module autonomously learns pairwise constraints among data samples by leveraging consensus information between latent representation and pseudo-labels, which are generated by the clustering algorithm. Consequently, there is no requirement for labeled images, offering a practical resolution to the challenge posed by the lack of sufficient supervised information in unsupervised clustering tasks. ICC-SPC’s effectiveness is validated through evaluations on multiple benchmark datasets. This contribution is significant, as we present a novel framework for unsupervised clustering by integrating contrastive learning with self-learning pairwise constraints.

---

PaperID: 439,   

Authors:  Guangsheng Yu, Xu Wang, Caijun Sun, Qin Wang, Ping Yu, Wei Ni, Ren Ping Liu

Title: IronForge: An Open, Secure, Fair, Decentralized Federated Learning

Abstract:
Federated learning (FL) offers an effective learning architecture to protect data privacy in a distributed manner. However, the inevitable network asynchrony, overdependence on a central coordinator, and lack of an open and fair incentive mechanism collectively hinder FL’s further development. We propose IronForge, a new generation of FL framework, that features a directed acyclic graph (DAG)-based structure, where nodes represent uploaded models, and referencing relationships between models form the DAG that guides the aggregation process. This design eliminates the need for central coordinators to achieve fully decentralized operations. IronForge runs in a public and open network and launches a fair incentive mechanism by enabling state consistency in the DAG. Hence, the system fits in networks where training resources are unevenly distributed. In addition, dedicated defense strategies against prevalent FL attacks on incentive fairness and data privacy are presented to ensure the security of IronForge. Experimental results based on a newly developed test bed FLSim highlight the superiority of IronForge to the existing prevalent FL frameworks under various specifications in performance, fairness, and security. To the best of our knowledge, IronForge is the first secure and fully decentralized FL (DFL) framework that can be applied in open networks with realistic network and training settings.

---

PaperID: 440,   

Authors:  Zhiming Liu, Jinhai Li, Xiao Zhang, Xi-Zhao Wang

Title: Incremental Incomplete Concept-Cognitive Learning Model: A Stochastic Strategy

Abstract:
Concept-cognitive learning is an emerging area of cognitive computing, which refers to continuously learning new knowledge by imitating the human cognition process. However, the existing research on concept-cognitive learning is still at the level of complete cognition as well as cognitive operators, which is far from the real cognition process. Meanwhile, the current classification algorithms based on concept-cognitive learning models (CCLMs) are not mature enough yet since their cognitive results highly depend on the cognition order of attributes. To address the above problems, this article presents a novel concept-cognitive learning method, namely, stochastic incremental incomplete concept-cognitive learning method (SI2CCLM), whose cognition process adopts a stochastic strategy that is independent of the order of attributes. Moreover, a new classification algorithm based on SI2CCLM is developed, and the analysis of the parameters and convergence of the algorithm is made. Finally, we show the cognitive effectiveness of SI2CCLM by comparing it with other concept-cognitive learning methods. In addition, the average accuracy of our model on 24 datasets is 82.02%, which is higher than the compared 20 classification algorithms, and the elapsed time of our model also has advantages.

---

PaperID: 441,   

Authors:  Jie Chen, Shengxiang Yang, Conor Fahy, Zhu Wang, Yinan Guo, Yingke Chen

Title: Online Sparse Representation Clustering for Evolving Data Streams

Abstract:
Data stream clustering can be performed to discover the patterns underlying continuously arriving sequences of data. A number of data stream clustering algorithms for finding clusters in arbitrary shapes and handling outliers, such as density-based clustering algorithms, have been proposed. However, these algorithms are often limited in their ability to construct and merge microclusters by measuring the Euclidean distances between high-dimensional data objects, e.g., transferring valuable knowledge from historical landmark windows to the current landmark window, and exploiting evolving subspace structures adaptively. We propose an online sparse representation clustering (OSRC) method to learn an affinity matrix for evaluating the relationships among high-dimensional data objects in evolving data streams. We first introduce a low-dimensional projection (LDP) into sparse representation to adaptively reduce the potential negative influence associated with the noise and redundancy contained in high-dimensional data. Then, we take advantage of the l_2,1 -norm optimization technique to choose the appropriate number of representative data objects and form a specific dictionary for sparse representation. The specific dictionary is integrated into sparse representation to adaptively exploit the evolving subspace structures of the high-dimensional data objects. Moreover, the data object representatives from the current landmark window can transfer valuable knowledge to the next landmark window. The experimental results based on a synthetic dataset and six benchmark datasets validate the effectiveness of the proposed method compared to that of state-of-the-art methods for data stream clustering.

---

PaperID: 442,   

Authors:  Huimin Zhang, Ronghu Chi, Biao Huang

Title: Data-Driven Internal Model Learning Control for Nonlinear Systems

Abstract:
A novel data-driven internal model learning control (DIMLC) strategy is developed for a nonlinear nonaffine system subject to unknown nonrepetitive uncertainties. At first, an iterative dynamic linearization (IDL) approach is employed for reformulating the nonlinear plant to an iterative linear data model (iLDM). Then, the nominal form of the IDL-based iLDM is used as an internal model of the nonlinear plant whose parameters are estimated by an iterative adaptive updating mechanism using only input–output (I/O) data. The equivalent feedback-principle-based internal model inversion is further applied to the subsequent controller design and analysis. The proposed DIMLC contains two parts. One is a nominal controller designed by the inversion of the internal model which achieves a perfect tracking of the target output; the other is a compensatory controller which offsets the uncertainties. The novel DIMLC is data-driven and does not require an explicit model. It can deal with model-plant mismatch and disturbances, enhancing the robustness against uncertainties. The theoretical results are verified by simulation study.

---

PaperID: 443,   

Authors:  Siying Zhu, Jiawei Zheng, Qianli Ma

Title: MR-Transformer: Multiresolution Transformer for Multivariate Time Series Prediction

Abstract:
Multivariate time series (MTS) prediction has been studied broadly, which is widely applied in real-world applications. Recently, transformer-based methods have shown the potential in this task for their strong sequence modeling ability. Despite progress, these methods pay little attention to extracting short-term information in the context, while short-term patterns play an essential role in reflecting local temporal dynamics. Moreover, we argue that there are both consistent and specific characteristics among multiple variables, which should be fully considered for MTS modeling. To this end, we propose a multiresolution transformer (MR-Transformer) for MTS prediction, modeling MTS from both the temporal and the variable resolution. Specifically, for the temporal resolution, we design a long short-term transformer. We first split the sequence into nonoverlapping segments in an adaptive way and then extract short-term patterns within segments, while long-term patterns are captured by the inherent attention mechanism. Both of them are aggregated together to capture the temporal dependencies. For the variable resolution, besides the variable-consistent features learned by long short-term transformer, we also design a temporal convolution module to capture the specific features of each variable individually. MR-Transformer enhances the MTS modeling ability by combining multiresolution features between both time steps and variables. Extensive experiments conducted on real-world time series datasets show that MR-Transformer significantly outperforms the state-of-the-art MTS prediction models. The visualization analysis also demonstrates the effectiveness of the proposed model.

---

PaperID: 444,   

Authors:  Chengyuan Mai, Yaomin Chang, Chuan Chen, Zibin Zheng

Title: Enhanced Scalable Graph Neural Network via Knowledge Distillation

Abstract:
Graph neural networks (GNNs) have achieved state-of-the-art performance in various graph representation learning scenarios. However, when applied to graph data in real world, GNNs have encountered scalability issues. Existing GNNs often have high computational load in both training and inference stages, making them incapable of meeting the performance needs of large-scale scenarios with a large number of nodes. Although several studies on scalable GNNs have developed, they either merely improve GNNs with limited scalability or come at the expense of reduced effectiveness. Inspired by knowledge distillation’s (KDs) achievement in preserving performances while balancing scalability in computer vision and natural language processing, we propose an enhanced scalable GNN via KD (KD-SGNN) to improve the scalability and effectiveness of GNNs. On the one hand, KD-SGNN adopts the idea of decoupled GNNs, which decouples feature transformation and feature propagation in GNNs and leverages preprocessing techniques to improve the scalability of GNNs. On the other hand, KD-SGNN proposes two KD mechanisms (i.e., soft-target (ST) distillation and shallow imitation (SI) distillation) to improve the expressiveness. The scalability and effectiveness of KD-SGNN are evaluated on multiple real datasets. Besides, the effectiveness of the proposed KD mechanisms is also verified through comprehensive analyses.

---

PaperID: 445,   

Authors:  Morgan B. Talbot, Rushikesh Zawar, Rohil Badkundri, Mengmi Zhang, Gabriel Kreiman

Title: Tuned Compositional Feature Replays for Efficient Stream Learning

Abstract:
Our brains extract durable, generalizable knowledge from transient experiences of the world. Artificial neural networks come nowhere close to this ability. When tasked with learning to classify objects by training on nonrepeating video frames in temporal order (online stream learning), models that learn well from shuffled datasets catastrophically forget old knowledge upon learning new stimuli. We propose a new continual learning algorithm, compositional replay using memory blocks (CRUMB), which mitigates forgetting by replaying feature maps reconstructed by combining generic parts. CRUMB concatenates trainable and reusable “memory block” vectors to compositionally reconstruct feature map tensors in convolutional neural networks (CNNs). Storing the indices of memory blocks used to reconstruct new stimuli enables memories of the stimuli to be replayed during later tasks. This reconstruction mechanism also primes the neural network to minimize catastrophic forgetting by biasing it toward attending to information about object shapes more than information about image textures and stabilizes the network during stream learning by providing a shared feature-level basis for all training examples. These properties allow CRUMB to outperform an otherwise identical algorithm that stores and replays raw images while occupying only 3.6% as much memory. We stress-tested CRUMB alongside 13 competing methods on seven challenging datasets. To address the limited number of existing online stream learning datasets, we introduce two new benchmarks by adapting existing datasets for stream learning. With only 3.7%–4.1% as much memory and 15%–43% as much runtime, CRUMB mitigates catastrophic forgetting more effectively than the state-of-the-art. Our code is available at https://github.com/MorganBDT/crumb.git

---

PaperID: 446,   

Authors:  Nisha Huang, Yuxin Zhang, Fan Tang, Chongyang Ma, Haibin Huang, Weiming Dong, Changsheng Xu

Title: DiffStyler: Controllable Dual Diffusion for Text-Driven Image Stylization

Abstract:
Despite the impressive results of arbitrary image-guided style transfer methods, text-driven image stylization has recently been proposed for transferring a natural image into a stylized one according to textual descriptions of the target style provided by the user. Unlike the previous image-to-image transfer approaches, text-guided stylization progress provides users with a more precise and intuitive way to express the desired style. However, the huge discrepancy between cross-modal inputs/outputs makes it challenging to conduct text-driven image stylization in a typical feed-forward CNN pipeline. In this article, we present DiffStyler, a dual diffusion processing architecture to control the balance between the content and style of the diffused results. The cross-modal style information can be easily integrated as guidance during the diffusion process step-by-step. Furthermore, we propose a content image-based learnable noise on which the reverse denoising process is based, enabling the stylization results to better preserve the structure information of the content image. We validate the proposed DiffStyler beyond the baseline methods through extensive qualitative and quantitative experiments. The code is available at https://github.com/haha-lisa/Diffstyler.

---

PaperID: 447,   

Authors:  Longjie Zhang, Yong Chen

Title: Finite-Time Adaptive Dynamic Programming for Affine-Form Nonlinear Systems

Abstract:
Inspired by the fusion of state optimization and finite-time convergence, the finite-time optimal control (FTOC) for the affine-form nonlinear systems is investigated in this article. To achieve optimal stability with finite response time, a novel finite-time adaptive dynamic programming (FTADP) is presented for the affine-form nonlinear systems. By mapping the value function into finite-time stability space with the transformation function, the Bellman equation with finite-time stability space is first obtained. Then, by solving the Hamilton–Jacobi–Bellman (HJB) equation, the new FTOC strategy is presented with the theoretical finite-time stability description. Furthermore, to solve the above optimal controller with nonlinearity characteristic, the novel adaptive dynamic programming (ADP) based on the finite-time critic-actor offline neural network (NN) approximation algorithm is implemented, and the corresponding finite-time convergence characteristic is illustrated theoretically. Eventually, the application analysis on the circuit systems shows that the proposed FTADP has superiority compared with general optimal control.

---

PaperID: 448,   

Authors:  Jing Wang, Jianhui Lv, Xin Geng

Title: Label Distribution Learning by Partitioning Label Distribution Manifold

Abstract:
Researchers have suggested leveraging label correlation to deal with the exponentially sized output space of label distribution learning (LDL). Among them, some have proposed to exploit local label correlation. They first partition the training set into different groups and then exploit local label correlation on each one. However, these works usually apply clustering algorithms, such as K -means, to split the training set and obtain the clustering results independent of label correlation. The structures (e.g., low rank and manifold) learned on such clusters may not efficiently capture label correlation. To solve this problem, we put forward a novel LDL method called LDL by partitioning label distribution manifold (LDL-PLDM). First, it jointly bipartitions the training set and learns the label distribution manifold to model label correlation. Second, it recurses until the reconstruction error of learning the label distribution manifold cannot be reduced. LDL-PLDM achieves label-correlation-related partition results, on which the learned label distribution manifold can better capture label correlation. We conduct extensive experiments to justify that LDL-PLDM statistically outperforms state-of-the-art LDL methods.

---

PaperID: 449,   

Authors:  Hao Wang, Jiahu Qin, Zhen Kan

Title: Shielded Planning Guided Data-Efficient and Safe Reinforcement Learning

Abstract:
Safe reinforcement learning (RL) has shown great potential for building safe general-purpose robotic systems. While many existing works have focused on post-training policy safety, it remains an open problem to ensure safety during training as well as to improve exploration efficiency. Motivated to address these challenges, this work develops shielded planning guided policy optimization (SPPO), a new model-based safe RL method that augments policy optimization algorithms with path planning and shielding mechanism. In particular, SPPO is equipped with shielded planning for guided exploration and efficient data collection via model predictive path integral (MPPI), along with an advantage-based shielding rule to keep the above processes safe. Based on the collected safe data, a task-oriented parameter optimization (TOPO) method is used for policy improvement, as well as the observation-independent latent dynamics enhancement. In addition, SPPO provides explicit theoretical guarantees, i.e., clear theoretical bounds for training safety, deployment safety, and the learned policy performance. Experiments demonstrate that SPPO outperforms baselines in terms of policy performance, learning efficiency, and safety performance during training.

---

PaperID: 450,   

Authors:  Zhi-Lin Zhao, Longbing Cao, Chang-Dong Wang

Title: Gray Learning From Non-IID Data With Out-of-Distribution Samples

Abstract:
The integrity of training data, even when annotated by experts, is far from guaranteed, especially for non-independent and identically distributed (non-IID) datasets comprising both in- and out-of-distribution samples. In an ideal scenario, the majority of samples would be in-distribution, while samples that deviate semantically would be identified as out-of-distribution and excluded during the annotation process. However, experts may erroneously classify these out-of-distribution samples as in-distribution, assigning them labels that are inherently unreliable. This mixture of unreliable labels and varied data types makes the task of learning robust neural networks notably challenging. We observe that both in- and out-of-distribution samples can almost invariably be ruled out from belonging to certain classes, aside from those corresponding to unreliable ground-truth labels. This opens the possibility of utilizing reliable complementary labels that indicate the classes to which a sample does not belong. Guided by this insight, we introduce a novel approach, termed gray learning (GL), which leverages both ground-truth and complementary labels. Crucially, GL adaptively adjusts the loss weights for these two label types based on prediction confidence levels. By grounding our approach in statistical learning theory, we derive bounds for the generalization error, demonstrating that GL achieves tight constraints even in non-IID settings. Extensive experimental evaluations reveal that our method significantly outperforms alternative approaches grounded in robust statistics.

---

PaperID: 451,   

Authors:  Lu Jin, Zechao Li, Yonghua Pan, Jinhui Tang

Title: Relational Consistency Induced Self-Supervised Hashing for Image Retrieval

Abstract:
This article proposes a new hashing framework named relational consistency induced self-supervised hashing (RCSH) for large-scale image retrieval. To capture the potential semantic structure of data, RCSH explores the relational consistency between data samples in different spaces, which learns reliable data relationships in the latent feature space and then preserves the learned relationships in the Hamming space. The data relationships are uncovered by learning a set of prototypes that group similar data samples in the latent feature space. By uncovering the semantic structure of the data, meaningful data-to-prototype and data-to-data relationships are jointly constructed. The data-to-prototype relationships are captured by constraining the prototype assignments generated from different augmented views of an image to be the same. Meanwhile, these data-to-prototype relationships are preserved to learn informative compact hash codes by matching them with these reliable prototypes. To accomplish this, a novel dual prototype contrastive loss is proposed to maximize the agreement of prototype assignments in the latent feature space and Hamming space. The data-to-data relationships are captured by enforcing the distribution of pairwise similarities in the latent feature space and Hamming space to be consistent, which makes the learned hash codes preserve meaningful similarity relationships. Extensive experimental results on four widely used image retrieval datasets demonstrate that the proposed method significantly outperforms the state-of-the-art methods. Besides, the proposed method achieves promising performance in out-of-domain retrieval tasks, which shows its good generalization ability. The source code and models are available at https://github.com/IMAG-LuJin/RCSH.

---

PaperID: 452,   

Authors:  Yifan Chen, Yang Zhao, Xuelong Li

Title: Adaptive Gait Feature Learning Using Mixed Gait Sequence

Abstract:
Gait recognition has become a mainstream technology for identification, as it can recognize the identity of subjects from a distance without any cooperation. However, when subjects wear coats (CL) or backpacks (BG), their gait silhouette will be occluded, which will lose some gait information and bring great difficulties to the identification. Another important challenge in gait recognition is that the gait silhouette of the same subject captured by different camera angles varies greatly, which will cause the same subject to be misidentified as different individuals under different camera angles. In this article, we try to overcome these problems from three aspects: data augmentation, feature extraction, and feature refinement. Correspondingly, we propose gait sequence mixing (GSM), multigranularity feature extraction (MFE), and feature distance alignment (FDA). GSM is a method that belongs to data enhancement, which uses the gait sequences in NM to assist in learning the gait sequences in BG or CL, thus reducing the influence of lost gait information in abnormal gait sequences (BG or CL). MFE explores and fuses different granularity features of gait sequences from different scales, and it can learn as much useful information as possible from incomplete gait silhouettes. FDA refines the extracted gait features with the help of the distribution of gait features in real world and makes them more discriminative, thus reducing the influence of various camera angles. Extensive experiments demonstrate that our method has better results than some state-of-the-art methods on CASIA-B and mini-OUMVLP. We also embed the GSM module and FDA module into some state-of-the-art methods, and the recognition accuracy of these methods is greatly improved.

---

PaperID: 453,   

Authors:  Birgit Hillebrecht, Benjamin Unger

Title: Rigorous a Posteriori Error Bounds for PDE-Defined PINNs

Abstract:
Prediction error quantification in machine learning has been left out of most methodological investigations of neural networks (NNs), for both purely data-driven and physics-informed approaches. Beyond statistical investigations and generic results on the approximation capabilities of NNs, we present a rigorous upper bound on the prediction error of physics-informed NNs (PINNs). This bound can be calculated without the knowledge of the true solution and only with a priori available information about the characteristics of the underlying dynamical system governed by a partial differential equation (PDE). We apply this a posteriori error bound exemplarily to four problems: the transport equation, the heat equation, the Navier–Stokes equation (NSE), and the Klein–Gordon equation.

---

PaperID: 454,   

Authors:  Yunzhi Ling, Feiping Nie, Weizhong Yu, Xuelong Li

Title: Discriminative and Robust Autoencoders for Unsupervised Feature Selection

Abstract:
Many recent research works on unsupervised feature selection (UFS) have focused on how to exploit autoencoders (AEs) to seek informative features. However, existing methods typically employ the squared error to estimate the data reconstruction, which amplifies the negative effect of outliers and can lead to performance degradation. Moreover, traditional AEs aim to extract latent features that capture intrinsic information of the data for accurate data recovery. Without incorporating explicit cluster structure-detecting objectives into the training criterion, AEs fail to capture the latent cluster structure of the data which is essential for identifying discriminative features. Thus, the selected features lack strong discriminative power. To address the issues, we propose to jointly perform robust feature selection and k -means clustering in a unified framework. Concretely, we exploit an AE with a l_2,1 -norm as a basic model to seek informative features. To improve robustness against outliers, we introduce an adaptive weight vector for the data reconstruction terms of AE, which assigns smaller weights to the data with larger errors to automatically reduce the influence of the outliers, and larger weights to the data with smaller errors to strengthen the influence of clean data. To enhance the discriminative power of the selected features, we incorporate k -means clustering into the representation learning of the AE. This allows the AE to continually explore cluster structure information, which can be used to discover more discriminative features. Then, we also present an efficient approach to solve the objective of the corresponding problem. Extensive experiments on various benchmark datasets are provided, which clearly demonstrate that the proposed method outperforms state-of-the-art methods.

---

PaperID: 455,   

Authors:  Wei Liu, Rongxin Cui, Yinglin Li, Shouxu Zhang

Title: Hybrid-Input Convolutional Neural Network-Based Underwater Image Quality Assessment

Abstract:
Since precisely sensing the underwater environment is a challenging prerequisite for safe and reliable underwater operation, interest in underwater image processing is growing at a rapid pace. In engineering applications, there are redundant underwater images addressed in real-time on the remotely operated vehicle (ROV). It puts the equipment or operators under great pressure. To relieve this pressure by transmitting images selectively according to the degradation degree, we propose an end-to-end hybrid-input convolutional neural network (HI-CNN) to predict the degradation of underwater images. First, we propose a feature extraction module to extract the features of original underwater images and saliency maps concurrently, which is composed of two branches with the same structure and shared parameters. Second, we design an end-to-end model to predict the quality scores of original images, which consists of a feature extraction module and a prediction module. Finally, we establish a real-world dataset to make the proposed model be duplicated in the practical underwater environment. Through several experiments, we demonstrate that the proposed model outperforms existing models in predicting underwater image quality.

---

PaperID: 456,   

Authors:  Shuhan Li, Xiaomeng Li, Xiaowei Xu, Kwang-Ting Cheng

Title: Dynamic Subcluster-Aware Network for Few-Shot Skin Disease Classification

Abstract:
This article addresses the problem of few-shot skin disease classification by introducing a novel approach called the subcluster-aware network (SCAN) that enhances accuracy in diagnosing rare skin diseases. The key insight motivating the design of SCAN is the observation that skin disease images within a class often exhibit multiple subclusters, characterized by distinct variations in appearance. To improve the performance of few-shot learning (FSL), we focus on learning a high-quality feature encoder that captures the unique subclustered representations within each disease class, enabling better characterization of feature distributions. Specifically, SCAN follows a dual-branch framework, where the first branch learns classwise features to distinguish different skin diseases, and the second branch aims to learn features, which can effectively partition each class into several groups so as to preserve the subclustered structure within each class. To achieve the objective of the second branch, we present a cluster loss to learn image similarities via unsupervised clustering. To ensure that the samples in each subcluster are from the same class, we further design a purity loss to refine the unsupervised clustering results. We evaluate the proposed approach on two public datasets for few-shot skin disease classification. The experimental results validate that our framework outperforms the state-of-the-art methods by around 2%–5% in terms of sensitivity, specificity, accuracy, and F1-score on the SD-198 and Derm7pt datasets.

---

PaperID: 457,   

Authors:  Pengzhen Ren, Kaidong Zhang, Hetao Zheng, Zixuan Li, Yuhang Wen, Fengda Zhu, Shikui Ma, Xiaodan Liang

Title: Surfer: A World Model-Based Framework for Vision-Language Robot Manipulation

Abstract:
Considering how to make the model accurately understand and follow natural language instructions and perform actions consistent with world knowledge is a key challenge in robot manipulation. This mainly includes human fuzzy instruction reasoning and the following of physical knowledge. Therefore, the embodied intelligence agent must have the ability to model world knowledge from training data. However, most existing vision and language robot manipulation methods mainly operate in less realistic simulators and language settings and lack explicit modeling of world knowledge. To bridge this gap, we introduce a novel and simple robot manipulation framework, called Surfer. It is based on the world model, treats robot manipulation as a state transfer of the visual scene, and decouples it into two parts: action and scene. Then, the generalization ability of the model on new instructions and new scenes is enhanced by explicit modeling of the action and scene prediction in multimodal information. In addition, we built a robot manipulation simulation platform that supports physics execution based on the MuJoCo physics engine. It can automatically generate demonstration training data and test data, effectively reducing labor costs. To conduct a comprehensive and systematic evaluation of the visual-language understanding and physical execution of the manipulation model, we also created a robotic manipulation benchmark with different difficulty levels, called SeaWave. It contains four visual-language manipulation tasks of different difficulty levels and can provide a standardized testing platform for embedded AI agents in multimodal environments. Overall, we hope Surfer can freely surf in the robot’s SeaWave benchmark. Extensive experiments show that Surfer consistently outperforms all baselines significantly in all manipulation tasks. On average, Surfer achieved a success rate of 54.74% on the defined four levels of manipulation tasks, exceeding the best baseline performance of 51.07%. The simulator, code, and benchmarks are released at https://pzhren.github.io/Surfer.

---

PaperID: 458,   

Authors:  Jinghao Wang, Zhang Li, Cong Sun, Yulan Guo, Zi Wang, Qifeng Yu

Title: Satellite Pose Set Estimation by Uncertainty-Guided Conformal Keypoint Detection

Abstract:
Satellite pose estimation constitutes a critical technology in the aerospace tasks. The tradeoff between accuracy and efficiency becomes paramount for successful mission execution, due to the limited computational resources of on-board systems. Existing methods predominantly provide single-point estimations, which fall short of fulfilling the uncertainty quantification requirements demanded by safety-critical space operations. To address these problems, we first propose uncertainty-guided conformal keypoint detection to predict keypoint inductive conformal prediction (IndCP) set and then design a uncertainty propagation strategy to obtain pose uncertainty set. Specifically, we build our method upon a transformer-based keypoint predictor, which directly outputs uncertainty-guided keypoints. We first propose a nonconformal function to generate keypoint IndCP set to cover the ground-truth keypoint with a certain probability. We then apply Monte Carlo to sample within the keypoint IndCP set and estimate the poses by solving the perspective-n-point (PnP) problem. The top-n poses with the smallest conformal reprojection error are used to construct a convex hull, which are defined as the pose uncertainty set. Furthermore, we take the mean of the top-n poses as the average pose. Experiments on the Spacecraft PosE Estimation challenge Dataset (SPEED) and LineMOD Occlusion (LMO) dataset show that not only the average pose demonstrates higher accuracy but also the pose uncertainty sets can cover the true pose with the certain probability.

---

PaperID: 459,   

Authors:  Yuntao Shou, Xiangyong Cao, Deyu Meng

Title: SpeGCL: Self-Supervised Graph Spectrum Contrastive Learning Without Positive Samples

Abstract:
Graph contrastive learning (GCL) has emerged as a powerful method for dealing with noise and fluctuations in graph-structured data, and can be applied to social networks and knowledge graphs. Although various graph augmentation strategies have emerged in the field of GCL, traditional graph convolutional network (GCN) mainly tends to preserve smooth features and has difficulty capturing fine-grained changes between different views. To address the above issue, we first construct Fourier graph neural network (FourierGNN) from the perspective of graph spectrum learning, which captures different frequency components by stacking multiple Fourier graph operations (FGO) layers in Fourier space. Then, we find that the difference between the high-frequency information of two augmented graphs should be larger than the difference between the low-frequency information. Next, we theoretically prove that focusing only on pushing negative pairs farther away can more effectively achieve performance advantages. By leveraging these discoveries, we propose a novel self-supervised graph spectrum contrastive learning framework, i.e., SpeGCL, and design an effective contrastive strategy to optimize this goal. We also provide a theoretical justification for the efficacy of using only negative samples in SpeGCL. Extensive experiments have been conducted on unsupervised, transfer, and semi-supervised learning tasks to show that SpeGCL outperforms existing state-of-the-art (SOTA) GCL methods.

---

PaperID: 460,   

Authors:  Yexin Liu, Weiming Zhang, Athanasios V. Vasilakos, Lin Wang

Title: Unsupervised Visible-Infrared ReID via Pseudo-Label Correction and Modality-Level Alignment

Abstract:
Unsupervised visible–infrared person reidentification (UVI-ReID) has recently gained great attention due to its potential for enhancing human detection in diverse environments without labeling. Previous methods utilize intramodality clustering and cross-modality feature matching to achieve UVI-ReID. However, there exist two challenges: 1) noisy pseudo-labels might be generated in the clustering process and 2) the cross-modality feature alignment via matching the marginal distribution of visible and infrared modalities may misalign the different identities from the two modalities. In this article, we first conduct a theoretical analysis where an interpretable generalization upper bound is introduced. Based on the analysis, we then propose a novel unsupervised cross-modality person reidentification framework (PRAISE). Specifically, to address the first challenge, we propose a pseudo-label correction (PLC) strategy that utilizes a beta mixture model (BMM) to predict the probability of misclustering-based network’s memory effect and rectifies the correspondence by adding a perceptual term to contrastive learning. Next, we introduce a modality-level alignment (MLA) strategy that generates paired visible–infrared latent features and reduces the modality gap by aligning the labeling function of visible and infrared features to learn identity-discriminative and modality-invariant features. Experimental results on two benchmark datasets demonstrate that our method achieves a state-of-the-art (SOTA) performance than the unsupervised visible-ReID methods.

---

PaperID: 461,   

Authors:  Zhijie Zhong, Zhiwen Yu, Xing Xi, Yue Xu, Wenming Cao, Yiyuan Yang, Kaixiang Yang, Jane You

Title: SimAD: A Simple Dissimilarity-Based Approach for Time-Series Anomaly Detection

Abstract:
Despite the prevalence of reconstruction-based deep learning methods, time-series anomaly detection (TSAD) remains a tremendous challenge. Existing approaches often struggle with limited temporal contexts, insufficient representation of normal patterns, and flawed evaluation metrics, all of which hinder their effectiveness in detecting anomalous behavior. To address these issues, we introduce a simple dissimilarity-based approach for time-series anomaly detection (SimAD). Specifically, SimAD first incorporates a patching-based feature extractor capable of processing extended temporal windows and employs the EmbedPatch encoder to fully integrate normal behavioral patterns. Second, we design an innovative ContrastFusion module in SimAD, which strengthens the robustness of anomaly detection by highlighting the distributional differences between normal and abnormal data. Third, we introduce two robust enhanced evaluation metrics, unbiased affiliation (UAff) and normalized affiliation (NAff), designed to overcome the limitations of existing metrics by providing better distinctiveness and semantic clarity. The reliability of these two metrics has been demonstrated by both theoretical and experimental analyses. Experiments conducted on seven diverse time-series datasets clearly demonstrate SimAD’s superior performance compared with state-of-the-art (SOTA) methods, achieving relative improvements of 19.85% on F1 , 4.44% on Aff-F1, 77.79% on NAff-F1, and 9.69% on AUC on six multivariate datasets. Code and pretrained models are available at https://github.com/EmorZz1G/SimAD

---

PaperID: 462,   

Authors:  Ye Liu, Hongshan Pu, Junjun Pan, Michael K. Ng, Hongmin Cai

Title: Anchor-Based Multiview Subspace Clustering With Anchor-wise and Class-wise Alignments

Abstract:
Multiview subspace clustering has shown promising performance in multimedia and data mining applications. However, its employment in large-scale datasets is limited due to its quadratic or even cubic computational complexity. The anchor graph strategy, which selects a few important samples (anchors) to represent the whole data for different views, has been introduced to address this challenge. These methods rely on a heuristic assumption that the correspondence and class structures between the sets of anchors across different views are the same. This assumption ignores the difference in the ordering of anchors with respect to their associated classes and the number of anchors belonging to the same class from different views. As a result, this can lead to unsatisfactory clustering results due to incorrect anchorwise and classwise alignments. To tackle this issue, this article proposes an anchor-based multiview subspace clustering with anchorwise and classwise alignments (AMCA2) method. Specifically, the proposed method simultaneously aligns and fuses multiple anchor graphs anchor wisely and class wisely via learning permutation matrices and utilizing the Hadamard product. To further enhance the clustering performance of AMCA2, we propose a novel anchor selection method called kernel anchor selection (KAS) to select more representative anchors. Extensive experiments on ten benchmark datasets are conducted to show the superiority and effectiveness of AMCA2 over the state-of-the-art methods.

---

PaperID: 463,   

Authors:  Yuqi Xiao, Muideen Adegoke, Chi-Sing Leung, Kwok Wa Leung

Title: Robust Fault-Aware Extreme Learning Machine Based on Maximum Correntropy

Abstract:
Extreme learning machine (ELM) is an effective and efficient neural model for universal approximation. However, its practical performance can degrade due to weight noise, node faults, and outliers. This brief introduces a robust ELM algorithm designed to address these issues and enhance network robustness. We first analyze the square error of the classic ELM, considering both weight noise and node faults. By integrating an outlier-resistant method, the maximum correntropy criterion (MCC), we derive a new objective function to bolster network resilience. This leads to the development of the robust fault-aware ELM (RFAELM) algorithm. The convergence property of RFAELM is rigorously proven. For validation, the proposed algorithm is evaluated in various noise and fault levels using eight different benchmark datasets. The simulation results, encompassing all imperfect conditions and datasets, verify the robustness and generalization of this new algorithm. Also, the new algorithm is compared with other robust ELM algorithms using different statistical measurements. The superior performance of RFAELM substantiates its significant improvement over existing algorithms.

---

PaperID: 464,   

Authors:  Zhixiu Lu, Hailong Li, Nehal A. Parikh, Jonathan R. Dillman, Lili He

Title: RadCLIP: Enhancing Radiologic Image Analysis Through Contrastive Language-Image Pretraining

Abstract:
The integration of artificial intelligence (AI) with radiology signifies a transformative era in medicine. Vision foundation models have been adopted to enhance radiologic imaging analysis. However, the inherent complexities of 2D and 3D radiologic data present unique challenges that existing models, which are typically pretrained on general nonmedical images, do not adequately address. To bridge this gap and harness the diagnostic precision required in radiologic imaging, we introduce radiologic contrastive language–image pretraining (RadCLIP): a cross-modal vision-language foundational model that utilizes a vision-language pretraining (VLP) framework to improve radiologic image analysis. Building on the contrastive language–image pretraining (CLIP) approach, RadCLIP incorporates a slice pooling mechanism designed for volumetric image analysis and is pretrained using a large, diverse dataset of radiologic image-text pairs. This pretraining effectively aligns radiologic images with their corresponding text annotations, resulting in a robust vision backbone for radiologic imaging. Extensive experiments demonstrate RadCLIP’s superior performance in both unimodal radiologic image classification and cross-modal image-text matching, underscoring its significant promise for enhancing diagnostic accuracy and efficiency in clinical settings. Our key contributions include curating a large dataset featuring diverse radiologic 2D/3D image-text pairs, pretraining RadCLIP as a vision-language foundation model on this dataset, developing a slice pooling adapter with an attention mechanism for integrating 2D images, and conducting comprehensive evaluations of RadCLIP on various radiologic downstream tasks.

---

PaperID: 465,   

Authors:  Tianyang Zhong, Yi Pan, Yutong Zhang, Yaonai Wei, Li Yang, Zhengliang Liu, Xiaozheng Wei, Wenjun Li, Junjie Yao, Chong Ma, Xi Jiang, Dinggang Shen, Junwei Han, Tuo Zhang

Title: ChatABL: Abductive Learning via Natural Language Interaction With ChatGPT

Abstract:
Large language models (LLMs) such as ChatGPT have recently demonstrated significant potential in mathematical abilities, providing a valuable reasoning paradigm consistent with human natural language. However, LLMs currently have difficulty in bridging perception, language understanding, and reasoning (PLR) capabilities due to incompatibility of the underlying information flow among them, making their reasoning ability not fully elicited and challenging to accomplish complicated reasoning tasks autonomously. To resolve the above problem, a novel method called ChatABL is proposed by integrating LLMs into an abductive learning (ABL) framework, capable of unifying the three abilities effectively in a more user-friendly and understandable manner. Initially, the proposed method uses LLMs to correct the incomplete logical facts for optimizing the perception module, by summarizing and reorganizing domain knowledge represented in natural language format. Then, the perception module also provides necessary logical reasoning materials for feeding LLMs. Finally, these parts are integrated into a dynamic closed-loop system by introducing the feedback form and automatic learning strategies to mutually promote their performance. As a testbed, the variable-length handwritten equation decipherment (HED), an abstract expression of the Mayan calendar decoding, is used to demonstrate that ChatABL has reasoning ability beyond most existing state-of-the-art methods, which has been well-supported by comparative studies. To the best of authors’ knowledge, the proposed ChatABL is the first attempt to explore a possible and novel avenue to approaching human-level cognitive ability via natural language interaction by means of ChatGPT.

---

PaperID: 466,   

Authors:  Yifan Zhang, Yang Yu, Hao Li, Anqi Wu, Xin Chen, Jinfang Liu, Ling-Li Zeng, Dewen Hu

Title: DMAE-EEG: A Pretraining Framework for EEG Spatiotemporal Representation Learning

Abstract:
Electroencephalography (EEG) plays a crucial role in neuroscience research and clinical practice, but it remains limited by nonuniform data, noise, and difficulty in labeling. To address these challenges, we develop a pretraining framework named DMAE-EEG, a denoising masked autoencoder for mining generalizable spatiotemporal representation from massive unlabeled EEG. First, we propose a novel brain region topological heterogeneity (BRTH) division method to partition the nonuniform data into fixed patches based on neuroscientific priors. Second, we design a denoised pseudo-label generator (DPLG), which utilizes a denoising reconstruction pretext task to enable the learning of generalizable representations from massive unlabeled EEG, suppressing the influence of noise and artifacts. Furthermore, we utilize an asymmetric autoencoder with self-attention as the backbone in the proposed DMAE-EEG, which captures long-range spatiotemporal dependencies and interactions from unlabeled EEG data across 14 public datasets. The proposed DMAE-EEG is validated on both generative (signal quality enhancement) and discriminative tasks (motion intention recognition). In the quality enhancement, DMAE-EEG outperforms existing statistical methods with normalized mean squared error (nMSE) reduction of 27.78%–50.00% under corruption levels of 25%, 50%, and 75%, respectively. In motion intention recognition, DMAE-EEG achieves a relative improvement of 2.71%–6.14% in intrasession classification balanced accuracy across 2–6 class motor execution and imagery tasks, outperforming state-of-the-art methods. Overall, the results suggest that the pretraining framework DMAE-EEG can capture generalizable spatiotemporal representations from massive unlabeled EEG and enhance the knowledge transferability across sessions, subjects, and tasks in various downstream scenarios, advancing EEG-aided diagnosis and brain–computer communication and control, and other clinical practice.

---

PaperID: 467,   

Authors:  Shenfu Zhang, Qiang Liu, Zhenhua Zhang, Rui Zhao, Liang Chen, Feng Shao, Xiangchao Meng

Title: Spatial-Spectral Heterogeneity-Aware Network for Hyperspectral and LiDAR Joint Classification

Abstract:
The integration of hyperspectral (HS) imagery and light detection and ranging (LiDAR) data for land cover classification has emerged as a prominent research focus. Despite the satisfactory classification accuracies achieved by existing methodologies, several unaddressed issues that remain warrant consideration. First, current approaches overlook the pronounced spectral and spatial heterogeneities in remote sensing (RS) images designated for multiclassification tasks, limiting the performance of classification models. Moreover, most existing studies amalgamate elevation features with other characteristics through simple addition and interaction operations, and they do not delve deeply into exploiting elevation height information, leading to an imbalance in the representation of elevation height. In light of the aforementioned issues, this article introduces a spatial–spectral heterogeneity-aware network (S2HANet) for the joint classification of HS and LiDAR data. Specifically, a shared spectral correction module (SSCM) is designed in the spectral branch to preliminarily alleviate the problem of large intraclass variance, followed by the use of a contrastive learning framework to enhance the intraclass compactness and interclass separability of spectral features. A multichannel signed distance discrimination module (MCSDDM) is developed to learn the distance relationships between intra- and interclass pixels and boundaries, and using prior boundary information to improve spatial boundary information. In addition, an elevation boost module (EBM) and an elevation injection module (EIM) are meticulously designed to phase-in elevation height information, further enhancing the utilization of elevation data and better facilitating the fusion of the two modalities. The proposed S2HANet has demonstrated exceptional classification performance across three opening benchmark datasets.

---

PaperID: 468,   

Authors:  Xinqiao Zhao, Mingjie Sun, Eng Gee Lim, Yao Zhao, Jimin Xiao

Title: SFBM: Shared Feature Bias Mitigating for Long-Tailed Image Recognition

Abstract:
Long-tailed distribution exists in real-world scenario and compromises the performance of recognition models. In this article, we point out that a neural network classifier has a shared feature bias, which tends to regard the shared features among different classes as head-class discriminative features, leading to misclassifications on tail-class samples under long-tailed scenarios. To solve this issue, we propose a shared feature bias mitigating (SFBM) framework. Specifically, we create two parallel classifiers trained concurrently with the baseline classifier, using our special training loss. The parallel classifier weight sums are then used for estimating the shared feature components in baseline classifier weights. Finally, we rectify the baseline classifier by removing the estimated shared feature components from it while supplementing the parallel classifier weights class by class to the rectified classifier weights, mitigating shared feature bias. Our proposed SFBM demonstrates broad compatibility with nearly all recognition methods while maintaining high computational efficiency, as it introduces no additional computation during inference. Extensive experiments on CIFAR10/100-LT, ImageNet-LT, and iNaturalist 2018 demonstrate that simply incorporating SFBM during the training phase consistently boosts the performance of various state-of-the-art methods by significant margins. The complete source code will be made publicly available at https://github.com/bzbz-bot/SFBM

---

PaperID: 469,   

Authors:  Jing Liang, Genyue Liu, Ying Bi, Mingyuan Yu, Mengnan Liu, Yaochu Jin

Title: Evolutionary Neural Architecture Search for Remote Sensing Image Classification

Abstract:
Remote sensing scene classification is a vital task in remote sensing image analysis with significant application potential. In recent years, convolutional neural network (CNN)-based methods have shown remarkable promise in classifying remote sensing scene images. However, these methods often require extensive trial and error and rely heavily on expert knowledge. To address these challenges, this article proposes a novel neural architecture search (NAS) approach that automatically designs CNNs for remote sensing scene classification. Specifically, an evolutionary algorithm (EA) is employed to search for well-structured basic modules, which are then combined to construct a new architecture. To further enhance the search process, a new population generation strategy is introduced to promote diversity and mitigate premature convergence. Additionally, a random forest-based selection mechanism is utilized to identify high-quality individuals based on estimated fitness values, effectively reducing computational complexity. The proposed approach is evaluated on three benchmark remote sensing scene datasets and compared with several widely used CNNs. The experimental results demonstrate that the proposed approach can discover CNN architectures that not only surpass state-of-the-art performance but also achieve this with fewer parameters and lower search cost.

---

PaperID: 470,   

Authors:  Haiteng Wang, Lei Ren, Yikang Li, Yuqing Wang

Title: MetaIndux-TS: Frequency-Aware AIGC Foundation Model for Industrial Time Series

Abstract:
Implementing advanced AI techniques in industrial manufacturing requires large volumes of annotated sensor data. Unfortunately, collecting such data is often impractical due to extreme environments and the manual burden of expert annotation. Recent advancements in artificial intelligence generated content (AIGC) have inspired the exploration of industrial time-series generation to mitigate data shortages. However, existing AIGC models encounter difficulties in generating industrial time series due to their complex temporal dynamics, multichannel intercolumn correlations, and diverse frequency characteristics. To address these challenges, we propose MetaIndux-TS, a frequency-informed AIGC foundation model based on diffusion model frameworks. This model is designed to generate industrial time-series data under a variety of working conditions, across different types of equipment, and with variable lengths. Specifically, MetaIndux-TS integrates dual-frequency cross-attention networks, transforming time series into the frequency domain to model multivariate dependencies and capture intricate temporal details. In addition, the contrastive synthesis layer is constructed to generate high-fidelity time series by comparing periodic and long-term trends with initial noisy sequences. Comprehensive experiments show that MetaIndux-TS outperforms state-of-the-art models (SSSD, Dit, and TabDDPM), achieving a 57.5% improvement in fidelity and 20.4% in predictive score. MetaIndux-TS exhibits zero-shot generation capabilities for samples under unseen conditions, offering the potential to address data collection challenges in extreme environments. Codes are available at: https://github.com/Dolphin-wang/MetaIndux

---

PaperID: 471,   

Authors:  Dimitrios Katsikas, Nikolaos Passalis, Anastasios Tefas

Title: Inducing Neural Collapse via Anticlasses and One-Cold Cross-Entropy Loss

Abstract:
While softmax cross-entropy (CE) loss is the standard objective for supervised classification, it primarily focuses on the ground-truth classes, ignoring the relationships between the nontarget, complementary classes. This leaves valuable information unexploited during optimization. In this work, we propose a novel loss function, one-cold CE (OCCE) loss, which addresses this limitation by structuring the activations of these complementary classes. Specifically, for each class, we define an anticlass, which consists of everything that is not part of the target class—this includes all complementary classes as well as out-of-distribution (OOD) samples, noise, or in general any instance that does not belong to the true class. By setting a uniform one-cold encoded distribution over the complementary classes as a target for each anticlass, we encourage the model to equally distribute activations across all nontarget classes. This approach promotes a symmetric geometric structure of classes in the final feature space, increases the degree of neural collapse (NC) during training, addresses the independence deficit problem of neural networks, and improves generalization. Our extensive evaluation shows that incorporating OCCE loss in the optimization objective consistently enhances performance across multiple settings, including classification, open-set recognition, and OOD detection.

---

PaperID: 472,   

Authors:  Sheng Liu, Shaobo Zhang, Fei Gao, Yuan Feng

Title: Highly Condensed All-MLP Architecture for Long-Term Human Motion Prediction

Abstract:
In artificial intelligence (AI) scenarios where computational resources are constrained, such as in autonomous driving systems, it is challenging to construct a lightweight model that can accurately predict human motion overextended duration. To tackle this challenge, we introduce a highly condensed all-multilayer perceptron (HCMLP) architecture that is engineered for supreme lightweight efficiency. This design facilitates extended-range motion predictions while maintaining uncompromised performance. First, the spatiotemporal dynamic perception (STDP) block enhances operational efficiency while maintaining a simple structure. In STDP, the distinct but parallel spatial multilayer perceptron (SMLP) and temporal multilayer perceptron (TMLP) simultaneously capture the spatial correlations between pose joints and the temporal dynamics of each joint. The subsequent dynamic aggregation (DA), coupled with the channel multilayer perceptron (CMLP), dynamically consolidates and refines spatial and temporal features, leading to improved predictive accuracy. Second, the multiterm union prediction (MTUP) block directly delivers precise predictions for periods ranging from 0 to 4000 ms, eliminating the need for repetitive short-term (ST) prediction iterations. Our experimental results on the Human3.6M, AMASS, 3DPW, and CMU-Mocap datasets demonstrate that HCMLP outperforms existing state-of-the-art (SOTA) methods in ST prediction, long-term (LT) prediction, and especially in extended and extra extended LT (ELT) predictions, all while utilizing the fewest parameters.

---

PaperID: 473,   

Authors:  Jing Zhao, Qimin Huang, Shanhu Wang, Shiliang Sun

Title: VB-Adapter: Variational Bayesian Adapter for Cross-Domain Speech Representation Learning

Abstract:
To leverage the abundant speech data available for pretraining, current models excel in generalization across diverse tasks. Nevertheless, real-world challenges emerge when addressing unfamiliar speech scenarios far from the pretrained speech, owing to the domain shift between pretraining (source) and fine-tuning (target) data. To overcome this barrier, we propose a variational Bayesian adapter (VB-Adapter) for cross-domain speech representation learning during fine-tuning. First, we establish a latent variable model to construct a desired posterior distribution after incorporating domain-specific knowledge to bridge the gap between the source and target domains. Then, an adaptive objective is presented to maximize the mutual information of the latent variables with and without domain-specific knowledge to facilitate model adaptation. Finally, we introduce contrastive learning on samples to optimize the lower bound of the above adaptive objective. Our experiments apply the VB-Adapter on transformers for dysarthric speech recognition (DSR) and the integration of Whisper-encoder and Llama for Mandarin speech recognition (MSR). The results reveal the effectiveness of VB-Adapter in modeling the uncertainties arising from domain shift and enhancing the robustness of speech representations.

---

PaperID: 474,   

Authors:  Oscar Pina, Verónica Vilaplana

Title: Layer-Wise Training of Graph Neural Networks With Self-Supervised Learning

Abstract:
Training graph neural networks (GNNs) on large graphs is challenging due to both the high memory and computational costs of end-to-end training and the scarcity of detailed node-level annotations. To address these challenges, we propose layer-wise regularized graph infomax (LRGI), a self-supervised learning algorithm inspired by predictive coding, a biologically motivated principle in which each layer is trained locally to predict its future inputs. LRGI trains GNNs layer by layer, decoupling their memory and time complexity from the network depth, thereby enabling scalable training on large graphs. In LRGI, each layer learns to predict the features propagated from its neighbors, allowing independent training of each layer. This approach, combined with regularization that promotes diverse representations, also helps mitigate oversmoothing in deep GNNs. Experiments on large inductive graph benchmarks demonstrate that LRGI achieves competitive performance compared to state-of-the-art end-to-end methods, while substantially improving efficiency.

---

PaperID: 475,   

Authors:  Xiao Liu, Xiaofeng Wang, Shouyi Wang, Haosong Gou, Zhengyong Wang, Chao Ren

Title: GBPG-Net: Global Background Prior-Guided Rain and Snow Image Restoration

Abstract:
The aim of image restoration in the presence of rain and snow effects is to eliminate these disturbances while retaining the underlying background structure. Most existing methods tend to directly learn the mapping from corrupted images to clean ones, often resulting in residual rain or snow artifacts and compromised background structures. In this work, both theoretical analysis and experimental findings confirm the robustness of the hue channel in HSV color space to rain and snow disturbances, even when extracted from corrupted images. Motivated by this insight, we propose to leverage the global clean background cues inherent in the hue channel to guide the network in preserving the image background structure and removing interference. To this end, we introduce the global background prior-guided network (GBPG-Net) for restoring rain and snow-affected images, which employs a triangular formation to facilitate continuous interaction and updating of the global background prior (GBP) with the image feature within the GBPG-unit, resulting in improved interference removal and background structure preservation. Specifically, the GBPG-Net incorporates the global clean background prior injector (GCBPI) to inject the GBP into the network. Subsequently, the prior-guided local detail excavation (PGLDE) module, built on GCBPI, further refines interference removal and structure preservation to process local details intricately. Finally, the prior-guided local-global aggregation (PGLGA) module aggregates global background features with local detailed features, enabling the network to better understand the overall content and subtle interference for more accurate reconstruction. Quantitative and qualitative evaluations on synthetic and real datasets demonstrate the effectiveness of the proposed GBPG-Net in deraining and desnowing tasks, highlighting its advantages over existing methods. The code and supplementary documentation are available at https://github.com/liux520/GBPG-Net

---

PaperID: 476,   

Authors:  Qingfeng Chen, Shiyuan Li, Yixin Liu, Shirui Pan, Geoffrey I. Webb, Shichao Zhang

Title: Uncertainty-Aware Graph Neural Networks: A Multihop Evidence Fusion Approach

Abstract:
Graph neural networks (GNNs) excel in graph representation learning by integrating graph structure and node features. Existing GNNs, unfortunately, fail to account for the uncertainty of class probabilities that vary with the depth of the model, leading to unreliable and risky predictions in real-world scenarios. To bridge the gap, in this article, we propose a novel evidence-fusing graph neural network (EFGNN) to achieve trustworthy prediction, enhance node classification accuracy, and make explicit the risk of wrong predictions. In particular, we integrate the evidence theory with multihop propagation-based GNN architecture to quantify the prediction uncertainty of each node with the consideration of multiple receptive fields. Moreover, a parameter-free cumulative belief fusion (CBF) mechanism is developed to leverage the changes in prediction uncertainty and fuse the evidence to improve the trustworthiness of the final prediction. To effectively optimize the EFGNN model, we carefully design a joint learning objective composed of evidence cross-entropy, dissonance coefficient, and false confident penalty. The experimental results on various datasets and theoretical analyses demonstrate the effectiveness of the proposed model in terms of accuracy and trustworthiness, as well as its robustness to potential attacks.

---

PaperID: 477,   

Authors:  Wenjia Meng, Qian Zheng, Long Yang, Yilong Yin, Gang Pan

Title: Off-OAB: Off-Policy Policy Gradient Method With Optimal Action-Dependent Baseline

Abstract:
The policy-based methods have achieved remarkable success in solving challenging reinforcement learning (RL) problems. Among these methods, the off-policy policy gradient (OPPG) methods are particularly important because they can benefit from off-policy data. However, these methods suffer from the high variance of the OPPG estimator, which results in poor sample efficiency during training. In this article, we propose an off-policy policy gradient method with the optimal action-dependent baseline (Off-OAB) to mitigate this variance issue. Specifically, this baseline maintains the OPPG estimator’s unbiasedness while theoretically minimizing its variance. To enhance practical computational efficiency, we design an approximated version of this optimal baseline. Utilizing this approximation, our method (Off-OAB) aims to decrease the OPPG estimator’s variance during policy optimization. We evaluate the proposed Off-OAB method on six representative tasks from OpenAI Gym and MuJoCo, where it demonstrably surpasses the state-of-the-art methods on the majority of these tasks.

---

PaperID: 478,   

Authors:  Yang Lu, Xiaolin Huang, Yizhou Chen, Mengke Li, Yan Yan, Chen Gong, Hanzi Wang

Title: MOOD: Leveraging Out-of-Distribution Data to Enhance Imbalanced Semi-Supervised Learning

Abstract:
The imbalanced semi-supervised learning (SSL) has emerged as a critical research area due to the prevalence of class imbalanced and partially labeled data in real-world scenarios. As the requirement for data volume increases, naturally collected datasets inevitably contain out-of-distribution (OOD) samples. However, the performance of existing imbalanced SSL methods experiences a marked deterioration with OOD data. In this article, we propose an imbalanced SSL method called mixup-OOD (MOOD) to address this issue. The core idea is to “turn waste into treasure,” exploring the potential of leveraging seemingly detrimental OOD data to expand the feature space, particularly for tail classes. Specifically, we first filter OOD data from unlabeled data, and then fuse it with labeled data to boost feature diversity for the tail classes. To avoid feature overlapping with OOD data, we develop a push-and-pull (PaP) loss to attract in-distribution (ID) instances toward respective class centroids while repelling OOD samples from them. Extensive experiments show that MOOD achieves superior performance compared with other state-of-the-art methods and exhibits robustness across data with different imbalanced ratios and OOD proportions. The source code is available at: https://github.com/xlhuang132/MOODv2

---

PaperID: 479,   

Authors:  Yongle Huang, Zedong Liu, Shijie Sun, Ningning Cui, Jianxin Li

Title: SFAN: Selective Filter and Alignment Network for Cross-Modal Retrieval

Abstract:
Bridging the gap between visual and textual modalities effectively has consistently been a key challenge in cross-modal retrieval. Fine-grained matching approaches improve performance by precisely aligning salient region features in visual modality with word embeddings in textual modality. However, how to effectively and efficiently filter out irrelevant features (e.g., irrelevant background regions and nonmeaningful prepositions) in multimodality remains a significant challenge. Furthermore, capturing key cross-modal relationships while minimizing misalignment interference is crucial for effective cross-modal retrieval. In this work, we propose a novel approach called the selective filter and alignment network (SFAN) to tackle these challenges. First, we propose modality-specific selective filter modules (SFMs) to selectively and implicitly filter out redundant information within each modality. We then propose the state-space models (SSMs)-based selective alignment module (SAM) to selectively capture key correspondences and reduce the disturbance of irrelevant associations. Finally, we utilize a fusion operation to combine these embeddings from both SFM and SAM to derive the final embeddings for similarity computation. Extensive experiments on the Flickr30k, MS-COCO, and MSR-VTT datasets reveal that our proposed SFAN can effectively learn robust patterns, significantly outperforming the state-of-the-art (SOTA) cross-modal retrieval methods by a wide margin.

---

PaperID: 480,   

Authors:  Kendong Liu, Mingtao Feng, Wei Zhao, Jingtao Sun, Weisheng Dong, Yaonan Wang, Ajmal Mian

Title: Pixel-Level Noise Mining for Weakly Supervised Salient Object Detection

Abstract:
Training a deep model for visual saliency detection requires the collection and labor-intensive annotation of overwhelmingly large data. We propose to learn saliency detection in a weakly supervised manner from single noisy label, which is easy to obtain from unsupervised handcrafted feature-based methods. However, deep networks tend to overfit such noises leading to a dramatic drop in accuracy. Given our goal, we address a natural question: can we identify outliers during network prediction and rectify the label noises? To this end, we propose a pixel-level noise mining framework for robust salient object detection (SOD) by exploiting its own knowledge, and without the need for external models. Specifically, during the early training stage, we progressively identify the outliers from a novel perspective during saliency detection, before the network overfits to the noisy labels, and generate a selection matrix in each iteration. Next, we adaptively rectify the label noises under the guidance of the selection matrix for better supervision in the later training stage. Extensive experiments on multiple benchmark datasets demonstrate the superiority of our method showing its ability to learn saliency detection comparable to state-of-the-art fully supervised methods. Furthermore, our approach outperforms existing weakly supervised methods utilizing single noisy label and surpasses the half of existing weakly supervised methods employing multiple noisy labels. Our approach, which trains with multiple noisy labels, outperforms all other methods employing multiple noisy labels across four major datasets. Furthermore, we also evaluate the generalization ability of our method on the multiclass semantic segmentation (SS) task. Our code is available at https://github.com/kendongdong/NoiseMining

---

PaperID: 481,   

Authors:  Feng Yan, Xiaoheng Jiang, Yang Lu, Jiale Cao, Mingliang Xu

Title: DefectSAM: Hierarchically Adapting SAM for Pixel-Wise Surface Defect Detection

Abstract:
Segment anything model (SAM) has recently demonstrated powerful segmentation ability for natural scene images (NSIs). However, the SAM exhibits limited performance in defect detection owing to the weak appearance of defects and cluttered backgrounds in industrial images. In this article, we propose a hierarchically adapting SAM for pixel-wise surface defect detection, named DefectSAM, which effectively modulates and decodes multilevel features of the encoder to capture defect information. Specifically, we introduce a learnable feature adaptation component between the image encoder and the decoder to modulate each level of features via the dual-feature adaptation unit. The dual-feature adaptation unit mainly includes the correlation-gated feature adaptation (CGFA) module and the mask-guided feature adaptation (MGFA) module. The CGFA exploits cross correlation spatial gating maps to adaptively incorporate a convolutional feature pyramid and Transformer features during feature adaptation, which is beneficial for capturing defect details. Moreover, the MGFA utilizes the mask prediction of high-level features as semantic guidance to select top-confidence foreground and background tokens for feature adaptation, focusing more on defect details and suppressing background noise. Extensive experiments on three defect detection datasets (i.e., MVTec AD, CrackSeg9k, ZJU-Leaper, and Magnetic tile) demonstrate that the proposed method achieves state-of-the-art performance with few learnable parameters, which greatly improves the generalization of SAM in defect detection.

---

PaperID: 482,   

Authors:  Kun Dai, Zhiqiang Jiang, Fuyuan Qiu, Dedong Liu, Tao Xie, Ke Wang, Ruifeng Li, Lijun Zhao

Title: HVLF: A Holistic Visual Localization Framework Across Diverse Scenes

Abstract:
Recently, integrating the multitask learning (MTL) paradigm into scene coordinate regression (SCoRe) techniques has achieved significant success in visual localization tasks. However, the feature extraction ability of existing frameworks is inherently constrained by the rigid weight activation strategy, which prevents each layer from concurrently capturing scene-universal features across diverse scenes and scene-particular attributes unique to each individual scene. In addition, the straightforward network architecture further exacerbates the issue of insufficient feature representation. To address these limitations, we introduce HVLF, a holistic framework that ensures flexible identification of both scene-universal and scene-particular attributes while integrating various attention mechanisms to enhance feature representation effectively. Technically, for the first issue, HVLF proposes a soft weight activation strategy (SWAS) equipped with polyhedral convolution to concurrently optimize scene-shared and scene-specific weights within each layer, which facilitates sufficient discernment of both scene-universal features and scene-particular attributes, thereby boosting the network’s capability for comprehensive scene perception. For the second issue, HVLF introduces a mixed attention perception module (MAPM) that incorporates channelwise, spatialwise, and elementwise attention mechanisms to perform multilevel feature fusion, hence extracting discriminative features to regress precise scene coordinates. Extensive experiments on indoor and outdoor datasets prove that HVLF realizes impressive localization performance. In addition, experiments conducted on 3-D object detection and feature matching tasks prove that the two proposed techniques are universal and can be seamlessly inserted into other methods.

---

PaperID: 483,   

Authors:  Yang Wang, Ya-Hui Jia, Wei-Neng Chen, Yi Mei

Title: Distance-Aware Attention Reshaping for Enhancing Generalization of Neural Solvers

Abstract:
Neural solvers (NSs) based on the attention mechanism have demonstrated remarkable effectiveness in solving routing problems like traveling salesman problems (TSPs) and vehicle routing problems (VRPs). However, in the generalization process, we find a phenomenon of the dispersion of attention scores in existing NSs, which leads to poor performance. To improve the generalization ability of NSs, this article proposes a distance-aware attention reshaping (DAR) method. Specifically, without increasing any parameter of the neural network (NN), we utilize the distance information between nodes to adjust attention scores. This enables an NS trained on small-scale instances with a certain distribution to make rational choices when solving large-scale problems with different distributions. Its effectiveness is verified both theoretically and empirically. Extensive experiments on the TSP, asymmetric TSP (ATSP), capacitated VRP (CVRP), VRP with time windows (VRPTW), capacitated arc routing problem (CARP), and knapsack problem (KP) demonstrate the advantages of our method. Our code is available at https://github.com/ftwangyang/DAR

---

PaperID: 484,   

Authors:  Qifeng Zhao, Xuchu Wang

Title: Unsupervised Multimanifold Cross-Guided Diffusion Deformable Registration for Cardiac MRI

Abstract:
Diffusion networks demonstrate remarkable robustness in extracting complex structural features across various domains of medical image processing. In the task of cardiac image registration, diffusion networks excel at reconstructing intricate structural details, thereby enabling effective representation of cardiac anatomical motion. In this article, we propose an unsupervised diffusion registration framework named MCG-Reg for 3-D cardiac magnetic resonance (MR) image registration, employing a multimanifold cross-fusion strategy. MCG-Reg comprises two components: the multimanifold cross-fusion (MCF) module and the weighted fusion codec (WFC) module. MCF module decouples the cardiac image, leveraging multifrequency and multiscale features for cross-attention (CA) calculation, and fuses with the edge image to enable adaptive focus gathering and edge perception capabilities in the model, thereby enhancing the effective aggregation of local and global features. WFC module further processes cardiac features by utilizing offset attention to capture large displacement information, while employing feature energy maps for residual connections to enhance the model’s attention perception ability, thus facilitating better topology maintenance and boundary constraint realization. The registration accuracy and model generalization of the proposed MCG-Reg are validated in publicly available ACDC, M&Ms, and CAP datasets. The experimental results verify that it achieves state-of-the-art performance in comparison to related methods, highlighting the significant potential of the proposed framework in cardiac image analysis applications.

---

PaperID: 485,   

Authors:  Zutao Jiang, Guian Fang, Jianhua Han, Guansong Lu, Hang Xu, Shengcai Liao, Xiaojun Chang, Xiaodan Liang

Title: RealignDiff: Boosting Text-to-Image Diffusion Model With Coarse-to-Fine Semantic Realignment

Abstract:
Recent advances in text-to-image diffusion models have achieved remarkable success in generating high-quality, realistic images from textual descriptions. However, these approaches have faced challenges in precisely aligning the generated visual content with the textual concepts described in the prompts. In this article, we propose a two-stage coarse-to-fine semantic realignment method, named RealignDiff, aimed at improving the alignment between text and images in text-to-image diffusion models. In the coarse semantic realignment phase, a novel caption reward, leveraging the BLIP-2 model, is proposed to evaluate the semantic discrepancy between the generated image caption and the given text prompt. Subsequently, the fine semantic realignment stage uses a local dense caption generation module and a reweighting attention modulation module to refine the previously generated images from a local semantic view. Experimental results on the MS-COCO and ViLG-300 datasets demonstrate that the proposed two-stage coarse-to-fine semantic realignment method outperforms other baseline realignment techniques by a substantial margin in both visual quality and semantic similarity with the input prompt.

---

PaperID: 486,   

Authors:  Mingyuan Luo, Xin Yang, Zhongnuo Yan, Yan Cao, Yuanji Zhang, Xindi Hu, Jin Wang, Haoxuan Ding, Wei Han, Litao Sun, Dong Ni

Title: MoNetV2: Enhanced Motion Network for Freehand 3-D Ultrasound Reconstruction

Abstract:
Three-dimensional ultrasound (US) aims to provide sonographers with the spatial relationships of anatomical structures, playing a crucial role in clinical diagnosis. Recently, deep-learning-based freehand 3-D US has made significant advancements. It reconstructs volumes by estimating transformations between images without external tracking. However, image-only reconstruction poses difficulties in reducing cumulative drift and further improving reconstruction accuracy, particularly in scenarios involving complex motion trajectories. In this context, we propose an enhanced motion network (MoNetV2) to enhance the accuracy and generalizability of reconstruction under diverse scanning velocities and tactics. First, we propose a sensor-based temporal and multibranch structure (TMS) that fuses image and motion information from a velocity perspective to improve image-only reconstruction accuracy. Second, we devise an online multilevel consistency constraint (MCC) that exploits the inherent consistency of scans to handle various scanning velocities and tactics. This constraint exploits scan-level velocity consistency (SVC), path-level appearance consistency (PAC), and patch-level motion consistency (PMC) to supervise interframe transformation estimation. Third, we distill an online multimodal self-supervised strategy (MSS) that leverages the correlation between network estimation and motion information to further reduce cumulative errors. Extensive experiments clearly demonstrate that MoNetV2 surpasses existing methods in both reconstruction quality and generalizability performance across three large datasets.

---

PaperID: 487,   

Authors:  Wenzhuo Liu, Fei Zhu, Cheng-Lin Liu

Title: Branch-Tuning: Balancing Stability and Plasticity for Continual Self-Supervised Learning

Abstract:
The self-supervised learning (SSL) has emerged as an effective paradigm for deriving general representations from vast amounts of unlabeled data. However, as real-world applications continually integrate new content, the high computational and resource demands of SSL necessitate continual learning (CL) rather than complete retraining. This poses a challenge in balancing between stability and plasticity when adapting to new information. In this article, we employ centered kernel alignment (CKA) for quantitatively analyzing model stability and plasticity, revealing the critical roles of batch normalization (BN) layers for stability and convolutional layers for plasticity. Motivated by this, we propose branch-tuning (BT), an efficient and straightforward method that achieves a balance between stability and plasticity in continual SSL. BT consists of branch expansion and compression and can be easily applied to various SSL methods without the need of modifying the original methods, retaining old data or models. We validate our method through experiments on various benchmark datasets, demonstrating its effectiveness and practical value in real-world scenarios. We hope our work offers new insights for future continual SSL research. The code will be made publicly available.

---

PaperID: 488,   

Authors:  Matteo Gambella, Jary Pomponi, Simone Scardapane, Manuel Roveri

Title: NACHOS: Neural Architecture Search for Hardware-Constrained Early-Exit Neural Networks

Abstract:
Early-exit neural networks (EENNs) endow a standard deep neural network (DNN) with early-exit classifiers (EECs) to provide predictions at intermediate points of the processing when enough confidence in classification is achieved. This leads to many benefits in terms of effectiveness and efficiency. Currently, the design of EENNs is carried out manually by experts, a complex and time-consuming task that requires accounting for many aspects, including the correct placement, the thresholding, and the computational overhead of the EECs. For this reason, the research is exploring the use of neural architecture search (NAS) to automate the design of EENNs. Currently, few comprehensive NAS solutions for EENNs have been proposed in the literature, and a fully automated, joint design strategy taking into consideration both the backbone and the EECs remains an open problem. To this end, this work presents neural architecture search for hardware-constrained early exit neural networks (NACHOS), the first NAS framework for the design of optimal EENNs satisfying constraints on the accuracy and the number of multiply and accumulate (MAC) operations performed by the EENNs at inference time. In particular, this provides the joint design of backbone and EECs to select a set of admissible (i.e., respecting the constraints) Pareto optimal solutions in terms of the best trade-off between the accuracy and the number of MACs. The results show that the models designed by NACHOS are competitive with the state-of-the-art EENNs. Additionally, this work investigates the effectiveness of two novel regularization terms designed for the optimization of the auxiliary classifiers of the EENN.

---

PaperID: 489,   

Authors:  Zhihao Yuan, Xu Yan, Zhuo Li, Xuhao Li, Yao Guo, Shuguang Cui, Zhen Li

Title: Toward Fine-Grained 3-D Visual Grounding Through Referring Textual Phrases

Abstract:
Recent progress in 3-D scene understanding has explored visual grounding [3D visual grounding (3DVG)] to localize a target object through a language description. However, existing methods only consider the dependency between the entire sentence and the target object, ignoring fine-grained relationships between contexts and nontarget ones. In this article, we extend 3DVG to a more fine-grained task, called 3D phrase-aware grounding (3DPAG). The 3DPAG task aims to localize the target objects in a 3-D scene by explicitly identifying all phrase-related objects and then conducting the reasoning according to contextual phrases. To tackle this problem, we manually labeled about 227 K phrase-level annotations using a self-developed platform, from 88 K sentences of widely used 3DVG datasets, i.e., Natural Reference in 3-D (Nr3D), Spatial Reference in 3-D (Sr3D), and ScanRefer. By tapping on our datasets, we can extend previous 3DVG methods to the fine-grained phrase-aware scenario. It is achieved through the proposed novel phrase-object alignment (POA) optimization and phrase-specific pretraining (PSP), boosting conventional 3DVG performance as well. Extensive results confirm significant improvements, i.e., previous state-of-the-art method achieves 3.9%, 3.5%, and 4.6% overall accuracy gains on Nr3D, Sr3D, and ScanRefer, respectively. Our datasets and platform are released in https://github.com/CurryYuan/PhraseRefer

---

PaperID: 490,   

Authors:  Shuchang Lyu, Qi Zhao, Hong Zhang, Guangliang Cheng, Chenguang Yang

Title: Online Self-Training Driven Attention-Guided Self-Mimicking Network for Semantic Segmentation

Abstract:
In the realm of semantic segmentation tasks, knowledge distillation (KD) has emerged as a prominent strategy, leveraging the transfer of mature knowledge from large teacher networks to enhance the performance of smaller student networks. However, existing methods often rely heavily on high-quality yet cumbersome teacher networks, leading to a complex training process. To address this challenge, we introduce a novel approach termed self-training driven attention-guided self-mimicking online ensemble network. Our proposed method begins by employing intermediate channel-joint attention maps to guide image augmentation. Both the original and augmented images are then input into the networks. Leveraging intermediate feature maps and predictive predictions generated from the two images, we employ KD to uncover invariant features. To further harness representation potential through learning from credible predictions, we introduce a self-training mechanism. This mechanism utilizes an exponential moving average (EMA)-teacher network constructed using the exponential moving average technique to generate feature maps and predicted posterior probabilities. The knowledge of the EMA-teacher is subsequently transferred to the student network through distillation. Extensive experiments and visualization analyses conducted on multiple benchmark datasets, including Cityscapes, Pascal VOC, CamVid, and ADE20k, validate the effectiveness of self-training driven attention-guided self-mimicking network (ST-ASMNet). The interpretability of our method is further validated through visualization and analysis. Our code will be publicly available.

---

PaperID: 491,   

Authors:  Hao Ding, Chengxing Jia, Zongzhang Zhang, Cong Guan, Feng Chen, Lei Yuan, Yang Yu

Title: Learning to Coordinate With Different Teammates via Team Probing

Abstract:
Coordinating with different teammates is essential in cooperative multiagent systems (MASs). However, most multiagent reinforcement learning (MARL) methods assume fixed team compositions, which leads to agents overfitting their training partners and failing to cooperate well with different teams during the deployment phase. A common way to mitigate the problem is to anticipate teammate behaviors and adapt policies accordingly during cooperation. However, these methods use the same policy for both collecting information for modeling teammates and maximizing cooperation performance. We argue that these two goals may conflict and reduce the effectiveness of both. In this work, we propose coordinating with different teammates via team probing (CDP), a novel approach that rapidly adapts to different teams by disentangling probing and adaptation phases. Specifically, we first generate a diverse population of teams as training partners with a novel value-based diversity objective. Then, we train a probing module to probe and reveal the coordination pattern of each team with policy-dynamics reconstruction and get a representation space of the population. Finally, we train a generalist meta-policy consisting of several expert policies with module selection based on the clustering of the learned representation space. We empirically show that CDP surpasses existing policy adaptation methods in various complex multiagent scenarios with both seen and unseen teammates.

---

PaperID: 492,   

Authors:  Fei Ye, Adrian G. Bors

Title: Training a Dynamic Growing Mixture Model for Lifelong Learning

Abstract:
Lifelong learning (LLL) defines a training paradigm that aims to continuously acquire and capture new concepts from a sequence of tasks without forgetting. Recently, dynamic expansion models (DEMs) have been proposed to address catastrophic forgetting under the LLL paradigm. However, the efficiency of DEMs lacks a thorough explanation based on theoretical analysis. In this article, we develop a new theoretical framework that interprets the forgetting process of the DEM as increasing the statistical discrepancy distance between the distribution of the probabilistic representation of the new data and the previously learned knowledge. The theoretical analysis shows that adding new components to a mixture model represents a trade-off between model complexity and its performance. Inspired by the theoretical analysis, we introduce a new DEM, called the growing mixture model (GMM), where generative data components are added according to the novelty of the incoming task information compared to what is already known. A new component selection mechanism considering the model’s already acquired knowledge is employed for updating new DEM’s components, promoting efficient future task learning. We also train a compact student model with samples drawn through the generative mechanisms of the GMM, aiming to accumulate cross-domain representations over time. By employing the student model, we can significantly reduce the number of parameters and make quick inferences during the testing phase.

---

PaperID: 493,   

Authors:  Yonghao Song, Yijun Wang, Huiguang He, Xiaorong Gao

Title: Recognizing Natural Images From EEG With Language-Guided Contrastive Learning

Abstract:
Electroencephalography (EEG), known for its convenient noninvasive acquisition but moderate signal-to-noise ratio, has recently gained much attention due to the potential to decode image information. However, previous works have not delivered sufficient evidence of this task, primarily limited by performance and biological plausibility. In this work, we first introduce a self-supervised framework to demonstrate the feasibility of recognizing images from EEG signals. Contrastive learning is leveraged to align the representations of EEG responses with image stimuli. Then, language descriptions of the stimuli generated by large language models (LLMs) help guide learning core semantic information. With the framework, we attain significantly above-chance results on the THINGS-EEG2 dataset, achieving a top-1 accuracy of 19.7% and a top-5 accuracy of 51.5% in challenging 200-way zero-shot tasks. Furthermore, we conduct thorough experiments to resolve the human visual responses with EEG from temporal, spatial, spectral, and semantic perspectives. These results provide evidence of feasibility and plausibility regarding EEG-based image recognition, substantiated by comparative studies with the THINGS-Magnetoencephalography (MEG) dataset. The findings offer valuable insights for neural decoding and real-world applications of brain-computer interfaces (BCIs), such as health care and robot control. The code is available at https://github.com/eeyhsong/NICE-LLM.

---

PaperID: 494,   

Authors:  Alberto Carlevaro, Teodoro Alamo, Fabrizio Dabbene, Maurizio Mongelli

Title: Probabilistic Safety Regions via Finite Families of Adjustable Classifiers

Abstract:
The supervised classification recognizes patterns in the data to separate classes of behaviors. Canonical solutions contain misclassification errors that are intrinsic to the numerical approximating nature of machine learning (ML). The data analyst may minimize the classification error on a class at the expense of increasing the error of the other classes. The error control of such a design phase is often done in a heuristic manner. In this article, it is key to develop theoretical foundations capable of providing probabilistic certifications to the obtained classifiers. In this perspective, we introduce the concept of probabilistic safety region to describe a subset of the input space in which the number of misclassified instances is probabilistically controlled. The notion of adjustable classifiers, a special class of classifiers that share the property of being controllable by a scalar parameter, is then exploited to link the tuning of ML with error control. Several tests and examples corroborate the approach. They are provided through the synthetic data in order to highlight all the steps involved, as well as notable benchmark datasets and a smart mobility application.

---

PaperID: 495,   

Authors:  Lei Zhao, Wing W. Y. Ng, Jianjun Zhang, Xiguang Wu

Title: An Innovative Multisource Teacher Collaborative Framework for Self-Knowledge Distillation

Abstract:
Self-knowledge distillation, abbreviated as SKD, exhibits greater computational efficiency than traditional knowledge distillation (KD) because it learns from its own predictions rather than from a pretrained teacher. Existing SKD methods diversify knowledge through auxiliary branches, data augmentation, historical models, and label smoothing. However, previous methods primarily extract knowledge from a single-source teacher, overlooking the diversity and complementarity of various types of teacher knowledge in model learning, thereby limiting performance improvements. In response to this challenge, we propose a pioneering paradigm termed multisource teacher collaboration for self-knowledge distillation (MSTCS-KD), which integrates knowledge from diverse types of teachers to complementarily enhance the model’s learning capability. We start by adding lightweight auxiliary branches with different structures in the shallow layers to build the student network, while also incorporating a teacher-guided attention mechanism to support adaptive learning. Then, we perform collaborative distillation by combining “heterogeneous knowledge” from the primary network’s deepest layers with “homogeneous knowledge” from the student’s outputs on augmented samples. This complementary distillation approach improves the model’s ability to learn features, generalize, and enhance trainability. Extensive experiments demonstrate that our method outperforms other state-of-the-art SKD methods across various network architectures and datasets.

---

PaperID: 496,   

Authors:  Zhiwen Xiao, Huanlai Xing, Rong Qu, Hui Li, Xinzhou Cheng, Lexi Xu, Li Feng, Qian Wan

Title: Heterogeneous Mutual Knowledge Distillation for Wearable Human Activity Recognition

Abstract:
Recently, numerous deep learning algorithms have addressed wearable human activity recognition (HAR), but they often struggle with efficient knowledge transfer to lightweight models for mobile devices. Knowledge distillation (KD) is a popular technique for model compression, transferring knowledge from a complex teacher to a compact student. Most existing KD algorithms consider homogeneous architectures, hindering performance in heterogeneous setups. This is an under-explored area in wearable HAR. To bridge this gap, we propose a heterogeneous mutual KD (HMKD) framework for wearable HAR. HMKD establishes mutual learning within the intermediate and output layers of both teacher and student models. To accommodate substantial structural differences between teacher and student, we employ a weighted ensemble feature approach to merge the features from their intermediate layers, enhancing knowledge exchange within them. Experimental results on the HAPT, WISDM, and UCI_HAR datasets show HMKD outperforms ten state-of-the-art KD algorithms in terms of classification accuracy. Notably, with ResNetLSTMaN as the teacher and MLP as the student, HMKD increases by 9.19% in MLP’s F_1 score on the HAPT dataset.

---

PaperID: 497,   

Authors:  Yuanchen Bei, Sheng Zhou, Jinke Shi, Yao Ma, Haishuai Wang, Jiajun Bu

Title: Guarding Graph Neural Networks for Unsupervised Graph Anomaly Detection

Abstract:
Unsupervised graph anomaly detection aims at identifying rare patterns that deviate from the majority in a graph without the aid of labels, which is important for a variety of real-world applications. Recent advances have utilized graph neural networks (GNNs) to learn effective node representations by aggregating information from neighborhoods. This is motivated by the hypothesis that nodes in the graph tend to exhibit consistent behaviors with their neighborhoods. However, such consistency can be disrupted by graph anomalies in multiple ways. Most existing methods directly employ GNNs to learn representations, disregarding the negative impact of graph anomalies on GNNs, resulting in suboptimal node representations and anomaly detection performance. While a few recent approaches have redesigned GNNs for graph anomaly detection under semi-supervised label guidance, how to address the adverse effects of graph anomalies on GNNs in unsupervised scenarios and learn effective representations for anomaly detection are still underexplored. To bridge this gap, in this article, we propose a simple, yet effective framework for guarding GNNs for unsupervised graph anomaly detection (G3AD). Specifically, G3AD first introduces two auxiliary networks along with correlation constraints to guard the GNNs against inconsistent information encoding. Furthermore, G3AD introduces an adaptive caching (AC) module to guard the GNNs from directly reconstructing the observed graph data that contains anomalies. Extensive experiments demonstrate that our G3AD can outperform 20 state-of-the-art methods on both synthetic and real-world graph anomaly datasets, with flexible generalization ability in different GNN backbones.

---

PaperID: 498,   

Authors:  Xiyuan Jin, Jing Wang, Xiaoyu Ou, Lei Liu, Youfang Lin

Title: Time-Series Contrastive Learning Against False Negatives and Class Imbalance

Abstract:
Self-supervised contrastive learning (SCL) has driven significant advancements in time-series representation learning. While recent studies built upon the information noise contrastive estimation (InfoNCE) loss framework focus on constructing appropriate positives and negatives, we theoretically analyze and identify two overlooked issues inherent in this approach: false negatives and class imbalance. To address these challenges, we propose a simple yet effective modification based on the SimCLR framework, integrating a multi-instance discrimination task to mitigate false negatives. Additionally, we introduce a graph-based interactive projection head and semantic consistency regularization, which enhances minority-class representations with minimal annotation cost. Extensive experiments on six real-world time-series datasets demonstrate that our approach consistently outperforms state-of-the-art methods, achieving up to 3.96% higher accuracy and 10.73% improvement in F1 -score, particularly benefiting imbalanced data scenarios.

---

PaperID: 499,   

Authors:  Ke Wang, Binghong Liu, Pandi Liu, Yungao Shi, Ping Guo, Yafei Li, Mingliang Xu

Title: Bi-PIL: Bidirectional Gradient-Free Learning Scheme for Multilayer Neural Networks

Abstract:
Training deep neural networks typically relies on gradient descent learning schemes, which is usually time-consuming, and the design of complex network architectures is often intractable. In this article, we explore the building of multilayer neural networks based on an efficient gradient-free learning scheme offering a potential solution to the architectural design. The proposed learning scheme encompasses both forward and backward training (BT) processes. In the forward process, the pseudoinverse learning (PIL) algorithm is employed to train a multilayer neural network, in which the network is dynamically constructed leveraging a layer-by-layer greedy strategy, enabling the automatic determination of the architecture across different hierarchies in a data-driven manner. The network architecture and connection weights determined in the forward training (FT) process are shared with the backward process which also conducts gradient-free learning to update the connection weights. After the bidirectional learning, a neural network comprising two twin subnetworks is obtained, and the fused features of subnetworks are used as inputs for downstream tasks. Comprehensive experiments and detailed analyses demonstrate the effectiveness and superiority of the proposed learning scheme.

---

PaperID: 500,   

Authors:  Zhengyi Zhong, Weidong Bao, Ji Wang, Jianguo Chen, Lingjuan Lyu, Wei Yang Bryan Lim

Title: SacFL: Self-Adaptive Federated Continual Learning for Resource-Constrained End Devices

Abstract:
The proliferation of end devices has led to a distributed computing paradigm, wherein on-device machine learning models continuously process diverse data generated by these devices. The dynamic nature of this data, characterized by continuous changes or data drift, poses significant challenges for on-device models. To address this issue, continual learning (CL) is proposed, enabling machine learning models to incrementally update their knowledge and mitigate catastrophic forgetting. However, the traditional centralized approach to CL is unsuitable for end devices due to privacy and data volume concerns. In this context, federated CL (FCL) emerges as a promising solution, preserving user data locally while enhancing models through collaborative updates. Aiming at the challenges of limited storage resources for CL, poor autonomy in task shift detection, and difficulty in coping with new adversarial tasks in the FCL scenario, we propose a novel FCL framework named self-adaptive federated CL (SacFL). \rm SacFL employs an encoder–decoder architecture to separate task-robust and task-sensitive components, significantly reducing storage demands by retaining lightweight task-sensitive components for resource-constrained end devices. Moreover, \rm SacFL leverages contrastive learning to introduce an autonomous data shift detection mechanism, enabling it to discern whether a new task has emerged and whether it is a benign task. This capability ultimately allows the device to autonomously trigger CL or attack defense strategy without additional information, which is more practical for end devices. Comprehensive experiments conducted on multiple text and image datasets, such as Cifar100 and THUCNews, have validated the effectiveness of \rm SacFL in both class-incremental and domain-incremental scenarios. Furthermore, a demo system has been developed to verify its practicality.

---

PaperID: 501,   

Authors:  Donghua Wang, Wen Yao, Tingsong Jiang, Xiaohu Zheng, Junqi Wu

Title: Improving the Transferability of Adversarial Examples by Feature Augmentation

Abstract:
Adversarial transferability is a significant property of adversarial examples, which renders the adversarial example capable of attacking unknown models. However, the models with different architectures on the same task would concentrate on different information, which weakens adversarial transferability. To enhance the adversarial transferability, input transformation-based attacks perform random transformation over input to find a better result that can resist such transformations, but these methods ignore the model discrepancy; ensemble attacks fuse multiple models to shrink the search space to ensure that the found adversarial examples work on these models, but ensemble attacks are resource-intensive. In this article, we propose a simple but effective feature augmentation attack (FAUG) method to improve adversarial transferability. We dynamically add random noise to intermediate features of the target model during the generation of adversarial examples, thereby avoiding overfitting the target model. Specifically, we first explore the noise tolerance of the model and disclose the discrepancy under different layers and noise strengths. Then, based on that analysis, we devise a dynamic random noise generation method, which determines noise strength according to the produced features in the mini-batch. Finally, we exploit the gradient-based attack algorithm on featureaugmented models, resulting in better adversarial transferability without introducing extra computation costs. Extensive experiments conducted on the ImageNet dataset across CNN and Transformer models corroborate the efficacy of our method, e.g., we achieve improvement of +30.67% and +5.57% on input transformation-based attacks and combination methods, respectively.

---

PaperID: 502,   

Authors:  Yakun Li, Shuhua Gao, Yiming Gao, Jian-Liang Wu, Jun-e Feng, Cheng Xiang

Title: Robust Controllability of Boolean Control Networks via Dynamic Programming

Abstract:
This article presents a novel dynamic programming approach to determine the robust controllability of Boolean control networks (BCNs) subject to stochastic disturbances. By applying Bellman’s optimality principle, we derive the recurrence relation for computing the optimal time matrix, a crucial concept characterizing robust reachability between two arbitrary states. We develop a finite-termination dynamic programming algorithm to calculate the optimal time matrix exactly and efficiently, with a rigorously certified iteration count. Sufficient and necessary conditions for robust controllability are then established based on the optimal time matrix. Furthermore, for any pair of reachable states, we construct time-optimal state feedback control laws to steer the system from the initial state to the target state, regardless of disturbances. Finally, extensive numerical experiments with biological networks validate the effectiveness of the proposed approach, showing significant improvements in computational efficiency. Additionally, we introduce a Q-learning-based algorithm and compare its performance, highlighting the advantages of our dynamic programming approach in terms of both efficiency and solution quality.

---

PaperID: 503,   

Authors:  Shiron Thalagala, Pak-Kin Wong, Xiaozheng Wang, Tianang Sun

Title: Broad Critic Deep Actor Reinforcement Learning for Continuous Control

Abstract:
In the domain of continuous control, deep reinforcement learning (DRL) demonstrates promising results. However, the dependence of DRL on deep neural networks (DNNs) results in the demand for extensive data and increased computational cost. To address this issue, a novel hybrid actor-critic reinforcement learning (RL) framework is introduced. The proposed framework integrates the broad learning system (BLS) with DNN, aiming to merge the strengths of both distinct architectural paradigms. Specifically, the critic network employs BLS for rapid value estimation via ridge regression, while the actor network retains the DNN structure to optimize policy gradients. This hybrid design is generalizable and can enhance existing actor-critic algorithms. To demonstrate its versatility, the proposed framework is integrated into three widely used actor-critic algorithms—deep deterministic policy gradient (DDPG), soft actor-critic (SAC), and twin delayed DDPG (TD3), resulting in BLS-augmented variants. The experimental results reveal that all BLS-enhanced versions surpass their original counterparts in terms of training efficiency and accuracy. These improvements highlight the suitability of the proposed framework for real-time control scenarios, where computational efficiency and rapid adaptation are critical.

---

PaperID: 504,   

Authors:  Shudong Huang, Wentao Feng, Chenwei Tang, Zhenan He, Caiyang Yu, Jiancheng Lv

Title: Partial Differential Equations Meet Deep Neural Networks: A Survey

Abstract:
Many problems in science and engineering can be mathematically modeled using partial differential equations (PDEs), which are essential for fields like computational fluid dynamics (CFD), molecular dynamics, and dynamical systems. Although traditional numerical methods like the finite difference/element method are widely used, their computational inefficiency, due to the large number of iterations required, has long been a challenge. Recently, deep learning (DL) has emerged as a promising alternative for solving PDEs, offering new paradigms beyond conventional methods. Despite the growing interest in techniques like physics-informed neural networks (PINNs), a systematic review of the diverse neural network (NN) approaches for PDEs is still missing. This survey fills that gap by categorizing and reviewing the current progress of deep NNs (DNNs) for PDEs. Unlike previous reviews focused on specific methods like PINNs, we offer a broader taxonomy and analyze applications across scientific, engineering, and medical fields. We also provide a historical overview, key challenges, and future trends, aiming to serve both researchers and practitioners with insights into how DNNs can be effectively applied to solve PDEs.

---

PaperID: 505,   

Authors:  Xinzhu Liu, Di Guo, Huaping Liu

Title: Semantics-Aware Hierarchical Decision Framework for Embodied Visual Room Rearrangement

Abstract:
In embodied visual room rearrangement, the agent needs to recover the scene state to the goal state through interacting with the environment based on the egocentric visual observations after the locations and states of some objects are changed. It has important application potential in the field of robotics. This task is challenging in visual perception, scene understanding, and action execution. Existing methods do not take full advantage of the semantic information and spatial relationship of objects in the scene perception and understanding process. To tackle the challenges and shortcomings of the current methods, we build a hierarchical decision framework based on the pretrained semantic scene representation and transformer-based scene memory to solve this task. The results in the unseen scenes demonstrate the effectiveness of the proposed model compared with other methods.

---

PaperID: 506,   

Authors:  Ziheng Jiao, Hongyuan Zhang, Xuelong Li

Title: Deep Graph Multi-View Representation Learning With Self-Augmented View Fusion

Abstract:
Some current researchers attempt to extend the graph neural network (GNN) on multi-view representation learning and learn the latent structure information among the data. Generally, they concatenate the features of each view and employ a single GNN to extract the representations of this concatenated feature. It causes that the within-view information may not be learned and the pivotal view will not be strengthened during the concatenation. Although some GNN models introduce the Siamese structure to extract the within-view information, the learned representation may not be informative since the Siamese GNNs share the same parameters. To overcome these issues, we propose a novel deep graph auto-encoder for multi-view representation learning. Among them, a self-augmented view-weight technique is theoretically devised for cross-view fusion, which can highlight the pivotal views and maintain the rest views. Then, GNNs of different views can learn the informative representation without sharing parameters. Furthermore, by fitting the fusion distribution with a neural layer, the model unifies these two individual procedures and achieve to extract the fusion representation end-to-end. Compared with numerous recently proposed methods, extensive experiments on clustering and recognition tasks demonstrate our superior performance.

---

PaperID: 507,   

Authors:  Yizhang Wang, Wei Pang, Di Wang, Witold Pedrycz

Title: One-Shot Secure Federated K-Means Clustering Based on Density Cores

Abstract:
Federated clustering (FC) performs well in independent and identically distributed (IID) scenarios, but it does not perform well in non-IID scenarios. In addition, existing methods lack proof of strict privacy protection. To address the above issues, we propose a new secure federated k-means clustering framework to achieve better clustering results under privacy requirements. Specifically, for the clients, we use cluster centers (representative points) generated by k-means to represent the corresponding clusters. These representative points can effectively preserve the structure of the local data and they are encrypted by differential privacy. For the server, we propose two methods to reprocess the uploaded encrypted representative points to obtain better final cluster centers, one uses k-means, and the other considers the improved density peaks (density cores) as final centers and then sends them back to the clients. Finally, each client assigns local data to their nearest centers. Experimental results show that the proposed methods perform better than several centralized (nonfederated) classical clustering algorithms [k-means, density-based spatial clustering of applications with noise (DBSCAN), and density peak clustering (DPC)] and state-of-the-art (SOTA) centralized clustering algorithms in most cases. In particular, the proposed algorithms perform better than the SOTA FC framework k-FED (ICML2021) and MUFC (ICLR2023).

---

PaperID: 508,   

Authors:  Yuan-Hung Kuan, Vignesh Narayanan, Jr-Shin Li

Title: Iterative Reservoir Computing Networks for Reconstructing Irregular Time Series

Abstract:
Time series data with missing entries are ubiquitous in a broad spectrum of practical and clinical applications, from climatology and cell biology to personalized medicine. This undesired structure arising either due to undesired artifacts (e.g., noise) or by design (e.g., asynchronous or aperiodic sampling in distributed sensors) results in irregularity in the temporal dimension and forms a bottleneck in data mining. Although extensive data science approaches have been proposed to address learning problems involving irregular data, the emphasis was largely placed on filling in the missing entries via interpolation and binning, or the methods were tailored to specific data analytic tasks. In this article, we develop a reservoir computing (RC)-based iterative learning method for recovering missing data in irregular time series generated by dynamical systems and networks. In particular, we formulate this learning task as a fixed-point iterative learning problem and develop a training procedure using an RC network (RCN). We find that when the irregular time series has “sufficient” samples to train an RCN within a tolerant training error then the missing samples in the time series can be recovered systematically. We also derive sufficient conditions with respect to the choices of the reservoir parameters that guarantee the convergence of the iterative procedure. We present several numerical experiments to demonstrate the efficacy of the developed iterative RCN approach. Specifically, we illustrate the capability of our approach to recover missing data in irregular time series generated by chaotic Rössler and Kuramoto-Sivashinsky (KS) systems. Finally, we also report the results of incorporating our approach in an irregular medical data classification task.

---

PaperID: 509,   

Authors:  Yachao Zhang, Yuxiang Lan, Yuan Xie, Cuihua Li, Yanyun Qu

Title: Cross-Cloud Consistency for Weakly Supervised Point Cloud Semantic Segmentation

Abstract:
Weakly supervised point cloud semantic segmentation is an increasingly active topic, because fully supervised learning acquires well-labeled point clouds and entails high costs. The existing weakly supervised methods either need meticulously designed data augmentation for self-supervised learning or ignore the negative effects of learning on pseudolabel noises. In this article, by designing different granularity of cross-cloud structures, we propose a cross-cloud consistency method for weakly supervised point cloud semantic segmentation which forms the expectation-maximum (EM) framework. Benefiting from the cross-cloud constraints, our method allows effective learning alternatively between refining pseudolabels and updating network parameters. Specifically, in E-step, we propose a pseudolabel selecting (PLS) strategy based on cross subcloud consistency, improving the credibility of selected pseudolabels explicitly. In M-step, a cross-scene contrastive regularization enforces cross-scene prototypes with the same label in different scenes to be more similar, while keeping prototypes with different labels to be a clear margin, reducing the noise fitting. Finally, we give some insight into the optimization of our method in the EM theoretical way. The proposed method is evaluated on three challenging datasets, where experimental results demonstrate that our method significantly outperforms state-of-the-art weakly supervised competitors. Our code is available online: https://github.com/Yachao-Zhang/Cross-Cloud-Consistency.

---

PaperID: 510,   

Authors:  Heng Xu, Tianqing Zhu, Lefeng Zhang, Wanlei Zhou, Wei Zhao

Title: Toward Efficient Target-Level Machine Unlearning Based on Essential Graph

Abstract:
Machine unlearning is an emerging technology that has come to attract widespread attention. A number of factors, including regulations and laws, privacy, and usability concerns, have resulted in this need to allow a trained model to forget some of its training data. Existing studies of machine unlearning mainly focus on unlearning requests that forget a cluster of instances or all instances from one class. While these approaches are effective in removing instances, they do not scale to scenarios where partial targets within an instance need to be forgotten. For example, one would like to only unlearn a person from all instances that simultaneously contain the person and other targets. Directly migrating instance-level unlearning to target-level unlearning will reduce the performance of the model after the unlearning process, or fail to erase information completely. To address these concerns, we have proposed a more effective and efficient unlearning scheme that focuses on removing partial targets from the model, which we name “target unlearning.” Specifically, we first construct an essential graph data structure to describe the relationships between all important parameters that are selected based on the model explanation method. After that, we simultaneously filter parameters that are also important for the remaining targets and use the pruning-based unlearning method, which is a simple but effective solution to remove information about the target that needs to be forgotten. Experiments with different training models on various datasets demonstrate the effectiveness of the proposed approach.

---

PaperID: 511,   

Authors:  Liyuan Liu, Yaohui Chen, Weifu Li, Yingjie Wang, Bin Gu, Feng Zheng, Hong Chen

Title: Generalization Bounds of Deep Neural Networks With τ-Mixing Samples

Abstract:
Deep neural networks (DNNs) have shown an astonishing ability to unlock the complicated relationships among the inputs and their responses. Along with empirical successes, some approximation analysis of DNNs has also been provided to understand their generalization performance. However, the existing analysis depends heavily on the independently identically distribution (i.i.d.) assumption of observations, which may be too ideal and often violated in real-world applications. To relax the i.i.d. assumption, this article develops the covering number-based concentration estimation to establish generalization bounds of DNNs with \tau -mixing samples, where the dependency between samples is much general including \alpha -mixing process as a special case. By assigning a specific parameter value to the \tau -mixing process, our results are consistent with the existing convergence analysis under the i.i.d. case. Experiments on simulated data validate the theoretical findings.

---

PaperID: 512,   

Authors:  Ruiyuan Kang, Panos Liatsis

Title: Physics-Driven Anomaly Detection and Correction for Spectroscopic Parameter Estimation

Abstract:
Machine learning (ML) techniques are popular in many parameter estimation tasks; however, they face challenges in the real-world deployment due to the lack of robustness to errors. ML estimators are not able to ascertain performance in the presence of noise, variations in the data distribution, and anomalies in the test samples. This work proposes a novel framework, surrogate-based physical error correction (SPEC), which addresses the unmet need for measurement reliability estimation and self-correction under process data uncertainty, by bringing together physics- and network-based optimization. The workings of SPEC are demonstrated using the paradigm of gas parameter estimation in the laser absorption spectroscopy (LAS). It operates in two modes, estimation and correction. During estimation, SPEC provides an initial state estimate, with estimation reliability being assessed by the physics-driven anomaly detection (PAD) module, which uses a hybrid error, combining a nondifferentiable reconstruction error, calculated through an ensemble network, and a differentiable feasibility error. When an estimate is flagged as unreliable, the correction mode is enabled. This network-based optimization algorithm delivers efficient and robust state correction by using a greedy ensemble search. SPEC’s performance is evaluated in a variety of experiments including outside-of-distribution and noisy data. Moreover, it offers reconfigurability through PAD configuration modification, eliminating the need for ML estimator retraining.

---

PaperID: 513,   

Authors:  Huiwei Lin, Shanshan Feng, Baoquan Zhang, Xutao Li, Yunming Ye

Title: HPCR: Holistic Proxy-Based Contrastive Replay for Online Continual Learning

Abstract:
Online continual learning (OCL), aimed at developing a neural network that continuously learns new data from a single pass over an online data stream, generally suffers from catastrophic forgetting (CF). Existing replay-based methods alleviate forgetting by replaying partial old data in a proxy-based or contrastive-based replay manner, each with its own shortcomings. Our previous work proposes a novel replay-based method called proxy-based contrastive replay (PCR), which handles the shortcomings by achieving complementary advantages of both replay manners. In this work, we further conduct gradient and limitation analysis of PCR. The analysis results show that PCR still can be further improved in feature extraction, generalization, and anti-forgetting capabilities of the model. Hence, we developed a more advanced method named holistic PCR (HPCR). HPCR consists of three components, each tackling one of the limitations of PCR. The contrastive component conditionally incorporates anchor-to-sample pairs to PCR, improving the feature extraction ability. The second is a temperature component that decouples the temperature coefficient into two parts based on their gradient impacts and sets different values for them to enhance the generalization ability. The third is a distillation component that constrains the learning process with additional loss terms to improve the anti-forgetting ability. Experiments on four datasets consistently demonstrate the superiority of HPCR over various state-of-the-art methods.

---

PaperID: 514,   

Authors:  Mingjin Zhang, Ke Yue, Jie Guo, Qiming Zhang, Jing Zhang, Xinbo Gao

Title: Computational Fluid Dynamic Network for Infrared Small Target Detection

Abstract:
Infrared small target detection (IRSTD) aims to identify and locate small targets amidst background noise. It is highly valuable in various practical application domains, such as maritime rescue and early warning systems deployed in challenging conditions such as harsh weather, low illumination, and long imaging distances. Different from existing works that either adopt well-designed backbone networks or devise specific modules to improve them from different aspects, in this article, we formulate the learning process of IRSTD from a novel perspective, i.e., the mechanism of pixel movement. Considering that the movement of pixels passing through the layers of the network for IRSTD can be analogized to the flow of particles in a fluid dynamic system, we propose a computational fluid dynamic network (CFD-Net) derived from computational fluid dynamics. Technically, we leverage the superiority of the unilateral difference equation with third-order accuracy and devise a unilateral differential residual structure as the backbone of CFD-Net. This design ensures that the pixel stream only flows in the forward direction. In addition, a switch-controlled multidirectional treatment tank (SMTT) is introduced to CFD-Net to dynamically guide the pixel stream to the appropriate path for different targets with varying shapes and orientations, facilitating learning robust target representation and improving detection performance. The proposed CFD-Net is evaluated on the IRSTD-1k and SIRST datasets and is found to outperform existing state-of-the-art (SOTA) methods.

---

PaperID: 515,   

Authors:  Zhenyu Chen, Lu Zhang, Ping Hu, Huchuan Lu, You He

Title: MaskTrack: Auto-Labeling and Stable Tracking for Video Object Segmentation

Abstract:
Video object segmentation (VOS) has witnessed notable progress due to the establishment of video training datasets and the introduction of diverse, innovative network architectures. However, video mask annotation is a highly intricate and labor-intensive task, as meticulous frame-by-frame comparisons are needed to ascertain the positions and identities of targets in the subsequent frames. Current VOS benchmarks often annotate only a few instances in each video to save costs, which, however, hinders the model’s understanding of the complete context of the video scenes. To simplify video annotation and achieve efficient dense labeling, we introduce a zero-shot auto-labeling strategy based on the segment anything model (SAM), enabling it to densely annotate video instances without access to any manual annotations. Moreover, although existing VOS methods demonstrate improving performance, segmenting long-term and complex video scenes remains challenging due to the difficulties in stably discriminating and tracking instance identities. To this end, we further introduce a new framework, MaskTrack, which excels in long-term VOS and also exhibits significant performance advantages in distinguishing instances in complex videos with densely packed similar objects. We conduct extensive experiments to demonstrate the effectiveness of the proposed method and show that without introducing image datasets for pretraining, it achieves excellent performance on both short-term (86.2% in YouTube-VOS val) and long-term (68.2% in LVOS val) VOS benchmarks. Our method also surprisingly demonstrates strong generalization ability and performs well in visual object tracking (VOT) (65.6% in VOTS2023) and referring VOS (RVOS) (65.2% in Ref YouTube VOS) challenges.

---

PaperID: 516,   

Authors:  Xiaosheng He, Fan Yang, Fayao Liu, Guosheng Lin

Title: Few-Shot Image Generation via Style Adaptation and Content Preservation

Abstract:
Training a generative model with limited data (e.g., 10) is a very challenging task. Many works propose to fine-tune a pretrained GAN model. However, this can easily result in overfitting. In other words, they manage to adapt the style but fail to preserve the content, where style denotes the specific properties that define a domain while content denotes the domain-irrelevant information that represents diversity. Recent works try to maintain a predefined correspondence to preserve the content, however, the diversity is still not enough and it may affect style adaptation. In this work, we propose a paired image reconstruction approach for content preservation. We propose to introduce an image translation module to GAN transferring, where the module teaches the generator to separate style and content, and the generator provides training data to the translation module in return. Qualitative and quantitative experiments show that our method consistently surpasses the state-of-the-art methods in a few-shot setting.

---

PaperID: 517,   

Authors:  Ya Zhang, Xiaoling Luo, Qian Sun, Yuchen Wang, Hong Qu, Zhang Yi

Title: Retina-Inspired Lightweight Spiking Convolutional Neural Network for Single-Image Dehazing

Abstract:
Suspended particles in hazy medium absorb and scatter light, severely degrading imaging quality. Numerous single-image dehazing methods have been proposed to reconstruct clear images from hazy ones. However, most of them focus on increasing depth and width to improve dehazing performance, which incurs high computation and energy costs. To address this issue, we propose a lightweight spiking convolutional neural network (CNN) referred to as retina-inspired spiking CNN (RI-SCNN) for the reconstruction of hazy images. Unlike conventional dehazing techniques, first, our proposed network simulates the hierarchical structure and cellular function of the retina and devises five network modules to efficiently encode and extract image features through ON and OFF roads. Furthermore, the linear reconstruction mechanism is introduced to integrate the outputs from different roads, adaptively preserving regions with optimal details and constructing a comprehensive visual representation. Finally, by the transformed atmospheric scattering formula, our network can generate the dehazy image. Incorporating the microscale spiking mechanism of the brain, the entire network leverages discrete binary spike trains for information encoding and transmission, directly trained by spiking surrogate gradient learning on integrate-and-fire (IF) neurons. Experimental results demonstrate the superiority of the proposed RI-SCNN in terms of quantitative dehazing performance, qualitative visual effect, energy efficiency, and run speed. Considering its lightweight architecture with ultralow computation and energy costs, the network is encouraged to be deployed in the visual sensor hardware to improve overall performance.

---

PaperID: 518,   

Authors:  Youngjoon Yu, Yeonju Kim, Yong Man Ro

Title: Advancing Causal Intervention in Image Captioning With Causal Prompt

Abstract:
This article introduces a novel approach, called causal prompting network (CPNet), to enhance the causal intervention in the context of image captioning. By leveraging visual prompt engineering in the feature space, this method aims to achieve superior performance in causal intervention tasks. Since CPNet is highly flexible and adaptable, it can be incorporated into any existing causal intervention-based image captioning framework. Specifically, two types of visual prompts—causal region of interest (RoI) prompt (CRP) and causal matching prompt (CMP)—are employed to refine the feature representations effectively. CRP is utilized on the RoI feature of the object feature to enhance RoI features with deconfounded causal features. Meanwhile, CMP is used to strengthen the contextual representation of confounders linked to image captioning tasks. To evaluate the proposed CPNet’s effectiveness, an extensive range of experiments are conducted on the popular microsoft common objects in context dataset (MS-COCO) and Flickr30k datasets, and the results are validated using the Karpathy split. Experimental results demonstrate that the proposed CPNet surpasses the performance of other state-of-the-art (SOTA) image captioning methods.

---

PaperID: 519,   

Authors:  Christopher Adnel, Islem Rekik

Title: FALCON: Feature-Label Constrained Graph Net Collapse for Memory-Efficient GNNs

Abstract:
Graph neural network (GNN) ushered in a new era of machine learning with interconnected datasets. While traditional neural networks can only be trained on independent samples, GNN allows for the inclusion of intersample interactions in the training process. This gain, however, incurs additional memory cost, rendering most GNNs unscalable for real-world applications involving vast and complicated networks with tens of millions of nodes (e.g., social circles, web graphs, and brain graphs). This means that storing the graph in the main memory can be difficult, let alone training the GNN model with significantly less GPU memory. While much of the recent literature has focused on either mini-batching GNN methods or quantization, graph reduction methods remain largely scarce. Furthermore, present graph reduction approaches have several drawbacks. First, most graph reduction focuses only on the inference stage (e.g., condensation, pruning, and distillation) and requires full graph GNN training, which does not reduce training memory footprint. Second, many methods focus solely on the graph’s structural aspect, ignoring the initial population feature-label distribution, resulting in a skewed postreduction label distribution. Here, we propose a feature-label constrained graph net collapse (FALCON) to address these limitations. Our three core contributions lie in: 1) designing FALCON, a topology-aware graph reduction technique that preserves feature-label distribution by introducing a K-means clustering with a novel dimension-normalized Euclidean distance; 2) implementation of FALCON with other state-of-the-art (SOTA) memory reduction methods (i.e., mini-batched GNN and quantization) for further memory reduction; and 3) extensive benchmarking and ablation studies against SOTA methods to evaluate FALCON memory reduction. Our comprehensive results show that FALCON can significantly collapse various public datasets (e.g., PPI and Flickr to as low as 34% of the total nodes) while keeping equal prediction quality across GNN models. Our FALCON code is available at https://github.com/basiralab/FALCON.

---

PaperID: 520,   

Authors:  Meng Zhang, Xing He, Tingwen Huang

Title: Matrix Neurodynamic Approaches for Rank Minimization: Finite/Fixed-Time Convergence Technique

Abstract:
This article presents two innovative matrix neurodynamic approaches (MNAs) designed to tackle the rank minimization problem. First, by introducing the matrix norm-normalized sign function, two variants of MNA are developed: finite-time converging MNA (FINt-MNA) and fixed-time converging MNA (FIXt-MNA). Then, the proposed approaches are shown to guarantee the existence and uniqueness of solutions, and based on Lyapunov analysis, it is demonstrated that the proposed approaches converge to the optimal solution within FINt and FIXt. In addition, upper bounds on the settling time are determined using finite-time and fixed-time lemmas, with subsequent analysis examining the influence of tunable parameters on these bounds for the two approaches through the control variable method. Finally, numerical examples and an image completion experiment confirm the effectiveness and superiority of the proposed approaches compared with the existing MNA and two classical approaches.

---

PaperID: 521,   

Authors:  Xiaoli Tang, Han Yu

Title: A Cost-Aware Utility-Maximizing Bidding Strategy for Auction-Based Federated Learning

Abstract:
Auction-based federated learning (AFL) has emerged as an efficient and fair approach to incentivize data owners (DOs) to contribute to federated model training, garnering extensive interest. However, the important problem of helping data consumers (DCs) bid for DOs in competitive AFL settings remains open. Existing work simply treats that the actual cost paid by a winning DC (i.e., the bid cost) is equal to the bid price offered by that DC itself. However, this assumption is inconsistent with the widely adopted generalized second-price (GSP) auction mechanism used in AFL, including in these existing works. Under a GSP auction, the winning DC does not pay its own proposed bid price. Instead, the bid cost for the winner is determined by the second-highest bid price among all participating DCs. To address this limitation, we propose a first-of-its-kind federated cost-aware bidding strategy (FedCA-Bidder) to help DCs maximize their utility under GSP auction-based federated learning (FL). It enables DCs to efficiently bid for DOs in competitive AFL markets, maximizing their utility and improving the resulting FL model accuracy. We first formulate the optimal bidding function under the GSP auction setting, and then demonstrate that it depends on utility estimation and market price modeling, which are interrelated. Based on this analysis, FedCA-Bidder jointly optimizes in a novel end-to-end framework, and then executes the proposed return on investment (ROI)-based method to determine the optimal bid price for each piece of the data resource. Through extensive experiments on six commonly adopted benchmark datasets, we show that FedCA-Bidder outperforms eight state-of-the-art methods, beating the best baseline by 4.39%, 4.56%, 1.33%, and 5.43% on average in terms of the total amount of data obtained, number of data samples per unit cost, total utility, and FL model accuracy, respectively.

---

PaperID: 522,   

Authors:  Pavel Sinha, Ioannis N. Psaromiligkos, Zeljko Zilic

Title: A Structurally Regularized CNN Architecture via Adaptive Subband Decomposition

Abstract:
We propose a generalized convolutional neural network (CNN) architecture that first decomposes the input signal into subbands by an adaptive filter bank structure, and then uses convolutional layers to extract features from each subband independently. Fully connected (FC) layers finally combine the extracted features to perform classification. The proposed architecture restrains each of the subband CNNs from learning using the entire input signal spectrum, resulting in structural regularization. Our proposed CNN architecture is fully compatible with the end-to-end learning mechanism of typical CNN architectures and learns the subband decomposition from the input dataset. We show that the proposed CNN architecture has attractive properties, such as robustness to input and weight-and-bias quantization noise, compared to regular full-band CNN architectures. Importantly, the proposed architecture significantly reduces computational costs, while maintaining state-of-the-art classification accuracy. Experiments on image classification tasks using the MNIST, CIFAR-10/100, Caltech-101, and ImageNet-2012 datasets show that the proposed architecture allows accuracy surpassing state-of-the-art results. On the ImageNet-2012 dataset, we achieved top-5 and top-1 validation set accuracy of 86.91% and 69.73%, respectively. Notably, the proposed architecture offers over 90% reduction in computation cost in the inference path and approximately 75% reduction in back-propagation (per iteration) with just a single-layer subband decomposition. With a two-layer subband decomposition, the computational gains are even more significant with comparable accuracy results to the single-layer decomposition.

---

PaperID: 523,   

Authors:  Songlin Fan, Wei Gao, Ge Li

Title: Point-MPP: Point Cloud Self-Supervised Learning From Masked Position Prediction

Abstract:
Masked autoencoding has gained momentum for improving fine-tuning performance in many downstream tasks. However, it tends to focus on low-level reconstruction details, lacking high-level semantics and resulting in weak transfer capability. This article presents a novel jigsaw puzzle solver inspired by the idea that predicting the positions of disordered point cloud patches provides more semantic information, similar to how children learn by solving jigsaw puzzles. Our method adopts the mask-then-predict paradigm, erasing the positions of selected point patches rather than their contents. We first partition input point clouds into irregular patches and randomly erase the positions of some patches. Then, a Transformer-based model is used to learn high-level semantic features and regress the positions of the masked patches. This approach forces the model to focus on learning transfer-robust semantics while paying less attention to low-level details. To tie the predictions within the encoding space, we further introduce a consistency constraint on their latent representations to encourage the encoded features to contain more semantic cues. We demonstrate that a standard Transformer backbone with our pretraining scheme can capture discriminative point cloud semantic information. Furthermore, extensive experiments indicate that our method outperforms the previous best competitor across six popular downstream vision tasks, achieving new state-of-the-art performance. Codes will be available at https://git.openi.org.cn/OpenPointCloud/Point-MPP.

---

PaperID: 524,   

Authors:  Yue Zhao, Maoguo Gong, Mingyang Zhang, A. K. Qin, Fenlong Jiang, Jianzhao Li

Title: SPCNet: Deep Self-Paced Curriculum Network Incorporated With Inductive Bias

Abstract:
The vulnerability to poor local optimum and the memorization of noise data limit the generalizability and reliability of massively parameterized convolutional neural networks (CNNs) on complex real-world data. Self-paced curriculum learning (SPCL), which models the easy-to-hard learning progression from human beings, is considered as a potential savior. In spite of the fact that numerous SPCL solutions have been explored, it still confronts two main challenges exactly in solving deep networks. By virtue of various designed regularizers, existing weighting schemes independent of the learning objective heavily rely on the prior knowledge. In addition, alternative optimization strategy (AOS) enables the tedious iterative training procedure, thus there is still not an efficient framework that integrates the SPCL paradigm well with networks. This article delivers a novel insight that attention mechanism allows for adaptive enhancement in the contribution of diverse instance information to the gradient propagation. Accordingly, we propose a general-purpose deep SPCL paradigm that incorporates the preferences of implicit regularizer for different samples into the network structure with inductive bias, which in turn is formalized as the self-paced curriculum network (SPCNet). Our proposal allows simultaneous online difficulty estimation, adaptive sample selection, and model updating in an end-to-end manner, which significantly facilitates the collaboration of SPCL to deep networks. Experiments on image classification and scene classification tasks demonstrate that our approach surpasses the state-of-the-art schemes and obtains superior performance.

---

PaperID: 525,   

Authors:  Zhi Jin, Chenxi Wang, Xing Luo

Title: Colorization-Inspired Customized Low-Light Image Enhancement by a Decoupled Network

Abstract:
Recently, numerous inspirational approaches have been proposed to enhance the visual quality of the images captured under poor lighting conditions. Simultaneously, in order to accommodate diverse user esthetics, researchers have explored customized operations within the enhancement process. However, most existing studies ignore the significance of the chrominance component, which often leads to unsatisfactory results in terms of color. To address this issue, we novelly decompose the low-light image enhancement (LLIE) task into the brightening and colorization subtasks and develop a decoupled network called CCNet for colorization-inspired customized enhancement. Specifically, the brightening subtask aims to restore images with normal contrast, less noise, and sharper details. While the colorization subtask utilizes the chrominance information from low-light images as color guidance to predict rich chrominance in enhanced images. Then, in the inference stage, users can adjust the color style or the saturation of color guidance to obtain customized results. Extensive experiments demonstrate that our proposed method achieves superior performance in both general and customized LLIE tasks—particularly in terms of improving chrominance components. Code is available at: https://github.com/FVL2020/CCNet.

---

PaperID: 526,   

Authors:  Yuqiang Li, Wenli Du, Xinjie Wang, Minglei Yang, Yunmeng Zhao

Title: Adaptive Robust Stochastic Configuration Networks for Near-Infrared Multivariate Analysis

Abstract:
Near-infrared (NIR) technology has gained wide acceptance in practical processes and is now the measurement of choice in many sectors. However, with increasing spectral dimensionality, it is challenging to establish a prediction model with satisfactory stability and generalization. Stochastic configuration networks (SCNs) based on supervisory learning mechanism have demonstrated significant advantages in developing nonlinear learners. However, existing incremental learning strategies make it difficult to achieve fast convergence while obtaining a suitable-scale network in high-dimensional spectra modeling. In addition, the linear or regularization weight estimation methods are vulnerable to outliers and noise in NIR analysis. To accelerate model construction and improve model performance in high-dimensional spectra analysis, the adaptive robust SCN (AR-SCN) algorithm is proposed in this work, which can perform adaptive incremental learning according to the prediction residual and robustly estimate the output weights by the global-local shrinkage strategy. Comparison results on three benchmark NIR datasets and real-world gasoline blending process verify the effectiveness of the proposed method. Compared with the state-of-the-art SCNs, the AR-SCN method can simultaneously improve the construction efficiency and robustness of SCNs.

---

PaperID: 527,   

Authors:  Baoshun Shi, Dan Li

Title: Provably Bounded Dynamic Sparsifying Transform Network for Compressive Imaging

Abstract:
Compressive imaging (CI) aims to recover the underlying image from the under-sampled observations. Recently, deep unfolded CI (DUCI) algorithms, which unfold the iterative algorithms into deep neural networks (DNNs), have achieved remarkable results. Theoretically, unfolding a convergent iterative algorithm could ensure a stable DUCI algorithm, i.e., its performance increases as the increasing stage. However, ensuring convergence often involves imposing constraints, such as bounded spectral norm or tight property, on the filter weights or sparsifying transform. Unfortunately, these constraints may compromise algorithm performance. To address this challenge, we present a provably bounded dynamic sparsifying transform network (BSTNet), which can be explicitly proven to be a bounded network without imposing constraints on the analysis sparsifying transform. Leveraging this advantage, the analysis sparsifying transform can be adaptively generated via a trainable DNN. Specifically, we elaborate a dynamic sparsifying transform generator capable of extracting multiple feature information from input instances, facilitating the creation of a faithful content-adaptive sparsifying transform. We explicitly demonstrate that the proposed BSTNet is a bounded network, and further embed it as the prior network into a DUCI framework to evaluate its performance on two CI tasks, i.e., spectral snapshot CI (SCI) and compressed sensing magnetic resonance imaging (CSMRI). Experimental results showcase that our DUCI algorithms can achieve competitive recovery quality compared to benchmark algorithms. Theoretically, we explicitly prove that the proposed BSTNet is bounded, and we provide a comprehensive theoretical convergence analysis of the proposed iteration algorithms.

---

PaperID: 528,   

Authors:  Xinxin Wang, Yongshan Zhang, Yicong Zhou

Title: Pseudo-Supervision Affinity Propagation for Efficient and Scalable Multiview Clustering

Abstract:
Anchor graph-based multiview clustering (AGMVC) demonstrates high efficiency and satisfactory performance. However, it still suffers from limitations such as single-structure similarity measurement, high time expenditure for large-scale anchor graph partitioning, and limited generalization ability. To alleviate the instability problem of single-structure information, this article proposes an anchor graph construction method that learns local and global (LG) structures simultaneously. To eliminate the need for graph partitioning and address the out-of-sample problem, we develop a landmark learning method to produce structural anchors, and further propose a pseudo-supervision affinity propagation (PSAP) framework. This framework jointly optimizes graph construction and landmark learning to disentangle the in-cluster distribution between samples and anchors while accelerating convergence. In addition, our framework introduces a clustering inference partition (CIP) strategy to directly output clustering results without the need for time-consuming postprocessing. Extensive experiments validate the efficiency and effectiveness of our framework. Our code is publicly available at https://github.com/W-Xinxin/PSAP.

---

PaperID: 529,   

Authors:  Zecheng Tang, Xiaolong Wu, Honggui Han

Title: Robust Reconstructed Neural Network Based on Spectral Elastic Activation

Abstract:
The presence of sparse samples poses a formidable challenge for neural networks (NNs) in shaping representative patterns due to the limited coverage of concentrative activation range in NNs. To address this issue, a robust reconstructed NN, based on spectral elastic activation (SEA-RRNN), is developed in this article. Primarily, a spectral elastic activation (SEA) is designed to broaden the original activation of NNs, which embeds a spectral increment scaled by the estimated outlier degree on the activation boundary. It enables SEA-RRNN to stretch the boundary of SEA to cover the features of sparse samples. Then, an adaptive robust gradient descent (ARGD) algorithm is introduced to update the parameters of SEA. By tuning the loss between error and correntropy with the estimated outlier degree, the ARGD algorithm establishes two loss functions with complementary distance sensitivity for different parameters of SEA, which alleviates the construction conflict of robust center and precise boundary in SEA-RRNN. Furthermore, the theoretical analysis of SEA-RRNN is provided to validate its convergence and robustness. Finally, the experimental results demonstrate that SEA-RRNN exhibits superior robustness compared to other NN models.

---

PaperID: 530,   

Authors:  Takashi Matsubara, Takehiro Aoshima, Ai Ishikawa, Takaharu Yaguchi

Title: Deep Energy-Based Discrete-Time Physical Model for Reproducing Energetic Behavior

Abstract:
Modeling and simulating physical phenomena, especially those governed by partial differential equations (PDEs), pose significant challenges in computational physics and scientific machine learning. While neural network approaches have made strides in learning continuous-time dynamics, they have struggled with discrete-time scenarios and often fail to adhere to fundamental laws of physics, such as the conservation of energy and mass. This study addresses this gap by introducing a novel deep energy-based discrete-time model. In the real world, energy-based modeling theories like Hamiltonian mechanics and the Landau theory are pivotal, as they support various laws of physics. By integrating differential geometric structures into neural networks as coefficient matrices, our model successfully simulates the conservation and dissipation laws of energy and mass. Furthermore, we propose an automatic discrete differentiation algorithm, which enables neural networks to utilize the discrete gradient method, ensuring adherence to these laws in discrete-time settings. This capability also facilitates the identification of such laws directly from data by learning matrices that represent geometric structures. These advantages are verified using simulation results of physical phenomena, namely the 1- and 2-D Korteweg–de Vries (KdV) equation and the Cahn–Hilliard equation.

---

PaperID: 531,   

Authors:  Jin-Rong Zhang, Wujun Wen, Shen-lan Liu, Gao Huang, Yun-Heng Li, Qifeng Li, Lin Feng

Title: End-to-End Streaming Video Temporal Action Segmentation With Reinforcement Learning

Abstract:
The streaming temporal action segmentation (STAS) task, a supplementary task of temporal action segmentation (TAS), has not received adequate attention in the field of video understanding. Existing TAS methods are constrained to offline scenarios due to their heavy reliance on multimodal features and complete contextual information. The STAS task requires the model to classify each frame of the entire untrimmed video sequence clip by clip in time, thereby extending the applicability of TAS methods to online scenarios. However, directly applying existing TAS methods to SATS tasks results in significantly poor segmentation outcomes. In this article, we thoroughly analyze the fundamental differences between STAS tasks and TAS tasks, attributing the severe performance degradation when transferring models to model bias and optimization dilemmas. We introduce an end-to-end streaming video TAS model with reinforcement learning (SVTAS-RL). The end-to-end modeling method mitigates the modeling bias introduced by the change in task nature and enhances the feasibility of online solutions. Reinforcement learning (RL) is utilized to alleviate the optimization dilemma. Through extensive experiments, the SVTAS-RL model significantly outperforms existing STAS models and achieves competitive performance to the state-of-the-art (SOTA) TAS model on multiple datasets under the same evaluation criteria, demonstrating notable advantages on the ultralong video dataset EGTEA. Our code is publicly available at https://github.com/Thinksky5124/SVTAS.

---

PaperID: 532,   

Authors:  Min Li, Yunlong Zhao, Qizhen Wang, Hanlin Gao, Gang Wang

Title: RBF NN-Enabled Adaptive Filter for Any Type of Noise

Abstract:
The brief proposes a radial basis function (RBF) neural network (NN)-enabled adaptive filter (AF) algorithm, which consists of two stages. The first stage is a data-driven (DD) preprocessing part, and the RBF NN is to fit the probability density function (pdf) of the noise. The second stage is a model-driven filtering part, the RBF NN works as the cost function of the adaptive filtering, and an adaptive gradient ascent algorithm is obtained by maximizing the RBF NN. Since the RBF NN can fit any pdf of the noise, the proposed algorithm can work well in Gaussian, sub-Gaussian or light-tailed (uniform), and super-Gaussian or heavy-tailed (multipeak, pulse, and skewness) noises. Theoretical analysis shows the mean-value stability and mean square performance. Simulations verify the effectiveness of the proposed algorithm.

---

PaperID: 533,   

Authors:  Qian Cui, Gang Feng, Xuesong Xu

Title: Q-Learning-Based Robust Control for Nonlinear Systems With Mismatched Perturbations

Abstract:
This brief presents a novel optimal control (OC) approach based on \mathcal Q -learning to address robust control challenges for uncertain nonlinear systems subject to mismatched perturbations. Unlike conventional methodologies that solve the robust control problem directly, our approach reformulates the problem by minimizing a value function that integrates perturbation information. The \mathcal Q -function is subsequently constructed by coupling the optimal value function with the Hamiltonian function. To estimate the parameters of the \mathcal Q -function, an integral reinforcement learning (IRL) technique is employed to develop a critic neural network (NN). Leveraging this parameterized \mathcal Q -function, we derive a model-free OC solution that generalizes the model-based formulation. Furthermore, using Lyapunov’s direct method, the resulting closed-loop system is guaranteed to have uniform ultimate bounded stability. A case study is presented to showcase the effectiveness and applicability of the proposed approach.

---

PaperID: 534,   

Authors:  Changtian Ying, Qi Li, Chen Wang, Donghua Yu

Title: DNRHP: Temporal Network Representation Learning via Hawkes Point Process

Abstract:
Graph neural networks (GNNs) have significantly advanced our ability to mine structured data, playing a central role in areas such as social networks and recommendation systems. However, while most GNN-based methods focus on learning node representations in static graphs, they often ignore the dynamic nature of real-world networks, limiting their applicability. Furthermore, existing dynamic representation learning methods using Hawkes point processes, while capable of modeling event sequences, are inherently transductive and tailored to specific scenarios with dual timescales and mixed event types, thus not fully generalizable. To bridge this gap, we introduce DNRHP, a novel framework for learning temporal network representations. Specifically, DNRHP integrates historical edge (HE) information with the network’s evolutionary properties, using the Hawkes point process to model edge formation. It captures not only the influence of past events on the likelihood of future connections but also the impact of the structural evolution of the network. The novelty of our model lies in its comprehensive consideration of the dynamics of network evolution and historical connectivity, allowing for a more accurate representation of nodes and their interactions over time. Extensive experiments on diverse real-world networks demonstrate the effectiveness of DNRHP, outperforming state-of-the-art baselines in terms of accuracy and efficiency for tasks such as node classification and link prediction.

---

PaperID: 535,   

Authors:  Tharindu Fernando, Darshana Priyasad, Sridha Sridharan, Clinton Fookes

Title: Decoupled and Explainable Associative Memory for Effective Knowledge Propagation

Abstract:
Long-term memory often plays a pivotal role in human cognition through the analysis of contextual information. Machine learning researchers have attempted to emulate this process through the development of memory-augmented neural networks (MANNs) to leverage indirectly related but resourceful historical observations during learning and inference. The area of MANN, however, is still in its infancy and significant research effort is required to enable machines to achieve performance close to the human cognition process. This article presents an innovative MANN framework for the advanced incorporation of historical knowledge into a predictive framework. Within the key-value memory structure, we propose to decouple the key representations from the learned value memory embeddings to offer improved associations between the inputs and latent memory embeddings. We argue that the keys should be static, sparse, and unique representations of a particular observation to offer robust input to memory associations, while the value embeddings could be trainable, dense latent vectors such that they can better capture historical knowledge. Moreover, we introduce a novel memory update procedure that preserves the explainability of the historical knowledge extraction process, which would enable the human end-users to interpret the deep machine learning model decisions, fostering their trust. With extensive experiments conducted on three different datasets using audio, text, and image modalities, we demonstrate that our proposed innovations collectively allow this framework to outperform the current state-of-the-art methods by significant margins, irrespective of the modalities or the downstream tasks. The code is available at https://github.com/tha725/DE-KVMN/tree/main.

---

PaperID: 536,   

Authors:  Yanguang Sun, Hanyu Xuan, Jian Yang, Lei Luo

Title: GLCONet: Learning Multisource Perception Representation for Camouflaged Object Detection

Abstract:
Recently, the biological perception has been a powerful tool for handling the camouflaged object detection (COD) task. However, most existing methods are heavily dependent on the local spatial information of diverse scales from convolutional operations to optimize initial features. A commonly neglected point in these methods is the long-range dependencies between feature pixels from different scale spaces that can help the model build a global structure of the object, inducing a more precise image representation. In this article, we propose a novel global-local collaborative optimization network called GLCONet. Technically, we first design a collaborative optimization strategy (COS) from the perspective of multisource perception to simultaneously model the local details and global long-range relationships, which can provide features with abundant discriminative information to boost the accuracy in detecting camouflaged objects. Furthermore, we introduce an adjacent reverse decoder (ARD) that contains cross-layer aggregation and reverse optimization to integrate complementary information from different levels for generating high-quality representations. Extensive experiments demonstrate that the proposed GLCONet method with different backbones can effectively activate potentially significant pixels in an image, outperforming 20 state-of-the-art (SOTA) methods on three public COD datasets. The source code is available at: https://github.com/CSYSI/GLCONet.

---

PaperID: 537,   

Authors:  Lei Ding, Can Chen, Maojiao Ye, Qing-Long Han

Title: Fully Distributed Nash Equilibrium Seeking: A Double-Layer Adaptive Approach

Abstract:
This article is concerned with fully distributed Nash equilibrium seeking in networked games under both undirected and directed communication graphs. New fully Nash equilibrium seeking strategies incorporating gradient-based optimization algorithms, consensus algorithms, and double-layer adaptive control laws are presented. In particular, the double-layer adaptive control laws are introduced to ensure that the control gains are not overlarge and free of dependence on any global information. This is achieved by adding a damping term to the adaptive parameter design such that the continuous increase in control gains is avoided. Theoretical analyses are conducted to prove that players’ actions can be convergent to the Nash equilibrium under the proposed strategies. Moreover, it is shown that the developed strategies can be extended to accommodate the players with heterogeneous linear dynamics. Finally, numerical examples are provided to illustrate the effectiveness of the proposed methods.

---

PaperID: 538,   

Authors:  Chao Li, Shaokang Dong, Shangdong Yang, Yujing Hu, Tianyu Ding, Wenbin Li, Yang Gao

Title: Multi-Task Multi-Agent Reinforcement Learning With Interaction and Task Representations

Abstract:
Multi-task multi-agent reinforcement learning (MT-MARL) is capable of leveraging useful knowledge across multiple related tasks to improve performance on any single task. While recent studies have tentatively achieved this by learning independent policies on a shared representation space, we pinpoint that further advancements can be realized by explicitly characterizing agent interactions within these multi-agent tasks and identifying task relations for selective reuse. To this end, this article proposes Representing Interactions and Tasks (RIT), a novel MT-MARL algorithm that characterizes both intra-task agent interactions and inter-task task relations. Specifically, for characterizing agent interactions, RIT presents the interactive value decomposition to explicitly take the dependency among agents into policy learning. Theoretical analysis demonstrates that the learned utility value of each agent approximates its Shapley value, thus representing agent interactions. Moreover, we learn task representations based on per-agent local trajectories, which assess task similarities and accordingly identify task relations. As a result, RIT facilitates the effective transfer of interaction knowledge across similar multi-agent tasks. Structurally, RIT develops universal policy structure for scalable multi-task policy learning. We evaluate RIT against multiple state-of-the-art baselines in various cooperative tasks, and its significant performance under both multi-task and zero-shot settings demonstrates its effectiveness.

---

PaperID: 539,   

Authors:  Kunlun Wu, Bo Peng, Donghai Zhai

Title: Boundary-Aware Axial Attention Network for High-Quality Pavement Crack Detection

Abstract:
Pavement crack detection is a practical and challenging task that has the ability to significantly reduce the burden of manual building and road maintenance in intelligent transportation systems. Existing methods mainly focus on addressing common crack diseases and are poor in generalizing to other conditions of crack detection due to diverse environmental factors (e.g., illumination), topology complexity, and intensity in-homogeneity. Moreover, the samples suffer from the severe foreground-background imbalance and the model is easily prone to overfitting on trained anomalies, resulting in unsatisfactory performance. To tackle the aforementioned challenges and achieve high-quality pavement crack detection, we propose an innovative approach termed boundary-aware axial attention network (BAAN), which is composed of multiple position-guided axial attention (PAA) modules in a hierarchical encoder-decoder architecture. Specifically, it learns efficient contextual information via decomposed multidimensional position-guided attention to capture more precise spatial structures, and the proposed boundary regularization module (BRM) mines more discriminative foreground-background relationships to regularize the ambiguous details between diverse spatial regions. Moreover, we propose a novel boundary refinement loss (BRL) to alleviate the challenges associated with regional losses (e.g., pixel-wise cross-entropy loss) in the context of heavily imbalanced crack detection problems. The proposed BAAN is evaluated on four crack datasets and experimental results indicate that the BAAN consistently outperforms the state-of-the-art methods with fewer computational requirements.

---

PaperID: 540,   

Authors:  Xi Yang, Wenjiao Dong, De Cheng, Nannan Wang, Xinbo Gao

Title: TIENet: A Tri-Interaction Enhancement Network for Multimodal Person Reidentification

Abstract:
Multimodal person reidentification (ReID), which aims to learn modality-complementary information by utilizing multimodal images simultaneously for person retrieval, is crucial for achieving all-time and all-weather monitoring. Existing methods try to address this issue through modality fusion to absorb complementary information. However, most of these methods are limited to the spatial domain only and usually overlook the intra-/intermodal interactions during feature fusion, resulting in insufficient learning of modality-specific and complementary information. To address these issues, we propose a tri-interaction enhancement network (TIENet), which contains three modules: spatial-frequency interaction (SFI), intermodal mask interaction (IMMI), and intramodal feature fusion (IMFF). Specifically, the SFI boosts the modality-specific representation by integrating the amplitude-guided attention mechanism into the phase space, combined with spatial-domain convolution to achieve fine-grained information learning. Meanwhile, the IMMI enhances the richness of the feature descriptors by embedding the intermodal relationships to preserve complementary information. Finally, the IMFF module considers the structure of the human body and integrates intramodal contextual information. Extensive experimental results demonstrate the effectiveness of our method, achieving superior performances on RGBNT201 and MARKET1501_RGBNT datasets.

---

PaperID: 541,   

Authors:  Hongsong Tang, Yingzhuo Liu, Letian Ni, Liuyu Xiang, Yaodong Yang, Ke Bi, Zhaofeng He

Title: Distributed Policy Space Response Oracles in Two-Player Zero-Sum Games

Abstract:
Policy space response oracle (PSRO) is a population-based algorithm that can be used to solve two-player zero-sum games. In the PSRO solution framework, optimizing policy diversity is crucial for addressing nontransitive game problems, helping the agent population avoid exploitation by unfamiliar opponents. In addition, while deep reinforcement learning is highly effective in solving complex game environments, its integration with PSRO remains fragmented and lacking in effective coordination. In this study, we propose distributed PSRO to efficiently solve complex game scenarios. To enhance diversity while managing optimization costs, we introduce TOP-K truncation, which prioritizes high-quality opponents and limits the size of the policy pool during sampling. This approach not only reduces interference from less effective strategies but also ensures computational efficiency by seamlessly integrating with our distributed training framework. We also design the distributed training framework to incorporate diversity estimation directly into the sampling process, achieving diversity optimization without incurring additional computational overhead. Furthermore, we introduce the opponent first (OF) method, which enhances decision-making by leveraging opponent information during interaction sampling. We perform experimental validation using a nontransitive mixture model and AlphaStar888 to confirm the effectiveness of the TOP-K truncation approach. Finally, we demonstrate the feasibility and efficiency of the distributed training framework and the OF approach in a Google Research Football 11 versus 11 scenario.

---

PaperID: 542,   

Authors:  Nana Yu, Jie Wang, Zihao Zhang, Yahong Han, Weiping Ding

Title: Semantic Prompt Enhancement for Semi-Supervised Low-Light Salient Object Detection

Abstract:
Most existing salient object detection (SOD) models are designed based on data collected in well-lit scenes, which is entirely inadequate for low-light conditions. Although recent models are designed for low-light conditions, they still have limitations. First, they simply integrate features without considering the impact of low-light scenes and fail to enhance the contextual information around salient objects. Second, in extremely dark scenes, it is difficult for the human eye to distinguish between the foreground and background, posing significant challenges for data labeling. To address these issues, we design a brightness Retinex enhancer (BRE) tailored for low-light SOD tasks and, for the first time, explore performing low-light SOD within a semi-supervised framework. By using sparse labeled semantic prompts to augment a large amount of unlabeled data, we mitigate the annotation burden while avoiding ineffective labeling in low-light conditions. More specifically, we first use Retinex decomposition to filter out the influence of illumination, while the semantic features extracted by a large model serve as semantic prompts to assist in enhancement. In addition, we introduce a context-guided encoder (CGE) to improve the model’s understanding of salient objects. Finally, both labeled and unlabeled data undergo joint consistency training between the shared decoder (SD) and the perturbation decoder. The semi-supervised model enhances low-light SOD performance while also alleviating the burden of data annotation. Extensive experiments demonstrate that, compared with state-of-the-art fully supervised SOD models, the proposed semi-supervised model achieves highly competitive results across multiple test datasets.

---

PaperID: 543,   

Authors:  Bei Pan, Kaoru Hirota, Yaping Dai, Zhiyang Jia, Shuai Shao, Jinhua She

Title: Learning Sequential Variation Information for Dynamic Facial Expression Recognition

Abstract:
A multiscale sequence information fusion (MSSIF) method is presented for dynamic facial expression recognition (DFER) in video sequences. It exploits multiscale information by integrating features from individual frames, subsequences, and entire sequences through a transformer-based architecture. This hierarchical feature fusion process includes deep feature extraction at the frame level to capture intricate visual details, intrasubsequence fusion using self-attention mechanisms for analyzing adjacent frames, and intersubsequence fusion to synthesize long-term emotional dynamics across time scales. The efficacy of MSSIF is demonstrated through extensive evaluation on three video datasets: eNTERFACE’05, BAUM-1s, and AFEW, where it achieves overall recognition accuracies of 60.1%, 60.7%, and 58.8%, respectively. These results substantiate MSSIF’s superior performance in accurately recognizing facial expressions by managing short and long-term dependencies within video sequences, making it a potent tool for real-world applications requiring nuanced dynamic facial expression detection.

---

PaperID: 544,   

Authors:  Yujie Wang, Weiwei Xu, Lei Zhu

Title: Efficient Linear Discriminant Analysis Based on Randomized Low-Rank Approaches

Abstract:
Linear discriminant analysis (LDA) faces challenges in practical applications due to the small sample size (SSS) problem and high computational costs. Various solutions have been proposed to address the SSS problem in both ratio trace LDA and trace ratio LDA (TRLDA). However, the iterative processing of large matrices often makes the computation process cumbersome. To address this issue, for TRLDA, we propose a novel random method that extracts orthogonal bases from matrices, allowing computations with small-sized matrices. This significantly reduces computational time without compromising accuracy. For ratio trace LDA, we introduce a fast generalized singular value decomposition (GSVD) algorithm, which demonstrates superior speed compared to MATLAB’s built-in GSVD algorithm in experiments. By integrating this new GSVD algorithm into ratio trace LDA, we propose FGSVD-LDA, which exhibits low computational complexity and good classification performance. The experimental results show that both methods effectively achieve dimensionality reduction and deliver satisfactory classification accuracy.

---

PaperID: 545,   

Authors:  Honggui Han, Chenxuan Sun, Xiaolong Wu, Hongyan Yang, Dezheng Zhao

Title: Self-Organizing Stacked Type-2 Fuzzy Neural Network With Rule Generalization

Abstract:
Type-2 fuzzy neural networks (T2FNNs) are particularly effective in dealing with nonlinear systems. However, they inevitably suffer from multicollinearity problems caused by the significant overlaps of the footprint uncertainty (FOU), which leads to generalization biases. To solve this challenge, a self-organizing stacked T2FNN with rule generalization (RG-SOST2FNN) is developed to boost its overall performance. First, a stacked technique with cosine smart priority is designed for T2FNN fusion. This technique employs multivariable cosine similarity to obtain sparse inputs, which selectively stacks multiple T2FNNs with non-collinear inputs to reduce collinearity dependence. Second, a dynamic stacked framework with a rule cluster generation mechanism is developed to achieve individual and batch rule adjustment. Then, a stacked structure with diversity is obtained to alleviate collinearity among rules by eliminating the singularity of the parameter matrix of FOUs. Third, a stacked risk mitigation algorithm is proposed to shape the fuzzy rule clusters (FRCs). Then, the parameters of FRCs are optimized using sparse gradient learning, which avoids the updating of collinear features to reduce the variance of parameter estimation. Finally, the simulation tests show that RG-SOST2FNN can achieve state-of-the-art performance even at high multicollinearity in complex systems.

---

PaperID: 546,   

Authors:  Ruixin Chen, Jianping Fan, Mei-Qin Wu, Rui Cheng, Jiawen Song

Title: G-Diff: A Graph-Based Decoding Network for Diffusion Recommender Model

Abstract:
The recommendation system is an effective approach to alleviate the information overload caused by the popularization of the Internet. Existing recommendation methods often use advanced deep learning algorithms to predict user preferences. The diffusion model is a deep generative model that has received much attention in recent years and has been successfully applied in recommendation systems. However, previous research has mainly used MLP in the reverse process of the diffusion model, which fails to fully utilize the collective signals of various items in the recommendation system. This article improves the diffusion recommendation model by introducing a carefully designed graph-based decoding network (GDN) in the reverse process. GDN improves recommendation performance by introducing relationships between items via the item-item graph. In addition, skip connections and normalization layers are implemented to maintain low-order neighbor information. Experiments are conducted to compare the proposed model with several state-of-the-art recommendation methods on three real-world datasets, which demonstrate the improvement of the proposed method over the diffusion recommendation model. Specifically, the proposed method outperforms the diffusion recommendation model with autoencoder (AE) by 21.67% on average. The contribution of each component of the proposed model is also illustrated by the ablation experiments. The implementation codes of the proposed model are available via https://github.com/crx1729/G-Diff.

---

PaperID: 547,   

Authors:  Hao Zhang, Chenglin Li, Wenrui Dai, Junni Zou, Hongkai Xiong

Title: Federated Learning Based on Model Discrepancy and Variance Reduction

Abstract:
In federated learning (FL), the heterogeneity of data and asynchronous participation of clients have been observed to induce the local client’s model discrepancy with high variance, leading to a slow and unstable convergence globally at the server. In this article, motivated by the usefulness of stale client updates, we first propose a general framework, named FedVR, to address this issue. In FedVR, we design an aggregate of both fresh and stale local model updates without additional communication overhead, which is computed at the server as a control variate to reduce the client variance incurred by data heterogeneity and client sampling. In order to further reduce the model discrepancy between local clients, we therefore propose FedMDVR, which broadcasts the designed control variate to all the active clients to help correct their local update directions toward the global optimum, i.e., stationary point of the global objective function. While in the global update at server, the client variance is also decreased as inherited from the variance reduction nature of FedVR. We theoretically prove the convergence of FedVR and FedMDVR in the general nonconvex settings. Through extensive experimental evaluations on several benchmark datasets, we also demonstrate that our proposed FedVR and FedMDVR not only accelerate the convergence by reducing the number of communication rounds required to achieve a certain target accuracy, but more importantly, can converge to a higher accuracy than the baseline algorithms.

---

PaperID: 548,   

Authors:  Sucheng Ren, Xiaomeng Li

Title: HResFormer: Hybrid Residual Transformer for Volumetric Medical Image Segmentation

Abstract:
Vision Transformer shows great superiority in medical image segmentation due to the ability to learn long-range dependency. For medical image segmentation from 3-D data, such as computed tomography (CT), existing methods can be broadly classified into 2-D-based and 3-D-based methods. One key limitation in 2-D-based methods is that the intraslice information is ignored, while the limitation in 3-D-based methods is the high computation cost and memory consumption, resulting in a limited feature representation for inner slice information. During the clinical examination, radiologists primarily use the axial plane and then routinely review both axial and coronal planes to form a 3-D understanding of anatomy. Motivated by this fact, our key insight is to design a hybrid model that can first learn fine-grained inner slice information and then generate a 3-D understanding of anatomy by incorporating 3-D information. We present a novel Hybrid Residual TransFormer (HResFormer) for 3-D medical image segmentation. Building upon standard 2-D and 3-D Transformer backbones, HResFormer involves two novel key designs: 1) a Hybrid Local-Global fusion Module (HLGM) to effectively and adaptively fuse inner slice information from 2-D Transformers and intraslice information from 3-D volumes for 3-D Transformers with local fine-grained and global long-range representation and 2) residual learning of the hybrid model, which can effectively leverage the inner slice and intraslice information for better 3-D understanding of anatomy. Experiments show that our HResFormer outperforms prior art on widely used medical image segmentation benchmarks. This article sheds light on an important but neglected way to design Transformers for 3-D medical image segmentation.

---

PaperID: 549,   

Authors:  Shashi Raj Pandey, Pierre Pinson, Petar Popovski

Title: Privacy-Aware Data Acquisition Under Data Similarity in Regression Markets

Abstract:
Data markets facilitate decentralized data exchange for applications such as prediction, learning, or inference. The design of these markets is challenged by varying privacy preferences and data similarity among data owners. Related works have often overlooked how data similarity impacts pricing and data value through statistical information leakage. We demonstrate that data similarity and privacy preferences are integral to market design and propose a query-response protocol using local differential privacy (LDP) for a two-party data acquisition mechanism. In our regression data market model, we analyze strategic interactions between privacy-aware owners and the learner as a Stackelberg game over the asked price and privacy factor. Finally, we numerically evaluate how data similarity affects market participation and traded data value.

---

PaperID: 550,   

Authors:  Yihong Leng, Jiaojiao Li, Rui Song, Yunsong Li, Qian Du

Title: Uncertainty-Guided Discriminative Priors Mining for Flexible Unsupervised Spectral Reconstruction

Abstract:
Existing supervised spectral reconstruction (SR) methods adopt paired RGB images and hyperspectral images (HSIs) to drive the overall paradigms. Nonetheless, in practice, “paired” requires higher device requirements such as specific well-calibrated dual cameras or more complex and exact registration processes among images with different time phases, widths, and spatial resolution. To tackle the above challenges, we propose a flexible uncertainty-aware unsupervised SR paradigm, which dynamically establishes the forceful and potent constraints with RGBs for driving unsupervised learning. As a specific plug-and-play tail in our paradigm, the uncertainty-aware saliency alignment module (USAM) calculates pixel- and spectralwise information entropy for uncertainty estimation, which attempts to represent the corresponding reflectivity or radiance to the light among different objects in various scenes, forcing the paradigm to adaptively explore the scene-agnostic prominent features. Furthermore, a progressively parallel network under our unsupervised paradigm is conducted to excavate discriminate structural and semantic priors of RGBs to assist in recovering dependable HSIs: 1) a learnable rank-guided structural representation (LRSR) flow is leveraged to characterize the latent structural priors via excavating nonzero elements in the full-rank matrix and further preserve evident boundaries in HSIs; and 2) a coarse-to-fine bandwise semantic perception (CBSP) flow is conducted to propagate perceptual bandwise affinity for aggregating and strengthening intrinsic interband dependencies, and further extract delicate semantic priors, which can recover plentiful contiguous spectral information in HSIs. Comprehensive quantitative and qualitative experimental results on three visual and two remote sensing benchmarks have shown the superiority and robustness of our method. We also conducted nine existing SR methods in our unsupervised paradigm to recover HSIs without any manual intervention, which proves the generality of our paradigm to some extent.

---

PaperID: 551,   

Authors:  Zhiqiang Pan, Honghui Chen, Wanyu Chen, Fei Cai, Xinwang Liu

Title: Time-Aware Graph Learning for Link Prediction on Temporal Networks

Abstract:
Link prediction on temporal networks aims to predict the future edges by modeling the dynamic evolution involved in the graph data. Previous methods relying on the node/edge attributes or the distance on the graph structure are not practical due to the deficiency of the attributes and the limitation of the explicit distance estimation, respectively. Moreover, the existing graph representation learning methods mostly rely on graph neural networks (GNNs), which cannot adequately take the dynamic correlations between nodes into consideration, leading to the generating of inferior node embeddings. Thus, we propose a time-aware graph (TAG) learning method for link prediction on temporal networks. We first conduct a theoretical causal analysis proving that the correlations between nodes are required to be unchanged for the temporal graph representation learning using GNNs. Then, we model the recent dynamic node correlations by designing an edge-dropping (ED) module and adopting a recent neighbor sampling (RNS) strategy so as to approximate the above condition. Besides, we also preserve the long-term stable node correlations by introducing additional self-supervisions using the contrastive learning. Comprehensive experiments were conducted on four public temporal network datasets, i.e., MathOverflow, StackOverflow, AskUbuntu, and SuperUser, demonstrate that TAG can achieve state-of-the-art performance in terms of average precision (AP) and area under the ROC curve (AUC). In addition, TAG can ensure high computational efficiency by making the temporal graph lightweight, letting it be practical in real-world applications.

---

PaperID: 552,   

Authors:  Pengfei Sun, Jibin Wu, Malu Zhang, Paul Devos, Dick Botteldooren

Title: Delayed Memory Unit: Modeling Temporal Dependency Through Delay Gate

Abstract:
Recurrent neural networks (RNNs) are widely recognized for their proficiency in modeling temporal dependencies, making them highly prevalent in sequential data processing applications. Nevertheless, vanilla RNNs are confronted with the well-known issue of gradient vanishing and exploding, posing a significant challenge for learning and establishing long-range dependencies. Additionally, gated RNNs tend to be over-parameterized, resulting in poor computational efficiency and network generalization. To address these challenges, this article proposes a novel delayed memory unit (DMU). The DMU incorporates a delay line structure along with delay gates into vanilla RNN, thereby enhancing temporal interaction and facilitating temporal credit assignment. Specifically, the DMU is designed to directly distribute the input information to the optimal time instant in the future, rather than aggregating and redistributing it over time through intricate network dynamics. Our proposed DMU demonstrates superior temporal modeling capabilities across a broad range of sequential modeling tasks, utilizing considerably fewer parameters than other state-of-the-art gated RNN models in applications such as speech recognition, radar gesture recognition, ECG waveform segmentation, and permuted sequential (PS) image classification.

---

PaperID: 553,   

Authors:  Ming Zhang, Zhe Chen, Vasile Palade, Tao Lu, Liya Wang, Junchi Zhang, Yanduo Zhang

Title: Double-Graph Representation With Relational Enhancement for Emotion-Cause Pair Extraction

Abstract:
The emotion-cause pair extraction (ECPE) task is to simultaneously extract emotions and causes as pairs (EC-pairs) from documents, which is important for natural language processing. Previous research tackled this task via a two-step approach, which first predicts separately the emotion and cause clauses, and then pairs them up by using a binary classifier. However, such a two-step approach may suffer from the possible propagation of errors, and it neglects the interaction between emotions and causes. In this article, an end-to-end double-graph method with relational enhancement (DGRE) is proposed to stimulate two relationship modes among clauses, i.e., semantic dependence and logical dependence. First, two united graph encoders are established to embed the semantic dependence into the representation of clauses and pairs. The first encoder is built on graph attention networks (GATs) for clause-level representation, the result of which is used by a relational graph convolutional network (RGCN) for the refinement of pair-level representation. Aiming to enhance the fitting ability of logical dependence, the emotion-type classification task is introduced into the multitask learning framework of GATs, which can effectively distinguish the logical relations between clauses according to their emotion types. Moreover, seven types of dependence relations have been designed for the node connections in RGCN, which emphasize the contextual interaction and clustering among neighboring nodes. Experiments on a benchmark Chinese corpus demonstrate that the proposed DGRE approach could effectively establish the communication mechanism between clauses and pairs from multiple perspectives, and comparisons with state-of-the-art (SOTA) models well validate its effectiveness.

---

PaperID: 554,   

Authors:  Lihong Pei, Yang Cao, Yu Kang, Zhenyi Xu, Qianming Liu

Title: Spatiotemporal Imputation of Traffic Emissions With Self-Supervised Diffusion Model

Abstract:
The comprehensive regulatory oversight of traffic emissions frequently encounters the missing not-at-random (MNAR) pattern, characterized by the long-term block missing in adjacent road segments, arising from insufficient monitoring points and nonuniform spatiotemporal distribution. The spatiotemporal block missing simultaneously disrupts the spatiotemporal correlation, introducing significant biases in spatiotemporal modeling for incomplete data. The emerging diffusion model recovers the information of the missing regions in a self-supervised manner and focuses on the generation process of the missing regions to address biases. However, the dynamics and spatiotemporal heterogeneity of traffic emissions limit its applicability in unknown spatiotemporal missing. To address this issue, this article proposes a novel progressive Diffusion Model-based framework for SpatioTemporal Imputation of traffic emissions (STI-dm). Specifically, a self-supervised masked training strategy is first devised to construct the nonlocal similarity prior of traffic emission data, explicitly introducing the MNAR missing mechanism for the diffusion process. Furthermore, an enhanced approach of noise injection and supervised denoising is adopted to rectify misconceptions of nonlocal alignment, decreasing modeling biases associated with incomplete data in the generation process. The imputation and prior modeling processes are progressively performed until obtaining stable results, and each of the preceding modeling processes benefits from the gradual improvement results in the other. Experimental evidence indicates that STI-dm surpasses the current state-of-the-art algorithms in scenarios with intricate spatiotemporal patterns and varying rates of missing data.

---

PaperID: 555,   

Authors:  Basit Alawode, Sajid Javed

Title: Learning Spatial-Temporal Regularized Tensor Sparse RPCA for Background Subtraction

Abstract:
Background subtraction in videos is a core challenge in computer vision, aiming to accurately identify moving objects. Robust principal component analysis (RPCA) has emerged as a promising unsupervised (US) paradigm for this task, showing strong performance on various benchmark datasets. Building on RPCA, tensor RPCA (TRPCA) variants have further enhanced background subtraction performance. However, current TRPCA methods often treat moving object pixels independently, lacking spatial-temporal structured-sparsity constraints. This limitation leads to performance degradation in scenarios with dynamic backgrounds, camouflage, and camera jitter. In this work, we introduce a novel spatial-temporal regularized tensor sparse RPCA algorithm to address these issues. By incorporating normalized graph-Laplacian matrices into the sparse component, we enforce spatial-temporal regularization. We construct two graphs—one across spatial locations and another across temporal slices—to guide regularization. By maximizing our objective function, we ensure that the tensor sparse component aligns with the spatiotemporal eigenvectors of the graph-Laplacian matrices, preserving disconnected moving object pixels. We formulate a new objective function and employ batch and online-based optimization methods to jointly optimize background-foreground separation and spatial-temporal regularization. Experimental evaluation on six publicly available datasets demonstrates the superior performance of our algorithm compared to existing methods.

---

PaperID: 556,   

Authors:  Lauren Nicole Delong, Ramon Fernández Mir, Jacques D. Fleuriot

Title: Neurosymbolic AI for Reasoning Over Knowledge Graphs: A Survey

Abstract:
Neurosymbolic artificial intelligence (AI) is an increasingly active area of research that combines symbolic reasoning methods with deep learning to leverage their complementary benefits. As knowledge graphs (KGs) are becoming a popular way to represent heterogeneous and multirelational data, methods for reasoning on graph structures have attempted to follow this neurosymbolic paradigm. Traditionally, such approaches have utilized either rule-based inference or generated representative numerical embeddings from which patterns could be extracted. However, several recent studies have attempted to bridge this dichotomy to generate models that facilitate interpretability, maintain competitive performance, and integrate expert knowledge. Therefore, we survey methods that perform neurosymbolic reasoning tasks on KGs and propose a novel taxonomy by which we can classify them. Specifically, we propose three major categories: 1) logically informed embedding approaches; 2) embedding approaches with logical constraints; and 3) rule-learning approaches. Alongside the taxonomy, we provide a tabular overview of the approaches and links to their source code, if available, for more direct comparison. Finally, we discuss the unique characteristics and limitations of these methods and then propose several prospective directions toward which this field of research could evolve.

---

PaperID: 557,   

Authors:  Fanghui Zhang, Haiyue Zhu, Yigang Cen, Shichao Kan, Linna Zhang, Prahlad Vadakkepat, Tong Heng Lee

Title: Low-Shot Unsupervised Visual Anomaly Detection via Sparse Feature Representation

Abstract:
Visual anomaly detection is an essential component in modern industrial manufacturing. Existing studies using notions of pairwise similarity distance between a test feature and nominal features have achieved great breakthroughs. However, the absolute similarity distance lacks certain generalizations, making it challenging to extend the comparison beyond the available samples. This limitation could potentially hamper anomaly detection performance in scenarios with limited samples. This article presents a novel sparse feature representation anomaly detection (SFRAD) framework, which formulates the anomaly detection as a sparse feature representation problem; and notably proposes an anomaly score by orthogonal matching pursuit (ASOMP) as a novel detection metric. Specifically, SFRAD calculates the Gaussian kernel distance between the test feature and its sparse representation in the nominal feature space for anomaly detection. Here, the orthogonal matching pursuit (OMP) algorithm is adopted to achieve the sparse feature representation. Moreover, to construct a low-redundancy memory bank storing the basis features for sparse representation, a novel basis feature sampling (BFS) algorithm is proposed by considering both the maximum coverage and the optimum feature representation simultaneously. As a result, SFRAD incorporates both the advantages of absolute similarity and linear representation; and this enhances the generalization in low-shot scenarios. Extensive experiments on the MVTec anomaly detection (MVTec AD), Kolektor surface-defect dataset (KolektorSDD), Kolektor surface-defect dataset 2 (KolektorSDD2), MVTec logical constraints anomaly detection (MVTec LOCO AD), Visual anomaly (VISA), Modified national institute of standards and technology (MNIST), and CIFAR-10 datasets demonstrate that our proposed SFRAD outperforms the previous methods and achieves state-of-the-art unsupervised anomaly detection performance. Notably, significantly improved outcomes and results have also been achieved on low-shot anomaly detection. Code is available at https://github.com/fanghuisky/SFRAD.

---

PaperID: 558,   

Authors:  Ruikai Yang, Fan He, Mingzhen He, Jie Yang, Xiaolin Huang

Title: Decentralized Kernel Ridge Regression Based on Data-Dependent Random Feature

Abstract:
Random feature (RF) has been widely used for node consistency in decentralized kernel ridge regression (KRR). Currently, the consistency is guaranteed by imposing constraints on coefficients of features, necessitating that the RFs on different nodes are identical. However, in many applications, data on different nodes vary significantly on the number or distribution, which calls for adaptive and data-dependent methods that generate different RFs. To tackle the essential difficulty, we propose a new decentralized KRR algorithm that pursues consensus on decision functions, which allows great flexibility and well adapts data on nodes. The convergence is rigorously given, and the effectiveness is numerically verified: by capturing the characteristics of the data on each node, while maintaining the same communication costs as other methods, we achieved an average regression accuracy improvement of 25.5% across six real-world datasets.

---

PaperID: 559,   

Authors:  Shuaibing Zhu, Hong Sang, Kai Zhang, Fanchao Kong, Jinhu Lü

Title: Synchronization of Intermittently Coupled Neural Networks With Coupling Delay

Abstract:
In recent years, the synchronization of coupled neural networks (CNNs) has been extensively studied. However, existing results heavily rely on assuming continuous couplings, overlooking the prevalence of intermittent couplings in reality. In this article, we address for the first time the synchronization challenge posed by intermittently CNNs (ICNNs) with coupling delay. To overcome the difficulties arising from intermittent couplings, we put forward a general piecewise delay differential inequality to characterize the dynamics during both coupled intervals and decoupled intervals. Based on the proposed inequality, we establish delay-independent synchronization criteria (DISCs) for ICNNs, enabling them to tackle general coupling delay. Notably, unlike previous studies, the achievement of synchronization in our approach does not rely on external control. Furthermore, for ICNNs that synchronize only under small delays, we formulate non-linear matrix inequality (LMI)-based delay-dependent synchronization criteria (DDSCs) that are computationally efficient and do not require delay differentiability. Finally, we provide illustrative examples to demonstrate our theoretical results.

---

PaperID: 560,   

Authors:  Geping Yang, Shusen Yang, Yiyang Yang, Xiang Chen, Can Chen, Zhiguo Gong, Zhifeng Hao

Title: SPGMVC: Multiview Clustering via Partitioning the Signed Prototype Graph

Abstract:
Multiview clustering (MVC) has been widely studied in machine learning and data mining for its capability of improving clustering performance by fusing the information from multiview data. In the past decade, a large number of MVC methods have made impressive progress, but most of them suffer from computational burdens, especially in large-scale tasks. Binary MVC (BMVC) is proposed to address this issue by representing the large-scale high-dimensional dataset as a group of consensus and low-dimensional binary codes. However, current BMVC-based approaches generate the clustering by executing binary k-means on the obtained binary codes, which fail to capture the embedded geometric information, leading to poor clustering performance. In addition, parameter selection is another “mission impossible” in unsupervised learning tasks including MVC. To tackle these challenges, a framework of multiview clustering via partitioning the signed prototype graph (SPGMVC) is proposed in this work. The SPGMVC framework offers several contributions. First, SPGMVC is designed as a unified framework for MVC. It combines effective technologies, such as consensus binary coding, code compression (CC), signed prototype graph (SPG) partitioning, and prototype-based cluster assignment. Second, SPGMVC partitions the signed graph (SG) based on the relationships between positive and negative edges. By capturing the underlying structure of the data, this partitioning strategy improves clustering accuracy (ACC). CC techniques are applied to reduce the graph’s scale, enabling further partitioning and enhancing computational efficiency. Third, SPGMVC employs an alternate minimizing strategy to efficiently handle the optimization problem. This strategy has nearly linear time and space complexity with respect to the data volume, making it suitable for large-scale tasks. Fourth, SPGMVC proposes an automatic parameter selection strategy, eliminating the need for extensive parameter exploration. Comprehensive experiments illustrate the superiority of our model. The implementation of SPGMVC is available at: https://github.com/gepingyang/PSGMVC.

---

PaperID: 561,   

Authors:  Jie Su, Peng Sun, Yuting Jiang, Zhenyu Wen, Fangda Guo, Yiming Wu, Zhen Hong, Haoran Duan, Yawen Huang, Rajiv Ranjan, Yefeng Zheng

Title: A Semantic-Consistent Few-Shot Modulation Recognition Framework for IoT Applications

Abstract:
The rapid growth of the Internet of Things (IoT) has led to the widespread adoption of the IoT networks in numerous digital applications. To counter physical threats in these systems, automatic modulation classification (AMC) has emerged as an effective approach for identifying the modulation format of signals in noisy environments. However, identifying those threats can be particularly challenging due to the scarcity of labeled data, which is a common issue in various IoT applications, such as anomaly detection for unmanned aerial vehicles (UAVs) and intrusion detection in the IoT networks. Few-shot learning (FSL) offers a promising solution by enabling models to grasp the concepts of new classes using only a limited number of labeled samples. However, prevalent FSL techniques are primarily tailored for tasks in the computer vision domain and are not suitable for the wireless signal domain. Instead of designing a new FSL model, this work suggests a novel approach that enhances wireless signals to be more efficiently processed by the existing state-of-the-art (SOTA) FSL models. We present the semantic-consistent signal pretransformation (ScSP), a parameterized transformation architecture that ensures signals with identical semantics exhibit similar representations. ScSP is designed to integrate seamlessly with various SOTA FSL models for signal modulation recognition and supports commonly used deep learning backbones. Our evaluation indicates that ScSP boosts the performance of numerous SOTA FSL models, while preserving flexibility.

---

PaperID: 562,   

Authors:  Pengfei Zhu, Jialu Li, Yu Wang, Bin Xiao, Jinglin Zhang, Wanyu Lin, Qinghua Hu

Title: Boosting Pseudo-Labeling With Curriculum Self-Reflection for Attributed Graph Clustering

Abstract:
Attributed graph clustering is an unsupervised learning task that aims to partition various nodes of a graph into distinct groups. Existing approaches focus on devising diverse pretext tasks to obtain suitable supervised information for representation learning, among which the predictive methods show great potential. However, these methods 1) generate auxiliary task bias toward the clustering target and 2) introduce label noise due to static thresholds. To address this issue, we propose a new self-supervised learning method, namely, pseudo-labeling with curriculum self-reflection (PLCSR), that learns reliable pseudo-labels by mining its information to achieve progressive processing of nodes in a self-reflection manner. First, a self-auxiliary encoder is constructed using the exponential moving average (EMA) of the original encoder’s parameters to replace the auxiliary tasks, which provides an additional perspective of finding highly confident pseudo-labels. Second, a curriculum selection strategy using dynamic thresholds is designed to take full advantage of graph nodes more accurately. Besides simple nodes with high confidence at the initial stage, nodes that yield consistent predictions from both encoders are then assigned pseudo-labels to avoid the under-learning problem. For the rest difficult nodes that are highly uncertain, we abstain from making judgments to minimize their adverse impact on the model. Extensive experiments have shown that PLCSR significantly outperforms the state-of-the-art predictive method CDRS, achieving more than 6% improvements in terms of clustering accuracy. The code is available at: https://github.com/Jillian555/PLCSR.

---

PaperID: 563,   

Authors:  Hang Shao, Lei Luo, Jianjun Qian, Mengkai Yan, Shangbing Gao, Jian Yang

Title: Video-Based Multiphysiological Disentanglement and Remote Robust Estimation for Respiration

Abstract:
Remote noncontact respiratory rate estimation by facial visual information has great research significance, providing valuable priors for health monitoring, clinical diagnosis, and anti-fraud. However, existing studies suffer from disturbances in epidermal specular reflections induced by head movements and facial expressions. Furthermore, diffuse reflections of light in the skin-colored subcutaneous tissue caused by multiple time-varying physiological signals independent of breathing are entangled with the intention of the respiratory process, leading to confusion in current research. To address these issues, this article proposes a novel network for natural light video-based remote respiration estimation. Specifically, our model consists of a two-stage architecture that progressively implements vital measurements. The first stage adopts an encoder-decoder structure to recharacterize the facial motion frame differences of the input video based on the gradient binary state of the respiratory signal during inspiration and expiration. Then, the obtained generative mapping, which is disentangled from various time-varying interferences and is only linearly related to the respiratory state, is combined with the facial appearance in the second stage. To further improve the robustness of our algorithm, we design a targeted long-term temporal attention module and embed it between the two stages to enhance the network’s ability to model the breathing cycle that occupies ultra many frames and to mine hidden timing change clues. We train and validate the proposed network on a series of publicly available respiration estimation datasets, and the experimental results demonstrate its competitiveness against the state-of-the-art breathing and physiological prediction frameworks.

---

PaperID: 564,   

Authors:  Shunxin Guo, Hongsong Wang, Shuxia Lin, Zhiqiang Kou, Xin Geng

Title: Addressing Skewed Heterogeneity via Federated Prototype Rectification With Personalization

Abstract:
Federated learning (FL) is an efficient framework designed to facilitate collaborative model training across multiple distributed devices while preserving user data privacy. A significant challenge of FL is data-level heterogeneity, i.e., skewed or long-tailed distribution of private data. Although various methods have been proposed to address this challenge, most of them assume that the underlying global data are uniformly distributed across all clients. This article investigates data-level heterogeneity FL with a brief review and redefines a more practical and challenging setting called skewed heterogeneous FL (SHFL). Accordingly, we propose a novel federated prototype rectification with personalization (FedPRP) which consists of two parts: federated personalization and federated prototype rectification. The former aims to construct balanced decision boundaries between dominant and minority classes based on private data, while the latter exploits both interclass discrimination and intraclass consistency to rectify empirical prototypes. Experiments on three popular benchmarks show that the proposed approach outperforms current state-of-the-art methods and achieves balanced performance in both personalization and generalization.

---

PaperID: 565,   

Authors:  Haoran Wang, Zeshen Tang, Yaoru Sun, Fang Wang, Siyu Zhang, Yeming Chen

Title: Guided Cooperation in Hierarchical Reinforcement Learning via Model-Based Rollout

Abstract:
Goal-conditioned hierarchical reinforcement learning (HRL) presents a promising approach for enabling effective exploration in complex, long-horizon reinforcement learning (RL) tasks through temporal abstraction. Empirically, heightened interlevel communication and coordination can induce more stable and robust policy improvement in hierarchical systems. Yet, most existing goal-conditioned HRL algorithms have primarily focused on the subgoal discovery, neglecting interlevel cooperation. Here, we propose a novel goal-conditioned HRL framework named Guided Cooperation via Model-Based Rollout (GCMR; code is available at https://github.com/HaoranWang-TJ/GCMR_ACLG_official), aiming to bridge interlayer information synchronization and cooperation by exploiting forward dynamics. First, the GCMR mitigates the state-transition error within off-policy correction via model-based rollout, thereby enhancing sample efficiency. Second, to prevent disruption by the unseen subgoals and states, lower level Q-function gradients are constrained using a gradient penalty with a model-inferred upper bound, leading to a more stable behavioral policy conducive to effective exploration. Third, we propose a one-step rollout-based planning, using higher level critics to guide the lower level policy. Specifically, we estimate the value of future states of the lower level policy using the higher level critic function, thereby transmitting global task information downward to avoid local pitfalls. These three critical components in GCMR are expected to facilitate interlevel cooperation significantly. Experimental results demonstrate that incorporating the proposed GCMR framework with a disentangled variant of hierarchical reinforcement learning guided by landmarks (HIGL), namely, adjacency constraint and landmark-guided planning (ACLG), yields more stable and robust policy improvement compared with various baselines and significantly outperforms previous state-of-the-art (SOTA) algorithms.

---

PaperID: 566,   

Authors:  Yiheng Lu, Maoguo Gong, Kaiyuan Feng, Jialu Liu, Ziyu Guan, Hao Li

Title: SAAF: Self-Adaptive Attention Factor-Based Taylor-Pruning on Convolutional Neural Networks

Abstract:
Nowadays, pruning techniques have drawn attention to convolutional neural networks (CNNs) for reducing the consumption of computation resources. In particular, the Taylor-based method simplifies the evaluation of importance for each filter as the product of the gradient and weight value of the output features, which outperforms other methods in reductions of parameters and floating point operations (FLOPs). However, the Taylor-based method sacrifices too much accuracy when the overall pruning rate is relatively large compared with other pruning algorithms. In this article, we propose a self-adaptive attention factor (SAAF) to improve the performance of the slimmed model when conventional Taylor-based pruning is utilized under higher pruning. Specifically, SAAF can be calculated by leveraging the remaining ratio of filters at the early pruning stage of the Taylor-based method, and then, some pruned filters can be recovered for improving the accuracy of the slimmed model in terms of SAAF. It means that SAAF can protect filters from being overslimmed to eliminate the degeneration of Taylor-based pruning when the pruning rate is large as well as can compress models apparently across various datasets. We test the efficiency of SAAF on VGG-16 and ResNet-50 with CIFAR-10, Tiny-ImageNet, ImageNet-1000, and remote sensing images. Our method outperforms the traditional Taylor-based method obviously in accuracy, and there are only tiny sacrifices in the reduction of parameters and FLOPs, which is better than other pruning methods.

---

PaperID: 567,   

Authors:  Qiuping Jiang, Jinguang Cheng, Zongwei Wu, Runmin Cong, Radu Timofte

Title: High-Precision Dichotomous Image Segmentation With Frequency and Scale Awareness

Abstract:
Dichotomous image segmentation (DIS) with rich fine-grained details within a single image is a challenging task. Despite the plausible results achieved by deep learning-based methods, most of them fail to segment generic objects when the boundary is cluttered with the background. In fact, the gradual decrease in feature map resolution during the encoding stage and the misleading texture clue may be the main issues. To handle these issues, we devise a novel frequency- and scale-aware deep neural network (FSANet) for high-precision DIS. The core of our proposed FSANet is twofold. First, a multimodality fusion (MF) module that integrates the information in spatial and frequency domains is adopted to enhance the representation capability of image features. Second, a collaborative scale fusion module (CSFM) which deviates from the traditional serial structures is introduced to maintain high resolution during the entire feature encoding stage. In the decoder side, we introduce hierarchical context fusion (HCF) and selective feature fusion (SFF) modules to infer the segmentation results from the output features of the CSFM module. We conduct extensive experiments on several benchmark datasets and compare our proposed method with existing state-of-the-art (SOTA) methods. The experimental results demonstrate that our FSANet achieves superior performance both qualitatively and quantitatively. The code will be made available at https://github.com/chasecjg/FSANet.

---

PaperID: 568,   

Authors:  Songtao Li, Shiqian Wu, Chang Tang, Junchi Zhang, Zushuai Wei

Title: Robust Nonnegative Matrix Factorization With Self-Initiated Multigraph Contrastive Fusion

Abstract:
Graph regularized nonnegative matrix factorization (GNMF) has been widely used in data representation due to its excellent dimensionality reduction. When it comes to clustering polluted data, GNMF inevitably learns inaccurate representations, leading to models that are unusually sensitive to outliers in the data. For example, in a face dataset, obscured by items such as a mask or glasses, there is a high probability that the graph regularization term incorrectly describes the association relationship for that sample, resulting in an incorrect elicitation in the matrix factorization process. In this article, a novel self-initiated unsupervised subspace learning method named robust nonnegative matrix factorization with self-initiated multigraph contrastive fusion (RNMF-SMGF) is proposed. RNMF-SMGF is capable of creating samples with different angles and learning different graph structures based on these different angles in a self-initiated method without changing the original data. In the process of subspace learning guided by graph regularization, these different graph structures are fused into a more accurate graph structure, along with entropy regularization, L_2,1/2 -norm constraints to facilitate the robust learning of the proposed model and the formation of different clusters in the low-dimensional space. To demonstrate the effectiveness of the proposed model in robust clustering, we have conducted extensive experiments on several benchmark datasets and demonstrated the effectiveness of the proposed method. The source code is available at: https://github.com/LstinWh/RNMF-SMGF/.

---

PaperID: 569,   

Authors:  Young Jae Lee, Jaehoon Kim, Youngjoon Park, Mingu Kwak, Seoung Bum Kim

Title: Masked and Inverse Dynamics Modeling for Data-Efficient Reinforcement Learning

Abstract:
In pixel-based deep reinforcement learning (DRL), learning representations of states that change because of an agent’s action or interaction with the environment poses a critical challenge in improving data efficiency. Recent data-efficient DRL studies have integrated DRL with self-supervised learning (SSL) and data augmentation to learn state representations from given interactions. However, some methods have difficulties in explicitly capturing evolving state representations or in selecting data augmentations for appropriate reward signals. Our goal is to explicitly learn the inherent dynamics that change with an agent’s intervention and interaction with the environment. We propose masked and inverse dynamics modeling (MIND), which uses masking augmentation and fewer hyperparameters to learn agent-controllable representations in changing states. Our method is comprised of a self-supervised multitask learning that leverages a transformer architecture, which captures the spatiotemporal information underlying in the highly correlated consecutive frames. MIND uses two tasks to perform self-supervised multitask learning: masked modeling and inverse dynamics modeling. Masked modeling learns the static visual representation required for control in the state, and inverse dynamics modeling learns the rapidly evolving state representation with agent intervention. By integrating inverse dynamics modeling as a complementary component to masked modeling, our method effectively learns evolving state representations. We evaluate our method by using discrete and continuous control environments with limited interactions. MIND outperforms previous methods across benchmarks and significantly improves data efficiency. The code is available at https://github.com/dudwojae/MIND.

---

PaperID: 570,   

Authors:  Chenyuan Feng, Daquan Feng, Guanxin Huang, Zuozhu Liu, Zhenzhong Wang, Xiang-Gen Xia

Title: Robust Privacy-Preserving Recommendation Systems Driven by Multimodal Federated Learning

Abstract:
Recommendation system (RS) is an important information filtering tool in nowadays digital era. With the growing concern on privacy, deploying RSs in a federated learning (FL) manner emerges as a promising solution, which can train a high-quality model on the premise that the server does not directly access sensitive user data. Nevertheless, some malicious clients can deduce user data by analyzing the uploaded model parameters. Even worse, some Byzantine clients can also send contaminated data to the server, causing blockage or failure of model convergence. In addition, most existing researches on federated recommendation algorithms only focus on unimodality learning, ignoring the assistance of multiple modality data to promote recommendation accuracy. Therefore, this article designs an FL-based privacy-preserving multimodal RS framework. To distinguish various modality data, an attention mechanism is introduced, wherein different weight ratios are assigned to various modal features. To further strengthen the privacy, local differential privacy (LDP) and personalized FL strategies are designed to identify malicious clients and bolster the resilience against Byzantine attacks. Finally, two multimodal datasets are established to verify the effectiveness of the proposed algorithm. The superiority of our proposed techniques is confirmed by the simulation results.

---

PaperID: 571,   

Authors:  Xiaolin Zhu, Dongli Wang, Jianxun Li, Rui Su, Qin Wan, Yan Zhou

Title: Dynamical Attention Hypergraph Convolutional Network for Group Activity Recognition

Abstract:
Recently, group activity recognition (GAR) has drawn growing interests in video analysis and computer vision communities. The current models of GAR tasks are often impractical in that they suppose that all interactions between actors are pairwise, which only models and leverages part of the information in real entire interactions. Motivated by this, we design a distinct dynamical attention hypergraph convolutional network framework, referred to as DAHGCN, for precise GAR, modeling the entire interactions and capturing the high-order relationships among involved actors in a real-life scenario. Specifically, to learn complementary feature representations for fine-grained GAR, a multilevel feature descriptor (MLFD) module is proposed. Furthermore, for learning higher order interaction relationships, we construct a DAHGCN to accommodate complex group interactions, which can dynamically change the topology of the hypergraph and learn these key representations by virtue of the “similarity-based shared nearest-neighbor (SSNN) clustering” and “attention mechanisms” on hypergraph. Finally, a multiscale temporal convolution (MSTC) module is utilized to explore various long-range temporal dynamic correlations across different frames. In addition, comprehensive experiments on three commonly used GAR datasets clearly demonstrate that, when compared with the state-of-the-art methods, our proposed method can achieve the most optimal performance.

---

PaperID: 572,   

Authors:  Yi Zhang, Fengyu Tian, Chuan Ma, Miaomiao Li, Hengfu Yang, Zhe Liu, En Zhu, Xinwang Liu

Title: Regularized Instance Weighting Multiview Clustering via Late Fusion Alignment

Abstract:
Multiview clustering has become a prominent research topic in data analysis, with wide-ranging applications across various fields. However, the existing late fusion multiview clustering (LFMVC) methods still exhibit some limitations, including variable importance and contributions and a heightened sensitivity to noise and outliers during the alignment process. To tackle these challenges, we propose a novel regularized instance weighting multiview clustering via late fusion alignment (R-IWLF-MVC), which considers the instance importance from various views, enabling information integration to be more effective. Specifically, we assign each sample an importance attribute to enable the learning process to focus more on the key sample nodes and avoid being influenced by noise or outliers, while laying the groundwork for the fusion of different views. In addition, we continue to employ late fusion alignment to integrate base clustering from various views and introduce a new regularization term with prior knowledge to ensure that the learning process does not deviate too much from the expected results. After that, we design a three-step alternating optimization strategy with proven convergence for the resultant problem. Our proposed approach has been extensively evaluated on multiple real-world datasets, demonstrating its superiority to state-of-the-art methods.

---

PaperID: 573,   

Authors:  Jiao Liu, Xinghua Li, Ximeng Liu, Haiyan Zhang, Yinbin Miao, Robert H. Deng

Title: DefendFL: A Privacy-Preserving Federated Learning Scheme Against Poisoning Attacks

Abstract:
Federated learning (FL) has become a popular mode of learning, allowing model training without the need to share data. Unfortunately, it remains vulnerable to privacy leakage and poisoning attacks, which compromise user data security and degrade model quality. Therefore, numerous privacy-preserving frameworks have been proposed, among which mask-based framework has certain advantages in terms of efficiency and functionality. However, it is more susceptible to poisoning attacks from malicious users, and current works lack practical means to detect such attacks within this framework. To overcome this challenge, we present DefendFL, an efficient, privacy-preserving, and poisoning-detectable mask-based FL scheme. We first leverage collinearity mask to protect users’ gradient privacy. Then, cosine similarity is utilized to detect masked gradients to identify poisonous gradients. Meanwhile, a verification mechanism is designed to detect the mask, ensuring the mask’s validity in aggregation and preventing poisoning attacks by intentionally changing the mask. Finally, we resist poisoning attacks by removing malicious gradients or lowering their weights in aggregation. Through security analysis and experimental evaluation, DefendFL can effectively detect and mitigate poisoning attacks while outperforming existing privacy-preserving detection works in efficiency.

---

PaperID: 574,   

Authors:  Shengjia Wang, Zhiguo Wang, Xi-Le Zhao, Xiaojing Shen

Title: An Unbalanced Optimal Transport-Based Approach for Robust Dictionary Learning

Abstract:
Dictionary learning (DL) is a pivotal task in machine learning and signal processing, involving extracting representative features from a given dataset. However, conventional DL models are known to be highly sensitive to outliers. To circumvent this issue, we introduce a new and robust DL model based on unbalanced optimal transport (UOT). Compared to DL models based on conventional robust distances and the Wasserstein distance, our model not only captures and leverages the structural information within the data but also demonstrates strong resilience to outliers. By employing the structure of the proposed robust DL model, we develop a novel hybrid block coordinate descent (BCD) algorithm. The proposed algorithm maintains computational tractability by exploiting special block structures of the subproblems. In addition, we establish the convergence of our algorithm without the Lipschitz smooth condition. Through extensive experimentation, we validate our theoretical results and demonstrate the effectiveness of the proposed method on synthetic data, MNIST data, Olivetti faces dataset, and hyperspectral images (HSIs) datasets.

---

PaperID: 575,   

Authors:  Sanket R. Jantre, Shrijita Bhattacharya, Tapabrata Maiti

Title: Spike-and-Slab Shrinkage Priors for Structurally Sparse Bayesian Neural Networks

Abstract:
Network complexity and computational efficiency have become increasingly significant aspects of deep learning. Sparse deep learning addresses these challenges by recovering a sparse representation of the underlying target function by reducing heavily overparameterized deep neural networks. Specifically, deep neural architectures compressed via structured sparsity (e.g., node sparsity) provide low-latency inference, higher data throughput, and reduced energy consumption. In this article, we explore two well-established shrinkage techniques, Lasso and Horseshoe, for model compression in Bayesian neural networks (BNNs). To this end, we propose structurally sparse BNNs, which systematically prune excessive nodes with the following: 1) spike-and-slab group Lasso (SS-GL) and 2) SS group Horseshoe (SS-GHS) priors, and develop computationally tractable variational inference, including continuous relaxation of Bernoulli variables. We establish the contraction rates of the variational posterior of our proposed models as a function of the network topology, layerwise node cardinalities, and bounds on the network weights. We empirically demonstrate the competitive performance of our models compared with the baseline models in prediction accuracy, model compression, and inference latency.

---

PaperID: 576,   

Authors:  Zhengdao Shao, Liansheng Zhuang, Yihong Huang, Houqiang Li, Shafei Wang

Title: Purified Policy Space Response Oracles for Symmetric Zero-Sum Games

Abstract:
Policy space response oracles (PSRO) is a promising tool to find an approximate Nash equilibrium (NE) in a two-player zero-sum game. It solves the equilibrium by iteratively expanding a small-scale meta-game formed by a restricted strategy population consisting of historical approximate best responses of the meta-games. However, since these best responses have a strong correlation with each other, existing PSRO and its variants often have the slow diversity growth of the strategy population, and thus suffer from poor exploration efficiency and slow convergence rate. To address this problem, this article proposes Purified PSRO, which deliberately maintains a pure strategy population formed by pure strategy bases of approximate best responses. A novel module namely non-best response suppression (NBRS) is introduced to calculate a pure strategy base with better orthogonality to expand the strategy population at each epoch. In this way, Purified PSRO can quickly increase the diversity of the strategy population, thus greatly enhance the efficiency of exploration. Theoretically, we prove the convergence of Purified PSRO. Moreover, we introduce an early stop module to reduce computation cost, and give the upper bound of the exploitability when the algorithm stops early. Extensive experiments on random games of skill (RGoS) and real-world meta-games show that Purified PSRO can consistently outperform existing SOTA methods, sometimes with a large margin.

---

PaperID: 577,   

Authors:  Xiyou Fu, Xi Zhou, Yawen Fu, Pan Liu, Sen Jia

Title: Progressive Semantic Enhancement Network for Hyperspectral and LiDAR Classification

Abstract:
The joint classification of hyperspectral image (HSI) and light detection and ranging (LiDAR) data is gaining attention for its improved classification accuracy. However, effectively integrating the rich spectral information of HSI and the elevation features of LiDAR has remained a challenge in multimodal fusion. This article proposes a novel approach called progressive semantic enhancement network (PSENet) for hyperspectral and LiDAR classification based on a progressive joint spatial-spectral attention mechanism. PSENet mainly comprises two modules: the spatial grouping constraint (SAGC) module and the spectral weighting constraint (SEWC) module. The SAGC module extracts multiscale features in the spatial domain, while the SEWC module focuses on enhancing semantic features in spectral dimension. By gradually utilizing spatial and spectral constraint modules to progressively enhance feature extraction, PSENet integrates affluent information for a more refined classification of ground objects. Based on experimental results, it has been demonstrated that PSENet outperforms several most advanced methods on three datasets. The SAGC and SEWC modules proposed in PSENet enable the effective integration of the spatial, spectral, and elevation information from HSI and LiDAR, providing a promising way to perform classification more accurately. The source codes of this work will be publicly available at http://szu-hsilab.com/.

---

PaperID: 578,   

Authors:  Ruohuan Fang, Guansong Pang, Wenjun Miao, Xiao Bai, Jin Zheng, Xin Ning

Title: Unsupervised Recognition of Unknown Objects for Open-World Object Detection

Abstract:
Open-world object detection (OWOD) extends object detection problem to a realistic and dynamic scenario, where a detection model is required to be capable of detecting both known and unknown objects and incrementally learning newly introduced knowledge. Current OWOD models detect the unknowns that exhibit similar features to the known objects, but they suffer from a severe label bias problem, i.e., they tend to detect all regions (including unknown object regions) that are dissimilar to the known objects as part of the background. To eliminate the label bias, this article proposes a novel module, namely reconstruction error-based Weibull (REW) model, that learns an unsupervised discriminative model for recognizing true unknown objects based on prior knowledge of object occurrence frequency via Weibull modeling. The resulting model can be further refined by another module of our method, called REW-enhanced object localization network (ROLNet), which iteratively extends pseudo-unknown objects to the unlabeled regions. Experimental results show that our method 1) significantly outperforms the prior SOTA in detecting unknown objects while maintaining competitive performance of detecting known object classes on the MS COCO dataset and 2) achieves better generalization ability on the LVIS and Objects365 datasets. Code is available at https://github.com/frh23333/mepu-owod

---

PaperID: 579,   

Authors:  Wenmian Yang, Lizhi Cheng, Mohamed Ragab, Min Wu, Sinno Jialin Pan, Zhenghua Chen

Title: A Virtual-Label-Based Hierarchical Domain Adaptation Method for Time-Series Classification

Abstract:
Unsupervised domain adaptation (UDA) is becoming a prominent solution for the domain-shift problem in many time-series classification tasks. With sequence properties, time-series data contain both local and sequential features, and the domain shift exists in both features. However, conventional UDA methods usually cannot distinguish those two features but mix them into one variable for direct alignment, which harms the performance. To address this problem, we propose a novel virtual-label-based hierarchical domain adaptation (VLH-DA) approach for time-series classification. Specifically, we first slice the original time-series data and introduce virtual labels to represent the type of each slice (called local patterns). With the help of virtual labels, we decompose the end-to-end (i.e., signal to time-series label) time-series task into two parts, i.e., signal sequence to local pattern sequence and local pattern sequence to time-series label. By decomposing the complex time-series UDA task into two simpler subtasks, the local features and sequential features can be aligned separately, making it easier to mitigate distribution discrepancies. Experiments on four public time-series datasets demonstrate that our VLH-DA outperforms all state-of-the-art (SOTA) methods.

---

PaperID: 580,   

Authors:  Lei Zhu, Xinliang Zhang, Hangzhou He, Qian Chen, Sha Li, Shuang Zeng, Yibao Zhang, Qiushi Ren, Yanye Lu

Title: Branches Mutual Promotion for End-to-End Weakly Supervised Semantic Segmentation

Abstract:
End-to-end weakly supervised semantic segmentation (E2E-WSSS) aims at optimizing a segmentation model in a single-stage training process based on only image annotations. Existing methods adopt an online-trained classification branch to provide pseudo annotations for supervising the segmentation branch. However, this strategy makes the classification branch dominate the whole concurrent training process, hindering these two branches from assisting each other. In our work, we treat these two branches equally by viewing them as diverse ways to generate the segmentation map, and add interactions on both their supervision and operation to achieve mutual promotion. For this purpose, a bidirectional supervision mechanism is elaborated to force the consistency between the outputs of these two branches. Thus, the segmentation branch can also give feedback to the classification branch to enhance the quality of localization seeds. Moreover, our method also designs interaction operations between these two branches to exchange their knowledge to assist each other. Experiments indicate our work outperforms existing end-to-end weakly supervised segmentation methods. Codes are available at https://github.com/zh460045050/BMP-WSSS.

---

PaperID: 581,   

Authors:  Mengke Lian, Zhenyuan Guo, Xiaoxuan Wang, Shiping Wen, Tingwen Huang

Title: Distributed Algorithms for Linear Equations Over General Directed Networks

Abstract:
This article deals with linear equations of the form Ax = b . By reformulating the original problem as an unconstrained optimization problem, we first provide a gradient-based distributed continuous-time algorithm over weight-balanced directed graphs, in which each agent only knows partial rows of the augmented matrix (A\; b) . The algorithm is also applicable to time-varying networks. By estimating a right-eigenvector corresponding to 0 eigenvalue of the out-Laplacian matrix in finite time, we further propose a distributed algorithm over weight-unbalanced communication networks. It is proved that each solution of the designed algorithms converges exponentially to an equilibrium point. Moreover, the convergence rate is given out clearly. For linear equations without solution, these algorithms are used to obtain a least-squares solution in approximate sense. These theoretical results are illustrated by four numerical examples.

---

PaperID: 582,   

Authors:  Kang Liu, Zhenhua Huang, Chang-Dong Wang, Beibei Gao, Yunwen Chen

Title: Fine-Grained Learning Behavior-Oriented Knowledge Distillation for Graph Neural Networks

Abstract:
Knowledge distillation (KD), as an effective compression technology, is used to reduce the resource consumption of graph neural networks (GNNs) and facilitate their deployment on resource-constrained devices. Numerous studies exist on GNN distillation, and however, the impacts of knowledge complexity and differences in learning behavior between teachers and students on distillation efficiency remain underexplored. We propose a KD method for fine-grained learning behavior (FLB), comprising two main components: feature knowledge decoupling (FKD) and teacher learning behavior guidance (TLBG). Specifically, FKD decouples the intermediate-layer features of the student network into two types: teacher-related features (TRFs) and downstream features (DFs), enhancing knowledge comprehension and learning efficiency by guiding the student to simultaneously focus on these features. TLBG maps the teacher model’s learning behaviors to provide reliable guidance for correcting deviations in student learning. Extensive experiments across eight datasets and 12 baseline frameworks demonstrate that FLB significantly enhances the performance and robustness of student GNNs within the original framework.

---

PaperID: 583,   

Authors:  Simiao Lai, Chang Liu, Dong Wang, Huchuan Lu

Title: Refocus the Attention for Parameter-Efficient Thermal Infrared Object Tracking

Abstract:
Introducing deep trackers to thermal infrared (TIR) tracking is hampered by the scarcity of large training datasets. To alleviate the predicament, a common approach is full fine-tuning (FFT) based on pretrained RGB parameters. Nevertheless, due to its inefficient training pattern and representation collapse risk, some parameter-efficient fine-tuning (PEFT) alternatives have been promoted recently. However, the existing PEFT algorithms typically follow a bottom-up way, where their attention solely relies on the input and lacks the capability of task-guided top-down attention, which provides the task-relevant representation such as the human visual perception system. In this article, we introduce ReFocus, a new PEFT method that adapts the pretrained RGB foundation tracking model to the downstream TIR tracking task through the guidance of high-level task-specific signals in a top-down attention manner. By freezing the entire foundation model and only training query-guided feature selection and top-down blocks, ReFocus achieves state-of-the-art (SOTA) TIR tracking performance while keeping training efficiency. Extensive experiments on five TIR tracking benchmarks demonstrate that ReFocus significantly improves the performance of the foundation tracker. Besides, further ablation studies show the effectiveness and flexible adaptability of the proposed method to lighter foundation models and different tracking frameworks. Compared to FFT and other bottom-up PEFT paradigms, such as head probe, low-rank adaptation (LoRA), and adapter, our method achieves comparable or superior performance with fewer training parameters and reveals the advantage of learning stability.

---

PaperID: 584,   

Authors:  Zhengrong Xiang, Pingchuan Li, Wencheng Zou, Choon Ki Ahn

Title: Data-Based Optimal Switching and Control With Admissibility Guaranteed Q-Learning

Abstract:
This article addresses the data-based optimal switching and control codesign for discrete-time nonlinear switched systems via a two-stage approximate dynamic programming (ADP) algorithm. Through offline policy improvement and policy evaluation, the proposed algorithm iteratively determines the optimal hybrid control policy using system input/output data. Moreover, a strict proof of the convergence is given for the two-stage ADP algorithm. Admissibility, an essential property of the hybrid control policy must be ensured for practical application. To this end, the properties of the hybrid control policies are analyzed and an admissibility criterion is obtained. To realize the proposed Q-learning algorithm, an actor-critic neural network (NN) structure that employs multiple NNs to approximate the Q-functions and control policies for different subsystems is adopted. By applying the proposed admissibility criterion, the obtained hybrid control policy is guaranteed to be admissible. Finally, two numerical simulations verify the effectiveness of the proposed algorithm.

---

PaperID: 585,   

Authors:  Xun Shen, Tinghui Ouyang, Kazumune Hashimoto, Yuhu Wu

Title: Sample-Based Continuous Approximate Method for Constructing Interval Neural Network

Abstract:
In safety-critical engineering applications, such as robust prediction against adversarial noise, it is necessary to quantify neural networks’ uncertainty. Interval neural networks (INNs) are effective models for uncertainty quantification, giving an interval of predictions instead of a single value for a corresponding input. This article formulates the problem of training an INN as a chance-constrained optimization problem. The optimal solution of the formulated chance-constrained optimization naturally forms an INN that gives the tightest interval of predictions with a required confidence level. Since the chance-constrained optimization problem is intractable, a sample-based continuous approximate method is used to obtain approximate solutions to the chance-constrained optimization problem. We prove the uniform convergence of the approximation, showing that it gives the optimal INN consistently with the original ones. Additionally, we investigate the reliability of the approximation with finite samples, giving the probability bound for violation with finite samples. Through a numerical example and an application case study of anomaly detection in wind power data, we evaluate the effectiveness of the proposed INN against existing approaches, including Bayesian neural networks, highlighting its capability to significantly improve the performance of applying INNs for regression and unsupervised anomaly detection.

---

PaperID: 586,   

Authors:  Yamin Sepehri, Pedram Pad, Ahmet Caner Yüzügüler, Pascal Frossard, L. Andrea Dunbar

Title: Hierarchical Training of Deep Neural Networks Using Early Exiting

Abstract:
Deep neural networks (DNNs) provide state-of-the-art accuracy for vision tasks, but they require significant resources for training. Thus, they are trained on cloud servers far from the edge devices that acquire the data. This issue increases communication cost, runtime, and privacy concerns. In this study, a novel hierarchical training method for DNNs is proposed that uses early exits in a divided architecture between edge and cloud workers to reduce the communication cost, training runtime, and privacy concerns. The method proposes a brand-new use case for early exits to separate the backward pass of neural networks between the edge and the cloud during the training phase. We address the issues of most available methods that, due to the sequential nature of the training phase, cannot train the levels of hierarchy simultaneously or they do it with the cost of compromising privacy. In contrast, our method can use both edge and cloud workers simultaneously, does not share the raw input data with the cloud, and does not require communication during the backward pass. Several simulations and on-device experiments for different neural network architectures demonstrate the effectiveness of this method. It is shown that the proposed method reduces the training runtime for VGG-16 and ResNet-18 architectures by 29% and 61% in CIFAR-10 classification and by 25% and 81% in Tiny ImageNet classification, respectively, when the communication with the cloud is done over a low bit rate channel. This gain in the runtime is achieved, while the accuracy drop is negligible. This method is advantageous for online learning of high-accuracy DNNs on sensor-holding low-resource devices such as mobile phones or robots as a part of an edge–cloud system, making them more flexible in facing new tasks and classes of data.

---

PaperID: 587,   

Authors:  Pengfei Zhu, Jialu Li, Zhe Dong, Qinghua Hu, Xiao Wang, Qilong Wang

Title: CCP-GNN: Competitive Covariance Pooling for Improving Graph Neural Networks

Abstract:
Graph neural networks (GNNs) have advanced graph classification tasks, where a global pooling to generate graph representations by summarizing node features plays a critical role in the final performance. Most of the existing GNNs are built with a global average pooling (GAP) or its variants, which however, take no full consideration of node specificity while neglecting rich statistics inherent in node features, limiting classification performance of GNNs. Therefore, this article proposes a novel competitive covariance pooling (CCP) based on observation of graph structures, i.e., graphs generally can be identified by a (small) key part of nodes. To this end, our CCP generates node-level second-order representations to explore rich statistics inherent in node features, which are fed to a competitive-based attention module for effectively discovering key nodes through learning node weights. Subsequently, our CCP aggregates node-level second-order representations in conjunction with node weights by summation to produce a covariance representation for each graph, while an iterative matrix normalization is introduced to consider geometry of covariances. Note that our CCP can be flexibly integrated with various GNNs (namely CCP-GNN) to improve the performance of graph classification with little computational cost. The experimental results on seven graph-level benchmarks show that our CCP-GNN is superior or competitive to state-of-the-arts. Our code is available at https://github.com/Jillian555/CCP-GNN.

---

PaperID: 588,   

Authors:  Kailing Guo, Zhenquan Lin, Canyang Chen, Xiaofen Xing, Fang Liu, Xiangmin Xu

Title: Compact Model Training by Low-Rank Projection With Energy Transfer

Abstract:
Low-rankness plays an important role in traditional machine learning but is not so popular in deep learning. Most previous low-rank network compression methods compress networks by approximating pretrained models and retraining. However, the optimal solution in the Euclidean space may be quite different from the one with low-rank constraint. A well-pretrained model is not a good initialization for the model with low-rank constraints. Thus, the performance of a low-rank compressed network degrades significantly. Compared with other network compression methods such as pruning, low-rank methods attract less attention in recent years. In this article, we devise a new training method, low-rank projection with energy transfer (LRPET), that trains low-rank compressed networks from scratch and achieves competitive performance. We propose to alternately perform stochastic gradient descent training and projection of each weight matrix onto the corresponding low-rank manifold. Compared to retraining on the compact model, this enables full utilization of model capacity since solution space is relaxed back to Euclidean space after projection. The matrix energy (the sum of squares of singular values) reduction caused by projection is compensated by energy transfer. We uniformly transfer the energy of the pruned singular values to the remaining ones. We theoretically show that energy transfer eases the trend of gradient vanishing caused by projection. In modern networks, a batch normalization (BN) layer can be merged into the previous convolution layer for inference, thereby influencing the optimal low-rank approximation (LRA) of the previous layer. We propose BN rectification to cut off its effect on the optimal LRA, which further improves the performance. Comprehensive experiments on CIFAR-10 and ImageNet have justified that our method is superior to other low-rank compression methods and also outperforms recent state-of-the-art pruning methods. For object detection and semantic segmentation, our method still achieves good compression results. In addition, we combine LRPET with quantization and hashing methods and achieve even better compression than the original single method. We further apply it in Transformer-based models to demonstrate its transferability. Our code is available at https://github.com/BZQLin/LRPET.

---

PaperID: 589,   

Authors:  Quan-Yong Fan, Meiying Cai, Bin Xu

Title: An Improved Prioritized DDPG Based on Fractional-Order Learning Scheme

Abstract:
Although deep deterministic policy gradient (DDPG) algorithm gets widespread attention as a result of its powerful functionality and applicability for large-scale continuous control, it cannot be denied that DDPG has problems such as low sample utilization efficiency and insufficient exploration. Therefore, an improved DDPG is presented to overcome these challenges in this article. Firstly, an optimizer based on fractional gradient is introduced into the algorithm network, which is conductive to increase the speed and accuracy of training convergence. On this basis, high-value experience replay based on weight-changed priority is proposed to improve sample utilization efficiency, and aiming to have a stronger exploration of the environment, an optimized exploration strategy for boundary action space is adopted. Finally, our proposed method is tested through the experiments of gym and pybullet platform. According to the results, our method speeds up the learning process, obtains higher average rewards in comparison with other algorithms.

---

PaperID: 590,   

Authors:  Bosen Lian, Wenqian Xue, Frank L. Lewis, Ali Davoudi

Title: Inverse Value Iteration and Q-Learning: Algorithms, Stability, and Robustness

Abstract:
This article proposes a data-driven model-free inverse Q-learning algorithm for continuous-time linear quadratic regulators (LQRs). Using an agent’s trajectories of states and optimal control inputs, the algorithm reconstructs its cost function that captures the same trajectories. This article first poses a model-based inverse value iteration scheme using the agent’s system dynamics. Then, an online model-free inverse Q-learning algorithm is developed to recover the agent’s cost function only using the demonstrated trajectories. It is more efficient than the existing inverse reinforcement learning (RL) algorithms as it avoids the repetitive RL in inner loops. The proposed algorithms do not need initial stabilizing control policies and solve for unbiased solutions. The proposed algorithm’s asymptotic stability, convergence, and robustness are guaranteed. Theoretical analysis and simulation examples show the effectiveness and advantages of the proposed algorithms.

---

PaperID: 591,   

Authors:  Xiaojing Ge, Rui Yan, Xiangbo Shu, Keke Chen, Wei Tian, Guo-Sen Xie

Title: Coarse-Fine Nested Network for Weakly Supervised Group Activity Recognition

Abstract:
Weakly supervised group activity recognition (WSGAR) aims at identifying the overall behavior of multiple persons without any fine-grained supervision information (including individual position and action label). Traditional methods usually adopt a person-to-whole way: detect persons via off-the-shelf detectors, obtain person-level features, and integrate into the group-level features for training the classifier. However, these methods are unflexible due to serious reliance on the quality of detectors. To get rid of the detector, recent works learn several prototype tokens from noisy grid features with learnable weights directly, which treat all the local visual information equally and bring in redundant and ambiguous information to some extent. To this end, we propose a novel coarse-fine nested network (CFNN) to coarsely localize the key visual patches of activity and further finely learn the local features, as well as the global features. Specifically, we design a nested interactor (NI) to progressively model the spatiotemporal interactions of the learnable global token. According to the cue of spatial interaction in NI, we localize several key visual patches via a new coarse-grained spatial localizer (CSL). Then, we finally encode these localized visual patches with the help of global spatiotemporal dependency via a new fine-grained spatiotemporal selector (FSS). Extensive experiments on Volleyball and NBA datasets demonstrate the effectiveness of the proposed CFNN compared with the existing competitive methods. Code is available at: https://github.com/gexiaojingshelby/CFNN

---

PaperID: 592,   

Authors:  Xing Zhao, Haoran Liang, Ronghua Liang

Title: Position Fusing and Refining for Clear Salient Object Detection

Abstract:
Multilevel feature fusion plays a pivotal role in salient object detection (SOD). High-level features present rich semantic information but lack object position information, whereas low-level features contain object position information but are mixed with noises such as backgrounds. Appropriately addressing the gap between low- and high-level features is important in SOD. We first propose a global position embedding attention (GPEA) module to minimize the discrepancy between multilevel features in this article. We extract the position information by utilizing the semantic information at high-level features to resist noises at low-level features. Object refine attention (ORA) module is introduced to refine features used to predict saliency maps further without any additional supervision and heighten discriminative regions near the salient object, such as boundaries. Moreover, we find that the saliency maps generated by the previous methods contain some blurry regions, and we design a pixel value (PV) loss to help the model generate saliency maps with improved clarity. Experimental results on five commonly used SOD datasets demonstrated that the proposed method is effective and outperforms the state-of-the-art approaches on multiple metrics.

---

PaperID: 593,   

Authors:  Wangli Hao, He Guan, Zhaoxiang Zhang

Title: VAG: A Uniform Model for Cross-Modal Visual-Audio Mutual Generation

Abstract:
Considering both audio and visual modalities is helpful for understanding a video. In the face of harsh environmental interference or signal packet loss, automatically compensating for audio and vision is a challenging task. We propose a dynamic cross-modal visual-audio mutual generation model (VAMG), which includes audio to visual conversion, visual to audio conversion, audio self-generation, and visual self-generation. VAMG jointly optimizes modal reconstruction and adversarial constraints, effectively solving the problems of structural alignment and signal compensation in incomplete videos. We conducted an instrument-oriented and pose-oriented cross-modal audio-visual mutual generation experiment on the sub-University of Rochester Musical Performance dataset to verify the effectiveness of the model.

---

PaperID: 594,   

Authors:  Liyan Zhang, Guodong Du, Fan Liu, Huawei Tu, Xiangbo Shu

Title: Global-Local Multiple Granularity Learning for Cross-Modality Visible-Infrared Person Reidentification

Abstract:
Cross-modality visible–infrared person reidentification (VI-ReID), which aims to retrieve pedestrian images captured by both visible and infrared cameras, is a challenging but essential task for smart surveillance systems. The huge barrier between visible and infrared images has led to the large cross-modality discrepancy and intraclass variations. Most existing VI-ReID methods tend to learn discriminative modality-sharable features based on either global or part-based representations, lacking effective optimization objectives. In this article, we propose a novel global–local multichannel (GLMC) network for VI-ReID, which can learn multigranularity representations based on both global and local features. The coarse- and fine-grained information can complement each other to form a more discriminative feature descriptor. Besides, we also propose a novel center loss function that aims to simultaneously improve the intraclass cross-modality similarity and enlarge the interclass discrepancy to explicitly handle the cross-modality discrepancy issue and avoid the model fluctuating problem. Experimental results on two public datasets have demonstrated the superiority of the proposed method compared with state-of-the-art approaches in terms of effectiveness.

---

PaperID: 595,   

Authors:  Seongmin Lee, Suwoong Heo, Sanghoon Lee

Title: DMESH: A Structure-Preserving Diffusion Model for 3-D Mesh Denoising

Abstract:
Denoising diffusion models have shown a powerful capacity for generating high-quality image samples by progressively removing noise. Inspired by this, we present a diffusion-based mesh denoiser that progressively removes noise from mesh. In general, the iterative algorithm of diffusion models attempts to manipulate the overall structure and fine details of target meshes simultaneously. For this reason, it is difficult to apply the diffusion process to a mesh denoising task that removes artifacts while maintaining a structure. To address this, we formulate a structure-preserving diffusion process. Instead of diffusing the mesh vertices to be distributed as zero-centered isotopic Gaussian distribution, we diffuse each vertex into a specific noise distribution, in which the entire structure can be preserved. In addition, we propose a topology-agnostic mesh diffusion model by projecting the vertex into multiple 2-D viewpoints to efficiently learn the diffusion using a deep network. This enables the proposed method to learn the diffusion of arbitrary meshes that have an irregular topology. Finally, the denoised mesh can be obtained via refinement based on 2-D projections obtained from reverse diffusion. Through extensive experiments, we demonstrate that our method outperforms the state-of-the-art mesh denoising methods in both quantitative and qualitative evaluations.

---

PaperID: 596,   

Authors:  Ruicong Zhi, Yicheng Meng, Junyi Hou, Jun Wan

Title: Dual Balanced Class-Incremental Learning With im-Softmax and Angular Rectification

Abstract:
Owing to the superior performances, exemplar-based methods with knowledge distillation (KD) are widely applied in class incremental learning (CIL). However, it suffers from two drawbacks: 1) data imbalance between the old/learned and new classes causes the bias of the new classifier toward the head/new classes and 2) deep neural networks (DNNs) suffer from distribution drift when learning sequence tasks, which results in narrowed feature space and deficient representation of old tasks. For the first problem, we analyze the insufficiency of softmax loss when dealing with the problem of data imbalance in theory and then propose the imbalance softmax (im-softmax) loss to relieve the imbalanced data learning, where we re-scale the output logits to underfit the head/new classes. For another problem, we calibrate the feature space by incremental-adaptive angular margin (IAAM) loss. The new classes form a complete distribution in feature space yet the old are squeezed. To recover the old feature space, we first compute the included angle of normalized features and normalized anchor prototypes, and use the angle distribution to represent the class distribution, then we replenish the old distribution with the deviation from the new. Each anchor prototype is predefined as a learnable vector for a designated class. The proposed im-softmax reduces the bias in the linear classification layer. IAAM rectifies the representation learning, reduces the intra-class distance, and enlarges the inter-class margin. Finally, we seamlessly combine the im-softmax and IAAM in an end-to-end training framework, called the dual balanced class incremental learning (DBL), for further improvements. Experiments demonstrate the proposed method achieves state-of-the-art (SOTA) performances on several benchmarks, such as CIFAR10, CIFAR100, Tiny-ImageNet, and ImageNet-100.

---

PaperID: 597,   

Authors:  Sina Ghiassian, Banafsheh Rafiee, Richard S. Sutton

Title: Off-Policy Prediction Learning: An Empirical Study of Online Algorithms

Abstract:
Off-policy prediction—learning the value function for one policy from data generated while following another policy—is one of the most challenging problems in reinforcement learning. This article makes two main contributions: 1) it empirically studies 11 off-policy prediction learning algorithms with emphasis on their sensitivity to parameters, learning speed, and asymptotic error and 2) based on the empirical results, it proposes two step-size adaptation methods called Step-size Ratchet and Soft Step-size Ratchet that help the algorithm with the lowest error from the experimental study learn faster. Many off-policy prediction learning algorithms have been proposed in the past decade, but it remains unclear which algorithms learn faster than others. In this article, we empirically compare 11 off-policy prediction learning algorithms with linear function approximation on three small tasks: the Collision task, the Rooms task, and the High Variance Rooms task. The Collision task is a small off-policy problem analogous to that of an autonomous car trying to predict whether it will collide with an obstacle. The Rooms and High Variance Rooms tasks are designed such that learning fast in them is challenging. In the Rooms task, the product of importance sampling ratios can be as large as 2^14 . To control the high variance caused by the product of the importance sampling ratios, step size should be set small, which, in turn, slows down learning. The High Variance Rooms task is more extreme in that the product of the ratios can become as large as 2^14 × 25 . The algorithms considered are Off-policy TD( \lambda ), five Gradient-TD algorithms, two Emphatic-TD algorithms, Vtrace, and variants of Tree Backup and ABQ that are applicable to the prediction setting. We found that the algorithms’ performance is highly affected by the variance induced by the importance sampling ratios. Tree Backup( \lambda ), Vtrace( \lambda ), and ABTD( \zeta ) are not affected by the high variance as much as other algorithms, but they restrict the effective bootstrapping parameter in a way that is too limiting for tasks where high variance is not present. We observed that Emphatic TD( \lambda ) tends to have lower asymptotic error than other algorithms but might learn more slowly in some cases. Based on the empirical results, we propose two step-size adaptation algorithms, which we collectively refer to as the Ratchet algorithms, with the same underlying idea: keep the step-size parameter as large as possible and ratchet it down only when necessary to avoid overshoot. We show that the Ratchet algorithms are effective by comparing them with other popular step-size adaptation algorithms, such as the Adam optimizer.

---

PaperID: 598,   

Authors:  Boyue Wang, Yujian Ma, Xiaoyan Li, Junbin Gao, Yongli Hu, Baocai Yin

Title: Bridging the Cross-Modality Semantic Gap in Visual Question Answering

Abstract:
The objective of visual question answering (VQA) is to adequately comprehend a question and identify relevant contents in an image that can provide an answer. Existing approaches in VQA often combine visual and question features directly to create a unified cross-modality representation for answer inference. However, this kind of approach fails to bridge the semantic gap between visual and text modalities, resulting in a lack of alignment in cross-modality semantics and the inability to match key visual content accurately. In this article, we propose a model called the caption bridge-based cross-modality alignment and contrastive learning model (CBAC) to address the issue. The CBAC model aims to reduce the semantic gap between different modalities. It consists of a caption-based cross-modality alignment module and a visual-caption (V-C) contrastive learning module. By utilizing an auxiliary caption that shares the same modality as the question and has closer semantic associations with the visual, we are able to effectively reduce the semantic gap by separately matching the caption with both the question and the visual to generate pre-alignment features for each, which are then used in the subsequent fusion process. We also leverage the fact that V-C pairs exhibit stronger semantic connections compared to question-visual (Q-V) pairs to employ a contrastive learning mechanism on visual and caption pairs to further enhance the semantic alignment capabilities of single-modality encoders. Extensive experiments conducted on three benchmark datasets demonstrate that the proposed model outperforms previous state-of-the-art VQA models. Additionally, ablation experiments confirm the effectiveness of each module in our model. Furthermore, we conduct a qualitative analysis by visualizing the attention matrices to assess the reasoning reliability of the proposed model.

---

PaperID: 599,   

Authors:  Kewei Wu, Wenjie Luo, Zhao Xie, Dan Guo, Zhao Zhang, Richang Hong

Title: Ensemble Prototype Network For Weakly Supervised Temporal Action Localization

Abstract:
Weakly supervised temporal action localization (TAL) aims to localize the action instances in untrimmed videos using only video-level action labels. Without snippet-level labels, this task should be hard to distinguish all snippets with accurate action/background categories. The main difficulties are the large variations brought by the unconstraint background snippets and multiple subactions in action snippets. The existing prototype model focuses on describing snippets by covering them with clusters (defined as prototypes). In this work, we argue that the clustered prototype covering snippets with simple variations still suffers from the misclassification of the snippets with large variations. We propose an ensemble prototype network (EPNet), which ensembles prototypes learned with consensus-aware clustering. The network stacks a consensus prototype learning (CPL) module and an ensemble snippet weight learning (ESWL) module as one stage and extends one stage to multiple stages in an ensemble learning way. The CPL module learns the consensus matrix by estimating the similarity of clustering labels between two successive clustering generations. The consensus matrix optimizes the clustering to learn consensus prototypes, which can predict the snippets with consensus labels. The ESWL module estimates the weights of the misclassified snippets using the snippet-level loss. The weights update the posterior probabilities of the snippets in the clustering to learn prototypes in the next stage. We use multiple stages to learn multiple prototypes, which can cover the snippets with large variations for accurate snippet classification. Extensive experiments show that our method achieves the state-of-the-art weakly supervised TAL methods on two benchmark datasets, that is, THUMOS’14, ActivityNet v1.2, and ActivityNet v1.3 datasets.

---

PaperID: 600,   

Authors:  Zhengming Chen, Jie Qiao, Feng Xie, Ruichu Cai, Zhifeng Hao, Keli Zhang

Title: Testing Conditional Independence Between Latent Variables by Independence Residuals

Abstract:
Conditional independence (CI) testing is an important problem, especially in causal discovery. Most testing methods assume that all variables are fully observable and then test the CI among the observed data. Such an assumption is often untenable beyond applications dealing with, e.g., psychological analysis about the mental health status and medical diagnosing (researchers need to consider the existence of latent variables in these scenarios); and typically adopted latent CI test schemes mainly suffer from robust or efficient issues. Accordingly, this article investigates the problem of testing CI between latent variables. To this end, we offer an auxiliary regression-based CI (AReCI) test by taking the measured variable as the surrogate variable of the latent variables to conduct the regression over the latent variables under the linear causal models, in which each latent variable has some certain measured variables. Specifically, given a pair of latent variables L_X and L_Y , and a corresponding latent variable set \mathcal L_O , L_X \mathrel \perp \mspace -10mu\perp L_Y | \mathcal L_O holds if and only if A_\L_X\-\omega _1^\intercal A^\prime _\\mathcal L_O\ and A_\L_Y\-\omega _2^\intercal A^\prime \prime _\\mathcal L_O\ are statistically independent, where A^\prime and A^\prime \prime are the two disjoint subset of the measured variable for the corresponding latent variables, A^\prime _\\mathcal L_O\ \cap A^\prime \prime _\\mathcal L_O\ =\emptyset , and \omega _1 is a parameter vector characterized from the cross covariance between A_\L_X\ and A^\prime _\\mathcal L_O\ , and \omega _2 is a parameter vector characterized from the cross covariance between A_\L_Y\ and A^\prime \prime _\\mathcal L_O\ . We theoretically show that the AReCI test is capable of addressing both Gaussian and non-Gaussian data. In addition, we find that the well-known partial correlation test can be seen as a special case of the AReCI test. Finally, we devise a causal discovery method by using the AReCI test as the CI test. The experimental results on synthetic and real-world data illustrate the effectiveness of our method.

---

PaperID: 601,   

Authors:  Cong Xu, Wei Zhang, Jun Wang, Min Yang

Title: Understanding Adversarial Robustness From Feature Maps of Convolutional Layers

Abstract:
The adversarial robustness of a neural network mainly relies on two factors: model capacity and antiperturbation ability. In this article, we study the antiperturbation ability of the network from the feature maps of convolutional layers. Our theoretical analysis discovers that larger convolutional feature maps before average pooling can contribute to better resistance to perturbations, but the conclusion is not true for max pooling. It brings new inspiration to the design of robust neural networks and urges us to apply these findings to improve existing architectures. The proposed modifications are very simple and only require upsampling the inputs or slightly modifying the stride configurations of downsampling operators. We verify our approaches on several benchmark neural network architectures, including AlexNet, VGG, RestNet18, and PreActResNet18. Nontrivial improvements in terms of both natural accuracy and adversarial robustness can be achieved under various attack and defense mechanisms. The code is available at https://github.com/MTandHJ/rcm.

---

PaperID: 602,   

Authors:  Qi Yao, Tengda Wei, Ping Lin, Linshan Wang

Title: Finite-Time Boundedness of Impulsive Delayed Reaction-Diffusion Stochastic Neural Networks

Abstract:
Considering the impulsive delayed reaction–diffusion stochastic neural networks (IDRDSNNs) with hybrid impulses, the finite-time boundedness (FTB) and finite-time contractive boundedness (FTCB) are investigated in this article. First, a novel delay integral inequality is presented. By integrating this inequality with the comparison principle, some sufficient conditions that ensure the FTB and FTCB of IDRDSNNs are obtained. This study demonstrates that the FTB of neural networks with hybrid impulses can be maintained, even in the presence of impulsive perturbations. And for a system that is not FTB due to impulsive perturbations, achieving FTB is possible through the implementation of appropriate impulsive control and optimization of the average impulsive intervals. In addition, to validate the practicality of our results, three illustrative examples are provided. In the end, these theoretical findings are successfully applied to image encryption.

---

PaperID: 603,   

Authors:  Qiujie Lv, Guanxing Chen, Ziduo Yang, Weihe Zhong, Calvin Yu-Chian Chen

Title: Meta-MolNet: A Cross-Domain Benchmark for Few Examples Drug Discovery

Abstract:
Predicting the pharmacological activity, toxicity, and pharmacokinetic properties of molecules is a central task in drug discovery. Existing machine learning methods are transferred from one resource rich molecular property to another data scarce property in the same scaffold dataset. However, existing models may produce fragile and highly uncertain predictions for new scaffold molecules. And these models were tested on different benchmarks, which seriously affected the quality of their evaluation results. In this article, we introduce Meta-MolNet, a collection of data benchmark and algorithms, which is a standard benchmark platform for measuring model generalization and uncertainty quantification capabilities. Meta-MolNet manages a wide range of molecular datasets with high ratio of molecules/scaffolds, which often leads to more difficult data shift and generalization problems. Furthermore, we propose a graph attention network based on cross-domain meta-learning, Meta-GAT, which uses bilevel optimization to learn meta-knowledge from the scaffold family molecular dataset in the source domain. Meta-GAT benefits from meta-knowledge that reduces the requirement of sample complexity to enable reliable predictions of new scaffold molecules in the target domain through internal iteration of a few examples. We evaluate existing methods as baselines for the community, and the Meta-MolNet benchmark demonstrates the effectiveness of measuring the proposed algorithm in domain generalization and uncertainty quantification. Extensive experiments demonstrate that the Meta-GAT model has state-of-the-art domain generalization performance and robustly estimates uncertainty under few examples constraints. By publishing AI-ready data, evaluation frameworks, and baseline results, we hope to see the Meta-MolNet suite become a comprehensive resource for the AI-assisted drug discovery community. Meta-MolNet is freely accessible at https://github.com/lol88/Meta-MolNet.

---

PaperID: 604,   

Authors:  Jinye Qu, Zeyu Gao, Tielin Zhang, Yanfeng Lu, Huajin Tang, Hong Qiao

Title: Spiking Neural Network for Ultralow-Latency and High-Accurate Object Detection

Abstract:
Spiking Neural Networks (SNNs) have attracted significant attention for their energy-efficient and brain-inspired event-driven properties. Recent advancements, notably Spiking-YOLO, have enabled SNNs to undertake advanced object detection tasks. Nevertheless, these methods often suffer from increased latency and diminished detection accuracy, rendering them less suitable for latency-sensitive mobile platforms. Additionally, the conversion of artificial neural networks (ANNs) to SNNs frequently compromises the integrity of the ANNs’ structure, resulting in poor feature representation and heightened conversion errors. To address the issues of high latency and low detection accuracy, we introduce two solutions: timestep compression and spike-time-dependent integrated (STDI) coding. Timestep compression effectively reduces the number of timesteps required in the ANN-to-SNN conversion by condensing information. The STDI coding employs a time-varying threshold to augment information capacity. Furthermore, we have developed an SNN-based spatial pyramid pooling (SPP) structure, optimized to preserve the network’s structural efficacy during conversion. Utilizing these approaches, we present the ultralow latency and highly accurate object detection model, SUHD. SUHD exhibits exceptional performance on challenging datasets like PASCAL VOC and MS COCO, achieving a remarkable reduction of approximately 750 times in timesteps and a 30% enhancement in mean average precision (mAP) compared to Spiking-YOLO on MS COCO. To the best of our knowledge, SUHD is currently the deepest spike-based object detection model, achieving ultralow timesteps for lossless conversion.

---

PaperID: 605,   

Authors:  Yuqing Chen, Zhencai Shen, Daoliang Li, Ping Zhong, Yingyi Chen

Title: Heterogeneous Domain Adaptation With Generalized Similarity and Dissimilarity Regularization

Abstract:
Heterogeneous domain adaptation (HDA) aims to address the transfer learning problems where the source domain and target domain are represented by heterogeneous features. The existing HDA methods based on matrix factorization have been proven to learn transferable features effectively. However, these methods only preserve the original neighbor structure of samples in each domain and do not use the label information to explore the similarity and separability between samples. This would not eliminate the cross-domain bias of samples and may mix cross-domain samples of different classes in the common subspace, misleading the discriminative feature learning of target samples. To tackle the aforementioned problems, we propose a novel matrix factorization-based HDA method called HDA with generalized similarity and dissimilarity regularization (HGSDR). Specifically, we propose a similarity regularizer by establishing the cross-domain Laplacian graph with label information to explore the similarity between cross-domain samples from the identical class. And we propose a dissimilarity regularizer based on the inner product strategy to expand the separability of cross-domain labeled samples from different classes. For unlabeled target samples, we keep their neighbor relationship to preserve the similarity and separability between them in the original space. Hence, the generalized similarity and dissimilarity regularization is built by integrating the above regularizers to facilitate cross-domain samples to form discriminative class distributions. HGSDR can more efficiently match the distributions of the two domains both from the global and sample viewpoints, thereby learning discriminative features for target samples. Extensive experiments on the benchmark datasets demonstrate the superiority of the proposed method against several state-of-the-art methods.

---

PaperID: 606,   

Authors:  Iyyakutti Iyappan Ganapathi, Fayaz Ali Dharejo, Sajid Javed, Syed Sadaf Ali, Naoufel Werghi

Title: Unsupervised Dual Transformer Learning for 3-D Textured Surface Segmentation

Abstract:
Analysis of the 3-D texture is indispensable for various tasks, such as retrieval, segmentation, classification, and inspection of sculptures, knit fabrics, and biological tissues. A 3-D texture represents a locally repeated surface variation (SV) that is independent of the overall shape of the surface and can be determined using the local neighborhood and its characteristics. Existing methods mostly employ computer vision techniques that analyze a 3-D mesh globally, derive features, and then utilize them for classification or retrieval tasks. While several traditional and learning-based methods have been proposed in the literature, only a few have addressed 3-D texture analysis, and none have considered unsupervised schemes so far. This article proposes an original framework for the unsupervised segmentation of 3-D texture on the mesh manifold. The problem is approached as a binary surface segmentation task, where the mesh surface is partitioned into textured and nontextured regions without prior annotation. The proposed method comprises a mutual transformer-based system consisting of a label generator (LG) and a label cleaner (LC). Both models take geometric image representations of the surface mesh facets and label them as texture or nontexture using an iterative mutual learning scheme. Extensive experiments on three publicly available datasets with diverse texture patterns demonstrate that the proposed framework outperforms standard and state-of-the-art unsupervised techniques and performs reasonably well compared to supervised methods.

---

PaperID: 607,   

Authors:  Qian Guo, Yi Guo, Jin Zhao

Title: HRCL: Hierarchical Relation Contrastive Learning for Low-Resource Relation Extraction

Abstract:
Low-resource relation extraction (LRE) aims to extract the relationships between given entities from natural language sentences in low-resource application scenarios, which has been an incredibly challenging task due to the limited annotated corpora. Existing studies either leverage self-training schemes to expand the scale of labeled data, while the error accumulation of pseudo-labels’ selection bias provoke the gradual drift problem in subsequent relation prediction, or utilize the instance-wise contrastive learning that fails to distinguish those sentence pairs with similar semantics. To alleviate these defects, this article introduces a novel contrastive learning framework called hierarchical relation contrastive learning (HRCL) for LRE. HRCL leverages task-related instruction description and schema-constrained as prompts to generate high-level relation representations. To enhance the efficacy of contrastive learning, we further employ hierarchical affinity propagation clustering (HiPC) to derive hierarchical signals from relational feature space with a hierarchy cross-attention (HCA) mechanism and effectively optimize pair-level relation features through relation-wise contrastive learning. Exhaustive experiments have been conducted on five public relation extraction (RE) datasets in low-resource settings. The results demonstrate the effectiveness and robustness of HRCL and outperform the current state-of-the-art (SOTA) model by 6.56% on average in terms of B3F1. Our source code is publicly available at https://github.com/Phevos75/HRCLRE.

---

PaperID: 608,   

Authors:  Jingchen Li, Haobin Shi, Huarui Wu, Chunjiang Zhao, Kao-Shing Hwang

Title: Eliminating Primacy Bias in Online Reinforcement Learning by Self-Distillation

Abstract:
Excessive invalid explorations at the beginning of training lead deep reinforcement learning process to fall into the risk of overfitting, further resulting in spurious decisions, which obstruct agents in the following states and explorations. This phenomenon is termed primacy bias in online reinforcement learning. This work systematically investigates the primacy bias in online reinforcement learning, discussing the reason for primacy bias, while the characteristic of primacy bias is also analyzed. Besides, to learn a policy generalized to the following states and explorations, we develop an online reinforcement learning framework, termed self-distillation reinforcement learning (SDRL), based on knowledge distillation, allowing the agent to transfer the learned knowledge into a randomly initialized policy at regular intervals, and the new policy network is used to replace the original one in the following training. The core idea for this work is distilling knowledge from the trained policy to another policy can filter biases out, generating a more generalized policy in the learning process. Moreover, to avoid the overfitting of the new policy due to excessive distillations, we add an additional loss in the knowledge distillation process, using L2 regularization to improve the generalization, and the self-imitation mechanism is introduced to accelerate the learning on the current experiences. The results of several experiments in DMC and Atari 100k suggest the proposal has the ability to eliminate primacy bias for reinforcement learning methods, and the policy after knowledge distillation can urge agents to get higher scores more quickly.

---

PaperID: 609,   

Authors:  Qi Xu, Sibo Liu, Xuming Ran, Yaxin Li, Jiangrong Shen, Huajin Tang, Jian K. Liu, Gang Pan, Qiang Zhang

Title: Robust Sensory Information Reconstruction and Classification With Augmented Spikes

Abstract:
Sensory information recognition is primarily processed through the ventral and dorsal visual pathways in the primate brain visual system, which exhibits layered feature representations bearing a strong resemblance to convolutional neural networks (CNNs), encompassing reconstruction and classification. However, existing studies often treat these pathways as distinct entities, focusing individually on pattern reconstruction or classification tasks, overlooking a key feature of biological neurons, the fundamental units for neural computation of visual sensory information. Addressing these limitations, we introduce a unified framework for sensory information recognition with augmented spikes. By integrating pattern reconstruction and classification within a single framework, our approach not only accurately reconstructs multimodal sensory information but also provides precise classification through definitive labeling. Experimental evaluations conducted on various datasets including video scenes, static images, dynamic auditory scenes, and functional magnetic resonance imaging (fMRI) brain activities demonstrate that our framework delivers state-of-the-art pattern reconstruction quality and classification accuracy. The proposed framework enhances the biological realism of multimodal pattern recognition models, offering insights into how the primate brain visual system effectively accomplishes the reconstruction and classification tasks through the integration of ventral and dorsal pathways.

---

PaperID: 610,   

Authors:  Zikun Zhou, Kaige Mao, Wenjie Pei, Hongpeng Wang, Yaowei Wang, Zhenyu He

Title: Reliability-Guided Hierarchical Memory Network for Scribble-Supervised Video Object Segmentation

Abstract:
This article aims to solve the video object segmentation (VOS) task in a scribble-supervised manner, in which VOS models are not only initialized with sparse target scribbles for inference but also trained by sparse scribble annotations. Thus, the annotation burdens for both initialization and training can be substantially lightened. The difficulties of scribble-supervised VOS lie in two aspects: 1) it demands a strong reasoning ability to carefully segment the target given only a sparse initial target scribble and 2) it necessitates learning dense prediction from sparse scribble annotations during training, requiring powerful learning capability. In this work, we propose a reliability-guided hierarchical memory network (RHMNet) for this task, which segments the target in a stepwise expanding strategy w.r.t. the memory reliability level. To be specific, RHMNet maintains a reliability-guided memory bank. It first uses the high-reliability memory to locate the region with high reliability belonging to the target, i.e., highly similar to the initial target scribble. Then, it expands the located high-reliability region to the entire target conditioned on the region itself and all existing memories. In addition, we propose a scribble-supervised learning mechanism to facilitate the model learning for dense prediction. It exploits the pixel-level relations within a single frame and the instance-level variations across multiple frames to take full advantage of the scribble annotations in sequence training samples. The favorable performance on four popular benchmarks demonstrates that our method is promising. Our project is available at: https://github.com/mkg1204/RHMNet-for-SSVOS.

---

PaperID: 611,   

Authors:  Hong Yin, Jiang Zhong, Rongzhen Li, Jiaxing Shang, Chen Wang, Xue Li

Title: High-Order Neighbors Aware Representation Learning for Knowledge Graph Completion

Abstract:
As a building block of knowledge acquisition, knowledge graph completion (KGC) aims at inferring missing facts in knowledge graphs (KGs) automatically. Previous studies mainly focus on graph convolutional network (GCN)-based KG embedding (KGE) to determine the representations of entities and relations, accordingly predicting missing triplets. However, most existing KGE methods suffer from limitations in predicting tail entities that are far away or even unreachable in KGs. This limitation can be attributed to the related high-order information being largely ignored. In this work, we focus on learning the information from the related high-order neighbors in KGs to improve the performance of prediction. Specifically, we first introduce a set of new nodes called pedalnodes to augment the KGs for facilitating message passing between related high-order entities, effectively injecting the information of high-order neighbors into entity representation. Additionally, we propose strength-guided graph neural networks to aggregate neighboring entity representations. To address the issue of transmitting irrelevant higher order information to entities through pedal nodes, which can potentially hurt entity representation, we further propose to dynamically integrate the aggregated representation of each node with its corresponding self-representation. Extensive experiments have been conducted on three benchmark datasets and the results demonstrate the superiority of our method compared to strong baseline models.

---

PaperID: 612,   

Authors:  Weixin An, Yuanyuan Liu, Fanhua Shang, Hongying Liu, Licheng Jiao

Title: DEs-Inspired Accelerated Unfolded Linearized ADMM Networks for Inverse Problems

Abstract:
Many research works have shown that the traditional alternating direction multiplier methods (ADMMs) can be better understood by continuous-time differential equations (DEs). On the other hand, many unfolded algorithms directly inherit the traditional iterations to build deep networks. Although they achieve superior practical performance and a faster convergence rate than traditional counterparts, there is a lack of clear insight into unfolded network structures. Thus, we attempt to explore the unfolded linearized ADMM (LADMM) from the perspective of DEs, and design more efficient unfolded networks. First, by proposing an unfolded Euler LADMM scheme and inspired by the trapezoid discretization, we design a new more accurate Trapezoid LADMM scheme. For the convenience of implementation, we provide its explicit version via a prediction–correction strategy. Then, to expand the representation space of unfolded networks, we design an accelerated variant of our Euler LADMM scheme, which can be interpreted as second-order DEs with stronger representation capabilities. To fully explore this representation space, we designed an accelerated Trapezoid LADMM scheme. To the best of our knowledge, this is the first work to explore a comprehensive connection with theoretical guarantees between unfolded ADMMs and first- (second-) order DEs. Finally, we instantiate our schemes as (A-)ELADMM and (A-)TLADMM with the proximal operators, and (A-)ELADMM-Net and (A-)TLADMM-Net with convolutional neural networks (CNNs). Extensive inverse problem experiments show that our Trapezoid LADMM schemes perform better than well-known methods.

---

PaperID: 613,   

Authors:  Qi Yu, Xijun Liang, Mengzhen Li, Ling Jian

Title: NGDE: A Niching-Based Gradient-Directed Evolution Algorithm for Nonconvex Optimization

Abstract:
Nonconvex optimization issues are prevalent in machine learning and data science. While gradient-based optimization algorithms can rapidly converge and are dimension-independent, they may, unfortunately, fall into local optimal solutions or saddle points. In contrast, evolutionary algorithms (EAs) gradually adapt the population of solutions to explore global optimal solutions. However, this approach requires substantial computational resources to perform numerous fitness function evaluations, which poses challenges for high-dimensional optimization in particular. This study introduces a novel nonconvex optimization algorithm, the niching-based gradient-directed evolution (NGDE) algorithm, designed specifically for high-dimensional nonconvex optimization. The NGDE algorithm generates potential solutions and divides them into multiple niches to explore distinct areas within the feasible region. Subsequently, each individual creates candidate offspring using the gradient-directed mutation operator we designed. The convergence properties of the NGDE algorithm are investigated in two scenarios: accessing the full gradient and approximating the gradient with mini-batch samples. The experimental studies demonstrate the superior performance of the NGDE algorithm in minimizing multimodal optimization functions. Additionally, when applied to train the neural networks of LeNet-5, NGDE shows significantly improved classification accuracy, especially in smaller training sizes.

---

PaperID: 614,   

Authors:  Yunhao Li, Zhen Xiao, Lin Yang, Dan Meng, Xin Zhou, Heng Fan, Libo Zhang

Title: AttMOT: Improving Multiple-Object Tracking by Introducing Auxiliary Pedestrian Attributes

Abstract:
Multiobject tracking (MOT) is a fundamental problem in computer vision with numerous applications, such as intelligent surveillance and automated driving. Despite the significant progress made in MOT, pedestrian attributes, such as gender, hairstyle, body shape, and clothing features, which contain rich and high-level information, have been less explored. To address this gap, we propose a simple, effective, and generic method to predict pedestrian attributes to support general reidentification (Re-ID) embedding. We first introduce attribute multi-object tracking (AttMOT), a large, highly enriched synthetic dataset for pedestrian tracking, containing over 80k frames and six million pedestrian identity switches (IDs) with different times, weather conditions, and scenarios. To the best of authors’ knowledge, AttMOT is the first MOT dataset with semantic attributes. Subsequently, we explore different approaches to fuse Re-ID embedding and pedestrian attributes, including attention mechanisms, which we hope will stimulate the development of attribute-assisted MOT. The proposed method attribute-assisted method (AAM) demonstrates its effectiveness and generality on several representative pedestrian MOT benchmarks, including MOT17 and MOT20, through experiments on the AttMOT dataset. When applied to the state-of-the-art trackers, AAM achieves consistent improvements in multi-object tracking accuracy (MOTA), higher order tracking accuracy (HOTA), association accuracy (AssA), IDs, and IDF1 scores. For instance, on MOT17, the proposed method yields a +1.1 MOTA, +1.7 HOTA, and +1.8 IDF1 improvement when used with FairMOT. To further encourage related research, we release the data and code at https://github.com/HengLan/AttMOT.

---

PaperID: 615,   

Authors:  Junhao Dong, Yuan Wang, Xiaohua Xie, Jianhuang Lai, Yew-Soon Ong

Title: Generalizable and Discriminative Representations for Adversarially Robust Few-Shot Learning

Abstract:
Few-shot image classification (FSIC) is beneficial for a variety of real-world scenarios, aiming to construct a recognition system with limited training data. In this article, we extend the original FSIC task by incorporating defense against malicious adversarial examples. This can be an arduous challenge because numerous deep learning-based approaches remain susceptible to adversarial examples, even when trained with ample amounts of data. Previous studies on this problem have predominantly concentrated on the meta-learning framework, which involves sampling numerous few-shot tasks during the training stage. In contrast, we propose a straightforward but effective baseline via learning robust and discriminative representations without tedious meta-task sampling, which can further be generalized to unforeseen adversarial FSIC tasks. Specifically, we introduce an adversarial-aware (AA) mechanism that exploits feature-level distinctions between the legitimate and the adversarial domains to provide supplementary supervision. Moreover, we design a novel adversarial reweighting training strategy to ameliorate the imbalance among adversarial examples. To further enhance the adversarial robustness without compromising discriminative features, we propose the cyclic feature purifier during the postprocessing projection, which can reduce the interference of unforeseen adversarial examples. Furthermore, our method can obtain robust feature embeddings that maintain superior transferability, even when facing cross-domain adversarial examples. Extensive experiments and systematic analyses demonstrate that our method achieves state-of-the-art robustness as well as natural performance among adversarially robust FSIC algorithms on three standard benchmarks by a substantial margin.

---

PaperID: 616,