Similar Journals
|
IEEE Transactions on Neural Networks and Learning Systems
Journal Prestige (SJR): 3.406  Citation Impact (citeScore): 9 Number of Followers: 53  Hybrid journal (It can contain Open Access articles)ISSN (Print) 2162-237X Published by IEEE  [228 journals]
|
- IEEE Transactions on Neural Networks and Learning Systems Publication
Information-
Free pre-print version: Loading...Rate this result: What is this?Please help us test our new pre-print finding feature by giving the pre-print link a rating.
A 5 star rating indicates the linked pre-print has the exact same content as the published article.
Pages: C2 - C2
PubDate: June 2023
Issue No: Vol. 34, No. 6 (2023)
-
- IEEE Computational Intelligence Society Information
-
Free pre-print version: Loading...Rate this result: What is this?Please help us test our new pre-print finding feature by giving the pre-print link a rating.
A 5 star rating indicates the linked pre-print has the exact same content as the published article.
Pages: C3 - C3
PubDate: June 2023
Issue No: Vol. 34, No. 6 (2023)
-
- IEEE Transactions on Neural Networks and Learning Systems Information for
Authors-
Free pre-print version: Loading...Rate this result: What is this?Please help us test our new pre-print finding feature by giving the pre-print link a rating.
A 5 star rating indicates the linked pre-print has the exact same content as the published article.
Pages: C4 - C4
PubDate: June 2023
Issue No: Vol. 34, No. 6 (2023)
-
- DPSNet: Multitask Learning Using Geometry Reasoning for Scene Depth and
Semantics-
Free pre-print version: Loading...Rate this result: What is this?Please help us test our new pre-print finding feature by giving the pre-print link a rating.
A 5 star rating indicates the linked pre-print has the exact same content as the published article.
Authors: Junning Zhang;Qunxing Su;Bo Tang;Cheng Wang;Yining Li;
Pages: 2710 - 2721
Abstract: Multitask joint learning technology continues gaining more attention as a paradigm shift and has shown promising performance in many applications. Depth estimation and semantic understanding from monocular images emerge as a challenging problem in computer vision. While the other joint learning frameworks establish the relationship between the semantics and depth from stereo pairs, the lack of learning camera motion renders the frameworks that fail to model the geometric structure of the image scene. We make a further step in this article by proposing a multitask learning method, namely DPSNet, which can jointly perform depth and camera pose estimation and semantic scene segmentation. Our core idea for depth and camera pose prediction is that we present the rigid semantic consistency loss to overcome the limitation of moving pixels from image reconstruction technology and further infer the segmentation of moving instances based on them. In addition, our proposed model performs semantic segmentation by reasoning the geometric correspondences between the pixel semantic outputs and the semantic labels at multiscale resolutions. Experiments on open-source datasets and a video dataset captured on a micro-smart car show the effectiveness of each component of DPSNet, and DPSNet achieves state-of-the-art results in all three tasks compared with the best popular methods. All our models and code are available at https://github.com/jn-z/DPSNet: Multitask Learning Using Geometry Reasoning for Scene Depth and semantics.
PubDate: June 2023
Issue No: Vol. 34, No. 6 (2023)
-
- Approximate Optimal Control for Nonlinear Systems With Periodic
Event-Triggered Mechanism-
Free pre-print version: Loading...Rate this result: What is this?Please help us test our new pre-print finding feature by giving the pre-print link a rating.
A 5 star rating indicates the linked pre-print has the exact same content as the published article.
Authors: Shengbo Wang;Shiping Wen;Kaibo Shi;Xiaojun Zhou;Tingwen Huang;
Pages: 2722 - 2731
Abstract: This article investigates the approximate optimal control problem for nonlinear affine systems under the periodic event triggered control (PETC) strategy. In terms of optimal control, a theoretical comparison of continuous control, traditional event-based control (ETC), and PETC from the perspective of stability convergence, concluding that PETC does not significantly affect the convergence rate than ETC. It is the first time to present PETC for optimal control target of nonlinear systems. A critic network is introduced to approximate the optimal value function based on the idea of reinforcement learning (RL). It is proven that the discrete updating time series from PETC can also be utilized to determine the updating time of the learning network. In this way, the gradient-based weight estimation for continuous systems is developed in discrete form. Then, the uniformly ultimately bounded (UUB) condition of controlled systems is analyzed to ensure the stability of the designed method. Finally, two illustrative examples are given to show the effectiveness of the method.
PubDate: June 2023
Issue No: Vol. 34, No. 6 (2023)
-
- Adaptive Neural Network Control for a Class of Nonlinear Systems With
Function Constraints on States-
Free pre-print version: Loading...Rate this result: What is this?Please help us test our new pre-print finding feature by giving the pre-print link a rating.
A 5 star rating indicates the linked pre-print has the exact same content as the published article.
Authors: Yan-Jun Liu;Wei Zhao;Lei Liu;Dapeng Li;Shaocheng Tong;C. L. Philip Chen;
Pages: 2732 - 2741
Abstract: In this article, the problem of tracking control for a class of nonlinear time-varying full state constrained systems is investigated. By constructing the time-varying asymmetric barrier Lyapunov function (BLF) and combining it with the backstepping algorithm, the intelligent controller and adaptive law are developed. Neural networks (NNs) are utilized to approximate the uncertain function. It is well known that in the past research of nonlinear systems with state constraints, the state constraint boundary is either a constant or a time-varying function. In this article, the constraint boundaries both related to state and time are investigated, which makes the design of control algorithm more complex and difficult. Furthermore, by employing the Lyapunov stability analysis, it is proven that all signals in the closed-loop system are bounded and the time-varying full state constraints are not violated. In the end, the effectiveness of the control algorithm is verified by numerical simulation.
PubDate: June 2023
Issue No: Vol. 34, No. 6 (2023)
-
- Adaptive NN-Based Event-Triggered Containment Control for Unknown
Nonlinear Networked Systems-
Free pre-print version: Loading...Rate this result: What is this?Please help us test our new pre-print finding feature by giving the pre-print link a rating.
A 5 star rating indicates the linked pre-print has the exact same content as the published article.
Authors: Yukan Zheng;Yuan-Xin Li;Wei-Wei Che;Zhongsheng Hou;
Pages: 2742 - 2752
Abstract: This article systematically addresses the distributed event-triggered containment control issues for multiagent systems subjected to unknown nonlinearities and external disturbances over a directed communication topology. Novel composite distributed adaptive neural network (NN) event-triggering conditions and event-triggered controller are raised meanwhile. Furthermore, the designed event-triggered controller is updated in an aperiodic way at the moment of event sampling, which saves the computation, resources, and transmission load. On the basis of the NN-based adaptive control techniques and event-triggered control strategies, the uniform ultimate bounded containment control can be achieved. In addition, the Zeno behavior is proven to be excluded. Simulation is presented to testify the effectiveness and advantages of the presented distributed containment control scheme.
PubDate: June 2023
Issue No: Vol. 34, No. 6 (2023)
-
- Joint Dynamic Manifold and Discriminant Information Learning for Feature
Extraction-
Free pre-print version: Loading...Rate this result: What is this?Please help us test our new pre-print finding feature by giving the pre-print link a rating.
A 5 star rating indicates the linked pre-print has the exact same content as the published article.
Authors: Xiaowei Zhao;Feiping Nie;Rong Wang;Xuelong Li;
Pages: 2753 - 2766
Abstract: Neighborhood reconstruction is a good recipe to learn the local manifold structure. Representation-based discriminant analysis methods normally learn the reconstruction relationship between each sample and all the other samples. However, reconstruction graphs constructed in these methods have three limitations: 1) they cannot guarantee the local sparsity of reconstruction coefficients; 2) heterogeneous samples may own nonzero coefficients; and 3) they learn the manifold information prior to the process of dimensionality reduction. Due to the existence of noise and redundant features in the original space, the prelearned manifold structure may be inaccurate. Accordingly, the performance of dimensionality reduction would be affected. In this article, we propose a joint model to simultaneously learn the affinity relationship, reconstruction relationship, and projection matrix. In this model, we actively assign neighbors for each sample and learn the inter-reconstruction coefficients between each sample and their neighbors with the same label information in the process of dimensionality reduction. Specifically, a sparse constraint is employed to ensure the sparsity of neighbors and reconstruction coefficients. The whitening constraint is imposed on the projection matrix to remove the relevance between features. An iterative algorithm is proposed to solve this method. Extensive experiments on toy data and public datasets show the superiority of the proposed method.
PubDate: June 2023
Issue No: Vol. 34, No. 6 (2023)
-
- C-DeepTrust: A Context-Aware Deep Trust Prediction Model in Online Social
Networks-
Free pre-print version: Loading...Rate this result: What is this?Please help us test our new pre-print finding feature by giving the pre-print link a rating.
A 5 star rating indicates the linked pre-print has the exact same content as the published article.
Authors: Qi Wang;Weiliang Zhao;Jian Yang;Jia Wu;Shan Xue;Qianli Xing;Philip S. Yu;
Pages: 2767 - 2780
Abstract: Trust prediction provides valuable support for decision making, information dissemination, and product promotion in online social networks. As a complex concept in the social network community, trust relationships among people can be established virtually based on: 1) their interaction behaviors, e.g., the ratings and comments that they provided; 2) the contextual information associated with their interactions, e.g., location and culture; and 3) the relative temporal features of interactions and the time periods when the trust relationships hold. Most of the existing works only focus on some aspects of trust, and there is not a comprehensive study of user trust development that considers and incorporates 1)–3) in trust prediction. In this article, we propose a context-aware deep trust prediction model C-DeepTrust to fill this gap. First, we conduct user feature modeling to obtain the user’s static and dynamic preference features in each context. Static user preference features are obtained from all the ratings and reviews that a user provided, while dynamic user preference features are obtained from the items rated/reviewed by the user in time series. The obtained context-aware user features are then combined and fed into the multilayer projection structure to further mine the context-aware latent features. Finally, the context-aware trust relationships between users are calculated by their context-aware feature vector cosine similarities according to the social homophily theory, which shows a pervasive property of social networks that trust relationships are more likely to be developed among similar people. Extensive experiments conducted on two real-world datasets show the superior performance of our approach compared with the representative baseline methods.
PubDate: June 2023
Issue No: Vol. 34, No. 6 (2023)
-
- Novel Discrete-Time Recurrent Neural Network for Robot Manipulator: A
Direct Discretization Technical Route-
Free pre-print version: Loading...Rate this result: What is this?Please help us test our new pre-print finding feature by giving the pre-print link a rating.
A 5 star rating indicates the linked pre-print has the exact same content as the published article.
Authors: Yang Shi;Wenhan Zhao;Shuai Li;Bin Li;Xiaobing Sun;
Pages: 2781 - 2790
Abstract: Controlling and processing of time-variant problem is universal in the fields of engineering and science, and the discrete-time recurrent neural network (RNN) model has been proven as an effective method for handling a variety of discrete time-variant problems. However, such model usually originates from the discretization research of continuous time-variant problem, and there is little research on the direct discretization method. To address the aforementioned problem, this article introduces a novel discrete-time RNN model for solving the discrete time-variant problem in a pioneering manner. Specifically, a discrete time-variant nonlinear system, which originates from the mathematical modeling of serial robot manipulator, is presented as a target problem. For solving the problem, first, the technique of second-order Taylor expansion is used to deal with the discrete time-variant nonlinear system, and the novel discrete-time RNN model is proposed subsequently. Second, the theoretical analyses are investigated and developed, which shows the convergence and precision of the proposed discrete-time RNN model. Furthermore, three distinct numerical experiments verify the excellent performance of the proposed discrete-time RNN model. In addition, a robot manipulator example further verifies the effectiveness and practicability of the proposed novel discrete-time RNN model.
PubDate: June 2023
Issue No: Vol. 34, No. 6 (2023)
-
- Comprehensive SNN Compression Using ADMM Optimization and Activity
Regularization-
Free pre-print version: Loading...Rate this result: What is this?Please help us test our new pre-print finding feature by giving the pre-print link a rating.
A 5 star rating indicates the linked pre-print has the exact same content as the published article.
Authors: Lei Deng;Yujie Wu;Yifan Hu;Ling Liang;Guoqi Li;Xing Hu;Yufei Ding;Peng Li;Yuan Xie;
Pages: 2791 - 2805
Abstract: As well known, the huge memory and compute costs of both artificial neural networks (ANNs) and spiking neural networks (SNNs) greatly hinder their deployment on edge devices with high efficiency. Model compression has been proposed as a promising technique to improve the running efficiency via parameter and operation reduction, whereas this technique is mainly practiced in ANNs rather than SNNs. It is interesting to answer how much an SNN model can be compressed without compromising its functionality, where two challenges should be addressed: 1) the accuracy of SNNs is usually sensitive to model compression, which requires an accurate compression methodology and 2) the computation of SNNs is event-driven rather than static, which produces an extra compression dimension on dynamic spikes. To this end, we realize a comprehensive SNN compression through three steps. First, we formulate the connection pruning and weight quantization as a constrained optimization problem. Second, we combine spatiotemporal backpropagation (STBP) and alternating direction method of multipliers (ADMMs) to solve the problem with minimum accuracy loss. Third, we further propose activity regularization to reduce the spike events for fewer active operations. These methods can be applied in either a single way for moderate compression or a joint way for aggressive compression. We define several quantitative metrics to evaluate the compression performance for SNNs. Our methodology is validated in pattern recognition tasks over MNIST, N-MNIST, CIFAR10, and CIFAR100 datasets, where extensive comparisons, analyses, and insights are provided. To the best of our knowledge, this is the first work that studies SNN compression in a comprehensive manner by exploiting all compressible components and achieves better results.
PubDate: June 2023
Issue No: Vol. 34, No. 6 (2023)
-
- Neural Network Algorithm With Reinforcement Learning for Parameters
Extraction of Photovoltaic Models-
Free pre-print version: Loading...Rate this result: What is this?Please help us test our new pre-print finding feature by giving the pre-print link a rating.
A 5 star rating indicates the linked pre-print has the exact same content as the published article.
Authors: Yiying Zhang;
Pages: 2806 - 2816
Abstract: This research focuses on the application of artificial neural networks (ANNs) on parameters extraction of photovoltaic (PV) models. Extracting parameters of the PV models accurately is crucial to control and optimize PV systems. Although many algorithms have been proposed to address this issue, how to extract the parameters of the PV models accurately and reliably is still a great challenge. Neural network algorithm (NNA) is a recently reported metaheuristic algorithm. NNA is inspired by ANNs. Benefiting from the unique structure of ANNs, NNA shows excellent global search ability. However, NNA faces the challenge of slow convergence rate and local optima stagnation in solving complex optimization problems. This article presents an improved NNA, named neural network algorithm with reinforcement learning (RLNNA), for extracting parameters of the PV models. In RLNNA, three strategies, namely modification factor with reinforcement learning (RL), transfer operator with historical population, and feedback operator, are designed to overcome the challenge of NNA. To verify the performance of RLNNA, it is employed to extract the parameters of the three PV models. Experimental results show that RLNNA can extract the parameters of the considered PV models with higher accuracy and stronger stability compared with NNA and the other 12 powerful algorithms, which fully indicates the effectiveness of the improved strategies.
PubDate: June 2023
Issue No: Vol. 34, No. 6 (2023)
-
- Topological Structure and Semantic Information Transfer Network for
Cross-Scene Hyperspectral Image Classification-
Free pre-print version: Loading...Rate this result: What is this?Please help us test our new pre-print finding feature by giving the pre-print link a rating.
A 5 star rating indicates the linked pre-print has the exact same content as the published article.
Authors: Yuxiang Zhang;Wei Li;Mengmeng Zhang;Ying Qu;Ran Tao;Hairong Qi;
Pages: 2817 - 2830
Abstract: Domain adaptation techniques have been widely applied to the problem of cross-scene hyperspectral image (HSI) classification. Most existing methods use convolutional neural networks (CNNs) to extract statistical features from data and often neglect the potential topological structure information between different land cover classes. CNN-based approaches generally only model the local spatial relationships of the samples, which largely limits their ability to capture the nonlocal topological relationship that would better represent the underlying data structure of HSI. In order to make up for the above shortcomings, a Topological structure and Semantic information Transfer network (TSTnet) is developed. The method employs the graph structure to characterize topological relationships and the graph convolutional network (GCN) that is good at processing for cross-scene HSI classification. In the proposed TSTnet, graph optimal transmission (GOT) is used to align topological relationships to assist distribution alignment between the source domain and the target domain based on the maximum mean difference (MMD). Furthermore, subgraphs from the source domain and the target domain are dynamically constructed based on CNN features to take advantage of the discriminative capacity of CNN models that, in turn, improve the robustness of classification. In addition, to better characterize the correlation between distribution alignment and topological relationship alignment, a consistency constraint is enforced to integrate the output of CNN and GCN. Experimental results on three cross-scene HSI datasets demonstrate that the proposed TSTnet performs significantly better than some state-of-the-art domain-adaptive approaches. The codes will be available from the website: https://github.com/YuxiangZhang-BIT/IEEE_TNNLS_TSTnet.
PubDate: June 2023
Issue No: Vol. 34, No. 6 (2023)
-
- Anomaly Detection With Representative Neighbors
-
Free pre-print version: Loading...Rate this result: What is this?Please help us test our new pre-print finding feature by giving the pre-print link a rating.
A 5 star rating indicates the linked pre-print has the exact same content as the published article.
Authors: Huawen Liu;Xiaodan Xu;Enhui Li;Shichao Zhang;Xuelong Li;
Pages: 2831 - 2841
Abstract: Identifying anomalies from data has attracted increasing attention in recent years due to its broad range of potential applications. Although many efforts have been made for anomaly detection, how to effectively handle high-dimensional data and how to exactly explore neighborhood information, a fundamental issue in anomaly detection, have not yet received sufficient concerns. To circumvent these challenges, in this article, we propose an effective anomaly detection method with representative neighbors for high-dimensional data. Specifically, it projects the high-dimensional data into a low-dimensional space via a sparse operation and explores representative neighbors with a self-representation learning technique. The neighborhood information is then transformed into similarity relations, making the data converge or disperse. Eventually, anomalies are discriminated by a tailored graph clustering technique, which can effectively reveal structural information of the data. Extensive experiments were conducted on ten public real-world datasets with 11 popular anomaly detection algorithms. The results show that the proposed method has encouraging and promising performance compared to the state-of-the-art anomaly detection algorithms.
PubDate: June 2023
Issue No: Vol. 34, No. 6 (2023)
-
- An Optimal Transport Analysis on Generalization in Deep Learning
-
Free pre-print version: Loading...Rate this result: What is this?Please help us test our new pre-print finding feature by giving the pre-print link a rating.
A 5 star rating indicates the linked pre-print has the exact same content as the published article.
Authors: Jingwei Zhang;Tongliang Liu;Dacheng Tao;
Pages: 2842 - 2853
Abstract: Deep neural networks (DNNs) have achieved state-of-the-art performance in various learning tasks, such as computer vision, natural language processing, and speech recognition. However, the fundamental theory of generalization still remains obscure in deep learning—why DNN models can generalize well, despite that they are heavily overparametrized in both depth and width' Recently, some work shows that traditional theory of analyzing the generalization error of learning models fails to explain the generalization of DNNs. The failure is mainly because of one simple fact that the worse case analysis of generalization error for learning models would be too loose for models with large parameter space, such as DNNs. In this work, we propose a new analysis of generalization in DNNs from an optimal transport perspective. Unlike traditional worse-case uniform convergence analysis in learning theory, our analysis of generalization error is dependent on both the learning algorithm and the data distribution and is the average-case analysis. Thus, our theory can be more practical and accurate to describe the generalization behavior of DNNs. More specifically, in this article, we try to answer a fundamental yet unsolved question in deep learning—why deeper models can generalize well than shallow models' The main contribution of this article can be summarized in four aspects. First, under a general learning framework, we derive upper bounds on the generalization error of learning algorithms by their algorithmic transport cost: the expected Wasserstein distance between the output hypothesis and the output hypothesis conditioned on an input example. We further provide several upper bounds on the algorithmic transport cost in terms of total variation distance, relative entropy, and Vapnik–Chervonenkis (VC) dimension. Moreover, we also study different conditions for loss functions under which the generalization error of a learni-g algorithm can be upper bounded by different probability metrics between distributions relating to the output hypothesis and/or the input data. Finally, under our established framework, we obtain our main results, showing that the generalization error in DNNs decreases exponentially to zero as the number of layers increases.
PubDate: June 2023
Issue No: Vol. 34, No. 6 (2023)
-
- Neural Network Model-Based Control for Manipulator: An Autoencoder
Perspective-
Free pre-print version: Loading...Rate this result: What is this?Please help us test our new pre-print finding feature by giving the pre-print link a rating.
A 5 star rating indicates the linked pre-print has the exact same content as the published article.
Authors: Zhan Li;Shuai Li;
Pages: 2854 - 2868
Abstract: Recently, neural network model-based control has received wide interests in kinematics control of manipulators. To enhance learning ability of neural network models, the autoencoder method is used as a powerful tool to achieve deep learning and has gained success in recent years. However, the performance of existing autoencoder approaches for manipulator control may be still largely dependent on the quality of data, and for extreme cases with noisy data it may even fail. How to incorporate the model knowledge into the autoencoder controller design with an aim to increase the robustness and reliability remains a challenging problem. In this work, a sparse autoencoder controller for kinematic control of manipulators with weights obtained directly from the robot model rather than training data is proposed for the first time. By encoding and decoding the control target though a new dynamic recurrent neural network architecture, the control input can be solved through a new sparse optimization formulation. In this work, input saturation, which holds for almost all practical systems but usually is ignored for analysis simplicity, is also considered in the controller construction. Theoretical analysis and extensive simulations demonstrate that the proposed sparse autoencoder controller with input saturation can make the end-effector of the manipulator system track the desired path efficiently. Further performance comparison and evaluation against the additive noise and parameter uncertainty substantiate robustness of the proposed sparse autoencoder manipulator controller.
PubDate: June 2023
Issue No: Vol. 34, No. 6 (2023)
-
- Improving the Accuracy of Spiking Neural Networks for Radar Gesture
Recognition Through Preprocessing-
Free pre-print version: Loading...Rate this result: What is this?Please help us test our new pre-print finding feature by giving the pre-print link a rating.
A 5 star rating indicates the linked pre-print has the exact same content as the published article.
Authors: Ali Safa;Federico Corradi;Lars Keuninckx;Ilja Ocket;André Bourdoux;Francky Catthoor;Georges G. E. Gielen;
Pages: 2869 - 2881
Abstract: Event-based neural networks are currently being explored as efficient solutions for performing AI tasks at the extreme edge. To fully exploit their potential, event-based neural networks coupled to adequate preprocessing must be investigated. Within this context, we demonstrate a 4-b-weight spiking neural network (SNN) for radar gesture recognition, achieving a state-of-the-art 93% accuracy within only four processing time steps while using only one convolutional layer and two fully connected layers. This solution consumes very little energy and area if implemented in event-based hardware, which makes it suited for embedded extreme-edge applications. In addition, we demonstrate the importance of signal preprocessing for achieving this high recognition accuracy in SNNs compared to deep neural networks (DNNs) with the same network topology and training strategy. We show that efficient preprocessing prior to the neural network is drastically more important for SNNs compared to DNNs. We also demonstrate, for the first time, that the preprocessing parameters can affect SNNs and DNNs in antagonistic ways, prohibiting the generalization of conclusions drawn from DNN design to SNNs. We demonstrate our findings by comparing the gesture recognition accuracy achieved with our SNN to a DNN with the same architecture and similar training. Unlike previously proposed neural networks for radar processing, this work enables ultralow-power radar-based gesture recognition for extreme-edge devices.
PubDate: June 2023
Issue No: Vol. 34, No. 6 (2023)
-
- Safety-Critical Containment Maneuvering of Underactuated Autonomous
Surface Vehicles Based on Neurodynamic Optimization With Control Barrier
Functions-
Free pre-print version: Loading...Rate this result: What is this?Please help us test our new pre-print finding feature by giving the pre-print link a rating.
A 5 star rating indicates the linked pre-print has the exact same content as the published article.
Authors: Nan Gu;Dan Wang;Zhouhua Peng;Jun Wang;
Pages: 2882 - 2895
Abstract: This article addresses the safety-critical containment maneuvering of multiple underactuated autonomous surface vehicles (ASVs) in the presence of multiple stationary/moving obstacles. In a complex marine environment, every ASV suffers from model uncertainties, external disturbances, and input constraints. A safety-critical control method is proposed for achieving a collision-free containment formation. Specifically, a fixed-time extended state observer is employed for estimating the model uncertainties and external disturbances. By estimating lumped disturbances in fixed time, nominal containment maneuvering control laws are designed in an Earth-fixed reference frame. Input-to-state safe control barrier functions (ISSf-CBFs) are constructed for mapping safety constraints on states to constraints on control inputs. A distributed quadratic optimization problem with the norm of control inputs as the objective function and ISSf-CBFs as constraints is formulated. A recurrent neural network-based neurodynamic optimization approach is adopted to solve the quadratic optimization problem for computing the forces and moments within the safety and input constraints in real time. It is proven that the error signals in the closed-loop control system are uniformly ultimately bounded and the multi-ASVs system is guaranteed for input-to-state safety. Simulation results are elaborated to substantiate the effectiveness of the proposed safety-critical control method for ASVs based on neurodynamic optimization with control barrier functions.
PubDate: June 2023
Issue No: Vol. 34, No. 6 (2023)
-
- Entropy Minimization Versus Diversity Maximization for Domain Adaptation
-
Free pre-print version: Loading...Rate this result: What is this?Please help us test our new pre-print finding feature by giving the pre-print link a rating.
A 5 star rating indicates the linked pre-print has the exact same content as the published article.
Authors: Xiaofu Wu;Suofei Zhang;Quan Zhou;Zhen Yang;Chunming Zhao;Longin Jan Latecki;
Pages: 2896 - 2907
Abstract: Entropy minimization has been widely used in unsupervised domain adaptation (UDA). However, existing works reveal that the use of entropy-minimization-only may lead to collapsed trivial solutions for UDA. In this article, we try to seek possible close-to-ideal UDA solutions by focusing on some intuitive properties of the ideal domain adaptation solution. In particular, we propose to introduce diversity maximization for further regulating entropy minimization. In order to achieve the possible minimum target risk for UDA, we show that diversity maximization should be elaborately balanced with entropy minimization, the degree of which can be finely controlled with the use of deep embedded validation in an unsupervised manner. The proposed minimal-entropy diversity maximization (MEDM) can be directly implemented by stochastic gradient descent without the use of adversarial learning. Empirical evidence demonstrates that MEDM outperforms the state-of-the-art methods on four popular domain adaptation datasets.
PubDate: June 2023
Issue No: Vol. 34, No. 6 (2023)
-
- Self-Unaware Adversarial Multi-Armed Bandits With Switching Costs
-
Free pre-print version: Loading...Rate this result: What is this?Please help us test our new pre-print finding feature by giving the pre-print link a rating.
A 5 star rating indicates the linked pre-print has the exact same content as the published article.
Authors: Amir Alipour-Fanid;Monireh Dabaghchian;Kai Zeng;
Pages: 2908 - 2922
Abstract: We study a family of adversarial (a.k.a. nonstochastic) multi-armed bandit (MAB) problems, wherein not only the player cannot observe the reward on the played arm (self-unaware player) but also it incurs switching costs when shifting to another arm. We study two cases: In Case 1, at each round, the player is able to either play or observe the chosen arm, but not both. In Case 2, the player can choose an arm to play and, at the same round, choose another arm to observe. In both cases, the player incurs a cost for consecutive arm switching due to playing or observing the arms. We propose two novel online learning-based algorithms each addressing one of the aforementioned MAB problems. We theoretically prove that the proposed algorithms for Case 1 and Case 2 achieve sublinear regret of $O(sqrt [{4}]{KT^{3}ln K})$ and $O(sqrt [{3}]{(K-1)T^{2}ln K})$ , respectively, where the latter regret bound is order-optimal in time, $K$ is the number of arms, and $T$ is the total number of rounds. In Case 2, we extend the player’s capability to multiple $m>1$ observations and show that more observations do not necessarily improve the regret bound due to incurring switching costs. However, we derive an upper bound for switching cost as $c leq 1/sqrt [{3}]{m^{2}}$ for which the regret bound is improved as the number of observations increases. Finally, through this study, we found that a generalized-version of our approach gives an interesting sublinear regret upper bound result of $tilde {O}left ({T^{frac {s+1}{s+2}}}right)$ for any self-unaware bandit player with $s$ number of binary decision dilemma before taking the action. To further validate and complement the theoretical findings, we conduct extensive performance evaluations over synthetic data constructed by nonstochastic MAB environment simulations and wireless spectrum measurement data collected in a real-world experiment.
PubDate: June 2023
Issue No: Vol. 34, No. 6 (2023)
-
- Primal–Dual Fixed Point Algorithms Based on Adapted Metric for
Distributed Optimization-
Free pre-print version: Loading...Rate this result: What is this?Please help us test our new pre-print finding feature by giving the pre-print link a rating.
A 5 star rating indicates the linked pre-print has the exact same content as the published article.
Authors: Huaqing Li;Zuqing Zheng;Qingguo Lü;Zheng Wang;Lan Gao;Guo-Cheng Wu;Lianghao Ji;Huiwei Wang;
Pages: 2923 - 2937
Abstract: This article considers distributed optimization by a group of agents over an undirected network. The objective is to minimize the sum of a twice differentiable convex function and two possibly nonsmooth convex functions, one of which is composed of a bounded linear operator. A novel distributed primal–dual fixed point algorithm is proposed based on an adapted metric method, which exploits the second-order information of the differentiable convex function. Furthermore, by incorporating a randomized coordinate activation mechanism, we propose a randomized asynchronous iterative distributed algorithm that allows each agent to randomly and independently decide whether to perform an update or remain unchanged at each iteration, and thus alleviates the communication cost. Moreover, the proposed algorithms adopt nonidentical stepsizes to endow each agent with more independence. Numerical simulation results substantiate the feasibility of the proposed algorithms and the correctness of the theoretical results.
PubDate: June 2023
Issue No: Vol. 34, No. 6 (2023)
-
- Synchronization of Delayed Complex Networks on Time Scales via
Aperiodically Intermittent Control Using Matrix-Based Convex Combination
Method-
Free pre-print version: Loading...Rate this result: What is this?Please help us test our new pre-print finding feature by giving the pre-print link a rating.
A 5 star rating indicates the linked pre-print has the exact same content as the published article.
Authors: Peng Wan;Zhigang Zeng;
Pages: 2938 - 2950
Abstract: This article reconsiders synchronization problem of linear complex networks with time-varying delay on time scales. For different types of time scales, aperiodically intermittent control scheme is established by using a matrix-based convex combination method, which has great potential in reducing control consumption and saving communication bandwidth. By employing a common Lyapunov function, aperiodically intermittent controllers are utilized successfully to achieve synchronization of linear delayed complex networks on special time scales onto an isolated node. Next, by constructing a special Lyapunov function with time-varying coefficients, sufficient criteria that consist of two linear matrix inequalities are demonstrated to make linear delayed complex networks on general time scales synchronized onto an isolated system with an exponential convergence rate given in advance. Due to delayed complex networks in this article defined on time scales, the proposed control schemes are applicable to continuous-time networks, their discrete-time forms, and any combination of them. Four numerical examples are offered to highlight the effectiveness and superiority of the proposed aperiodically intermittent control schemes at last.
PubDate: June 2023
Issue No: Vol. 34, No. 6 (2023)
-
- Learning Social Spatio-Temporal Relation Graph in the Wild and a Video
Benchmark-
Free pre-print version: Loading...Rate this result: What is this?Please help us test our new pre-print finding feature by giving the pre-print link a rating.
A 5 star rating indicates the linked pre-print has the exact same content as the published article.
Authors: Haoran Wang;Licheng Jiao;Fang Liu;Lingling Li;Xu Liu;Deyi Ji;Weihao Gan;
Pages: 2951 - 2964
Abstract: Social relations are ubiquitous and form the basis of social structure in our daily life. However, existing studies mainly focus on recognizing social relations from still images and movie clips, which are different from real-world scenarios. For example, movie-based datasets define the task as the video classification, only recognizing one relation in the scene. In this article, we aim to study the problem of social relation recognition in an open environment. To close the gap, we provide the first video dataset collected from real-life scenarios, named social relation in the wild (SRIW), where the number of people can be huge and vary, and each pair of relations needs to be recognized. To overcome new challenges, we propose a spatio-temporal relation graph convolutional network (STRGCN) architecture, utilizing correlative visual features to recognize social relations intuitively. Our method decouples the task into two classification tasks: person-level and pair-level relation recognition. Specifically, we propose a person behavior and character module to encode moving and static features in two explicit ways. Then we take them as node features to build a relation graph with meaningful edges in a scene. Based on the relation graph, we introduce the graph convolutional network (GCN) and local GCN to encode social relation features which are used for both recognitions. Experimental results demonstrate the effectiveness of the proposed framework, achieving 83.1% and 40.8% mAP in person-level and pair-level classification. Moreover, the study also contributes to the practicality in this field.
PubDate: June 2023
Issue No: Vol. 34, No. 6 (2023)
-
- Batch-Based Learning Consensus of Multiagent Systems With Faded
Neighborhood Information-
Free pre-print version: Loading...Rate this result: What is this?Please help us test our new pre-print finding feature by giving the pre-print link a rating.
A 5 star rating indicates the linked pre-print has the exact same content as the published article.
Authors: Ganggui Qu;Dong Shen;Xinghuo Yu;
Pages: 2965 - 2977
Abstract: This article addresses the batch-based learning consensus for linear and nonlinear multiagent systems (MASs) with faded neighborhood information. The motivation comes from the observation that agents exchange information via wireless networks, which inevitably introduces random fading effect and channel additive noise to the transmitted signals. It is therefore of great significance to investigate how to ensure the precise consensus tracking to a given reference leader using heavily contaminated information. To this end, a novel distributed learning consensus scheme is proposed, which consists of a classic distributed control structure, a preliminary correction mechanism, and a separated design of learning gain and regulation matrix. The influence of biased and unbiased randomness is discussed in detail according to the convergence rate and consensus performance. The iterationwise asymptotic consensus tracking is strictly established for linear MAS first to demonstrate the inherent principles for the effectiveness of the proposed scheme. Then, the results are extended to nonlinear systems with nonidentical initialization condition and diverse gain design. The obtained results show that the distributed learning consensus scheme can achieve high-precision tracking performance for an MAS under unreliable communications. The theoretical results are verified by two illustrative simulations.
PubDate: June 2023
Issue No: Vol. 34, No. 6 (2023)
-
- FTA-GAN: A Computation-Efficient Accelerator for GANs With Fast
Transformation Algorithm-
Free pre-print version: Loading...Rate this result: What is this?Please help us test our new pre-print finding feature by giving the pre-print link a rating.
A 5 star rating indicates the linked pre-print has the exact same content as the published article.
Authors: Wendong Mao;Peixiang Yang;Zhongfeng Wang;
Pages: 2978 - 2992
Abstract: Nowadays, generative adversarial network (GAN) is making continuous breakthroughs in many machine learning tasks. The popular GANs usually involve computation-intensive deconvolution operations, leading to limited real-time applications. Prior works have brought several accelerators for deconvolution, but all of them suffer from severe problems, such as computation imbalance and large memory requirements. In this article, we first introduce a novel fast transformation algorithm (FTA) for deconvolution computation, which well solves the computation imbalance problem and removes the extra memory requirement for overlapped partial sums. Besides, it can reduce the computation complexity for various types of deconvolutions significantly. Based on FTA, we develop a fast computing core (FCC) and the corresponding computing array so that the deconvolution can be efficiently computed. We next optimize the dataflow and storage scheme to further reuse on-chip memory and improve the computation efficiency. Finally, we present a computation-efficient hardware architecture for GANs and validate it on several GAN benchmarks, such as deep convolutional GAN (DCGAN), energy-based GAN (EBGAN), and Wasserstein GAN (WGAN). The experimental results show that our design can reach 2211 GOPS under 185-MHz working frequency on Intel Stratix 10SX field-programmable gate array (FPGA) board with satisfactory visual results. In brief, the proposed design can achieve more than $2times $ hardware efficiency improvement over previous designs, and it can reduce the storage requirement drastically.
PubDate: June 2023
Issue No: Vol. 34, No. 6 (2023)
-
- Event-Triggered Impulsive Fault-Tolerant Control for Memristor-Based RDNNs
With Actuator Faults-
Free pre-print version: Loading...Rate this result: What is this?Please help us test our new pre-print finding feature by giving the pre-print link a rating.
A 5 star rating indicates the linked pre-print has the exact same content as the published article.
Authors: Ruimei Zhang;Hongxia Wang;Ju H. Park;Peisong He;Xiangpeng Xie;
Pages: 2993 - 3004
Abstract: This article focuses on designing an event-triggered impulsive fault-tolerant control strategy for the stabilization of memristor-based reaction–diffusion neural networks (RDNNs) with actuator faults. Different from the existing memristor-based RDNNs with fault-free environments, actuator faults are considered here. A hybrid event-triggered and impulsive (HETI) control scheme, which combines the advantages of event-triggered control and impulsive control, is newly proposed. The hybrid control scheme can effectively accommodate the actuator faults, save the limited communication resources, and achieve the desired system performance. Unlike the existing Lyapunov–Krasovskii functionals (LKFs) constructed on sampling intervals or required to be continuous, the introduced LKF here is directly constructed on event-triggered intervals and can be discontinuous. Based on the LKF and the HETI control scheme, new stabilization criteria are derived for memristor-based RDNNs. Finally, numerical simulations are presented to verify the effectiveness of the obtained results and the merits of the HETI control method.
PubDate: June 2023
Issue No: Vol. 34, No. 6 (2023)
-
- Jerk-Level Zhang Neurodynamics Equivalency of Bound Constraints, Equation
-
Free pre-print version: Loading...Rate this result: What is this?Please help us test our new pre-print finding feature by giving the pre-print link a rating.
A 5 star rating indicates the linked pre-print has the exact same content as the published article.
Authors: Yunong Zhang;Zhenyu Li;Min Yang;Liangjie Ming;Jinjin Guo;
Pages: 3005 - 3018
Abstract: Equivalency is a powerful approach that can transform an original problem into another problem that is relatively more ready to be resolved. In recent years, Zhang neurodynamics equivalency (ZNE), in the form of neurodynamics or recurrent neural networks (RNNs), has been investigated, abstracted, and proposed as a process that can equivalently solve equations at different levels. After long-term research, we have noticed that the ZNE can not only work with equations, but also inequations. Thus, the ZNE of inequation type is proposed, proved, and applied in this study. The ZNE of inequation type can transform different-level bound constraints into unified-level bound constraints. Applications of the jerk-level ZNE of bound constraints, equation constraints, and objective indices ultimately build up effective time-varying quadratic-programming schemes for cyclic motion planning and control (CMPC) of single and dual robot-arm systems. In addition, as an effective time-varying quadratic-programming solver, a projection neural network (PNN) is introduced. Experimental results with single and dual robot-arm systems substantiate the correctness and efficacy of ZNE and especially the ZNE of inequation type. Comparisons with conventional methods also exhibit the superiorities of ZNE.
PubDate: June 2023
Issue No: Vol. 34, No. 6 (2023)
-
- Privacy Enhancing Machine Learning via Removal of Unwanted Dependencies
-
Free pre-print version: Loading...Rate this result: What is this?Please help us test our new pre-print finding feature by giving the pre-print link a rating.
A 5 star rating indicates the linked pre-print has the exact same content as the published article.
Authors: Mert Al;Semih Yagli;Sun-Yuan Kung;
Pages: 3019 - 3033
Abstract: The rapid rise of IoT and Big Data has facilitated copious data-driven applications to enhance our quality of life. However, the omnipresent and all-encompassing nature of the data collection can generate privacy concerns. Hence, there is a strong need to develop techniques that ensure the data serve only the intended purposes, giving users control over the information they share. To this end, this article studies new variants of supervised and adversarial learning methods, which remove the sensitive information in the data before they are sent out for a particular application. The explored methods optimize privacy-preserving feature mappings and predictive models simultaneously in an end-to-end fashion. Additionally, the models are built with an emphasis on placing little computational burden on the user side so that the data can be desensitized on device in a cheap manner. Experimental results on mobile sensing and face datasets demonstrate that our models can successfully maintain the utility performances of predictive models while causing sensitive predictions to perform poorly.
PubDate: June 2023
Issue No: Vol. 34, No. 6 (2023)
-
- Finite-Time Synchronization of Neural Networks With Infinite Discrete
Time-Varying Delays and Discontinuous Activations-
Free pre-print version: Loading...Rate this result: What is this?Please help us test our new pre-print finding feature by giving the pre-print link a rating.
A 5 star rating indicates the linked pre-print has the exact same content as the published article.
Authors: Yin Sheng;Zhigang Zeng;Tingwen Huang;
Pages: 3034 - 3043
Abstract: This article investigates finite-time synchronization of neural networks (NNs) with infinite discrete time-varying delays and discontinuous activations (DDNNs). By virtue of theory of differential inclusions, comparison strategies, and inequality techniques, finite-time synchronization of the underlying DDNNs can be developed via a discontinuous state feedback control law, and the synchronous settling time can be estimated. The delayed state feedback controller and finite-time stability theorem are not employed during the analysis. As a special case, finite-time synchronization of NNs with bounded delays and discontinuous activations is given. Finally, two examples are provided to illustrate the validity of the theories.
PubDate: June 2023
Issue No: Vol. 34, No. 6 (2023)
-
- Multilabel Feature Selection: A Local Causal Structure Learning Approach
-
Free pre-print version: Loading...Rate this result: What is this?Please help us test our new pre-print finding feature by giving the pre-print link a rating.
A 5 star rating indicates the linked pre-print has the exact same content as the published article.
Authors: Kui Yu;Mingzhu Cai;Xingyu Wu;Lin Liu;Jiuyong Li;
Pages: 3044 - 3057
Abstract: Multilabel feature selection plays an essential role in high-dimensional multilabel learning tasks. Existing multilabel feature selection approaches mainly either explore the feature–label and feature–feature correlations or the label–label and feature–feature correlations. A few of them are able to deal with all three types of correlations simultaneously. To address this problem, in this article, we formulate multilabel feature selection as a local causal structure learning problem and propose a novel algorithm, M2LC. By learning the local causal structure of each class label, M2LC considers three types of feature relationships simultaneously and is scalable to high-dimensional datasets as well. To tackle false discoveries caused by the label–label correlations, M2LC consists of two novel error-correction subroutines to correct those false discoveries. Through local causal structure learning, M2LC learns the causal mechanism behind data, and thus, it can select causally informative features and visualize common features shared by class labels and specific features owned by an individual class label using the learned causal structures. Extensive experiments have been conducted to evaluate M2LC in comparison with the state-of-the-art multilabel feature selection algorithms.
PubDate: June 2023
Issue No: Vol. 34, No. 6 (2023)
-
- QBox: Partial Transfer Learning With Active Querying for Object Detection
-
Free pre-print version: Loading...Rate this result: What is this?Please help us test our new pre-print finding feature by giving the pre-print link a rating.
A 5 star rating indicates the linked pre-print has the exact same content as the published article.
Authors: Ying-Peng Tang;Xiu-Shen Wei;Borui Zhao;Sheng-Jun Huang;
Pages: 3058 - 3070
Abstract: Object detection requires plentiful data annotated with bounding boxes for model training. However, in many applications, it is difficult or even impossible to acquire a large set of labeled examples for the target task due to the privacy concern or lack of reliable annotators. On the other hand, due to the high-quality image search engines, such as Flickr and Google, it is relatively easy to obtain resource-rich unlabeled datasets, whose categories are a superset of those of target data. In this article, to improve the target model with cost-effective supervision from source data, we propose a partial transfer learning approach QBox to actively query labels for bounding boxes of source images. Specifically, we design two criteria, i.e., informativeness and transferability, to measure the potential utility of a bounding box for improving the target model. Based on these criteria, QBox actively queries the labels of the most useful boxes from the source domain and, thus, requires fewer training examples to save the labeling cost. Furthermore, the proposed query strategy allows annotators to simply labeling a specific region, instead of the whole image, and, thus, significantly reduces the labeling difficulty. Extensive experiments are performed on various partial transfer benchmarks and a real COVID-19 detection task. The results validate that QBox improves the detection accuracy with lower labeling cost compared to state-of-the-art query strategies for object detection.
PubDate: June 2023
Issue No: Vol. 34, No. 6 (2023)
-
- Joint Feature Selection and Extraction With Sparse Unsupervised Projection
-
Free pre-print version: Loading...Rate this result: What is this?Please help us test our new pre-print finding feature by giving the pre-print link a rating.
A 5 star rating indicates the linked pre-print has the exact same content as the published article.
Authors: Jingyu Wang;Lin Wang;Feiping Nie;Xuelong Li;
Pages: 3071 - 3081
Abstract: Feature selection and feature extraction, in the field of data dimensionality reduction, are the two main strategies. Nevertheless, each of these two strategies has its own advantages and disadvantages. The features chosen by feature selection method have complete physical meaning. However, feature selection cannot reveal the implicit structural information of the samples. In this article, the methods proposed by us combine both feature selection and feature extraction, called joint feature selection and extraction with sparse unsupervised projection (SUP) and graph optimization SUP (GOSUP). A constraint on the number of nonzero rows of the projection matrix is added, which ensures the sparsity of the projection matrix, and only the features corresponding to the nonzero rows of the projection matrix are selected for the feature extraction procedure. We invoke a newly proposed algorithm to tackle this constrained optimization problem. A new concept of “purification matrix” is invented, the use of which could better eliminate meaningless information of samples in subspace. The performance on several datasets verifies the effectiveness of the proposed method for data dimensionality reduction.
PubDate: June 2023
Issue No: Vol. 34, No. 6 (2023)
-
- Globally Localized Multisource Domain Adaptation for Cross-Domain Fault
Diagnosis With Category Shift-
Free pre-print version: Loading...Rate this result: What is this?Please help us test our new pre-print finding feature by giving the pre-print link a rating.
A 5 star rating indicates the linked pre-print has the exact same content as the published article.
Authors: Yong Feng;Jinglong Chen;Shuilong He;Tongyang Pan;Zitong Zhou;
Pages: 3082 - 3096
Abstract: Deep learning has demonstrated splendid performance in mechanical fault diagnosis on condition that source and target data are identically distributed. In engineering practice, however, the domain shift between source and target domains significantly limits the further application of intelligent algorithms. Despite various transfer techniques proposed, either they focus on single-source domain adaptation (SDA) or they utilize multisource domain globally or locally, which both cannot address the cross-domain diagnosis effectively, especially with category shift. To this end, we propose globally localized multisource DA for cross-domain fault diagnosis with category shift. Specifically, we construct a GlocalNet to fuse multisource information comprehensively, which consists of a feature generator and three classifiers. By optimizing the Wasserstein discrepancy of classifiers locally and accumulative higher order multisource moment globally, multisource DA is achieved from domain and class levels thus to reduce the shift on domain and category. To refine the classifier at sample level, a distilling strategy is presented. Finally, an adaptive weighting policy is employed for reliable result. To evaluate the effectiveness, the proposed method is compared with multiple methods on four bearing vibration datasets. Experimental results indicate the superiority and practicability of the proposed method for cross-domain fault diagnosis.
PubDate: June 2023
Issue No: Vol. 34, No. 6 (2023)
-
- Spatiotemporal Sequence Prediction With Point Processes and
Self-Organizing Decision Trees-
Free pre-print version: Loading...Rate this result: What is this?Please help us test our new pre-print finding feature by giving the pre-print link a rating.
A 5 star rating indicates the linked pre-print has the exact same content as the published article.
Authors: Oguzhan Karaahmetoglu;Suleyman Serdar Kozat;
Pages: 3097 - 3110
Abstract: We study the spatiotemporal prediction problem and introduce a novel point-process-based prediction algorithm. Spatiotemporal prediction is extensively studied in machine learning literature due to its critical real-life applications, such as crime, earthquake, and social event prediction. Despite these thorough studies, specific problems inherent to the application domain are not yet fully explored. Here, we address the nonstationary spatiotemporal prediction problem on both densely and sparsely distributed sequences. We introduce a probabilistic approach that partitions the spatial domain into subregions and models the event arrivals in each region with interacting point processes. Our algorithm can jointly learn the spatial partitioning and the interaction between these regions through a gradient-based optimization procedure. Finally, we demonstrate the performance of our algorithm on both simulated data and two real-life datasets. We compare our approach with baseline and state-of-the-art deep learning-based approaches, where we achieve significant performance improvements. Moreover, we also show the effect of using different parameters on the overall performance through empirical results and explain the procedure for choosing the parameters.
PubDate: June 2023
Issue No: Vol. 34, No. 6 (2023)
-
- Bayesian Matrix Factorization for Semibounded Data
-
Free pre-print version: Loading...Rate this result: What is this?Please help us test our new pre-print finding feature by giving the pre-print link a rating.
A 5 star rating indicates the linked pre-print has the exact same content as the published article.
Authors: Oumayma Dalhoumi;Nizar Bouguila;Manar Amayri;Wentao Fan;
Pages: 3111 - 3123
Abstract: Bayesian non-negative matrix factorization (BNMF) has been widely used in different applications. In this article, we propose a novel BNMF technique dedicated to semibounded data where each entry of the observed matrix is supposed to follow an Inverted Beta distribution. The model has two parameter matrices with the same size as the observation matrix which we factorize into a product of excitation and basis matrices. Entries of the corresponding basis and excitation matrices follow a Gamma prior. To estimate the parameters of the model, variational Bayesian inference is used. A lower bound approximation for the objective function is used to find an analytically tractable solution for the model. An online extension of the algorithm is also proposed for more scalability and to adapt to streaming data. The model is evaluated on five different applications: part-based decomposition, collaborative filtering, market basket analysis, transactions prediction and items classification, topic mining, and graph embedding on biomedical networks.
PubDate: June 2023
Issue No: Vol. 34, No. 6 (2023)
-
- Intermittent Learning Through Operant Conditioning for Cyber-Physical
Systems-
Free pre-print version: Loading...Rate this result: What is this?Please help us test our new pre-print finding feature by giving the pre-print link a rating.
A 5 star rating indicates the linked pre-print has the exact same content as the published article.
Authors: Prachi Pratyusha Sahoo;Aris Kanellopoulos;Kyriakos G. Vamvoudakis;
Pages: 3124 - 3134
Abstract: This article presents a novel scheme, namely, an intermittent learning scheme based on Skinner’s operant conditioning techniques that approximates the optimal policy while decreasing the usage of the communication buses transferring information. While traditional reinforcement learning schemes continuously evaluate and subsequently improve, every action taken by a specific learning agent based on received reinforcement signals, this form of continuous transmission of reinforcement signals and policy improvement signals can cause overutilization of the system’s inherently limited resources. Moreover, the highly complex nature of the operating environment for cyber-physical systems (CPSs) creates a gap for malicious individuals to corrupt the signal transmissions between various components. The proposed schemes will increase uncertainty in the learning rate and the extinction rate of the acquired behavior of the learning agents. In this article, we investigate the use of fixed/variable interval and fixed/variable ratio schedules in CPSs along with their rate of success and loss in their optimal behavior incurred during intermittent learning. Simulation results show the efficacy of the proposed approach.
PubDate: June 2023
Issue No: Vol. 34, No. 6 (2023)
-
- COMIRE: A Consistence-Based Mislabeled Instances Removal Method
-
Free pre-print version: Loading...Rate this result: What is this?Please help us test our new pre-print finding feature by giving the pre-print link a rating.
A 5 star rating indicates the linked pre-print has the exact same content as the published article.
Authors: Xiaokun Pu;Chunguang Li;Hui-Liang Shen;
Pages: 3135 - 3145
Abstract: Training neural network classifiers (NNCs) usually requires all instances to be correctly labeled, which is difficult and/or expensive to satisfy in some practical applications. When label noise is present, mislabeled data will severely mislead the training of NNCs, resulting in poor generalization performance. In this work, we address the label noise issue by removing mislabeled instances from the training data. A COnsistence-based Mislabeled Instances REmoval (COMIRE) method is proposed. The main idea is based on the observation that during the training of the NNC, the training loss and the model’s prediction uncertainty of correctly labeled instances show similar trends, while those of mislabeled instances have quite different trends. Thus, the consistency between the two trends can be used to distinguish correctly labeled instances from mislabeled ones. On this basis, an iteration scheme is introduced to further increase the separability between the two types of data. Experimental results show that COMIRE can effectively identify the mislabeled instances. Moreover, the classification performance is significantly improved after removing the identified instances from the noisy training data.
PubDate: June 2023
Issue No: Vol. 34, No. 6 (2023)
-
- Training Provably Robust Models by Polyhedral Envelope Regularization
-
Free pre-print version: Loading...Rate this result: What is this?Please help us test our new pre-print finding feature by giving the pre-print link a rating.
A 5 star rating indicates the linked pre-print has the exact same content as the published article.
Authors: Chen Liu;Mathieu Salzmann;Sabine Süsstrunk;
Pages: 3146 - 3160
Abstract: Training certifiable neural networks enables us to obtain models with robustness guarantees against adversarial attacks. In this work, we introduce a framework to obtain a provable adversarial-free region in the neighborhood of the input data by a polyhedral envelope, which yields more fine-grained certified robustness than existing methods. We further introduce polyhedral envelope regularization (PER) to encourage larger adversarial-free regions and thus improve the provable robustness of the models. We demonstrate the flexibility and effectiveness of our framework on standard benchmarks; it applies to networks of different architectures and with general activation functions. Compared with state of the art, PER has negligible computational overhead; it achieves better robustness guarantees and accuracy on the clean data in various settings.
PubDate: June 2023
Issue No: Vol. 34, No. 6 (2023)
-
- Data-Driven Bipartite Formation for a Class of Nonlinear MIMO Multiagent
Systems-
Free pre-print version: Loading...Rate this result: What is this?Please help us test our new pre-print finding feature by giving the pre-print link a rating.
A 5 star rating indicates the linked pre-print has the exact same content as the published article.
Authors: Jiaqi Liang;Xuhui Bu;Lizhi Cui;Zhongsheng Hou;
Pages: 3161 - 3173
Abstract: The bipartite formation control for the nonlinear discrete-time multiagent systems with signed digraph is considered in this article, in which the dynamics of the agents are completely unknown and multi-input multi-output (MIMO). First, the unknown nonlinear dynamic is converted into the compact-form dynamic linearization (CFDL) data model with a pseudo-Jacobian matrix (PJM). Based on the structurally balanced signed graph, a distance-based formation term is constructed and a bipartite formation model-free adaptive control (MFAC) protocol is designed. By employing the measured input and output data of the agents, the theoretical analysis is developed to prove the bounded-input bounded-output stability and the asymptotic convergence of the formation tracking error. Finally, the effectiveness of the proposed protocol is verified by two numerical examples.
PubDate: June 2023
Issue No: Vol. 34, No. 6 (2023)
-
- DIET-SNN: A Low-Latency Spiking Neural Network With Direct Input Encoding
and Leakage and Threshold Optimization-
Free pre-print version: Loading...Rate this result: What is this?Please help us test our new pre-print finding feature by giving the pre-print link a rating.
A 5 star rating indicates the linked pre-print has the exact same content as the published article.
Authors: Nitin Rathi;Kaushik Roy;
Pages: 3174 - 3182
Abstract: Bioinspired spiking neural networks (SNNs), operating with asynchronous binary signals (or spikes) distributed over time, can potentially lead to greater computational efficiency on event-driven hardware. The state-of-the-art SNNs suffer from high inference latency, resulting from inefficient input encoding and suboptimal settings of the neuron parameters (firing threshold and membrane leak). We propose DIET-SNN, a low-latency deep spiking network trained with gradient descent to optimize the membrane leak and the firing threshold along with other network parameters (weights). The membrane leak and threshold of each layer are optimized with end-to-end backpropagation to achieve competitive accuracy at reduced latency. The input layer directly processes the analog pixel values of an image without converting it to spike train. The first convolutional layer converts analog inputs into spikes where leaky-integrate-and-fire (LIF) neurons integrate the weighted inputs and generate an output spike when the membrane potential crosses the trained firing threshold. The trained membrane leak selectively attenuates the membrane potential, which increases activation sparsity in the network. The reduced latency combined with high activation sparsity provides massive improvements in computational efficiency. We evaluate DIET-SNN on image classification tasks from CIFAR and ImageNet datasets on VGG and ResNet architectures. We achieve top-1 accuracy of 69% with five timesteps (inference latency) on the ImageNet dataset with $12times $ less compute energy than an equivalent standard artificial neural network (ANN). In addition, DIET-SNN performs 20– $500times $ faster inference compared to other state-of-the-art SNN models.
PubDate: June 2023
Issue No: Vol. 34, No. 6 (2023)
-
- Knowledge Distillation Classifier Generation Network for Zero-Shot
Learning-
Free pre-print version: Loading...Rate this result: What is this?Please help us test our new pre-print finding feature by giving the pre-print link a rating.
A 5 star rating indicates the linked pre-print has the exact same content as the published article.
Authors: Yunlong Yu;Bin Li;Zhong Ji;Jungong Han;Zhongfei Zhang;
Pages: 3183 - 3194
Abstract: In this article, we present a conceptually simple but effective framework called knowledge distillation classifier generation network (KDCGN) for zero-shot learning (ZSL), where the learning agent requires recognizing unseen classes that have no visual data for training. Different from the existing generative approaches that synthesize visual features for unseen classifiers’ learning, the proposed framework directly generates classifiers for unseen classes conditioned on the corresponding class-level semantics. To ensure the generated classifiers to be discriminative to the visual features, we borrow the knowledge distillation idea to both supervise the classifier generation and distill the knowledge with, respectively, the visual classifiers and soft targets trained from a traditional classification network. Under this framework, we develop two, respectively, strategies, i.e., class augmentation and semantics guidance, to facilitate the supervision process from the perspectives of improving visual classifiers. Specifically, the class augmentation strategy incorporates some additional categories to train the visual classifiers, which regularizes the visual classifier weights to be compact, under supervision of which the generated classifiers will be more discriminative. The semantics-guidance strategy encodes the class semantics into the visual classifiers, which would facilitate the supervision process by minimizing the differences between the generated and the real-visual classifiers. To evaluate the effectiveness of the proposed framework, we have conducted extensive experiments on five datasets in image classification, i.e., AwA1, AwA2, CUB, FLO, and APY. Experimental results show that the proposed approach performs best in the traditional ZSL task and achieves a significant performance improvement on four out of the five datasets in the generalized ZSL task.
PubDate: June 2023
Issue No: Vol. 34, No. 6 (2023)
-
- Optimal Adaptive Control of Uncertain Nonlinear Continuous-Time Systems
With Input and State Delays-
Free pre-print version: Loading...Rate this result: What is this?Please help us test our new pre-print finding feature by giving the pre-print link a rating.
A 5 star rating indicates the linked pre-print has the exact same content as the published article.
Authors: Rohollah Moghadam;Sarangapani Jagannathan;
Pages: 3195 - 3204
Abstract: In this article, an actor-critic neural network (NN)-based online optimal adaptive regulation of a class of nonlinear continuous-time systems with known state and input delays and uncertain system dynamics is introduced. The temporal difference error (TDE), which is dependent upon state and input delays, is derived using actual and estimated value function and via integral reinforcement learning. The NN weights of the critic are tuned at every sampling instant as a function of the instantaneous integral TDE. A novel identifier, which is introduced to estimate the control coefficient matrices, is utilized to obtain the estimated control policy. The boundedness of the state vector, critic NN weights, identification error, and NN identifier weights are shown through the Lyapunov analysis. Simulation results are provided to illustrate the effectiveness of the proposed approach.
PubDate: June 2023
Issue No: Vol. 34, No. 6 (2023)
-
- LMFFNet: A Well-Balanced Lightweight Network for Fast and Accurate
Semantic Segmentation-
Free pre-print version: Loading...Rate this result: What is this?Please help us test our new pre-print finding feature by giving the pre-print link a rating.
A 5 star rating indicates the linked pre-print has the exact same content as the published article.
Authors: Min Shi;Jialin Shen;Qingming Yi;Jian Weng;Zunkai Huang;Aiwen Luo;Yicong Zhou;
Pages: 3205 - 3219
Abstract: Real-time semantic segmentation is widely used in autonomous driving and robotics. Most previous networks achieved great accuracy based on a complicated model involving mass computing. The existing lightweight networks generally reduce the parameter sizes by sacrificing the segmentation accuracy. It is critical to balance the parameters and accuracy for real-time semantic segmentation. In this article, we propose a lightweight multiscale-feature-fusion network (LMFFNet) mainly composed of three types of components: split-extract-merge bottleneck (SEM-B) block, feature fusion module (FFM), and multiscale attention decoder (MAD), where the SEM-B block extracts sufficient features with fewer parameters. FFMs fuse multiscale semantic features to effectively improve the segmentation accuracy and the MAD well recovers the details of the input images through the attention mechanism. Without pretraining, LMFFNet-3-8 achieves 75.1% mean intersection over union (mIoU) with 1.4 M parameters at 118.9 frames/s using RTX 3090 GPU. More experiments are investigated extensively on various resolutions on other three datasets of CamVid, KITTI, and WildDash2. The experiments verify that the proposed LMFFNet model makes a decent tradeoff between segmentation accuracy and inference speed for real-time tasks. The source code is publicly available at https://github.com/Greak-1124/LMFFNet.
PubDate: June 2023
Issue No: Vol. 34, No. 6 (2023)
-
- Stability Analysis of Delayed Recurrent Neural Networks via a Quadratic
Matrix Convex Combination Approach-
Free pre-print version: Loading...Rate this result: What is this?Please help us test our new pre-print finding feature by giving the pre-print link a rating.
A 5 star rating indicates the linked pre-print has the exact same content as the published article.
Authors: Shasha Xiao;Zhanshan Wang;Yufeng Tian;
Pages: 3220 - 3225
Abstract: This brief addresses the stability analysis problem of a class of delayed recurrent neural networks (DRNNs). In previously published studies, the slope information of activation function (SIAF) is just reflected in three slope information matrices, i.e., the upper and lower boundary matrices and the maximum norm matrix. In practice, there are $2^{n}$ possible combination cases on the slope information matrices. To exploit more information about SIAF, first, an activation function separation method is proposed to derive $n$ slope-information-based uncertainties (SIBUs) containing SIAF; second, a quadratic matrix convex combination approach is proposed to dispose $n$ SIBUs using $2^{n}$ combination slope information matrices. Third, a stability criterion with less conservatism is established based on the proposed approach. Finally, two simulation examples are used to testify the validity of theoretical results.
PubDate: June 2023
Issue No: Vol. 34, No. 6 (2023)
-
- Synchronization and Control for Multiweighted and Directed Complex
Networks-
Free pre-print version: Loading...Rate this result: What is this?Please help us test our new pre-print finding feature by giving the pre-print link a rating.
A 5 star rating indicates the linked pre-print has the exact same content as the published article.
Authors: Xiwei Liu;
Pages: 3226 - 3233
Abstract: The study of complex networks with multiweights (CNMWs) has been a hot topic recently. For a network with a single weight, previous studies have shown that they can promote synchronization, but for CNMWs, there are no rigorous analyses about the role of coupling matrices. In this brief, the complex network is allowed to be directed, which is the main difference with previous studies and may make the synchronization analysis difficult for multiple couplings. At first, we prove that if the inner coupling matrices are all diagonal, then synchronization can be realized only if the weighted sum (or union) of multiple coupling matrices is strongly connected, which bridges the gap between single-weighted and multiweighted networks. Moreover, we also consider the case that inner coupling matrices are positive definite but not diagonal. We design two techniques for this hard problem. One technique is to decompose inner coupling matrices into diagonal matrices and residual matrices. The other one is to measure the similarity between outer coupling matrices. In virtue of the normalized left eigenvectors (NLEVecs) corresponding to the zero eigenvalue of coupling matrices, we prove that if the Chebyshev distance between NLEVec is less than some value, defined as the allowable deviation bound, then the synchronization and control will be realized with sufficiently large coupling strengths. Furthermore, adaptive rules are also designed for coupling strength.
PubDate: June 2023
Issue No: Vol. 34, No. 6 (2023)
-
- Parameter-Free Loss for Class-Imbalanced Deep Learning in Image
Classification-
Free pre-print version: Loading...Rate this result: What is this?Please help us test our new pre-print finding feature by giving the pre-print link a rating.
A 5 star rating indicates the linked pre-print has the exact same content as the published article.
Authors: Jie Du;Yanhong Zhou;Peng Liu;Chi-Man Vong;Tianfu Wang;
Pages: 3234 - 3240
Abstract: Current state-of-the-art class-imbalanced loss functions for deep models require exhaustive tuning on hyperparameters for high model performance, resulting in low training efficiency and impracticality for nonexpert users. To tackle this issue, a parameter-free loss (PF-loss) function is proposed, which works for both binary and multiclass-imbalanced deep learning for image classification tasks. PF-loss provides three advantages: 1) training time is significantly reduced due to NO tuning on hyperparameter(s); 2) it dynamically pays more attention on minority classes (rather than outliers compared to the existing loss functions) with NO hyperparameters in the loss function; and 3) higher accuracy can be achieved since it adapts to the changes of data distribution in each mini-batch instead of the fixed hyperparameters in the existing methods during training, especially when the data are highly skewed. Experimental results on some classical image datasets with different imbalance ratios (IR, up to 200) show that PF-loss reduces the training time down to 1/148 of that spent by compared state-of-the-art losses and simultaneously achieves comparable or even higher accuracy in terms of both G-mean and area under receiver operating characteristic (ROC) curve (AUC) metrics, especially when the data are highly skewed.
PubDate: June 2023
Issue No: Vol. 34, No. 6 (2023)
-
