Publisher: Elsevier   (Total: 3161 journals)

 A  B  C  D  E  F  G  H  I  J  K  L  M  N  O  P  Q  R  S  T  U  V  W  X  Y  Z  

        1 2 3 4 5 6 7 8 | Last   [Sort by number of followers]   [Restore default list]

Showing 1 - 200 of 3161 Journals sorted alphabetically
Academic Pediatrics     Hybrid Journal   (Followers: 39, SJR: 1.655, CiteScore: 2)
Academic Radiology     Hybrid Journal   (Followers: 26, SJR: 1.015, CiteScore: 2)
Accident Analysis & Prevention     Partially Free   (Followers: 106, SJR: 1.462, CiteScore: 3)
Accounting Forum     Hybrid Journal   (Followers: 28, SJR: 0.932, CiteScore: 2)
Accounting, Organizations and Society     Hybrid Journal   (Followers: 43, SJR: 1.771, CiteScore: 3)
Achievements in the Life Sciences     Open Access   (Followers: 7)
Acta Anaesthesiologica Taiwanica     Open Access   (Followers: 6)
Acta Astronautica     Hybrid Journal   (Followers: 446, SJR: 0.758, CiteScore: 2)
Acta Automatica Sinica     Full-text available via subscription   (Followers: 2)
Acta Biomaterialia     Hybrid Journal   (Followers: 30, SJR: 1.967, CiteScore: 7)
Acta Colombiana de Cuidado Intensivo     Full-text available via subscription   (Followers: 3)
Acta de Investigación Psicológica     Open Access   (Followers: 2)
Acta Ecologica Sinica     Open Access   (Followers: 11, SJR: 0.18, CiteScore: 1)
Acta Histochemica     Hybrid Journal   (Followers: 5, SJR: 0.661, CiteScore: 2)
Acta Materialia     Hybrid Journal   (Followers: 322, SJR: 3.263, CiteScore: 6)
Acta Mathematica Scientia     Full-text available via subscription   (Followers: 5, SJR: 0.504, CiteScore: 1)
Acta Mechanica Solida Sinica     Full-text available via subscription   (Followers: 9, SJR: 0.542, CiteScore: 1)
Acta Oecologica     Hybrid Journal   (Followers: 12, SJR: 0.834, CiteScore: 2)
Acta Otorrinolaringologica (English Edition)     Full-text available via subscription  
Acta Otorrinolaringológica Española     Full-text available via subscription   (Followers: 2, SJR: 0.307, CiteScore: 0)
Acta Pharmaceutica Sinica B     Open Access   (Followers: 2, SJR: 1.793, CiteScore: 6)
Acta Psychologica     Hybrid Journal   (Followers: 26, SJR: 1.331, CiteScore: 2)
Acta Sociológica     Open Access   (Followers: 1)
Acta Tropica     Hybrid Journal   (Followers: 6, SJR: 1.052, CiteScore: 2)
Acta Urológica Portuguesa     Open Access   (Followers: 1)
Actas Dermo-Sifiliograficas     Full-text available via subscription   (Followers: 3, SJR: 0.374, CiteScore: 1)
Actas Dermo-Sifiliográficas (English Edition)     Full-text available via subscription   (Followers: 2)
Actas Urológicas Españolas     Full-text available via subscription   (Followers: 3, SJR: 0.344, CiteScore: 1)
Actas Urológicas Españolas (English Edition)     Full-text available via subscription   (Followers: 1)
Actualites Pharmaceutiques     Full-text available via subscription   (Followers: 7, SJR: 0.19, CiteScore: 0)
Actualites Pharmaceutiques Hospitalieres     Full-text available via subscription   (Followers: 3)
Acupuncture and Related Therapies     Hybrid Journal   (Followers: 8)
Acute Pain     Full-text available via subscription   (Followers: 15, SJR: 2.671, CiteScore: 5)
Ad Hoc Networks     Hybrid Journal   (Followers: 11, SJR: 0.53, CiteScore: 4)
Addictive Behaviors     Hybrid Journal   (Followers: 18, SJR: 1.29, CiteScore: 3)
Addictive Behaviors Reports     Open Access   (Followers: 9, SJR: 0.755, CiteScore: 2)
Additive Manufacturing     Hybrid Journal   (Followers: 13, SJR: 2.611, CiteScore: 8)
Additives for Polymers     Full-text available via subscription   (Followers: 22)
Advanced Drug Delivery Reviews     Hybrid Journal   (Followers: 188, SJR: 4.09, CiteScore: 13)
Advanced Engineering Informatics     Hybrid Journal   (Followers: 13, SJR: 1.167, CiteScore: 4)
Advanced Powder Technology     Hybrid Journal   (Followers: 17, SJR: 0.694, CiteScore: 3)
Advances in Accounting     Hybrid Journal   (Followers: 9, SJR: 0.277, CiteScore: 1)
Advances in Agronomy     Full-text available via subscription   (Followers: 17, SJR: 2.384, CiteScore: 5)
Advances in Anesthesia     Full-text available via subscription   (Followers: 30, SJR: 0.126, CiteScore: 0)
Advances in Antiviral Drug Design     Full-text available via subscription   (Followers: 2)
Advances in Applied Mathematics     Full-text available via subscription   (Followers: 12, SJR: 0.992, CiteScore: 1)
Advances in Applied Mechanics     Full-text available via subscription   (Followers: 12, SJR: 1.551, CiteScore: 4)
Advances in Applied Microbiology     Full-text available via subscription   (Followers: 24, SJR: 2.089, CiteScore: 5)
Advances In Atomic, Molecular, and Optical Physics     Full-text available via subscription   (Followers: 15, SJR: 0.572, CiteScore: 2)
Advances in Biological Regulation     Hybrid Journal   (Followers: 4, SJR: 2.61, CiteScore: 7)
Advances in Botanical Research     Full-text available via subscription   (Followers: 1, SJR: 0.686, CiteScore: 2)
Advances in Cancer Research     Full-text available via subscription   (Followers: 35, SJR: 3.043, CiteScore: 6)
Advances in Carbohydrate Chemistry and Biochemistry     Full-text available via subscription   (Followers: 9, SJR: 1.453, CiteScore: 2)
Advances in Catalysis     Full-text available via subscription   (Followers: 5, SJR: 1.992, CiteScore: 5)
Advances in Cell Aging and Gerontology     Full-text available via subscription   (Followers: 5)
Advances in Cellular and Molecular Biology of Membranes and Organelles     Full-text available via subscription   (Followers: 14)
Advances in Chemical Engineering     Full-text available via subscription   (Followers: 29, SJR: 0.156, CiteScore: 1)
Advances in Child Development and Behavior     Full-text available via subscription   (Followers: 11, SJR: 0.713, CiteScore: 1)
Advances in Chronic Kidney Disease     Full-text available via subscription   (Followers: 11, SJR: 1.316, CiteScore: 2)
Advances in Clinical Chemistry     Full-text available via subscription   (Followers: 26, SJR: 1.562, CiteScore: 3)
Advances in Colloid and Interface Science     Full-text available via subscription   (Followers: 21, SJR: 1.977, CiteScore: 8)
Advances in Computers     Full-text available via subscription   (Followers: 14, SJR: 0.205, CiteScore: 1)
Advances in Dermatology     Full-text available via subscription   (Followers: 16)
Advances in Developmental Biology     Full-text available via subscription   (Followers: 14)
Advances in Digestive Medicine     Open Access   (Followers: 13)
Advances in DNA Sequence-Specific Agents     Full-text available via subscription   (Followers: 7)
Advances in Drug Research     Full-text available via subscription   (Followers: 26)
Advances in Ecological Research     Full-text available via subscription   (Followers: 45, SJR: 2.524, CiteScore: 4)
Advances in Engineering Software     Hybrid Journal   (Followers: 30, SJR: 1.159, CiteScore: 4)
Advances in Experimental Biology     Full-text available via subscription   (Followers: 9)
Advances in Experimental Social Psychology     Full-text available via subscription   (Followers: 52, SJR: 5.39, CiteScore: 8)
Advances in Exploration Geophysics     Full-text available via subscription   (Followers: 2)
Advances in Fluorine Science     Full-text available via subscription   (Followers: 9)
Advances in Food and Nutrition Research     Full-text available via subscription   (Followers: 68, SJR: 0.591, CiteScore: 2)
Advances in Fuel Cells     Full-text available via subscription   (Followers: 17)
Advances in Genetics     Full-text available via subscription   (Followers: 21, SJR: 1.354, CiteScore: 4)
Advances in Genome Biology     Full-text available via subscription   (Followers: 12, SJR: 12.74, CiteScore: 13)
Advances in Geophysics     Full-text available via subscription   (Followers: 8, SJR: 1.193, CiteScore: 3)
Advances in Heat Transfer     Full-text available via subscription   (Followers: 26, SJR: 0.368, CiteScore: 1)
Advances in Heterocyclic Chemistry     Full-text available via subscription   (Followers: 11, SJR: 0.749, CiteScore: 3)
Advances in Human Factors/Ergonomics     Full-text available via subscription   (Followers: 26)
Advances in Imaging and Electron Physics     Full-text available via subscription   (Followers: 3, SJR: 0.193, CiteScore: 0)
Advances in Immunology     Full-text available via subscription   (Followers: 37, SJR: 4.433, CiteScore: 6)
Advances in Inorganic Chemistry     Full-text available via subscription   (Followers: 10, SJR: 1.163, CiteScore: 2)
Advances in Insect Physiology     Full-text available via subscription   (Followers: 2, SJR: 1.938, CiteScore: 3)
Advances in Integrative Medicine     Hybrid Journal   (Followers: 6, SJR: 0.176, CiteScore: 0)
Advances in Intl. Accounting     Full-text available via subscription   (Followers: 3)
Advances in Life Course Research     Hybrid Journal   (Followers: 9, SJR: 0.682, CiteScore: 2)
Advances in Lipobiology     Full-text available via subscription   (Followers: 1)
Advances in Magnetic and Optical Resonance     Full-text available via subscription   (Followers: 8)
Advances in Marine Biology     Full-text available via subscription   (Followers: 21, SJR: 0.88, CiteScore: 2)
Advances in Mathematics     Full-text available via subscription   (Followers: 17, SJR: 3.027, CiteScore: 2)
Advances in Medical Sciences     Hybrid Journal   (Followers: 9, SJR: 0.694, CiteScore: 2)
Advances in Medicinal Chemistry     Full-text available via subscription   (Followers: 6)
Advances in Microbial Physiology     Full-text available via subscription   (Followers: 5, SJR: 1.158, CiteScore: 3)
Advances in Molecular and Cell Biology     Full-text available via subscription   (Followers: 26)
Advances in Molecular and Cellular Endocrinology     Full-text available via subscription   (Followers: 8)
Advances in Molecular Toxicology     Full-text available via subscription   (Followers: 7, SJR: 0.182, CiteScore: 0)
Advances in Nanoporous Materials     Full-text available via subscription   (Followers: 5)
Advances in Oncobiology     Full-text available via subscription   (Followers: 2)
Advances in Organ Biology     Full-text available via subscription   (Followers: 2)
Advances in Organometallic Chemistry     Full-text available via subscription   (Followers: 18, SJR: 1.875, CiteScore: 4)
Advances in Parallel Computing     Full-text available via subscription   (Followers: 7, SJR: 0.174, CiteScore: 0)
Advances in Parasitology     Full-text available via subscription   (Followers: 6, SJR: 1.579, CiteScore: 4)
Advances in Pediatrics     Full-text available via subscription   (Followers: 27, SJR: 0.461, CiteScore: 1)
Advances in Pharmaceutical Sciences     Full-text available via subscription   (Followers: 19)
Advances in Pharmacology     Full-text available via subscription   (Followers: 17, SJR: 1.536, CiteScore: 3)
Advances in Physical Organic Chemistry     Full-text available via subscription   (Followers: 10, SJR: 0.574, CiteScore: 1)
Advances in Phytomedicine     Full-text available via subscription  
Advances in Planar Lipid Bilayers and Liposomes     Full-text available via subscription   (Followers: 3, SJR: 0.109, CiteScore: 1)
Advances in Plant Biochemistry and Molecular Biology     Full-text available via subscription   (Followers: 11)
Advances in Plant Pathology     Full-text available via subscription   (Followers: 6)
Advances in Porous Media     Full-text available via subscription   (Followers: 5)
Advances in Protein Chemistry     Full-text available via subscription   (Followers: 19)
Advances in Protein Chemistry and Structural Biology     Full-text available via subscription   (Followers: 20, SJR: 0.791, CiteScore: 2)
Advances in Psychology     Full-text available via subscription   (Followers: 69)
Advances in Quantum Chemistry     Full-text available via subscription   (Followers: 7, SJR: 0.371, CiteScore: 1)
Advances in Radiation Oncology     Open Access   (Followers: 3, SJR: 0.263, CiteScore: 1)
Advances in Small Animal Medicine and Surgery     Hybrid Journal   (Followers: 3, SJR: 0.101, CiteScore: 0)
Advances in Space Biology and Medicine     Full-text available via subscription   (Followers: 7)
Advances in Space Research     Full-text available via subscription   (Followers: 430, SJR: 0.569, CiteScore: 2)
Advances in Structural Biology     Full-text available via subscription   (Followers: 6)
Advances in Surgery     Full-text available via subscription   (Followers: 13, SJR: 0.555, CiteScore: 2)
Advances in the Study of Behavior     Full-text available via subscription   (Followers: 37, SJR: 2.208, CiteScore: 4)
Advances in Veterinary Medicine     Full-text available via subscription   (Followers: 20)
Advances in Veterinary Science and Comparative Medicine     Full-text available via subscription   (Followers: 15)
Advances in Virus Research     Full-text available via subscription   (Followers: 6, SJR: 2.262, CiteScore: 5)
Advances in Water Resources     Hybrid Journal   (Followers: 56, SJR: 1.551, CiteScore: 3)
Aeolian Research     Hybrid Journal   (Followers: 6, SJR: 1.117, CiteScore: 3)
Aerospace Science and Technology     Hybrid Journal   (Followers: 394, SJR: 0.796, CiteScore: 3)
AEU - Intl. J. of Electronics and Communications     Hybrid Journal   (Followers: 8, SJR: 0.42, CiteScore: 2)
African J. of Emergency Medicine     Open Access   (Followers: 6, SJR: 0.296, CiteScore: 0)
Ageing Research Reviews     Hybrid Journal   (Followers: 12, SJR: 3.671, CiteScore: 9)
Aggression and Violent Behavior     Hybrid Journal   (Followers: 487, SJR: 1.238, CiteScore: 3)
Agri Gene     Hybrid Journal   (Followers: 1, SJR: 0.13, CiteScore: 0)
Agricultural and Forest Meteorology     Hybrid Journal   (Followers: 18, SJR: 1.818, CiteScore: 5)
Agricultural Systems     Hybrid Journal   (Followers: 32, SJR: 1.156, CiteScore: 4)
Agricultural Water Management     Hybrid Journal   (Followers: 46, SJR: 1.272, CiteScore: 3)
Agriculture and Agricultural Science Procedia     Open Access   (Followers: 4)
Agriculture and Natural Resources     Open Access   (Followers: 3)
Agriculture, Ecosystems & Environment     Hybrid Journal   (Followers: 58, SJR: 1.747, CiteScore: 4)
Ain Shams Engineering J.     Open Access   (Followers: 5, SJR: 0.589, CiteScore: 3)
Air Medical J.     Hybrid Journal   (Followers: 8, SJR: 0.26, CiteScore: 0)
AKCE Intl. J. of Graphs and Combinatorics     Open Access   (SJR: 0.19, CiteScore: 0)
Alcohol     Hybrid Journal   (Followers: 12, SJR: 1.153, CiteScore: 3)
Alcoholism and Drug Addiction     Open Access   (Followers: 12)
Alergologia Polska : Polish J. of Allergology     Full-text available via subscription   (Followers: 1)
Alexandria Engineering J.     Open Access   (Followers: 2, SJR: 0.604, CiteScore: 3)
Alexandria J. of Medicine     Open Access   (Followers: 1, SJR: 0.191, CiteScore: 1)
Algal Research     Partially Free   (Followers: 11, SJR: 1.142, CiteScore: 4)
Alkaloids: Chemical and Biological Perspectives     Full-text available via subscription   (Followers: 2)
Allergologia et Immunopathologia     Full-text available via subscription   (Followers: 1, SJR: 0.504, CiteScore: 1)
Allergology Intl.     Open Access   (Followers: 5, SJR: 1.148, CiteScore: 2)
Alpha Omegan     Full-text available via subscription   (SJR: 3.521, CiteScore: 6)
ALTER - European J. of Disability Research / Revue Européenne de Recherche sur le Handicap     Full-text available via subscription   (Followers: 11, SJR: 0.201, CiteScore: 1)
Alzheimer's & Dementia     Hybrid Journal   (Followers: 55, SJR: 4.66, CiteScore: 10)
Alzheimer's & Dementia: Diagnosis, Assessment & Disease Monitoring     Open Access   (Followers: 6, SJR: 1.796, CiteScore: 4)
Alzheimer's & Dementia: Translational Research & Clinical Interventions     Open Access   (Followers: 6, SJR: 1.108, CiteScore: 3)
Ambulatory Pediatrics     Hybrid Journal   (Followers: 5)
American Heart J.     Hybrid Journal   (Followers: 58, SJR: 3.267, CiteScore: 4)
American J. of Cardiology     Hybrid Journal   (Followers: 67, SJR: 1.93, CiteScore: 3)
American J. of Emergency Medicine     Hybrid Journal   (Followers: 48, SJR: 0.604, CiteScore: 1)
American J. of Geriatric Pharmacotherapy     Full-text available via subscription   (Followers: 13)
American J. of Geriatric Psychiatry     Hybrid Journal   (Followers: 15, SJR: 1.524, CiteScore: 3)
American J. of Human Genetics     Hybrid Journal   (Followers: 39, SJR: 7.45, CiteScore: 8)
American J. of Infection Control     Hybrid Journal   (Followers: 29, SJR: 1.062, CiteScore: 2)
American J. of Kidney Diseases     Hybrid Journal   (Followers: 37, SJR: 2.973, CiteScore: 4)
American J. of Medicine     Hybrid Journal   (Followers: 50)
American J. of Medicine Supplements     Full-text available via subscription   (Followers: 3, SJR: 1.967, CiteScore: 2)
American J. of Obstetrics and Gynecology     Hybrid Journal   (Followers: 264, SJR: 2.7, CiteScore: 4)
American J. of Ophthalmology     Hybrid Journal   (Followers: 67, SJR: 3.184, CiteScore: 4)
American J. of Ophthalmology Case Reports     Open Access   (Followers: 5, SJR: 0.265, CiteScore: 0)
American J. of Orthodontics and Dentofacial Orthopedics     Full-text available via subscription   (Followers: 6, SJR: 1.289, CiteScore: 1)
American J. of Otolaryngology     Hybrid Journal   (Followers: 25, SJR: 0.59, CiteScore: 1)
American J. of Pathology     Hybrid Journal   (Followers: 32, SJR: 2.139, CiteScore: 4)
American J. of Preventive Medicine     Hybrid Journal   (Followers: 30, SJR: 2.164, CiteScore: 4)
American J. of Surgery     Hybrid Journal   (Followers: 39, SJR: 1.141, CiteScore: 2)
American J. of the Medical Sciences     Hybrid Journal   (Followers: 12, SJR: 0.767, CiteScore: 1)
Ampersand : An Intl. J. of General and Applied Linguistics     Open Access   (Followers: 7)
Anaerobe     Hybrid Journal   (Followers: 4, SJR: 1.144, CiteScore: 3)
Anaesthesia & Intensive Care Medicine     Full-text available via subscription   (Followers: 67, SJR: 0.138, CiteScore: 0)
Anaesthesia Critical Care & Pain Medicine     Full-text available via subscription   (Followers: 25, SJR: 0.411, CiteScore: 1)
Anales de Cirugia Vascular     Full-text available via subscription   (Followers: 1)
Anales de Pediatría     Full-text available via subscription   (Followers: 3, SJR: 0.277, CiteScore: 0)
Anales de Pediatría (English Edition)     Full-text available via subscription  
Anales de Pediatría Continuada     Full-text available via subscription  
Analytic Methods in Accident Research     Hybrid Journal   (Followers: 6, SJR: 4.849, CiteScore: 10)
Analytica Chimica Acta     Hybrid Journal   (Followers: 44, SJR: 1.512, CiteScore: 5)
Analytica Chimica Acta : X     Open Access  
Analytical Biochemistry     Hybrid Journal   (Followers: 214, SJR: 0.633, CiteScore: 2)
Analytical Chemistry Research     Open Access   (Followers: 13, SJR: 0.411, CiteScore: 2)
Analytical Spectroscopy Library     Full-text available via subscription   (Followers: 14)
Anesthésie & Réanimation     Full-text available via subscription   (Followers: 2)
Anesthesiology Clinics     Full-text available via subscription   (Followers: 25, SJR: 0.683, CiteScore: 2)
Angiología     Full-text available via subscription   (SJR: 0.121, CiteScore: 0)
Angiologia e Cirurgia Vascular     Open Access   (Followers: 1, SJR: 0.111, CiteScore: 0)
Animal Behaviour     Hybrid Journal   (Followers: 236, SJR: 1.58, CiteScore: 3)
Animal Feed Science and Technology     Hybrid Journal   (Followers: 7, SJR: 0.937, CiteScore: 2)
Animal Reproduction Science     Hybrid Journal   (Followers: 7, SJR: 0.704, CiteScore: 2)
Annales d'Endocrinologie     Full-text available via subscription   (Followers: 3, SJR: 0.451, CiteScore: 1)

        1 2 3 4 5 6 7 8 | Last   [Sort by number of followers]   [Restore default list]

Similar Journals
Journal Cover
Neurocomputing
Journal Prestige (SJR): 1.073
Citation Impact (citeScore): 4
Number of Followers: 7  
 
  Hybrid Journal Hybrid journal (It can contain Open Access articles)
ISSN (Print) 0925-2312
Published by Elsevier Homepage  [3161 journals]
  • Integrating manifold ranking with boundary expansion and corners
           clustering for saliency detection of home scene
    • Abstract: Publication date: 28 February 2020Source: Neurocomputing, Volume 379Author(s): Zhongli Wang, Guohui TianAbstractIn this paper, we propose a novel framework for saliency detection of home scene by exploiting manifold ranking, boundary expansion, and corners clustering. Our proposed method firstly combines color cues in RGB and CIELab to select image boundary seeds, and exclude the ones which might be located at salient objects as much as possible. Then, we utilize the boundary seeds on each image boundary as the queries of manifold ranking to compute saliency and integrate them for a background-based saliency map. For the foreground-based saliency detection. Boundary expansion combined with background-based saliency map highlights foreground regions, which are regarded as queries for a foreground-based saliency map. Moreover, we achieve center prior saliency map through multi-scale Harris corner detection and corners clustering to further highlight salient regions and suppress background regions. Finally, we integrate the three saliency maps via the proposed unified framework for a more accurate and smooth saliency map. Both qualitative and quantitative experimental results indicate that our proposed method can deliver better performance than several state-of-the-art saliency detection methods as a whole.
       
  • A forest of trees with principal direction specified oblique split on
           random subspace
    • Abstract: Publication date: 28 February 2020Source: Neurocomputing, Volume 379Author(s): Fei Wang, Quan Wang, Feiping Nie, Weizhong Yu, Rong Wang, Zhongheng LiAbstractNo matter whether they are univariate or multivariate decision forests, most of previous decision forests determine their partition hyperplanes at split nodes by exhaustive search from candidates or by random generation, which makes some dent in either efficiency or accuracy. In this paper, we propose a new oblique/multivariate decision forest, a forest of trees with principal direction specified oblique split on random subspace (FPDS), where each split of trees is uniquely deterministic once the random feature subspace is determined, the largest principal direction of Principal Component Analysis (PCA) on the sample data at the corresponding split node and the median value of all the current sample points’ projections on the largest principal direction directly specified as the normal direction and the cut-point of the partition hyperplane. This method avoids either tediously searching for the optimal split or casually randomly generating the split. The heuristic method to obtain the hyperplanes guarantees accuracy of trees, and the random feature subspace selection adequately ensures the diversity among individual trees in the forest. In addition, each tree of the FPDS uses the whole training set instead of the sampling subset. Therefore, the only randomness factor in the FPDS derives from the random feature subspace selection, which to some extent enhances the robustness. It proves that the proposed forest FPDS is an alternative classifier which can match or even outperform the existing ensemble classifiers or other classifiers.
       
  • Anatomical context protects deep learning from adversarial perturbations
           in medical imaging
    • Abstract: Publication date: 28 February 2020Source: Neurocomputing, Volume 379Author(s): Yi Li, Huahong Zhang, Camilo Bermudez, Yifan Chen, Bennett A. Landman, Yevgeniy VorobeychikAbstractDeep learning has achieved impressive performance across a variety of tasks, including medical image processing. However, recent research has shown that deep neural networks are susceptible to small adversarial perturbations in the image. We study the impact of such adversarial perturbations in medical image processing where the goal is to predict an individual’s age based on a 3D MRI brain image. We consider two models: a conventional deep neural network, and a hybrid deep learning model which additionally uses features informed by anatomical context. We find that we can introduce significant errors in predicted age by adding imperceptible noise to an image, can accomplish this even for large batches of images using a single perturbation, and that the hybrid model is much more robust to adversarial perturbations than the conventional deep neural network. Our work highlights limitations of current deep learning techniques in clinical applications, and suggests a path forward.
       
  • Person re-identification with dictionary learning regularized by
           stretching regularization and label consistency constraint
    • Abstract: Publication date: 28 February 2020Source: Neurocomputing, Volume 379Author(s): Huafeng Li, Weiyan Zhou, Zhengtao Yu, Biao Yang, Huaiping JinAbstractPerson re-identification (PRID) is a rather challenging task due to the ambiguity of visual appearance. In this paper, we develop a dictionary-based projection transformation learning approach, where the idea of metric learning and dictionary learning are introduced into a unified framework to make full use of their respective advantages. More specifically, to cope with the challenge caused by dramatic changes in visual appearance, we first project the image features of pedestrian into a discriminative subspace to make the same person from different views with the same coding coefficients. Moreover, we develop a new stretch regularization to make the distance between different pedestrian images larger than that of the same pedestrian images so as to reduce the similarity exhibited by different pedestrian images. Additionally, we develop a label consistency constraint and integrate it into the dictionary learning and then we obtain the ensemble learning model of identity discriminator and dictionary. As a result, the coding coefficient and the corresponding label are bridged and the supervision from the labeled samples is also better exploited. Experimental results on five popular person re-identification benchmarks indicate that the approach developed in this paper has higher identification performance than some state-of-the-art methods.
       
  • Spike train analysis in a digital neuromorphic system of cutaneous
           mechanoreceptor
    • Abstract: Publication date: 28 February 2020Source: Neurocomputing, Volume 379Author(s): Fatemeh Yavari, Mahmood Amiri, Fereydoon Nowshiravan Rahatabad, Egidio Falotico, Cecilia LaschiAbstractIn this research, we develop a neuromorphic system to study neural signaling at the level of first order tactile afferents which are slowly adapting type I (SA1) and rapidly adapting type I (RA1) mechanoreceptors. Considering, the linearized Izhikevich model, two digital circuits are developed for both afferents and are executed on the field programmable gate array (FPGA). After implementation of the digital circuits, we investigate how much information is encoded by this hardware-based neuromorphic system. Indeed, the artificial spiking sequences are evoked by applying different force profiles to the sensor connected to the FPGA. Next, the obtained neural responses are classified based on the two fundamental neural coding for brain information processing: spike timing and rate coding. Considering temporal coding, k-nearest neighbors (kNN), support vector machine (SVM) and Decision Tree algorithms are used for forces recognition using acquired artificial spike patterns. The results of classification show that the digital RA1 is susceptible to signal variations, while the digital SA1, on the other hand, is sensitive to the ramp and hold inputs. Furthermore, these responses are better distinguishable to different stimuli when both artificial SA1 and RA1 afferents are regarded. These results, which are functionally compatible with biological observations, yield the promise for fabrication and development of new tactile sensing modules to be employed in bio-robotic and prosthetic applications.
       
  • Multi-scale feature fusion residual network for Single Image
           Super-Resolution
    • Abstract: Publication date: 28 February 2020Source: Neurocomputing, Volume 379Author(s): Jinghui Qin, Yongjie Huang, Wushao WenAbstractWe have witnessed great success of Single Image Super-Resolution (SISR) with convolutional neural networks (CNNs) in recent years. However, most existing Super-Resolution (SR) networks fail to utilize the multi-scale features of low-resolution (LR) images to further improve the representation capability for more accurate SR. In addition, most of them do not exploit the hierarchical features across networks for the final reconstruction. In this paper, we propose a novel multi-scale feature fusion residual network (MSFFRN) to fully exploit image features for SISR. Based on the residual learning, we propose a multi-scale feature fusion residual block (MSFFRB) with multiple intertwined paths to adaptively detect and fuse image features at different scales. Furthermore, the outputs of each MSFFRB and the shallow features are used as the hierarchical features for global feature fusion. Finally, we recover the high-resolution image based on the fused global features. Extensive experiments on four standard benchmarks demonstrate that our MSFFRN achieves better accuracy and visually pleasing than the current state-of-the-art methods.
       
  • Robust approximations of low-rank minimization for tensor completion
    • Abstract: Publication date: 28 February 2020Source: Neurocomputing, Volume 379Author(s): Shangqi Gao, Xiahai ZhuangAbstractMotivated by the nuclear norm of tensors and nonconvex approximations of matrix rank, we propose three robust approximations of multi-linear rank for tensor completion. For each method, we develop an efficient algorithm to solve the corresponding optimization problem. Besides, we prove that every cluster point of the sequence, generated by the respective algorithm, is a stationary point. To obtain a more robust reconstruction, we design an updating rule of parameters for each method. Our empirical experiments on real-world data show that the proposed methods deliver state-of-the-art performance in the reconstruction of low-rank tensors.
       
  • DeepANF: A deep attentive neural framework with distributed representation
           for chromatin accessibility prediction
    • Abstract: Publication date: 28 February 2020Source: Neurocomputing, Volume 379Author(s): Yanbu Guo, Dongming Zhou, Rencan Nie, Xiaoli Ruan, Weihua LiAbstractThe identification of chromatin accessibility is a significant part of the genomics and genetics. However, high-throughput experimental techniques are costly and impractical for systematic identification of accessibility. Many computational methods were proposed to predict the functional regions of chromatin purely relying on DNA sequences, but they could not take full advantage of sequence information to capture hidden complex motifs among DNA sequences. Recently, deep learning algorithms have been incorporated into the chromatin accessibility predication and achieved the remarkable results. Nevertheless, there still exists a problem in chromatin accessibility prediction as how to effectively represent the complex features merely from DNA sequences. Thus, developing efficient computational methods is becoming increasingly urgent to identify functional regions of the genome. In this paper, combining convolutional and gated recurrent unit neural networks with attention mechanism, we develop a discriminative computational framework DeepANF to adaptively extract hidden pattern features and identify the chromatin accessibility based on distributed representation of DNA sequences. To verify the efficacy of the DeepANF framework, we conduct extensive experiments on five large scale datasets, and experimental results reveal that our framework not only consistently outperforms these published methods for chromatin accessibility prediction tasks, but also extracts more discriminative features from pure DNA sequences than published methods, especially on MCF-7 dataset.
       
  • Memristive autapse involving magnetic coupling and excitatory autapse
           enhance firing
    • Abstract: Publication date: 28 February 2020Source: Neurocomputing, Volume 379Author(s): Ge Zhang, Daqing Guo, Fuqiang Wu, Jun MaAbstractAutapse is the synaptic coupling of a neuron's axon to its own dendrite. Considering recent experimental evidence regarding functional excitatory autapses, we explored how the excitatory autapse influences electrical activity in a neuronal model with ion-channel effects. We found that the excitatory autaptic current is able to enhance firing rates. Based on the chemical features of the autapse, we further proposed a memristive autapse involving magnetic coupling, and compared the memristive autapse and the excitatory autapse using the bifurcation analysis and fast/slow decomposition. Our results identified that both of these two types of autapses exhibit similar geometric dynamic properties in terms of burst regulation. Comparing with the excitatory autapse, the memristive autapse involves in a stronger spiking modulation capability, ensuring the neuron to accommodate strong external inputs. Overall, these findings suggested that the memristive system is expected to be usable to mimic biological synapses for advances in neuromorphic computing.
       
  • Event-triggered bipartite consensus for high-order multi-agent systems
           with input saturation
    • Abstract: Publication date: 28 February 2020Source: Neurocomputing, Volume 379Author(s): Yuling Xu, Jinhuan Wang, Yingwen Zhang, Yong XuAbstractIn this paper, the bipartite consensus problem for high-order multi-agent systems (MASs) with input saturation is investigated by distributed event-triggered control and low-gain feedback technique. The underlying topological graph of the system is divided into undirected graph and directed graph. Distributed event-triggered control strategy is proposed to guarantee the semi-global bipartite consensus of MASs. The lower bound of time interval between any two consecutive triggering instants ensures that the Zeno behavior can be excluded for each agent. Finally, two simulation examples are presented to illustrate the effectiveness of the theoretical results.
       
  • Comprehensive design and analysis of time-varying delayed zeroing neural
           network and its application to matrix inversion
    • Abstract: Publication date: 28 February 2020Source: Neurocomputing, Volume 379Author(s): Qiuyue Zuo, Lin Xiao, Kenli LiAbstractTime delays, which inevitably occur in the circuit implementations of zeroing neural networks (ZNNs), are believed to be one of the most primary causes for their instability and oscillation. To handle this problem, a time-varying delayed zeroing neural network (TVDZNN) is for the first time proposed in this paper to consider the potential influence of time-varying delays for matrix inversion. Moreover, the proposed TVDZNN model is theoretically proved to achieve the global exponential convergence for solving matrix inversion under the Lipschitz condition. Besides, a less conservative sufficient condition is presented to relax the restriction on design parameter γ. Moreover, for expediting the convergence speed of the TVDZNN model, two different time-varying parameters have been used to replace its fixed parameter γ, and the improved version is called the varying-parameter TVDZNN (VP-TVDZNN) model. At last, computer simulations further demonstrate the efficiency of the proposed TVDZNN and VP-TVDZNN models for matrix inversion in a time-varying delayed environment.
       
  • Dynamic interaction networks for image-text multimodal learning
    • Abstract: Publication date: 28 February 2020Source: Neurocomputing, Volume 379Author(s): Wenshan Wang, Pengfei Liu, Su Yang, Weishan ZhangAbstractRecently, there is a surge of interest in image-text multimodal representation learning, and many neural network based models have been proposed aiming to capture the interaction between two modalities with different forms of functions. Despite their success, a potential limitation of these methods is insufficient to model all kinds of interactions with a set of static parameters. To alleviate this problem, we present a dynamic interaction network, in which the parameters of the interaction function are dynamically generated by a meta network. Additionally, to provide necessary multimodal features that the meta network needs, we propose a new neural module called Multimodal Transformer. Experimentally, we not only make a comprehensively quantitative evaluation on four image-text tasks, but also show some interpretable analyses of our models, revealing the internal working mechanism of the dynamic parameter learning.
       
  • Unsupervised depth estimation from monocular videos with hybrid
           geometric-refined loss and contextual attention
    • Abstract: Publication date: 28 February 2020Source: Neurocomputing, Volume 379Author(s): Mingliang Zhang, Xinchen Ye, Xin Fan, Wei ZhongAbstractMost existing methods based on convolutional neural networks (CNNs) are supervised, which require a large amount of ground-truth data for training. Recently, some unsupervised methods utilize stereo image pairs as input by transforming depth estimation into a view synthesis problem, but need stereo camera as an additional equipment for data acquisition. Therefore, we use more available monocular videos captured from monocular camera as our input, and propose an unsupervised learning framework to predict scene depth maps from monocular video frames. First, we design a novel unsupervised hybrid geometric-refined loss, which can explicitly explore more accurate geometric relationship between the input color image and the predicted depth map, and preserve depth boundaries and fine structures in depth maps. Then, we design a contextual attention module to capture nonlocal dependencies along the spatial and channel dimensions in a dual path, which can improve the ability of feature representation and further preserve fine depth details. In addition, we also utilize an adversarial loss to discriminate synthetic or realistic color images by training a discriminator so as to produce realistic results. Experimental results demonstrate that the proposed framework achieves comparable or even better results than those trained with monocular videos or stereo image pairs.
       
  • Fuzzy granularity neighborhood extreme clustering
    • Abstract: Publication date: 28 February 2020Source: Neurocomputing, Volume 379Author(s): Shuliang Xu, Shenglan Liu, Lin FengAbstractClustering is an important method for data analysis. Up to now, how to develop an efficient clustering algorithm is still a critical issue. Unsupervised extreme learning machine is an effective neural network learning method which has a fast training speed. In this paper, a fuzzy granularity neighborhood extreme clustering algorithm which is based on extreme learning machine is proposed. We use fuzzy neighborhood rough set to develop a new feature selection method to eliminate redundant attributes and introduce the adaptive adjustment mechanism to solve the parameters of unsupervised extreme learning machine. Different from the existing clustering algorithms, the proposed algorithm can obtain a clustering result with minimum intra-cluster distance and maximum inter-cluster distance. The proposed algorithm and comparison algorithms are executed on the synthetic data sets and real data sets. The experimental results show that the proposed algorithm outperforms the comparison algorithms on the most data sets and the proposed algorithm is effective for clustering task.
       
  • Global exponential stability analysis of discrete-time BAM neural networks
           with delays: A mathematical induction approach
    • Abstract: Publication date: 28 February 2020Source: Neurocomputing, Volume 379Author(s): Er-yong Cong, Xiao Han, Xian ZhangAbstractThe problem of global exponential stability analysis for discrete-time bidirectional associative memory (BAM) neural networks with time-varying delays is investigated. By using the mathematical induction method, a novel exponential stability criterion in the form of linear matrix inequalities is firstly established. Then stability criteria depending upon only the system parameters are derived, which can easily checked by using the standard toolbox software (e.g., MATLAB). The proposed approach is directly based on the definition of global exponential stability, and it does not involve the construct of any Lyapunov–Krasovskii functional or auxiliary function. For a class of special cases, it is theoretical proven that the less conservative stability criteria can be obtained by using the proposed approach than ones in literature. Moreover, several numerical examples are also provided to demonstrate the effectiveness of the proposed method.
       
  • Fixed-time event-triggered synchronization of a multilayer
           Kuramoto-oscillator network
    • Abstract: Publication date: 28 February 2020Source: Neurocomputing, Volume 379Author(s): Jia Sun, Jian Liu, Yuanda Wang, Yao Yu, Changyin SunAbstractThis paper investigates the synchronization problem of the Kuramoto-oscillator network with non-identical oscillators. The fixed-time event-triggered synchronization control strategies are developed for phase agreement and frequency synchronization under both continuous and intermittent communication. With the developed fixed-time controller, the synchronization can be achieved within a pre-defined time for any initial phase of each oscillator. The event-triggered mechanism avoids continuous controller update and data transmission, which significantly saves the computation and communication resources. Furthermore, theoretical analysis shows that the fixed-time convergence can be guaranteed and the Zeno behavior is avoided by the proposed methods. The numerical simulations of each situation also verify the effectiveness of the proposed synchronization control strategies.
       
  • Adaptive NN event-triggered control for path following of underactuated
           vessels with finite-time convergence
    • Abstract: Publication date: 28 February 2020Source: Neurocomputing, Volume 379Author(s): Meilin Li, Tieshan Li, Xiaoyang Gao, Qihe Shan, C. L. Philip Chen, Yang XiaoAbstractThis paper investigates the problem of path following of underactuated marine surface vessels (MSVs) with uncertain nonlinear dynamics. First, the tracking target of the vessel is reduced from tracking the earth-fixed position to tracking the line-of-sight (LOS) angle by LOS method. Then, by employing the radial basis function neural network (RBFNN) to deal with the uncertain nonlinear dynamics, an adaptive NN fast power reaching law is developed for the path following problem based on the backstepping design methodology. Thereafter, the event-triggered technique is incorporated into the control design to synthesize an adaptive NN event-triggered controller with the fast power reaching convergence rate. By combining with the presented event-triggered mechanism, the controller is only updated when the triggering condition is satisfied. Therefore, both the update frequency of the controller and actuator loss are greatly reduced comparing with the traditional time-triggered controller. Theoretical analysis via Lyapunov method indicates that the tracking error can converge to zero within a finite time, meanwhile it also shows that Zeno behavior can be avoided. Simulation results with comparations illustrate the validity and superiority of the proposed controller.
       
  • A model with length-variable attention for spoken language understanding
    • Abstract: Publication date: 28 February 2020Source: Neurocomputing, Volume 379Author(s): Cong Xu, Qing Li, Dezheng Zhang, Jiarui Cui, Zhenqi Sun, Hao ZhouAbstractIntent detection (ID) and slot filling (SF) are important components in spoken language understanding (SLU) of a dialogue system. The most widely used method is pipeline manner which detects the user’s intent at first, then labels the slots. For the purpose of addressing error propagate, some researchers combine these two tasks together by ID and SF joint model. However, the joint models usually perform well only on one of these tasks due to the different values of the trade-off parameter. We therefore propose an encoder-decoder model with a new tag scheme which unifies these two tasks into one sequence labeling task. In our model, the process of slot filling can receive an intent information and the performance about multiple tags of a word has been improved. Moreover, we show a length-variable attention which can selectively look at a subset of source sentence in the sequence labeling model. Experimental results on two datasets display that the proposed model with length-variable attention outperforms over other joint models. Besides, our method will automatically find the balance between two tasks and achieve better overall performances.
       
  • PAC-Bayes and domain adaptation
    • Abstract: Publication date: 28 February 2020Source: Neurocomputing, Volume 379Author(s): Pascal Germain, Amaury Habrard, François Laviolette, Emilie MorvantAbstractWe provide two main contributions in PAC-Bayesian theory for domain adaptation where the objective is to learn, from a source distribution, a well-performing majority vote on a different, but related, target distribution. Firstly, we propose an improvement of the previous approach we proposed in [1], which relies on a novel distribution pseudodistance based on a disagreement averaging, allowing us to derive a new tighter domain adaptation bound for the target risk. While this bound stands in the spirit of common domain adaptation works, we derive a second bound (introduced in [2]) that brings a new perspective on domain adaptation by deriving an upper bound on the target risk where the distributions’ divergence—expressed as a ratio—controls the trade-off between a source error measure and the target voters’ disagreement. We discuss and compare both results, from which we obtain PAC-Bayesian generalization bounds. Furthermore, from the PAC-Bayesian specialization to linear classifiers, we infer two learning algorithms, and we evaluate them on real data.
       
  • Progressive Operational Perceptrons with Memory
    • Abstract: Publication date: 28 February 2020Source: Neurocomputing, Volume 379Author(s): Dat Thanh Tran, Serkan Kiranyaz, Moncef Gabbouj, Alexandros IosifidisAbstractGeneralized Operational Perceptron (GOP) was proposed to generalize the linear neuron model used in the traditional Multilayer Perceptron (MLP) by mimicking the synaptic connections of biological neurons showing nonlinear neurochemical behaviours. Previously, Progressive Operational Perceptron (POP) was proposed to train a multilayer network of GOPs which is formed layer-wise in a progressive manner. While achieving superior learning performance over other types of networks, POP has a high computational complexity. In this work, we propose POPfast, an improved variant of POP that signicantly reduces the computational complexity of POP, thus accelerating the training time of GOP networks. In addition, we also propose major architectural modications of POPfast that can augment the progressive learning process of POP by incorporating an information preserving, linear projection path from the input to the output layer at each progressive step. The proposed extensions can be interpreted as a mechanism that provides direct information extracted from the previously learned layers to the network, hence the term “memory”. This allows the network to learn deeper architectures and better data representations. An extensive set of experiments in human action, object, facial identity and scene recognition problems demonstrates that the proposed algorithms can train GOP networks much faster than POPs while achieving better performance compared to original POPs and other related algorithms.
       
  • Food det: Detecting foods in refrigerator with supervised transformer
           network
    • Abstract: Publication date: 28 February 2020Source: Neurocomputing, Volume 379Author(s): Yousong Zhu, Xu Zhao, Chaoyang Zhao, Jinqiao Wang, Hanqing LuAbstractMost of existing methods mainly focus on the food image recognition which assumes that one food image contains only one food item. However, in this paper, we present a system to detect a diversity of foods in refrigerator where multiple food items may exist. In view of the refrigerator environment, we propose a food detection framework based on the supervised transformer network. More specifically, the supervised transformer network, dotted as RectNet, is first proposed to automatically select the irregular food regions and transform them to the frontal views. Then, based on the rectified food images, we further propose an end-to-end detection network that predicts the categories and locations of food items. The proposed detection network, called Lite Fully Convolutional Network (LiteFCN), is evolved from the advanced object detection algorithm Faster R-CNN while several significant improvements are tailored to achieve a higher accuracy and keep inference time efficiency. To validate the effectiveness of each component of our method, we build a real-world refrigerator dataset with 80 classes. Extensive experiments demonstrate that our methods achieve the state-of-the-art results, which improves the baseline by a large margin, e.g., 3–5% in terms of F-measure. We also show that the proposed detection network achieve a competitive result on the public PASCAL VOC2007 dataset, which outperforms the Faster R-CNN by 2.3% with a higher speed.
       
  • Autonomous deep learning: A genetic DCNN designer for image classification
    • Abstract: Publication date: 28 February 2020Source: Neurocomputing, Volume 379Author(s): Benteng Ma, Xiang Li, Yong Xia, Yanning ZhangAbstractRecent years have witnessed the breakthrough success of deep convolutional neural networks (DCNNs) in image classification and other vision applications. DCNNs have distinct advantages over traditional solutions in providing a uniform feature extraction-classification framework to free users from troublesome handcrafted feature extraction. However, DCNNs are far from autonomous, since their performance relies heavily on the handcrafted architectures, which also requires a lot expertise and experience to design, and cannot be continuously improved once the tuning of hyper-parameters converges. In this paper, we propose an autonomous and continuous learning (ACL) algorithm to generate automatically a DCNN architecture for each given vision task. We first partition a DCNN into multiple stacked meta convolutional blocks and fully connected blocks, each of which may contain the operations of convolution, pooling, fully connection, batch normalization, activation and drop out, and thus convert the architecture into an integer code. Then, we use genetic evolutionary operations, including selection, mutation and crossover to evolve a population of DCNN architectures. We have evaluated this algorithm on six image classification tasks, i.e., MNIST, Fashion-MNIST, EMNIST-Letters, EMNIST-Digits, CIFAR10 and CIFAR100. Our results indicate that the proposed ACL algorithm is able to evolve the DCNN architecture continuously if more time cost is allowed and can find a suboptimal DCNN architecture, whose performance is comparable to the state of the art.
       
  • Adaptive neural dynamic surface control of mechanical systems using
           integral terminal sliding mode
    • Abstract: Publication date: 28 February 2020Source: Neurocomputing, Volume 379Author(s): Jafar Keighobadi, Mehran Hosseini-Pishrobat, Javad FarajiAbstractThis paper studies the robust tracking control problem of fully-actuated mechanical systems using a novel integral dynamics surface control (DSC) method. We replace the conventional DSC error surfaces with new nonlinear integral surfaces to generate a quasi-terminal sliding mode (TSM) in the tracking error trajectories. Then, we follow the recursive, Lyapunov-based design procedure of the DSC to obtain the control law. The resultant quasi-TSM adjusts the error convergence rate according to the distance from the origin. To achieve robustness against structural variations of the mechanical system as well as external disturbances, we use nonlinear damping combined with a radial basis function neural network (RBFNN) approximator. The RBFNN adaptively identifies the upper-bound of the uncertainty/disturbances to prevent conservative, high-gain control inputs. Moreover, we use raised-cosine basis functions, which have compact supports, to improve the computational efficiency of the RBFNN. Through Lyapunov-based stability analysis, we show the boundedness and ultimate boundedness of the closed-loop system as well as the TSM-induced convergence of the tracking errors. Detailed numerical simulations support the efficacy of the proposed control method.
       
  • Data-driven simulation of pedestrian collision avoidance with a
           nonparametric neural network
    • Abstract: Publication date: 28 February 2020Source: Neurocomputing, Volume 379Author(s): Rafael F. Martin, Daniel R. ParisiAbstractData-driven simulation of pedestrian dynamics is an incipient and promising approach for building reliable microscopic pedestrian models. We propose a methodology based on generalized regression neural networks, which does not have to deal with a huge number of free parameters as in the case of multilayer neural networks. Although the method is general, we focus on the one pedestrian - one obstacle problem. Experimental data were collected in a motion capture laboratory providing high-precision trajectories. The proposed model allows us to simulate the trajectory of a pedestrian avoiding an obstacle from any direction. Together with the methodology specifications, we provide the data set needed for performing the simulations of this kind of pedestrian dynamic system.
       
  • Recurrent convolutional neural network: A new framework for remaining
           useful life prediction of machinery
    • Abstract: Publication date: 28 February 2020Source: Neurocomputing, Volume 379Author(s): Biao Wang, Yaguo Lei, Tao Yan, Naipeng Li, Liang GuoAbstractDeep learning is becoming more appealing in remaining useful life (RUL) prediction of machines, because it is able to automatically build the mapping relationship between the raw data and the corresponding RUL by representation learning. Among deep learning models, convolutional neural networks (CNNs) are gaining special attention because of its powerful ability in dealing with time-series signals, and have achieved promising results in current studies. These studies, however, suffer from the two limitations: (1) The temporal dependencies of different degradation states are not considered during network construction; and (2) The uncertainty of RUL prediction results cannot be obtained. To overcome the above-mentioned limitations, a new framework named recurrent convolutional neural network (RCNN) is proposed in this paper for RUL prediction of machinery. In RCNN, recurrent convolutional layers are first constructed to model the temporal dependencies of different degradation states. Then, variational inference is used to quantify the uncertainty of RCNN in RUL prediction. The proposed RCNN is evaluated using vibration data from accelerated degradation tests of rolling element bearings and sensor data from life testing of milling cutters, and compared with some state-of-the-art prognostics approaches. Experimental results demonstrate the effectiveness and superiority of RCNN in improving the accuracy and convergence of RUL prediction. More importantly, RCNN is able to provide a probabilistic RUL prediction result, which breaks the inherent limitation of CNNs and facilitates maintenance decision making.
       
  • Bootstrap dual complementary hashing with semi-supervised re-ranking for
           image retrieval
    • Abstract: Publication date: 28 February 2020Source: Neurocomputing, Volume 379Author(s): Xing Tian, Xiancheng Zhou, Wing W.Y. Ng, Jiayong Li, Hui WangAbstractWith the rapid growth of multimedia data on the Internet, content-based image retrieval becomes a key technique for the Internet development. Hashing methods are efficient and effective for image retrieval. Dual Complementary Hashing (DCH) is one such method, which uses multiple hash tables and has good performance. However, DCH utilizes wrongly hashed image pairs to train the following hash table and discards correctly hashed image pairs. Therefore, the number of image pairs utilized for training the following hash tables will decrease rapidly. Moreover, each hash function in a hash table of DCH is trained by correcting the errors caused by its preceding one instead of holistically considering errors made by all previous hash functions. These restrictions significantly reduce the training efficiency and the overall performance of DCH. In this paper, we propose a new hashing method for image retrieval, Bootstrap Dual Complementary Hashing with semi-supervised Re-ranking (BDCHR). It is a semi-supervised multi-hashing method consisting of two parts: bootstrap DCH and semi-supervised re-ranking. The first part relieves the restrictions of DCH while the second part further enhances the image retrieval performance. Experimental results show that BDCHR yields better performance than other state-of-the-art multi-hashing methods.
       
  • Feature concatenation multi-view subspace clustering
    • Abstract: Publication date: 28 February 2020Source: Neurocomputing, Volume 379Author(s): Qinghai Zheng, Jihua Zhu, Zhongyu Li, Shanmin Pang, Jun Wang, Yaochen LiAbstractMulti-view clustering is a learning paradigm based on multi-view data. Since statistic properties of different views are diverse, even incompatible, few approaches implement multi-view clustering based on the concatenated features straightforward. However, feature concatenation is a natural way to combine multi-view data. To this end, this paper proposes a novel multi-view subspace clustering approach dubbed Feature Concatenation Multi-view Subspace Clustering (FCMSC), which boosts the clustering performance by exploring the consensus information of multi-view data. Specifically, multi-view data are concatenated into a joint representation firstly, then, l2,1-norm is integrated into the objective function to deal with the sample-specific and cluster-specific corruptions of multiple views. Moreover, a graph regularized FCMSC is also proposed in this paper to explore both the consensus information and complementary information of multi-view data for clustering. It is noteworthy that the obtained coefficient matrix is not derived by simply applying the Low-Rank Representation (LRR) to concatenated features directly. Finally, an effective algorithm based on the Augmented Lagrangian Multiplier (ALM) is designed to optimize the objective functions. Comprehensive experiments on six real-world datasets illustrate the superiority of the proposed methods over several state-of-the-art approaches for multi-view clustering.
       
  • Automatic processing of Z-transform artificial neural networks using
           parallel programming
    • Abstract: Publication date: 28 February 2020Source: Neurocomputing, Volume 379Author(s): M. Szymczyk, P. SzymczykAbstractThe article describes the process of computing the Z-transform neural network on the basis of input and output signals of analyzed object. Parallel algorithms for performing these calculations are presented and different parallel architectures with different number of processors showing their advantages and limitations are analyzed.
       
  • Bayesian capsule networks for 3D human pose estimation from single 2D
           images
    • Abstract: Publication date: 28 February 2020Source: Neurocomputing, Volume 379Author(s): Iván Ramírez, Alfredo Cuesta-Infante, Emanuele Schiavi, Juan José PantrigoAbstractDeep Bayesian Networks are a hot topic in Deep Learning because this approach makes it possible to minimize both the epistemic and the homoscedastic uncertainty at the same time self balancing multiple and complementary losses for a given task, simply by employing standard operations such as dropout, mean squared error or cross-entropy. On the other hand, Capsule networks are a novel DNN architecture that offer a richer representation because each concept is represented by a number of different vectors. The bayesian formulation of the Capsule networks is still an open problem that we address in this paper. We present a bayesian formulation of Capsule networks and compare its performance against the state-of-the-art for the ill-posed regression problem of estimating the 3D human pose from a single 2D image. The results show that our network is very competitive with a much more straightforward solution. To enable fair comparisons the source code is openly available at https://github.com/rollervan/BCN_3DPose/
       
  • SARPNET: Shape attention regional proposal network for liDAR-based 3D
           object detection
    • Abstract: Publication date: 28 February 2020Source: Neurocomputing, Volume 379Author(s): Yangyang Ye, Houjin Chen, Chi Zhang, Xiaoli Hao, Zhaoxiang ZhangAbstractReal-time 3D object detection is a fundamental technique in numerous applications, such as autonomous driving, unmanned aerial vehicles (UAV) and robot vision. However, current LiDAR-based 3D object detection algorithms allocate inadequate attention to the inhomogeneity of LiDAR point clouds and the shape encoding capability of regional proposal schemes. This paper introduces a novel 3D object detection network called the Shape Attention Regional Proposal Network (SARPNET), which deploys a new low-level feature encoder to remedy the sparsity and inhomogeneity of LiDAR point clouds with an even sample method, and embodies a shape attention mechanism that learns the statistic 3D shape priors of objects and uses them to spatially enhance semantic embeddings. Experimental results show that the proposed one-stage method outperforms state-of-the-art one-stage and even two-stage methods on the KITTI 3D object detection benchmark. It achieved a BEV AP of (87.26%, 62.80%), 3D AP of (75.64%, 60.43%), and orientation AP of (88.86%, 71.01%) for the detection of cars and cyclists, respectively. Besides, the method is the third winner in the nuScenes 3D Detection challenge of CVPR2019 Workshop on Autonomous Driving (WAD).
       
  • Neural network based integral sliding mode optimal flight control of near
           space hypersonic vehicle
    • Abstract: Publication date: 28 February 2020Source: Neurocomputing, Volume 379Author(s): Rongsheng Xia, Mou Chen, Qingxian Wu, Yuhui WangAbstractIn this paper, based on the integral sliding mode method and adaptive dynamic programming (ADP) algorithm, a robust optimal tracking control scheme is presented for near space hypersonic vehicle (NSHV) system in the presence of unknown modeling error, external disturbance, and input saturation. Firstly, combining neural network, auxiliary system and integral sliding mode methods, an adaptive integral sliding mode control (AISMC) law is designed to guarantee system trajectories tend to a defined integral sliding surface and the effects of modeling uncertainty, external disturbance, and control input saturation are eliminated. Then, the robust optimal tracking control problem of original system is converted into the optimal control problem of a nominal system, and an ADP method with single critic network is utilized to acquire the corresponding optimal controller. Furthermore, Lyapunov analysis method shows that the overall control input which contains AISMC law and optimal controller can ensure all the signals in closed-loop system are stable in the sense of uniform ultimate boundedness (UUB). Finally, simulation results about attitude flight control of NSHV are given to verify the effectiveness of the proposed control scheme.
       
  • Introspection unit in memory network: Learning to generalize inference in
           OOV scenarios
    • Abstract: Publication date: 28 February 2020Source: Neurocomputing, Volume 379Author(s): Qichuan Yang, Zhiqiang He, Zhiqiang Zhan, Yang Zhang, Rang Li, Changjian HuAbstractInference in natural language processing (NLP) is a tough task. Although plenty of models have been proposed in recent years, they are usually restricted to infer within a limited vocabulary or handcrafted training templates. In this paper, we propose the introspection unit (IU), a new neural module which can be incorporated with memory networks to deal with inference tasks in out of vocabulary (OOV) and rare named entities (RNEs) scenarios. Specifically, when encountering a new word, IU compares its part-of-speech context with the training dataset to extract a similar sample, and then embeds the new word into a target position to construct a simulated sample. The target position is located by the result of part-of-speech tagging. Finally, using the simulated sample, IU helps memory networks to learn the context and characteristic of the new word. In experiments, we evaluate the effectiveness of IU with the memory network on four inference datasets: a name OOV dataset, a place OOV dataset, a more challenging synthetical mixture OOV dataset and a realistic dialogue dataset. The experimental results demonstrate that IU effectively generalizes the inference ability of memory networks to OOV scenarios and improves the inference accuracies significantly. Furthermore, we visualize both the introspection process and the effect of IU in word embeddings and memories.
       
  • A renewable fusion fault diagnosis network for the variable speed
           conditions under unbalanced samples
    • Abstract: Publication date: 28 February 2020Source: Neurocomputing, Volume 379Author(s): Kun Xu, Shunming Li, Xingxing Jiang, Zenghui An, Jinrui Wang, Tianyi YuAbstractDeep learning technology has been gradually applied to solve a variety of fault diagnosis problems because of its outstanding feature learning and nonlinear classification abilities. However, few deep learning network models can be applied to both variable speed conditions and unbalanced samples scenarios in fault diagnosis, especially in extreme cases where fault samples are missing. And most of the fault diagnosis models do not have the ability to update automatically as the collected fault data increases. To deal with the above problems, a deep learning model named renewable fusion fault diagnosis network (RFFDN) is proposed. The network has three main parts: improved feature classification network; second order statistics fusion network and unbalanced feature comparison network. Moreover, these three networks are simultaneously organized on a two-branch convolutional neural network (CNN) architecture with fused data input, so as to facilitate the network to learn the depth nonlinear domain invariant features. Finally, the RFFDN model and other mainstream fault diagnosis models are tested on two different datasets. The results show that the RFFDN model not simply achieves high diagnostic accuracy in diagnosis results, but also extracts the domain invariant features at variable speed conditions under unbalanced samples, and accurately classifies the new faults. These prove that the model can not only be applied to a variety of operating modes, but can be updated as more data are collected as well, which is of great significance to the field of fault diagnosis.
       
  • Modified gradient neural networks for solving the time-varying Sylvester
           equation with adaptive coefficients and elimination of matrix inversion
    • Abstract: Publication date: 28 February 2020Source: Neurocomputing, Volume 379Author(s): Shan Liao, Jiayong Liu, Xiuchun Xiao, Dongyang Fu, Guancheng Wang, Long JinAbstractIn scientific and engineering fields, the solutions to many problems can be transformed into finding the solutions to Sylvester equation, for which various computational methods (e.g., recurrent neural network, RNN) have been presented and investigated. RNN models are frequently used to solve computational problems due to the prevalent exploitation of the gradient-based RNN. However, the overlong convergent time and the too large residual error restrict the widespread applications of the RNN model in solving time-varying problems. Further, a special type of RNN named zeroing neural network (ZNN) is able to solve the time-varying Sylvester equation, which breaks the limitations mentioned above, but fails to handle complex time-varying problems owing to the sharp increment of the calculated amount in matrix inversion involved. To remedy the limitation, a modified gradient-based RNN (MGRNN) model is proposed to generate more accurate computational solutions with less convergent time and adaptive coefficients for solving the time-varying Sylvester equation, which replaces the matrix inversion problem with the matrix transposition problem. Besides, theoretical analyses and mathematical verifications are presented to validate the efficiency and superiority of the proposed MGRNN model compared with the traditional gradient-based recurrent neural network (GRNN) and ZNN models. Furthermore, simulation experiments are conducted to substantiate the properties of the newly proposed MGRNN model for solving the time-varying Sylvester equation.
       
  • Robust stochastic block model
    • Abstract: Publication date: 28 February 2020Source: Neurocomputing, Volume 379Author(s): Zhijuan Xu, Xueyan Liu, Xianjuan Cui, Ximing Li, Bo YangAbstractThe family of stochastic block model (SBM) is a mainstay to detect network structures, especially for the exploratory networks analysis without any prior. However, the real-world networks often contain many noisy nodes that have abnormal behaviors or go against the certain patterns. This creates the so-called noise problem, resulting in lower performance of SBMs in real applications. To alleviate this problem, we propose a novel Robust Stochastic Block Model (RSBM). The proposed method can model the noisy nodes in the network and maintain the ability of SBM in structure analysis. RSBM is inferred using variational Bayesian expectation maximization. We evaluate RSBM on both synthetic and real-world networks, and empirical results demonstrate that our RSBM outperforms the state-of-the-art baseline models in the structural partitioning task.
       
  • Human skeleton mutual learning for person re-identification
    • Abstract: Publication date: Available online 16 January 2020Source: NeurocomputingAuthor(s): Ziyang Wang, Dan Wei, Xiaoqiang Hu, Yiping LuoAbstractPerson re-identification refers to matching people across non-overlapping camera views on different locations and at different times. In the case of changes in perspective, light, background, veil, and person's clothing, traditional method can't achieve person recognition effectively and reliably. In this paper, we propose a novel biometric metric learning method named Human Skeleton Mutual Learning person re-identification (HSMLP-Reid). The purpose of HSML person re-identification method (HSMLP-Reid) largely aims to use the new pedestrian local segmentation method proposed in this paper combined with the global skeleton information to solve the influence of background and local posture change. Firstly, bottom-up method is used to estimate the pedestrian posture and skeleton, and the joint points of the pedestrians will be marked in this process. A new local segmentation method proposed in this paper named joint segmentation is used to locally segment pedestrians and perform local block matching. Furthermore, we learn the global skeleton information by defined joint distances from the pedestrian 2D skeleton estimation by the bottom-up method and use global skeleton information to global skeleton matching. Finally, we use local match and global skeleton match for mutual learning. Local match based on pedestrian nodes and global skeleton match based on pedestrian skeleton are based on biometrics. We learn the classification loss and metric learning loss to train model. Metric loss includes global skeletal distance and local block metric distance. Extensive experimental results on the large-scale Market1501, CUHK03 and CUHK-SYSU data sets demonstrate that the proposed method achieves consistently superior performance and outperforms most of the state-of-the-art methods.
       
  • Integration of an Actor-Critic Model and Generative Adversarial Networks
           for a Chinese Calligraphy Robot
    • Abstract: Publication date: Available online 16 January 2020Source: NeurocomputingAuthor(s): Ruiqi Wu, Changle Zhou, Fei Chao, Longzhi Yang, Chih-Min Lin, Changjing ShangAbstractAs a combination of robotic motion planning and Chinese calligraphy culture, robotic calligraphy plays a significant role in the inheritance and education of Chinese calligraphy culture. Most existing calligraphy robots focus on enabling the robots to learn writing through human participation, such as human-robot interactions and manually designed evaluation functions. However, because of the subjectivity of art aesthetics, these existing methods require a large amount of implementation work from human engineers. In addition, the written results cannot be accurately evaluated. To overcome these limitations, in this paper, we propose a robotic calligraphy model that combines a generative adversarial network (GAN) and deep reinforcement learning to enable a calligraphy robot to learn to write Chinese character strokes directly from images captured from Chinese calligraphic textbooks. In our proposed model, to automatically establish an aesthetic evaluation system for Chinese calligraphy, a GAN is first trained to understand and reconstruct stroke images. Then, the discriminator network is independently extracted from the trained GAN and embedded into a variant of the reinforcement learning method, the “actor-critic model”, as a reward function. Thus, a calligraphy robot adopts the improved actor-critic model to learn to write multiple character strokes. The experimental results demonstrate that the proposed model successfully allows a calligraphy robot to write Chinese character strokes based on input stroke images. The performance of our model, compared with the state-of-the-art deep reinforcement learning method, shows the efficacy of the combination approach. In addition, the key technology in this work shows promise as a solution for robotic autonomous assembly.
       
  • Hybrid-Loss Supervision for Deep Neural Network
    • Abstract: Publication date: Available online 16 January 2020Source: NeurocomputingAuthor(s): Qishang Cheng, Hongliang Li, Qingbo Wu, King Ngi NganAbstractMulti-loss-joint-optimization has been proven to be valid in computer vision literature. However, the learned deep sub-features usually fit their disjoint constraints, which yield confrontation and spatial inconsistency among the sub-features with nonshared FC layers. In this paper, we propose a Hybrid-Loss Supervision (HLS) framework in order to obtain smoother and more spatially consistent features with shared FC layers. First, we analyze the shortcomings of the monitoring with single-loss in the existing framework theoretically. Then, we selected two notable loss functions (e.g., Center loss and Weighted loss) to instantiate the HLS framework by linear combination. By instantiating the framework with two standard loss functions, the network has learned more compact intra-class deep features and uniform inter-class deep features. The HLS framework can significantly boost the efficiency of existing convolution networks for both image classification task and object detection task without increasing network parameters and computational complexity. Extensive experimental results on different vision tasks demonstrate consistent improvement can be achieved across a variety of datasets (e.g., CIFAR-10/100, ImageNet-2012, PASCAL VOC and MS-COCO) and different convolutional neural network architectures.
       
  • Empirical Mode Decomposition Based Multi-objective Deep Belief Network for
           Short-term Power Load Forecasting
    • Abstract: Publication date: Available online 15 January 2020Source: NeurocomputingAuthor(s): Chaodong Fan, Changkun Ding, Jinhua Zheng, Leyi Xiao, Zhaoyang AiAbsrtactWith the rapid development of power grid data, the data generated by the operation of the power system is increasingly complex, and the amount of data increases exponentially. In order to fully exploit and utilize the deep relationship between data to achieve accurate prediction of power load, this paper proposes an Empirical Mode Decomposition Based Multi-objective Deep Belief Network prediction method (EMD-MODBN). In the training process of DBN, a multi-objective optimization model is constructed aiming at accuracy and diversity, and MOEA/D is used to optimize the parameters of the model to enhance the generalization ability of the prediction model. Finally, the final load forecasting results are obtained by summing up the weighted outputs of each forecasting model with ensemble learning method. The experimental results show that compared with several current better load forecasting methods, this method has obvious advantages in prediction accuracy and generalization ability.
       
  • Multivariate Time Series Forecasting via Attention-based Encoder-Decoder
           Framework
    • Abstract: Publication date: Available online 15 January 2020Source: NeurocomputingAuthor(s): Shengdong Du, Tianrui Li, Yan Yang, Shi-Jinn HorngAbstractTime series forecasting is an important technique to study the behavior of temporal data and forecast future values, which is widely applied in many fields, e.g. air quality forecasting, power load forecasting, medical monitoring, and intrusion detection. In this paper, we firstly propose a novel temporal attention encoder-decoder model to deal with the multivariate time series forecasting problem. It is an end-to-end deep learning structure that integrates the traditional encode context vector and temporal attention vector for jointly temporal representation learning, which is based on bi-directional long short-term memory networks (Bi-LSTM) layers with temporal attention mechanism as the encoder network to adaptively learning long-term dependency and hidden correlation features of multivariate temporal data. Extensive experimental results on five typical multivariate time series datasets showed that our model has the best forecasting performance compared with baseline methods.
       
  • Stochastic reconstruction of 3D porous media from 2D images using
           generative adversarial networks
    • Abstract: Publication date: Available online 15 January 2020Source: NeurocomputingAuthor(s): Andrea Valsecchi, Sergio Damas, Cristina Tubilleja, Javier ArechaldeAbstractMicro computed tomography (CT) provides petrophysics laboratories with the ability to image three dimensional porous media at pore scale. However, evaluating flow properties requires the acquisition of a large number of representative images, which is often unfeasible. Stochastic reconstruction methods are algorithms able to generate novel, realistic rock images from a small sample, thus avoiding a large acquisition process. A more convenient approach would use only two dimensional images, replacing 3D scans with images of the rock cuttings made during the drilling. This would extend the technique to media having pores smaller than the resolution of the micro-CT, but that can be imaged by microscopy.We introduce a novel method for 2D-to-3D reconstruction of the structure of porous media by applying generative adversarial neural networks. We compare several measures of pore morphology between simulated and acquired images. Experiments include bead pack, Berea sandstone, and Ketton limestone images. Results show that our GANs-based method can reconstruct three-dimensional images of porous media at different scales that are representative of the morphology of the original images. Also, compared to classical stochastic methods of image reconstruction, the generation of multiple images is much faster.
       
  • Output based transfer learning with least squares support vector machine
           and its application in bladder cancer prognosis
    • Abstract: Publication date: Available online 15 January 2020Source: NeurocomputingAuthor(s): Guanjin Wang, Guangquan Zhang, Kup-Sze Choi, Kin-Man Lam, Jie LuAbstractTwo dilemmas frequently occur in many real-world clinical prognoses. First, the on-hand data cannot be put entirely into the existing prediction model, since the features from new data do not perfectly match those of the model. As a result, some unique features collected from the patients in the current domain of interest might be wasted. Second, the on-hand data is not sufficient enough to learn a new prediction model. To overcome these challenges, we propose an output-based transfer learning approach with least squares support vector machine (LS-SVM) to make the maximum use of the small dataset and guarantee an enhanced generalization capability. The proposed approach can learn a current domain of interest with limited samples effectively by leveraging the knowledge from the predicted outputs of the existing model in the source domain. Also, the extent of output knowledge transfer from the source domain to the current one can be automatically and rapidly determined using a proposed fast leave-one-out cross validation strategy. The proposed approach is applied to a real-world clinical dataset to predict 5-year overall and cancer-specific mortality of bladder cancer patients after radical cystectomy. The experimental results indicate that the proposed approach achieves better classification performances than the other comparative methods and has the potential to be implemented into the real-world context to deal with small data problems in cancer prediction and prognosis.
       
  • Multi-label symbolic value partitioning through random walks
    • Abstract: Publication date: Available online 15 January 2020Source: NeurocomputingAuthor(s): Liu-Ying Wen, Chao-Guang Luo, Wei-Zhi Wu, Fan MinAbstractFeature selection and symbolic value partitioning are effective knowledge reduction techniques in the field of data mining. A large body of feature selection methods has been proposed for multi-label data. By contrast, symbolic value partitioning for such data has not been studied. In this paper, we propose the multi-label symbolic value partitioning through random walks algorithm with two stages. In the first stage, an undirected weighted graph is constructed for each attribute. Each node corresponds to an attribute value and the weight of each edge corresponds to the similarity between two nodes. Similarity is defined based on the attribute value distribution for each label. In the second stage, a random walk algorithm is used to cluster attribute values. The average weight serves as the separation operator to sharpen the inter-cluster edges. We tested the new algorithm and seven popular feature selection algorithms on 13 datasets. The experimental results demonstrate the effectiveness of the proposed algorithm in reducing the data size and improving classification accuracy.
       
  • A Cross-modal Adaptive Gated Fusion Generative Adversarial Network for
           RGB-D Salient Object Detection
    • Abstract: Publication date: Available online 15 January 2020Source: NeurocomputingAuthor(s): Zhengyi Liu, Wei Zhang, Peng ZhaoAbstractSalient object detection in RGB-D images aims to identify the most attractive objects in a pair of color and depth images for the observer. As an important branch of salient object detection, it focuses on solving the following two major challenges: how to achieve cross-modal fusion that is efficient and beneficial for salient object detection; how to effectively extract the information of depth image with relatively poor quality. This paper proposes a cross-modal adaptive gated fusion generative adversarial network for RGB-D salient object detection by using color and depth images. Specifically, the generator network adopts double-stream encoder-decoder network and receives RGB and depth images at the same time. The proposed depthwise separable residual convolution module is used to deal with deep semantic information, and the processed feature is combined with side-output features of the encoder network progressively. In order to compensate for the shortcoming of poor quality of the depth image, the proposed method adds the cross-modal guidance from the side-output features of the RGB stream to the decoder network of depth stream. The discriminator network adaptively fuses the features of double streams using a gated fusion module, then sends the gated fusion saliency map to the discriminator to distinguish the similarity from ground-truth map. Adversarial learning forms the better generator network and discriminator network, and the gated fusion saliency map generated by the best generator network is served as final result. Experiments on five publicly RGB-D datasets demonstrate the effect of cross-modal fusion, depthwise separable residual convolution and adaptive gated fusion. Compared with the state-of-the-art methods, our method achieves the better performance.
       
  • Dialogue Systems with Audio Context
    • Abstract: Publication date: Available online 14 January 2020Source: NeurocomputingAuthor(s): Tom Young, Vlad Pandelea, Soujanya Poria, Erik CambriaAbstractResearch on building dialogue systems that converse with humans naturally has recently attracted a lot of attention. Most work on this area assumes text-based conversation, where the user message is modeled as a sequence of words in a vocabulary. Real-world human conversation, in contrast, involves other modalities, such as voice, facial expression and body language, which can influence the conversation significantly in certain scenarios. In this work, we explore the impact of incorporating the audio features of the user message into generative dialogue systems. Specifically, we first design an auxiliary response retrieval task for audio representation learning. Then, we use word-level modality fusion to incorporate the audio features as additional context to our main generative model. Experiments show that our audio-augmented model outperforms the audio-free counterpart on perplexity, response diversity and human evaluation.
       
  • A Novel Patch-based Nonlinear Matrix Completion Algorithm for Image
           Analysis through Convolutional Neural Network
    • Abstract: Publication date: Available online 14 January 2020Source: NeurocomputingAuthor(s): Mingming Yang, Songhua XuAbstractMatrix completion is extensively studied due to its wide applications in science and technology. In this paper, we concentrate our study on the matrix completion problem for image analysis tasks due to their immense importance and pervasive use in many fields. A rich collection of models has been proposed to capture both linear and nonlinear relationships latent in a matrix. Even though nonlinear models possess more powerful matrix completion capabilities than their linear counterparts, these models generally carry higher model complexity and tuning difficulties. To take advantage of the superior discriminative power offered by nonlinear models while curbing its deployment overheads, this paper proposes a novel nonlinear matrix completion model utilizing a deep learning-based approach. In contrast to existing nonlinear models, the new model carefully explores and exploits spatial locality among adjacent matrix elements exhibited in patches of various sizes and locations in a target matrix. Building upon this idea, a new patch-based nonlinear matrix completion algorithm is designed. The algorithm leverages a convolutional neural network to learn the predictive relationship between a matrix element and its surrounding elements through an end-to-end trainable fashion, leading to a capable and easy-to-deploy nonlinear matrix completion solution. To identify an optimal patch size suited for tackling a given matrix completion task without exhaustively enumerating all candidate patch sizes, the new algorithm is coupled with a fast stochastic search procedure, yielding a good trade-off between computational efficiency and accuracy. Extensive experiments are conducted to validate the effectiveness and advantages of the proposed algorithm for nonlinear matrix completion problem in comparison with a series of state-of-the-art algorithms. Experimental results consistently demonstrate the superiority of the new algorithm in completing images with a variety of random and textual noises.
       
  • Adversarial dictionary learning for a robust analysis and modelling of
           spontaneous neuronal activity
    • Abstract: Publication date: Available online 14 January 2020Source: NeurocomputingAuthor(s): Eirini Troullinou, Grigorios Tsagkatakis, Ganna Palagina, Maria Papadopouli, Stelios Manolis Smirnakis, Panagiotis TsakalidesAbstractThe field of neuroscience is experiencing rapid growth in the complexity and quantity of the recorded neural activity allowing us unprecedented access to its dynamics in different brain areas. The objective of this work is to discover directly from the experimental data rich and comprehensible models for brain function that will be concurrently robust to noise. Considering this task from the perspective of dimensionality reduction, we develop an innovative, robust to noise dictionary learning framework based on adversarial training methods for the identification of patterns of synchronous firing activity as well as within a time lag. We employ real-world binary datasets describing the spontaneous neuronal activity of laboratory mice over time, and we aim to their efficient low-dimensional representation. The results on the classification accuracy for the discrimination between the clean and the adversarial-noisy activation patterns obtained by an SVM classifier highlight the efficacy of the proposed scheme compared to other methods, and the visualization of the dictionary’s distribution demonstrates the multifarious information that we obtain from it.
       
  • Multi-view Feature Selection via Nonnegative Structured Graph Learning
    • Abstract: Publication date: Available online 14 January 2020Source: NeurocomputingAuthor(s): Xiangpin Bai, Lei Zhu, Cheng Liang, Jingjing Li, Xiushan Nie, Xiaojun ChangAbstractGraph-based solutions have achieved state-of-the-art performance on unsupervised multi-view feature selection. However, existing methods generally characterize the sample similarities first by constructing multiple fixed graphs with manually determined parameters, and then perform the feature selection on a composite one. They will suffer from two severe problems: 1) The fixed graphs may be unreliable as the raw multi-view features usually contain adverse noises and cannot accurately capture the intrinsic sample relations. 2) The graph construction and feature selection are separate and independent, the two-step learning may lead to sub-optimal performance. To tackle these problems, in this paper, we propose an effective unsupervised multi-view feature selection method, dubbed as Nonnegative Structured Graph Learning (NSGL). Specifically, we develop a unified learning framework, which directly learns the structured graph from the raw features by imposing a rank constraint, and simultaneously performs adaptive feature selection with exploiting the complementarity of multi-view features. Besides, we introduce the pseudo label learning to extract the discriminative semantic information in unsupervised scenarios and steer the graph learning process. The informative features are finally selected by forcing the feature selection matrix to be sparse in rows with sparse regression. To solve the challenging optimization problem, we first transform the formulated problem into an equivalent one that can be tackled more easily, and then develop an efficient alternate optimization algorithm guaranteed with convergence to calculate the solution iteratively. Extensive experiments on several widely tested benchmarks demonstrate the superiority of NSGL compared with several state-of-the-art approaches.
       
  • Robust Stochastic Configuration Network Multi-Output Modeling of Molten
           Iron Quality in Blast Furnace Ironmaking
    • Abstract: Publication date: Available online 13 January 2020Source: NeurocomputingAuthor(s): Jin Xie, Ping ZhouAbstractBlast furnace ironmaking (BFI) is currently the most widely used method of pig iron smelting. In order to achieve efficient and reasonable control, how to quickly and accurately obtain the molten iron quality (MIQ) model is a key issue. Aiming at this problem, this paper applies robust stochastic configuration networks (RSCNs) based on kernel density estimation (KDE) into the BFI modeling to obtain the MIQ model with good modeling accuracy and strong robustness quickly and effectively. Firstly, the network model is incrementally constructed by adding neurons one by one using the conventional SCNs algorithm. Secondly, in order to solve the problem of insufficient robustness of conventional SCNs, kernel density estimation algorithm is introduced to obtain the corresponding probability density estimates of each training set, and it's used as the penalty weight introduced into constructing process of conventional SCNs. At the same time, the network output weight is obtained by an improved method to solve the problem that the output weight of the conventional RSCNs is abnormal in the multi-output modeling application. Finally, modeling experiments based on actual industrial data of BFI production verified that RSCNs can achieve good modeling accuracy and strong robust performance.
       
  • Landmark Map: An Extension of the Self-Organizing Map for a User-Intended
           Nonlinear Projection
    • Abstract: Publication date: Available online 13 January 2020Source: NeurocomputingAuthor(s): Akinari OnishiAbstractA self-organizing map (SOM) is an unsupervised artificial neural network that is widely used in, e.g., data mining and visualization. Supervised and semi-supervised learning methods have been proposed for the SOM. However, their teacher labels do not describe the relationship between the data and the location of nodes. This study proposes a landmark map (LAMA), which is an extension of SOMs that utilizes several landmarks, e.g., pairs of nodes and data points. LAMA is designed to obtain a user-intended nonlinear projection to achieve, e.g., the landmark-oriented data visualization. To reveal the learning properties of LAMA, the Zoo dataset from the UCI Machine Learning Repository, the McDonald’s dataset from Kaggle, and an artificial formant dataset were analyzed. The analysis results of the Zoo dataset indicated that LAMA could provide a new data view such as the landmark-centered data visualization. McDonald’s dataset analysis demonstrated menu recommendation examples based on a few designated items. Furthermore, the artificial formant data analysis revealed that LAMA successfully provided the intended nonlinear projection associating articular movement with vertical and horizontal movement of a computer cursor. Potential applications of LAMA include data mining, recommendation systems, and human–computer interaction.
       
  • PolSAR image classification via a novel semi-supervised recurrent
           complex-valued convolution neural network
    • Abstract: Publication date: Available online 13 January 2020Source: NeurocomputingAuthor(s): Wen Xie, Gaini Ma, Feng Zhao, Hanqiang Liu, Lu ZhangAbstractDue to that polarimetric synthetic aperture radar (PolSAR) data suffers from missing labeled samples and complex-valued data, this article presents a novel semi-supervised PolSAR terrain classification method named recurrent complex-valued convolution neural network (RCV-CNN) which combines semi-supervised learning and complex-valued convolution neural network (CV-CNN). The proposed method only needs a small number of labeled samples to achieve good classification results. First, a Wishart classifier is used to select some reliable PolSAR samples. Then, two new semi-supervised deep classification model RCV-CNN1 and RCV-CNN2 have been proposed to improve PolSAR image classification accuracy. Moreover, our proposed methods could solve the problem of network overfitting phenomenon to some extend when the number of training samples is too small. Finally, three real PolSAR dataset are applied to verify the effectiveness of our algorithms. Compared with the other five state-of-the-art methods, the proposed RCV-CNN1 and RCV-CNN2 classification models show good performance in accuracy and generalization.
       
  • A Simple Saliency Detection Approach via Automatic Top-Down Feature Fusion
    • Abstract: Publication date: Available online 13 January 2020Source: NeurocomputingAuthor(s): Yu Qiu, Yun Liu, Hui Yang, Jing XuAbstractIt is widely accepted that the top sides of convolutional neural networks (CNNs) convey high-level semantic features, and the bottom sides contain low-level details. Therefore, most of recent salient object detection methods aim at designing effective fusion strategies for side-output features. Although significant progress has been achieved in this direction, the network architectures become more and more complex, which will make the future improvement difficult and heavily engineered. Moreover, the manually designed fusion strategies would be sub-optimal due to the large search space of possible solutions. To address above problems, we propose an Automatic Top-Down Fusion (ATDF) method, in which the global information at the top sides are flowed into bottom sides to guide the learning of low layers. We design a novel valve module and add it at each side to control the coarse semantic information flowed into a specific bottom side. Through these valve modules, each bottom side at the top-down pathway is expected to receive necessary top information. We also design a generator to improve the prediction capability of fused deep features for saliency detection. We perform extensive experiments to demonstrate that ATDF is simple yet effective and thus opens a new path for saliency detection.
       
  • Systematic Evaluation of Deep Face Recognition Methods
    • Abstract: Publication date: Available online 13 January 2020Source: NeurocomputingAuthor(s): Mingyu You, Xuan Han, Yangliu Xu, Li LiAbstractFace recognition is an important task in both academia and industry. With the development of deep convolutional neural networks, many deep face recognition methods have been proposed and have achieved remarkable results. However, these methods show great diversity among their datasets, network architectures, loss functions, and parameter learning strategies. For those who want to apply these technologies to establish a deep face recognition system, it is bewildering to evaluate which improvements are more suitable and effective.This study systematically summarizes and evaluates the state-of-the-art face recognition methods. However, unlike general reviews, on the basis of a survey, this study presents a comprehensive evaluation framework and measures the effects of multifarious settings in five components, including data augmentation, network architecture, loss function, network training, and model compression.Based on the experimental results, the influences of these five components on the deep face recognition are summarized. In terms of the datasets, a high sample-identity ratio is conducive to generalization, but it leads to increased difficulty for the training to converge. For the network architecture, deep ResNet has an advantage over other designs. Various normalization operations in the network are also necessary. For the loss function, whose performance is closely related to network design and training conditions. The angle-margin loss has a higher upper bound performance, but the traditional Euclidean-margin loss has a stable performance in limited training condition and shallower network. In terms of the training strategy, the step-declining learning rate and large batch size are recommended for recognition tasks. Furthermore, this study compares several popular model compression methods and shows that MobileNet has advantages over the others in terms of both compression ratio and robustness. Finally, a detailed list of recommended settings is provided.
       
  • Neighbor Similarity and Soft-label Adaptation for Unsupervised
           Cross-dataset Person Re-identification
    • Abstract: Publication date: Available online 13 January 2020Source: NeurocomputingAuthor(s): Yiru Zhao, Hongtao LuAbstractMost of the existing person re-identification algorithms rely on supervised model learning from a large number of labeled training data per-camera-pair. However, the manual annotations often require expensive human labor, which limits the application of supervised methods for large-scale real-world deployments. To address this problem, we formulate a Neighbor Similarity and Soft-label Adaptation (NSSA) algorithm to transfer the supervised information from source domain to a new unlabeled target dataset. Specifically, we introduce a distance metric on the target domain, which incorporates inner-domain neighbor similarity and inter-domain soft-label adapted from source domain. The unlabeled samples which are close in this metric are considered to share the same pseudo-id and further selected to fine-tune the model. The training is performed iteratively to incorporate more credible sample pairs and incrementally improve the model. Extensive experimental results validate the superiority of our proposed NSSA algorithm, which significantly outperforms the state-of-the-art unsupervised and domain adaptation re-identification methods.
       
  • Conformal Prediction Based Active Learning by Linear Regression
           Optimization
    • Abstract: Publication date: Available online 13 January 2020Source: NeurocomputingAuthor(s): Sergio Matiz, Kenneth E. BarnerAbstractConformal prediction uses the degree of strangeness (nonconformity) of data instances to determine the confidence values of new predictions. We propose a conformal prediction based active learning algorithm, referred to as CPAL-LR, to improve the performance of pattern classification algorithms. CPAL-LR uses a novel query function that determines the relevance of unlabeled instances through the solution of a constrained linear regression model, incorporating uncertainty, diversity, and representativeness in the optimization problem. Furthermore, we present a nonconformity measure that produces reliable confidence values. CPAL-LR is implemented in conjunction with support vector machines, sparse coding algorithms, and convolutional networks. Experiments conducted on face and object recognition databases demonstrate that CPAL-LR improves the classification performance of a variety classifiers, outperforming previously proposed active learning techniques, while producing reliable confidence values.
       
  • Effective and Scalable Causal Partitioning Based on Low-order Conditional
           Independent Tests
    • Abstract: Publication date: Available online 13 January 2020Source: NeurocomputingAuthor(s): Chuanxu Yan, Shuigeng ZhouAbstractRecovering causal relationships from observed data is crucial to a variety of applications. Due to the curse of dimensionality, general causal discovery methods such as constraint-based methods and functional model based methods are not quite effective and efficient for large and high-dimensional data sets. Thus, some causal partitioning methods have been proposed to handle this problem. However, existing causal partitioning methods rely on high-order conditional independent (CI) tests, which makes them inefficient in handling dense causal graphs. Therefore, high-dimensionality is still a big challenge to these methods. In this work, we propose a new split-and-merge strategy to enable effective and scalable causality discovery. Different from the existing methods, our method uses only low-order CI tests, can get more accurate results and is applicable to various scenarios. We provide both theoretic analysis and empirical evaluation on the proposed method. Experiments on various real-world causal graphs show that the proposed method outperforms the stat-of-the-art method in terms of accuracy, efficiency and scalability. For high-dimensional cases, our method is much faster than the counterpart by one to three orders of magnitudes.
       
  • Emission Stations Location Selection Based on Conditional Measurement GAN
           Data
    • Abstract: Publication date: Available online 13 January 2020Source: NeurocomputingAuthor(s): Zhenyi Xu, Yu Kang, Yang CaoAbstractUrban vehicle emission monitoring can help make suggestions for the pollution emission control, and can protect public health. However, it is hard to get an overview of the vehicle emission in the city scale due to the sparse emission remote sensing stations in city, and selecting the appropriate stations locations which can reflect the emission variation in the given region mostly is another challenge. The existing methods solve the problem by spatial interpolation based on geographical statistics methods, without considering that urban vehicle emission varies by locations non-linearly and depends on many complex external factors. To tackle the spatial sparsity of vehicle remote sensing data, we design a data augmentation strategy based on Generative Adversarial Networks (GAN), which leverages prior model COPERT with conditional measurement data to filling the missing entries at other locations. With this strategy, we can generate realistic emission data while accelerating the training process. In addition, to address the emission stations location selection problem, we design a novel location selection strategy based on Spearman’s rank correlation coefficients, which leverages the realistic data generated to discover the grids with maximum link correlation for the pre-deployed station. Finally, we present experiments with the remote emission sensing data in Hefei, and the results demonstrate that our proposed model can mimics the real vehicle emission distribution in the given region effectively.
       
  • A Power-Type Varying Gain Discrete-Time Recurrent Neural Network For
           Solving Time-Varying Linear System
    • Abstract: Publication date: Available online 13 January 2020Source: NeurocomputingAuthor(s): Zhijun Zhang, Wenwei Lin, Lunan Zheng, Pengchao Zhang, Xilong Qu, Yue FengAbstractMany practical engineering problems can be described as an online time-varying linear system (TVLS), and thus solving TVLS is very important in control theory and control engineering. In this paper, a novel power-type varying gain discrete-time recurrent neural network (PVG-DTRNN) is proposed to solve the TVLS problem. Compared with the state-of-art method, i.e., the fixed-parameter discrete-time zeroing neural network (FP-DTZNN), the proposed PVG-DTRNN has better convergent rate and higher accuracy. To do so, a vector error function is firstly defined. Secondly, a power-type gain implicit dynamic model is derived and needs to be further discretized. Thirdly, by using Euler forward-difference rule, a discretized dynamic model is designed. In order to get the explicit dynamic model, the Broyden-Fletcher-Goldfarb-Shanno (BFGS) quasi-Newton method is utilized to estimate the inverse of the Hessian matrix. Comparisons of computer simulations verify the effectiveness and superiority of the proposed PVG-DTRNN models.
       
  • Alpha divergence minimization in multi-class Gaussian process
           classification
    • Abstract: Publication date: 22 February 2020Source: Neurocomputing, Volume 378Author(s): Carlos Villacampa-Calvo, Daniel Hernández-LobatoAbstractThis paper analyzes the minimization of α-divergences in the context of multi-class Gaussian process classification. For this task, several methods are explored, including memory and computationally efficient variants of the Power Expectation Propagation algorithm, which allow for efficient training using stochastic gradients and mini-batches. When these methods are used for training, very large datasets (several millions of instances) can be considered. The proposed methods are also very general as they can interpolate between other popular approaches for approximate inference based on Expectation Propagation (EP) (α → 1) and Variational Bayes (VB) (α → 0) simply by varying the α parameter. An exhaustive empirical evaluation analyzes the generalization properties of each of the proposed methods for different values of the α parameter. The results obtained show that one can do better than EP and VB by considering intermediate values of α.
       
  • DCGSA: A global self-attention network with dilated convolution for crowd
           density map generating
    • Abstract: Publication date: 22 February 2020Source: Neurocomputing, Volume 378Author(s): Liping Zhu, Chengyang Li, Bing Wang, Kun Yuan, Zhongguo YangAbstractDue to non-uniform density and variations in scale and perspective, estimating crowd count in crowded scenes in different degree is an extremely challenging task. The deep learning models mostly use pooling operation so that the density map of original resolution is obtained through the last upsampling. This paper aims to solve the problem of losing local spatial information by pooling in density map estimation. Therefore, we propose a dilated convolution neural network with global self-attention, named DCGSA. Especially, we introduce a Global Self-Attention module (GSA) to provide global context as guidance of low-level features to select person location details and a Pyramid Dilated Convolution module (PDC) that extracts channel-wise and pixel-wise features more precisely. Extensive experiments on several crowd datasets show that our method achieves lower crowd counting error and better density maps compared to the recent state-of-the-art methods. In particular, our method also performs well on the sparse dataset UCSD.
       
  • Non-overlapping classification of hyperspectral imagery based on
           set-to-sets distance
    • Abstract: Publication date: 22 February 2020Source: Neurocomputing, Volume 378Author(s): Xianghai Cao, Meiru Ren, Jing Zhao, Hongxia Lu, Licheng JiaoAbstractSpectral–spatial classification methods can improve the classification accuracy of hyperspectral imagery (HSI) dramatically. However, this improvement is caused in part by the overlap between training set and test set. In this paper, a novel non-overlapping spectral-spatial classification framework based on set-to-sets distance is proposed. First, each class is sampled using a controlled random sampling method, which is regarded as a training set. The image is segmented into many superpixels and each superpixel is taken as a test set. In order to ensure that the test set and the training set are not overlapped with each other, the training pixels contained in each test superpixel are deleted. Then, the training set of each class is compressed into a more compact set to reduce the computational complexity and kernel trick is used to make samples be approximately linearly separable. Finally, each test set is modeled as a convex hull, this hull is represented collaboratively with all training sets. With the resolved representation coefficients, the distance between the test set and each training set can be calculated for classification. Experimental results based on three real HSI data sets demonstrate the superiority of the proposed method to state-of-the-art algorithms under the non-overlapping sampling strategy.
       
  • Off-policy synchronous iteration IRL method for multi-player zero-sum
           games with input constraints
    • Abstract: Publication date: 22 February 2020Source: Neurocomputing, Volume 378Author(s): He Ren, Huaguang Zhang, Yunfei Mu, Jie DuanAbstractIn this paper, a novel synchronous off-policy method is given to solve multi-player zero-sum (ZS) game under the condition that the knowledge of system data are completely unknown, the actuators of controls are constrained and the disturbances are bounded simultaneously. The cost functions are built by nonquadratic functions to reflect the constrained properties of inputs. The integral reinforcement learning (IRL) technology is employed to solve Hamilton–Jacobi–Bellman equation, so that the system dynamics are not necessary anymore. The obtained value function is proved to converge to the optimal game values. And the equivalent of traditional policy iteration (PI) algorithm and the proposed algorithm is given in solving the multi-player ZS game with constrained inputs. Three neural networks in this paper are utilized, the critic neural network (CNN) to approach the cost function, the action neural network (ANN) to approach the control policies and the disturbance neural networks (DNN) to approach the disturbances are utilized. Finally, a simulation example is given to demonstrate the convergence and performance of the proposed algorithm.
       
  • Global exponential dissipativity of neutral-type BAM inertial neural
           networks with mixed time-varying delays
    • Abstract: Publication date: 22 February 2020Source: Neurocomputing, Volume 378Author(s): Liyan Duan, Jigui Jian, Baoxian WangAbstractThis paper considers the global exponential dissipativity of neutral-type BAM inertial neural networks with mixed time-varying delays. Firstly, we transform the proposed BAM inertial neural networks to usual one. Secondly, by establishing a new neutral-type differential inequality and employing Lyapunov method and analytical techniques, some novel sufficient conditions in accordance with algebraic and linear matrix inequalities are obtained for the global exponential dissipativity of the addressed neural networks. Moreover, the globally exponentially attractive sets and the exponential convergence rate index are also assessed. Finally, the effectiveness of the obtained results is illustrated by some examples with numerical simulations.
       
  • Deep Gabor convolution network for person re-identification
    • Abstract: Publication date: 22 February 2020Source: Neurocomputing, Volume 378Author(s): Yuan Yuan, Jian’an Zhang, Qi WangAbstractPerson re-identification is an import problem in computer vision fields and more and more deep neural network models have been developed for representation learning in this task due to their good performance. However, compared with hand-crafted feature representations, deep learned features cannot not be interpreted easily. To meet this demand, motivated by the Gabor filters’ good interpretability and the deep neural network models’ reliable learning ability, we propose a new convolution module for deep neural networks based on Gabor function (Gabor convolution). Compared with classical convolution module, every parameter in the proposed Gabor convolution kernel has a specific meaning while classical one has not. The Gabor convolution module has a good texture representation ability and is effective when it is embedded in the low layers of a network. Besides, in order to make the proposed Gabor module meaningful, a new loss function designed for this module is proposed as a regularizer of total loss function. By embedding the Gabor convolution module to the Resnet-50 network, we show that it has a good representation learning ability for person re-identification. And experiments on three widely used person re-identification datasets show favorable results compared with the state-of-the-arts.
       
  • Enhancing multi-view clustering through common subspace integration by
           considering both global similarities and local structures
    • Abstract: Publication date: 22 February 2020Source: Neurocomputing, Volume 378Author(s): Wanlin Weng, Weiwei Zhou, Jiazhou Chen, Hong Peng, Hongmin CaiAbstractMulti-view clustering seeks to partition objects based on various observations by utilizing cross-views to provide a complementary description of the same objects. It remains challenging to effectively fuse the multi-view data with various dimensions as well as different structures into a new yet highly informative form, thus facilitating adequate assignment of the objects. To tackle the issue, we propose a common subspace integration (CSI) model. The CSI actively learns a common subspace by jointly preserving the local geometry of each view, while incorporating a global partition information to enhance its separability during the learning process. It can be easily generalized to its kernel version, thereby popularizing its general usages. An effective alternative optimization scheme is designed to solve the proposed model. Extensive experiments on six real-world datasets were conducted to demonstrate its superiority by comparing with the twelve state-of-art methods.
       
  • 3D-SSD: Learning hierarchical features from RGB-D images for amodal 3D
           object detection
    • Abstract: Publication date: 22 February 2020Source: Neurocomputing, Volume 378Author(s): Qianhui Luo, Huifang Ma, Li Tang, Yue Wang, Rong XiongAbstractThis paper aims at developing a faster and more accurate solution to the amodal 3D object detection problem for indoor scenarios. The solution is achieved through a novel neural network structure which takes a pair of RGB-D images as input and delivers oriented 3D bounding boxes as the output. Such network, named 3D-SSD, has two components: hierarchical feature fusion and multi-layer prediction. The hierarchical feature fusion combines multi-scale appearance and geometric features learned from RGB-D images, which is later utilized in the multi-layer prediction for object detection. Both the accuracy and the efficiency can be improved by exploiting 2.5D representations in a synergistic way. To specifically address the shape variance of different objects, a set of 3D anchor boxes with varying physical sizes are attached to every location on the prediction layers. While testing, the category scores for 3D anchor boxes are generated with adjusted positions, sizes and orientations, leading to the final detections using non-maximum suppression. Comprehensive experiments have been performed on publicly accessible dataset of SUN RGB-D and NYUV2. The results show the proposed algorithm is the first 3D detector that runs in near real-time on the challenging datasets with competitive performance to the state-of-the-art methods. The 3D-SSD gets 37.1% mAP on the SUN RGB-D dataset at around 5.6 fps, which outperforms the state-of-the-art Deep Sliding Shape by 10.2% mAP and around 109 ×  faster. For an efficient model setting with a rate of 9.3 fps, 3D-SSD still gets an accuracy of 37% on mAP. Further, experiments also suggest the proposed approach achieves comparable accuracy and is about 477 ×  faster than the state-of-art method on the NYUv2 dataset even with a smaller input image size.
       
  • Effective piecewise planar modeling based on sparse 3D points and
           convolutional neural network
    • Abstract: Publication date: 22 February 2020Source: Neurocomputing, Volume 378Author(s): Wei Wang, Wei Gao, Zhanyi HuAbstractPiecewise planar stereo methods can approximately reconstruct the complete structures of a scene by overcoming challenging difficulties (e.g., poorly textured regions) that pixel-level stereo methods cannot resolve. In this paper, a novel plane assignment cost is first constructed by incorporating scene structure priors and high-level image features obtained by convolutional neural network (CNN). Then, the piecewise planar scene structures are reconstructed in a progressive manner that jointly optimizes image regions (or superpixels) and their associated planes, followed by a global plane assignment optimization under a Markov Random Field (MRF) framework. Experimental results on a variety of urban scenes confirm that the proposed method can effectively reconstruct the complete structures of a scene from only sparse three-dimensional (3D) points with high efficiency and accuracy and can achieve superior results compared with state-of-the-art methods.
       
  • Automatic fetal brain extraction from 2D in utero fetal MRI slices using
           deep neural network
    • Abstract: Publication date: 22 February 2020Source: Neurocomputing, Volume 378Author(s): Jinpeng Li, Yishan Luo, Lin Shi, Xin Zhang, Ming Li, Bing Zhang, Defeng WangAbstractBackgroundIn utero fetal MRI has been developing in common medical prenatal practice for nearly two decades. But the applications and research on fetal MRI still lag behind due to the lack of specialized image processing and analysis tools. Brain extraction, as an initial preprocessing step for many brain MRI-based processing methods, is an important basis for accurate fetal MRI analysis. However, it is very challenging to automatically extract fetal brains from fetal MRI due to the large variation in fetal brains across different gestational weeks and complex maternal tissues surrounding the fetal brains.MethodWe proposed a novel two-step framework using the deep learning method for solving the challenging problem of automatic fetal brain extraction in 2D in utero fetal MRI slices. The proposed framework consisted of two fully convolutional network (FCN) models, i.e., a shallow FCN and an extra deep multi-scale FCN (M-FCN). The first shallow FCN rapidly located the fetal brain and extracted the region of interest (ROI) containing the brain. Then, within the brain ROI, the M-FCN further refined the segmentation and produced the final brain mask by leveraging the multi-scale information and residual learning blocks. Dilated convolutional layers were employed in both FCNs to control the size of feature maps and increase the field of view.ResultEighty-eight 2D fetal MRIs were collected for experiments. We compared our method with the state-of-the-art methods on extracting fetal brains. It has been evaluated that our proposed framework outperformed the other methods in both fetal brain localization and segmentation tasks. With the proposed method, we located the fetal brain with an accuracy of 100%. The brain segmentation performance was measured based on the overlap between the automatic segmentations and the manual segmentations. Our proposed method achieved an average of 0.958 Dice score, 0.950 sensitivity rate, and 0.968 precision on the testing dataset, and it took an average of 6 s to process one fetal MRI stack on a workstation with TITAN X GPU and i7-6700 CPU.ConclusionIn this paper, we proposed an effective and efficient deep learning framework for automatic fetal brain extraction from fetal MRI. It has been validated with solid experiments that the proposed method can be used as a practical and useful tool in clinical practice and neuroscience research.
       
  • Integral sliding mode synchronization control for Markovian jump inertial
           memristive neural networks with reaction–diffusion terms
    • Abstract: Publication date: 22 February 2020Source: Neurocomputing, Volume 378Author(s): Xiaona Song, Jingtao Man, Shuai Song, Junwei LuAbstractThis paper is concerned with the synchronization problem for a class of inertial memristive neural networks (IMNNs), where the Markovian jump parameters and reaction–diffusion terms are involved. First, by choosing a suitable variable substitution, the original second-order differential systems can be transformed into the first-order ones. Then, based on the sliding mode control scheme, a specific sliding mode function that contains some mode-dependent integral terms and an integral term of the symbol function is proposed, so that the computational difficulties in handling Markovian jump IMNNs with reaction–diffusion terms can be solved. The synchronization criterion in forms of algebraic inequalities can be obtained by the ingenious use of some inequality techniques. Finally, three examples are provided to verify the feasibility, practicability and superiority of the proposed theoretical results.
       
  • Semantic-based padding in convolutional neural networks for improving the
           performance in natural language processing. A case of study in sentiment
           analysis
    • Abstract: Publication date: 22 February 2020Source: Neurocomputing, Volume 378Author(s): Maite Giménez, Javier Palanca, Vicent BottiAbstractIn this work, a methodology for applying semantic-based padding in Convolutional Neural Networks for Natural Language Processing tasks is proposed. Semantic-based padding takes advantage of the unused space required for having a fixed-size input matrix in a Convolutional Network effectively, using words present in the sentence. The methodology proposed has been evaluated intensively in Sentiment Analysis tasks using a variety of word embeddings. In all the experimentation carried out the proposed semantic-based padding improved the results achieved when no padding strategy is applied. Moreover, when the model used a pre-trained word embeddings, the performance of the state of the art has been surpassed.
       
  • Event-triggered ADP control of a class of non-affine continuous-time
           nonlinear systems using output information
    • Abstract: Publication date: 22 February 2020Source: Neurocomputing, Volume 378Author(s): Yang Yang, Chuang Xu, Dong Yue, Xiangnan Zhong, Xuefeng Si, Jie TanAbstractAn event-triggered adaptive dynamic programming (ADP) approach is proposed for a class of non-affine continuous-time nonlinear systems with unknown internal states. A neural networks (NNs)-based observer is designed to reconstruct internal states of the system using output information, and then, by the estimation signals, an output feedback ADP control approach is developed in an event-triggered manner. The proposed approach samples the states and updates the control signal only when the triggered condition is violated, and critic NNs are designed to approximate the performance index. Compared with the traditional ADP one under a fixed sampling mechanism, the event-triggered control approach reduces the computation resource and transmission load in the learning process. The stability analysis of the closed-loop system is provided based on the Lyapunov’s theorem. Two simulation results also verify the theoretical claims.
       
  • An expectation-maximization based single-beacon underwater navigation
           method with unknown ESV
    • Abstract: Publication date: 22 February 2020Source: Neurocomputing, Volume 378Author(s): Hong-De Qin, Xiang Yu, Zhong-Ben Zhu, Zhong-Chao DengAbstractNavigation performance in a single-beacon underwater navigation system considerably depends on the accuracy of the slant-range measurement. Ranges are usually obtained based on a presumed or known effective sound velocity (ESV). Because it is difficult to accurately determine the ESV between the pinger and the receiver, traditional methods are usually affected by large-range measurement errors that lead to large positioning errors. In this study, we use the expectation maximization (EM) method, which is widely used for parameter identification, to estimate the unknown ESV by treating it as a model parameter. We propose an EM-based, single-beacon navigation method that incorporates the Kalman filter into the EM frame. Numerical examples using simulated and field data indicate that navigation accuracy can be significantly improved when the proposed EM-based method is implemented, and the estimated ESV is in good agreement with its true value.
       
  • Aggregating diverse deep attention networks for large-scale plant species
           identification
    • Abstract: Publication date: 22 February 2020Source: Neurocomputing, Volume 378Author(s): Haixi Zhang, Zhenzhong Kuang, Xianlin Peng, Guiqing He, Jinye Peng, Jianping FanAbstractIn this paper, a novel fusion method is proposed to deal with large-scale plant species identification by aggregating diverse outputs from multiple deep networks, where each deep network focus on one subset of the whole plant species. Firstly, a fixed plant taxonomy is constructed for organizing large number of fine-grained plant species hierarchically and it is further used as a guideline to help generating diverse but overlapped task groups. Secondly, an attention-based deep hierarchical multi-task learning (AHMTL) algorithm is proposed to recognize fine-grained plant species belonging to the same task group effectively by learning more discriminative deep features and classifiers jointly. Finally, we fuse all outputs from multiple deep networks to obtain the final high-level feature representation and give the prediction probability for each plant species. The experimental results have proved the effectiveness of our proposed method on large-scale plant species identification.
       
  • Diverse frequency band-based convolutional neural networks for tonic cold
           pain assessment using EEG
    • Abstract: Publication date: 22 February 2020Source: Neurocomputing, Volume 378Author(s): Mingxin Yu, Yichen Sun, Bofei Zhu, Lianqing Zhu, Yingzi Lin, Xiaoying Tang, Yikang Guo, Guangkai Sun, Mingli DongAbstractThe purpose of this study is to present a novel classification framework, called diverse frequency band-based Convolutional Neural Networks (DFB-based ConvNets), which can objectively identify tonic cold pain states. To achieve this goal, scalp EEG data were recorded from 32 subjects under cold stimuli conditions. The proposed DFB-based ConvNets model is capable of classifying three classes of tonic pain: No pain, Moderate Pain, and Severe Pain. Firstly, the proposed method utilizes diverse frequency band-based inputs to learn temporal representations from different frequency bands of Electroencephalogram (EEG) which are expected to have more discriminative power. Then the derived features are concatenated to form a feature vector, which is fed into a fully-connected network for performing the classification task. Experimental results demonstrate that the proposed method successfully discriminates the tonic cold pain states. To show the superiority of the DFB-based ConvNets classifier, we compare our results with the state-of-the-art classifiers and show it has a competitive classification accuracy (97.37%). Moreover, these promising results may pave the way to use DFB-based ConvNets in clinical pain research.
       
  • Deep neural networks compression learning based on multiobjective
           evolutionary algorithms
    • Abstract: Publication date: 22 February 2020Source: Neurocomputing, Volume 378Author(s): Junhao Huang, Weize Sun, Lei HuangAbstractThis work addresses the problem of deep neural network compression, which is a promising technique to reduce the number of network parameters and/or speed up the network evaluation process significantly. Most of the existing methods rely on domain experts’ experience for the selection of hyperparameters such as the rank and sparsity ratio of the weight matrix in order to get an appropriate compression result without serious performance downgrade. However, they usually suffer from heavy computational loads due to the large number of tests in revealing the best hyperparameters. In this work, we propose an efficient approach to network compression from the perspective of multiobjective evolution. The contributions in the paper are twofolds: (1) We build a multiobjective compression learning model that considers the model classification error rate and compression rate as two objectives in the optimization, which can provide a compromise of the tradeoffs between these two objectives. (2) A mechanism for approximate compressed model generation is devised in the framework of expensive multiobjective optimization, which is able to reduce the high model training costs involved in the optimization process. Experiments are carried out to confirm the superiority of the proposed algorithm.
       
  • Prediction of blood glucose concentration for type 1 diabetes based on
           echo state networks embedded with incremental learning
    • Abstract: Publication date: 22 February 2020Source: Neurocomputing, Volume 378Author(s): Ning Li, Jianyong Tuo, Youqing Wang, Menghui WangAbstractValid prediction of blood glucose concentration can help people to manage diabetes mellitus, alert hypoglycemia/hyperglycemia, exploit artificial pancreas, and plan a treatment program. Along the development of continuous glucose monitoring system (CGMS), the massive historical data require a new modeling framework based on a data-driven perspective. Studies indicate that the glucose time series (i.e., CGMS readings) involve chaotic properties; therefore, echo state networks (ESN) and its improved variants are proposed to establish subject-specific prediction models owing to their superiority in processing chaotic systems. This study mainly has two innovations: (1) a novel combination of incremental learning and ESN is developed to obtain a suitable network structure through partial optimization of parameters; (2) a feedback ESN is proposed to excavate the relationship of different predictions. These methods are assessed on ten patients with diabetes mellitus. Experimental results substantiate that the proposed methods achieve superior prediction performance in terms of four evaluation metrics compared with three conventional methods.
       
  • Passive browser identification with multi-scale Convolutional Neural
           Networks
    • Abstract: Publication date: 22 February 2020Source: Neurocomputing, Volume 378Author(s): Saeid Samizade, Chao Shen, Chengxiang Si, Xiaohong GuanAbstractBrowser identification is the act of recognizing web traffic through surveillance despite the use of encryption or anonymizing software. Although previous work has reported some promising results, browser fingerprinting is still an emerging technique and has not reached an acceptable level of performance. This paper presents a novel approach by using deep-convolutional-neural-network-based (deep CNN) learning model to extract the complete shape of traffic I/O graph signal in obtaining stable traffic characteristics, employing nonlinear multi-class classification algorithms to perform the task of browser identification. The approach is evaluated on a new dataset collected across a large number of websites. Extensive experimental results show that traffic characteristics which are learned from I/O graph by deep CNN are much more stable and discriminative than the metrics those are obtained from the early studies, and the approach achieves a practically useful level of performance with significant precision and recall. Additional experiments on the depth of deep CNN are provided to further examine the applicability of our approach. Our dataset is publicly available to facilitate future research.
       
  • Some necessary and sufficient conditions for containment of second-order
           multi-agent systems with sampled position data
    • Abstract: Publication date: 22 February 2020Source: Neurocomputing, Volume 378Author(s): Yifan Liu, Housheng SuAbstractThe containment control problem using only sampled position data for second-order multi-agent systems (MAS) is studied in this paper. We first consider the delay-free case of the containment control protocol and obtain a necessary and sufficient condition regarding the network structure and the system parameters. Then, we obtain a similar necessary and sufficient condition for the case with time delay of the containment control protocol. Finally, several numerical simulations are performed to verify the correctness of the theoretical results.
       
  • Beyond EM: A faster Bayesian linear regression algorithm without matrix
           inversions
    • Abstract: Publication date: 22 February 2020Source: Neurocomputing, Volume 378Author(s): Ying TangAbstractThe Bayesian linear regression is a useful tool for many scientific communities. This paper presents a novel algorithm for solving the Bayesian linear regression problem with Gaussian priors, which shares the same spirit as the gradient based methods. In addition, the standard scheme for this task, the Expectation Maximization (EM) algorithm, involves matrix inversions but our proposed algorithm is free of. Numerical experiments demonstrate that the proposed algorithm performs as well as the gradient based and EM algorithms in term of precision, but runs significantly faster than the gradient based and EM algorithms. Due to its matrix-inversion-free nature, the algorithm of this paper is a viable alternative to the competing methods available in the literature.
       
  • Video summarization via block sparse dictionary selection
    • Abstract: Publication date: 22 February 2020Source: Neurocomputing, Volume 378Author(s): Mingyang Ma, Shaohui Mei, Shuai Wan, Junhui Hou, Zhiyong Wang, David Dagan FengAbstractThe explosive growth of video data has raised new challenges for many video processing tasks such as video browsing and retrieval, hence, effective and efficient video summarization (VS) is urgently demanded to automatically summarize a video into a succinct version. Recent years have witnessed the advancements of sparse representation based approaches for VS. However, video frames are analyzed individually for keyframe selection in existing methods, which could lead to redundancy among selected keyframes and poor robustness to outlier frames. Due to that adjacent frames are visually similar, candidate keyframes often occur in temporal blocks, in addition to sparse presence. Therefore, in this paper, the block-sparsity of candidate keyframes is taken into consideration, by which the VS problem is formulated as a block sparse dictionary selection model. Moreover, a simultaneous block version of Orthogonal Matching Pursuit (SBOMP) algorithm is designed for model optimization. Two keyframe selection strategies are also explored for each block. Experimental results on two benchmark datasets, namely VSumm and TVSum datasets, demonstrate that the proposed SBOMP based VS method clearly outperforms several state-of-the-art sparse representation based methods in terms of F-score, redundancy among keyframes and robustness to outlier frames.
       
  • Efficient precise weight tuning protocol considering variation of the
           synaptic devices and target accuracy
    • Abstract: Publication date: 22 February 2020Source: Neurocomputing, Volume 378Author(s): Hyeongsu Kim, Jong-Ho Bae, Suhwan Lim, Sung-Tae Lee, Young-Tak Seo, Dongseok Kwon, Byung-Gook Park, Jong-Ho LeeAbstractIn off-chip training, we can improve the inference accuracy of hardware-based neural networks by reducing the conductance (weight) variation of synaptic devices through precise program/erase control in weight mapping. However, the precise weight tuning protocol (PWTP) requires a significant amount of time because it requires repeated read-verify-write cycles for each synapse device. In this paper, we propose an efficient PWTP method to significantly reduce the weight mapping time by greatly reducing the number of synaptic devices to which PWTP should be applied. In the proposed method, the effect of weight variation of synaptic devices on the inference accuracy of neural networks depends largely on which layer the synaptic devices belong to. Using our layer-selection method, the required percentage of PWTP-applied synaptic devices is can be reduced by up to 2600 times compared to that of the conventional method where PWTP is applied to all or part of the synaptic devices which are simply ranked by the weight magnitude. Also, three criteria of variation sensitivity are evaluated and compared in the method of selecting synaptic devices to which PWTP is applied within the selected layer.
       
  • Abstractive meeting summarization by hierarchical adaptive segmental
           network learning with multiple revising steps
    • Abstract: Publication date: 22 February 2020Source: Neurocomputing, Volume 378Author(s): Jiyuan Zheng, Zhou Zhao, Zehan Song, Min Yang, Jun Xiao, Xiaohui Yanive meeting summarization is a challenging problem in natural language understanding, which automatically generates the condensed summary covering the important points in the meeting conversation. However, the existing abstractive summarization works mainly focus on the structured text documents, which may be ineffectively applied to the meeting summarization task due to the lack of modeling the unstructured long-form conversational contents. In this paper, we consider the problem of abstractive meeting summarization from the viewpoint of hierarchical adaptive segmental encoder-decoder network learning. We propose the hierarchical neural encoder based on adaptive recurrent networks to learn the semantic representation of meeting conversation with adaptive conversation segmentation. We then devise a multi-step revising mechanism to refine the learned semantic representation. We finally develop the reinforced decoder network to generate the high-quality summaries for abstractive meeting summarization. We conduct the extensive experiments on the well-known AMI meeting conversation dataset to validate the effectiveness of our proposed method.
       
  • Robust optimal graph clustering
    • Abstract: Publication date: 22 February 2020Source: Neurocomputing, Volume 378Author(s): Fei Wang, Lei Zhu, Cheng Liang, Jingjing Li, Xiaojun Chang, Ke LuAbstractMost graph-based clustering methods separate the graph construction and clustering into two independent processes. The manually pre-constructed graph may not be suitable for the subsequent clustering. Moreover, as real world data generally contains noises and outliers, the similarity graph directly learned from them will be unreliable and further impair the subsequent clustering performance. To tackle the problems, in this paper, we propose a novel clustering framework where a robust graph is learned with noise removal, and simultaneously, with desirable clustering structure. To this end, we first learn a discriminative representation of data samples via sparse reconstruction. Then, a robust graph is automatically constructed with adaptive neighbors to each data sample. Simultaneously, a reasonable rank constraint is imposed on the Laplacian matrix of similarity graph to pursue the ideal clustering structure, where the number of connected components in the learned graph is exactly equal to the number of clusters. We finally derive an alternate optimization algorithm guaranteed with convergence to solve the formulated unified learning framework to achieve better prediction accuracy. Experiments on both synthetic and real datasets demonstrate the superior performance of the proposed method compared with several state-of-the-art clustering techniques.
       
  • A Pareto-smoothing method for causal inference using generalized Pareto
           distribution
    • Abstract: Publication date: 22 February 2020Source: Neurocomputing, Volume 378Author(s): Fujin Zhu, Jie Lu, Adi Lin, Guangquan ZhangAbstractCausal inference aims to estimate the treatment effect of an intervention on the target outcome variable and has received great attention across fields ranging from economics and statistics to machine learning. Observational causal inference is challenging because the pre-treatment variables may influence both the treatment and the outcome, resulting in confounding bias. The classic inverse propensity weighting (IPW) estimator is theoretically able to eliminate the confounding bias. However, in observational studies, the propensity scores used in the IPW estimator must be estimated from finite observational data and may be subject to extreme values, leading to the problem of highly variable importance weights, which consequently makes the estimated causal effect unstable or even misleading. In this paper, by reframing the IPW estimator in the importance sampling framework, we propose a Pareto-smoothing method to tackle this problem. The generalized Pareto distribution (GPD) from extreme value theory is used to fit the upper tail of the estimated importance weights and to replace them using the order statistics of the fitted GPD. To validate the performance of the new method, we conducted extensive experiments on simulated and semi-simulated datasets. Compared with two existing methods for importance weight stabilization, i.e., weight truncation and self-normalization, the proposed method generally achieves better performance in settings with a small sample size and high-dimensional covariates. Its application on a real-world heath dataset indicates its utility in estimating causal effects for program evaluation.
       
  • Reducibilities of hyperbolic neural networks
    • Abstract: Publication date: 22 February 2020Source: Neurocomputing, Volume 378Author(s): Masaki KobayashiAbstractClifford algebra includes the real and complex numbers. The hyperbolic numbers also belong to Clifford algebra. In recent years, neural networks (NNs) are extended using Clifford algebra, and hyperbolic NNs have been proposed. Since the hyperbolic numbers have zero divisors, it is difficult to analyze the hyperbolic NNs. Thus, the reducibilities of hyperbolic NNs have never been revealed. In this work, the reducibilities of hyperbolic NNs are studied. The reducibilities are tightly related to learning process. In the real-valued and complex-valued NNs, there exist three types of reducibilities. In the hyperbolic NNs, there exists another type of reducibilities, and it has been difficult to determine all the reducibilities of hyperbolic NNs. It is proved that hyperbolic NNs have another reducibility, referred to as hyperbola-reducibility, and all the reducibities of hyperbolic NNs are determined. In addition, the inherent singularities of hyperbolic NNs are revealed. These facts are expected to improve the learning process of hyperbolic NNs in future.
       
  • Mechanisms of dynamical complexity changes in patterns of sensory neurons
           under antinociceptive effect emergence
    • Abstract: Publication date: 22 February 2020Source: Neurocomputing, Volume 378Author(s): O.E. DickAbstractThe aim is to elucidate mechanisms of changes in dynamical complexity of impulse activity patterns of sensory neurons during the emergence of the antinociceptive response on the painful stimulus. To solve the problem we used the method of bifurcation analysis that enabled us to determine relations between parameters of the nociceptive neuron model and a type of the solution before and during the emergence of the antinociceptive effect. We have shown that the mechanism of the suppression of ectopic bursting discharges is the base for the changes in the dynamical complexity of patterns in the nociceptive neurons. Under the conditions of the potassium blocking, the molecular mechanism of suppression of these discharges can be connected entirely with the modification of the activation gating system of slow NaV1.8 sodium channels by the specific action of comenic acid that is a drug substance of a new non-opioid analgesic anoceptin. The obtained results explain one of the possible molecular mechanisms of the analgesic suppression of the neuropathic pain.
       
  • Impact of fully connected layers on performance of convolutional neural
           networks for image classification
    • Abstract: Publication date: 22 February 2020Source: Neurocomputing, Volume 378Author(s): S.H. Shabbeer Basha, Shiv Ram Dubey, Viswanath Pulabaigari, Snehasis MukherjeeAbstractThe Convolutional Neural Networks (CNNs), in domains like computer vision, mostly reduced the need for handcrafted features due to its ability to learn the problem-specific features from the raw input data. However, the selection of dataset-specific CNN architecture, which mostly performed by either experience or expertise is a time-consuming and error-prone process. To automate the process of learning a CNN architecture, this paper attempts at finding the relationship between Fully Connected (FC) layers with some of the characteristics of the datasets. The CNN architectures, and recently datasets also, are categorized as deep, shallow, wide, etc. This paper tries to formalize these terms along with answering the following questions. (i) What is the impact of deeper/shallow architectures on the performance of the CNN w.r.t. FC layers', (ii) How the deeper/wider datasets influence the performance of CNN w.r.t. FC layers', and (iii) Which kind of architecture (deeper/shallower) is better suitable for which kind of (deeper/wider) datasets. To address these findings, we have performed experiments with four CNN architectures having different depths. The experiments are conducted by varying the number of FC layers. We used four widely used datasets including CIFAR-10, CIFAR-100, Tiny ImageNet, and CRCHistoPhenotypes to justify our findings in the context of image classification problem. The source code of this work is available at https://github.com/shabbeersh/Impact-of-FC-layers.
       
  • Selective ensemble of multiple local model learning for nonlinear and
           nonstationary systems
    • Abstract: Publication date: 22 February 2020Source: Neurocomputing, Volume 378Author(s): Tong Liu, Sheng Chen, Shan Liang, Chris J. HarrisAbstractThis paper proposes a selective ensemble of multiple local model learning for modeling and identification of nonlinear and nonstationary systems, in which the set of local linear models are self adapted to capture the newly emerging process characteristics and the prediction of the process output is also self adapted based on an optimally selected ensemble of subset linear local models. Specifically, our selective ensemble of multiple local model learning approach performs the model adaptation at two levels. At the level of local model adaptation, a newly emerging process state in the incoming data is automatically identified and a new local linear model is fitted to this newly emerged process state. At the level of online prediction, a subset of candidate local linear models are optimally selected and the prediction of the process output is computed as an optimal linear combiner of the selected subset local linear models. Two case studies involving chaotic time series prediction and modeling of a real-world industrial microwave heating process are used to demonstrate the effectiveness of our proposed approach, in comparison with other existing methods for modeling and identification of nonlinear and time-varying systems.
       
  • Unsupervised feature selection via adaptive hypergraph regularized latent
           representation learning
    • Abstract: Publication date: 22 February 2020Source: Neurocomputing, Volume 378Author(s): Deqiong Ding, Xiaogao Yang, Fei Xia, Tiefeng Ma, Haiyun Liu, Chang TangAbstractDue to the rapid development of multimedia technology, a large number of unlabelled data with high dimensionality need to be processed. The high dimensionality of data not only increases the computation burden of computer hardware, but also hinders algorithms to obtain optimal performance. Unsupervised feature selection, which is regarded as a means of dimensionality reduction, has been widely recognized as an important and challenging pre-step for many machine learning and data mining tasks. However, we observe that there are at least two issues in previous unsupervised feature selection methods. Firstly, traditional unsupervised feature selection algorithms usually assume that the data instances are identically distributed and there is no dependency between them. However, the data instances are not only associated with high dimensional features but also inherently interconnected with each other. Secondly, the traditional similarity graph used in previous methods can only describe the pair-wise relations of data, but cannot capture the high-order relations, so that the complex structures implied in the data cannot be sufficiently exploited. In this work, we propose a robust unsupervised feature selection method which embeds the latent representation learning into feature selection. Instead of measuring the feature importances in original data space, the feature selection is carried out in the learned latent representation space which is more robust to noises. In order to capture the local manifold geometrical structure of original data in a high-order manner, a hypergraph is adaptively learned and embedded into the resultant model. An efficient alternating algorithm is developed to optimize the problem. Experimental results on eight benchmark data sets demonstrate the effectiveness of the proposed method.
       
  • Streaking artifacts suppression for cone-beam computed tomography with the
           residual learning in neural network
    • Abstract: Publication date: 22 February 2020Source: Neurocomputing, Volume 378Author(s): Fuqiang Yang, Dinghua Zhang, Hua Zhang, Kuidong Huang, You Du, Mingxuan TengAbstractThis study aims to address and test a new residual learning algorithm in neural network applied to the projection data to generate high qualified imaging by reducing the streaking artifacts in cone-beam computed tomography (CBCT). Since the streaking artifacts have a large relationship with the noise on the projection, a residual objective upon Poisson noise corresponding to the image was proposed. As the prior, the convolution neural network (CNN) was constructed to residual learning based on the simulated label and exploited to eliminate the artifacts in the slice. To illustrate the robustness and applicability of CNN, the proposed method is evaluated using CBCT images. For the simulated projection, the PSNR and SSIM of the proposed method were dramatically increased by 15.4% and 85.9% of that with raw projection; for the true projection, the PSNR and SSIM were increased by 14.9% and 56.2%, respectively. Study results show effective results, and the proposed method is practical and attractive as a preferred solution to CT streaking artifacts suppression.
       
  • Bayesian neural multi-source transfer learning
    • Abstract: Publication date: 22 February 2020Source: Neurocomputing, Volume 378Author(s): Rohitash Chandra, Arpit KapoorAbstractAlthough the use of deep learning and neural networks techniques are gaining popularity, there remain a number of challenges when multiple sources of information and data need to be combined. Although transfer learning and data fusion methodologies try to address this challenge, they lack robust uncertainty quantification which is crucial for decision making. Bayesian inference provides a rigorous approach for uncertainty quantification in decision making. Uncertainty quantification using Bayesian inference takes into consideration uncertainty associated with model parameters, as well as, the uncertainty in combining multiple sources of data. In this paper, we present a Bayesian framework for transfer learning using neural networks that considers single and multiple sources of data. We use existence of prior distributions to define the dependency between different data sources in a multi-source Bayesian transfer learning framework. We use Markov Chain Monte-Carlo method to obtain samples from the posterior distribution that consider the knowledge from the source datasets as priors. The results show that the framework provides a robust probabilistic approach for decision making with accuracy that is similar to gradient-based learning methods. Moreover, the results are comparable to related machine learning methods used for transfer learning in the literature.
       
  • Eye localization based on weight binarization cascade convolution neural
           network
    • Abstract: Publication date: 22 February 2020Source: Neurocomputing, Volume 378Author(s): Zhen-Tao Liu, Si-Han Li, Min Wu, Wei-Hua Cao, Man Hao, Lin-Bo XianAbstractEye localization is a key step in the field of face recognition and analysis, which is the premise and breakthrough of drowsiness estimation and auxiliary driving. An eye localization method based on Weight Binarization Cascade Convolution Neural Network (WBCCNN) is proposed, in which the WBCCNN is composed of four levels and the weight is constrained by binarization. It predicts eye positions from coarse-to-fine to improve the performance of eye localization, and binary network not only helps to reduce the storage size of the model, but also speeds up the operation. Experiments on eye localization are performed using Labeled Faces in the Wild (LFW), BioID, and Labeled Face Parts in the Wild (LFPW) Databases, from which the results show that the average detection errors of left eye and right eye by our method are 0.66% and 0.71% on LFW, respectively. The operation speed of binary network is approximately as twice as that of non-binary. In addition, our method requires less storage capacity, which maintains higher performance on BioID and LFPW, compared to some state-of-the-art works.
       
  • Efficient representations of EEG signals for SSVEP frequency recognition
           based on deep multiset CCA
    • Abstract: Publication date: 22 February 2020Source: Neurocomputing, Volume 378Author(s): Qianqian Liu, Yong Jiao, Yangyang Miao, Cili Zuo, Xingyu Wang, Andrzej Cichocki, Jing JinAbstractCanonical correlation analysis (CCA) has been widely used for frequency recognition in steady-state visual evoked potential (SSVEP) based brain–computer interfaces (BCIs). However, linear CCA-based methods may be insufficient given the complexity of EEG signals. A nonlinear feature extraction method based on deep multiset CCA (DMCCA) is proposed for SSVEP recognition to fully utilize the real EEG and constructed sine–cosine signals. In DMCCA, neural networks are trained to learn the nonlinear representations of multiple sets of EEG signals at the same frequency by maximizing the overall correlation within the representations and reference signals. Therefore, reference signals are augmented with the extracted features for frequency recognition. Finally, the proposed method is evaluated using SSVEP signals collected from 10 subjects. DMCCA-based method outperforms others in terms of classification accuracy compared with CCA- and multiset CCA-based methods. The proposed DMCCA-based method has substantial potential for improving the recognition performance of SSVEP signals.
       
  • Data-Independent Feature Learning with Markov Random Fields in
           Convolutional Neural Networks
    • Abstract: Publication date: 22 February 2020Source: Neurocomputing, Volume 378Author(s): Yao Peng, Richard Hankins, Hujun YinAbstractIn image classification, deriving robust image representations is a key process that determines the performance of vision systems. Numerous image features and descriptors have been developed manually over the years. As an alternative, however, deep neural networks, in particular convolutional neural networks (CNNs), have become popular for learning image features or representations from data and have demonstrated remarkable performance in many real-world applications. But CNNs often require huge amount of labelled data, which may be prohibitive in many applications, as well as long training times. This paper considers an alternative, data-independent means of obtaining features for CNNs. The proposed framework makes use of the Markov random field (MRF) and self-organising map (SOM) to generate basic features and model both intra- and inter-image dependencies. Various MRF textures are synthesized first, and are then clustered by a convolutional translation-invariant SOM, to form generic image features. These features can be directly applied as early convolutional filters of the CNN, leading to a new way of deriving effective features for image classification. The MRF framework also offers a theoretical and transparent way to examine and determine the influence of image features on performance of CNNs. Comprehensive experiments on the MNIST, rotated MNIST, CIFAR-10 and CIFAR-100 datasets were conducted with results outperforming most state-of-the-art models of similar complexity.
       
  • Single image dehazing based on fusion strategy
    • Abstract: Publication date: 22 February 2020Source: Neurocomputing, Volume 378Author(s): Fan Guo, Xin Zhao, Jin Tang, Hui Peng, Lijue Liu, Beiji ZouAbstractIn this paper, we propose a deep convolutional network for single image dehazing based on derived image fusion strategy. Instead of estimating the transmission map and atmospheric light as previously performed, we directly generate a haze-free image by the proposed end-to-end trainable neural network. We derive five maps from the original hazy image based on the characteristics of the hazy scene to improve the dehazing performance. First, the exposure map (EM) and saliency map (SM) complement each other to focus on details in far-away and near-region scenes. Second, the white balance map (BM) and gamma correction map (GM) are employed to recover the latent colour and intensity components of the scene. Finally, the haze veil map (VM) is introduced to enhance the global image contrast. To efficiently blend the five derived maps, we propose a U-shaped deep convolutional network consisting of encoder and decoder layers to generate a haze-free image. The convolutional layers transferred from the pretrained ResNet50 are used as encoder layers for hierarchical feature extraction. Two efficient blocks, named the cascaded residual block and the channel compression block, are proposed in the network for better dehazing performance. The final dehazed result is generated by combining the significant features of the different derived maps. Additionally, perceptual loss is introduced for better visual quality. The experimental results for both synthetic and natural hazy images demonstrate that our algorithm performs comparably or even better than state-of-the-art methods in terms of the peak signal-to-noise ratio (PSNR), structure similarity (SSIM) and visual quality.
       
  • Small universal asynchronous spiking neural P systems with multiple
           channels
    • Abstract: Publication date: 22 February 2020Source: Neurocomputing, Volume 378Author(s): Xiaoxiao Song, Hong Peng, Jun Wang, Guimin Ning, Zhang SunAbstractResearchers have proposed spiking neural P systems with multiple channels (SNP-MC systems), as a variant of spiking neural P systems (SN P systems), with channel labels distinguishing different synapses. This work focuses on small universal SNP-MC systems working on asynchronously mode, where the use of enabled rules is not obligatory. We construct an asynchronous SNP-MC system using only 38 neurons and preserving its universality for computing functions. It is also proved, as small universal number generators, an asynchronous SNP-MC system needs 41 neurons. In comparing with the existing literature, asynchronous SNP-MC system needs fewer neurons than any other small asynchronous SN P system and even some synchronous SN P systems. The results show that our use of multiple channels well compensates for the computing lost when removing synchronization from SNP-MC systems.
       
  • A structure-guided approach to the prediction of natural image saliency
    • Abstract: Publication date: 22 February 2020Source: Neurocomputing, Volume 378Author(s): Haoran Liang, Ming Jiang, Ronghua Liang, Qi ZhaoAbstractThe structure of a scene provides global contextual information in directing gaze and complements local object information in saliency prediction. In this study, we explore how visual attention can be affected by scene structures, namely openness, depth and perspective. We first build an eye tracking dataset with 2500 natural scene images and collect gaze data via both eye tracking and mouse tracking. We make observations on scene layout properties and propose a set of scene structural features relating to visual attention. The set of complementary features are then integrated for saliency prediction. Our features are independent of and can work together with many computational modules, and this work demonstrates the use of Multiple kernel learning (MKL) as an example to integrate the features at low- and high-levels. Experimental results demonstrate that our model outperforms existing methods and our scene structural features can improve the performance of other saliency models in outdoor scenes.
       
 
JournalTOCs
School of Mathematical and Computer Sciences
Heriot-Watt University
Edinburgh, EH14 4AS, UK
Email: journaltocs@hw.ac.uk
Tel: +00 44 (0)131 4513762
 


Your IP address: 100.26.179.196
 
Home (Search)
API
About JournalTOCs
News (blog, publications)
JournalTOCs on Twitter   JournalTOCs on Facebook

JournalTOCs © 2009-