Similar Journals
|
Machine Learning and Knowledge Extraction
Number of Followers: 17  Open Access journalISSN (Online) 2504-4990 Published by MDPI  [258 journals]
|
- MAKE, Vol. 5, Pages 346-358: Generalized Persistence for Equivariant
Operators in Machine Learning
Authors: Mattia G. Bergomi, Massimo Ferri, Alessandro Mella, Pietro Vertechi
First page: 346
Abstract: Artificial neural networks can learn complex, salient data features to achieve a given task. On the opposite end of the spectrum, mathematically grounded methods such as topological data analysis allow users to design analysis pipelines fully aware of data constraints and symmetries. We introduce an original class of neural network layers based on a generalization of topological persistence. The proposed persistence-based layers allow the users to encode specific data properties (e.g., equivariance) easily. Additionally, these layers can be trained through standard optimization procedures (backpropagation) and composed with classical layers. We test the performance of generalized persistence-based layers as pooling operators in convolutional neural networks for image classification on the MNIST, Fashion-MNIST and CIFAR-10 datasets.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2023-03-24
DOI: 10.3390/make5020021
Issue No: Vol. 5, No. 2 (2023)
- MAKE, Vol. 5, Pages 359-383: 3t2FTS: A Novel Feature Transform Strategy to
Classify 3D MRI Voxels and Its Application on HGG/LGG Classification
Authors: Abdulsalam Hajmohamad, Hasan Koyuncu
First page: 359
Abstract: The distinction between high-grade glioma (HGG) and low-grade glioma (LGG) is generally performed with two-dimensional (2D) image analyses that constitute semi-automated tumor classification. However, a fully automated computer-aided diagnosis (CAD) can only be realized using an adaptive classification framework based on three-dimensional (3D) segmented tumors. In this paper, we handle the classification section of a fully automated CAD related to the aforementioned requirement. For this purpose, a 3D to 2D feature transform strategy (3t2FTS) is presented operating first-order statistics (FOS) in order to form the input data by considering every phase (T1, T2, T1c, and FLAIR) of information on 3D magnetic resonance imaging (3D MRI). Herein, the main aim is the transformation of 3D data analyses into 2D data analyses so as to applicate the information to be fed to the efficient deep learning methods. In other words, 2D identification (2D-ID) of 3D voxels is produced. In our experiments, eight transfer learning models (DenseNet201, InceptionResNetV2, InceptionV3, ResNet50, ResNet101, SqueezeNet, VGG19, and Xception) were evaluated to reveal the appropriate one for the output of 3t2FTS and to design the proposed framework categorizing the 210 HGG–75 LGG instances in the BraTS 2017/2018 challenge dataset. The hyperparameters of the models were examined in a comprehensive manner to reveal the highest performance of the models to be reached. In our trails, two-fold cross-validation was considered as the test method to assess system performance. Consequently, the highest performance was observed with the framework including the 3t2FTS and ResNet50 models by achieving 80% classification accuracy for the 3D-based classification of brain tumors.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2023-04-06
DOI: 10.3390/make5020022
Issue No: Vol. 5, No. 2 (2023)
- MAKE, Vol. 5, Pages 384-399: A Diabetes Prediction System Based on
Incomplete Fused Data Sources
Authors: Zhaoyi Yuan, Hao Ding, Guoqing Chao, Mingqiang Song, Lei Wang, Weiping Ding, Dianhui Chu
First page: 384
Abstract: In recent years, the diabetes population has grown younger. Therefore, it has become a key problem to make a timely and effective prediction of diabetes, especially given a single data source. Meanwhile, there are many data sources of diabetes patients collected around the world, and it is extremely important to integrate these heterogeneous data sources to accurately predict diabetes. For the different data sources used to predict diabetes, the predictors may be different. In other words, some special features exist only in certain data sources, which leads to the problem of missing values. Considering the uncertainty of the missing values within the fused dataset, multiple imputation and a method based on graph representation is used to impute the missing values within the fused dataset. The logistic regression model and stacking strategy are applied for diabetes training and prediction on the fused dataset. It is proved that the idea of combining heterogeneous datasets and imputing the missing values produced in the fusion process can effectively improve the performance of diabetes prediction. In addition, the proposed diabetes prediction method can be further extended to any scenarios where heterogeneous datasets with the same label types and different feature attributes exist.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2023-04-10
DOI: 10.3390/make5020023
Issue No: Vol. 5, No. 2 (2023)
- MAKE, Vol. 5, Pages 400-417: Lottery Ticket Search on Untrained Models
with Applied Lottery Sample Selection
Authors: Ryan Bluteau, Robin Gras
First page: 400
Abstract: In this paper, we present a new approach to improve tabular datasets by applying the lottery ticket hypothesis to tabular neural networks. Prior approaches were required to train the original large-sized model to find these lottery tickets. In this paper we eliminate the need to train the original model and discover lottery tickets using networks a fraction of the model’s size. Moreover, we show that we can remove up to 95% of the training dataset to discover lottery tickets, while still maintaining similar accuracy. The approach uses a genetic algorithm (GA) to train candidate pruned models by encoding the nodes of the original model for selection measured by performance and weight metrics. We found that the search process does not require a large portion of the training data, but when the final pruned model is selected it can be retrained on the full dataset, even if it is often not required. We propose a lottery sample hypothesis similar to the lottery ticket hypotheses where a subsample of lottery samples of the training set can train a model with equivalent performance to the original dataset. We show that the combination of finding lottery samples alongside lottery tickets can allow for faster searches and greater accuracy.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2023-04-18
DOI: 10.3390/make5020024
Issue No: Vol. 5, No. 2 (2023)
- MAKE, Vol. 5, Pages 418-430: A Reinforcement Learning Approach for
Scheduling Problems with Improved Generalization through Order Swapping
Authors: Deepak Vivekanandan, Samuel Wirth, Patrick Karlbauer, Noah Klarmann
First page: 418
Abstract: The scheduling of production resources (such as associating jobs to machines) plays a vital role for the manufacturing industry not only for saving energy, but also for increasing the overall efficiency. Among the different job scheduling problems, the Job Shop Scheduling Problem (JSSP) is addressed in this work. JSSP falls into the category of NP-hard Combinatorial Optimization Problem (COP), in which solving the problem through exhaustive search becomes unfeasible. Simple heuristics such as First-In, First-Out, Largest Processing Time First and metaheuristics such as taboo search are often adopted to solve the problem by truncating the search space. The viability of the methods becomes inefficient for large problem sizes as it is either far from the optimum or time consuming. In recent years, the research towards using Deep Reinforcement Learning (DRL) to solve COPs has gained interest and has shown promising results in terms of solution quality and computational efficiency. In this work, we provide an novel approach to solve the JSSP examining the objectives generalization and solution effectiveness using DRL. In particular, we employ the Proximal Policy Optimization (PPO) algorithm that adopts the policy-gradient paradigm that is found to perform well in the constrained dispatching of jobs. We incorporated a new method called Order Swapping Mechanism (OSM) in the environment to achieve better generalized learning of the problem. The performance of the presented approach is analyzed in depth by using a set of available benchmark instances and comparing our results with the work of other groups.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2023-04-29
DOI: 10.3390/make5020025
Issue No: Vol. 5, No. 2 (2023)
- MAKE, Vol. 5, Pages 431-448: Artificial Intelligence-Based Prediction of
Spanish Energy Pricing and Its Impact on Electric Consumption
Authors: Marcos Hernández Rodríguez, Luis Gonzaga Baca Baca Ruiz, David Criado Criado Ramón, María del Carmen Pegalajar Pegalajar Jiménez
First page: 431
Abstract: The energy supply sector faces significant challenges, such as the ongoing COVID-19 pandemic and the ongoing conflict in Ukraine, which affect the stability and efficiency of the energy system. In this study, we highlight the importance of electricity pricing and the need for accurate models to estimate electricity consumption and prices, with a focus on Spain. Using hourly data, we implemented various machine learning models, including linear regression, random forest, XGBoost, LSTM, and GRU, to forecast electricity consumption and prices. Our findings have important policy implications. Firstly, our study demonstrates the potential of using advanced analytics to enhance the accuracy of electricity price and consumption forecasts, helping policymakers anticipate changes in energy demand and supply and ensure grid stability. Secondly, we emphasize the importance of having access to high-quality data for electricity demand and price modeling. Finally, we provide insights into the strengths and weaknesses of different machine learning algorithms for electricity price and consumption modeling. Our results show that the LSTM and GRU artificial neural networks are the best models for price and consumption modeling with no significant difference.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2023-05-02
DOI: 10.3390/make5020026
Issue No: Vol. 5, No. 2 (2023)
- MAKE, Vol. 5, Pages 448-459: Tree-Structured Model with Unbiased Variable
Selection and Interaction Detection for Ranking Data
Authors: Yu-Shan Shih, Yi-Hung Kung
First page: 448
Abstract: In this article, we propose a tree-structured method for either complete or partial rank data that incorporates covariate information into the analysis. We use conditional independence tests based on hierarchical log-linear models for three-way contingency tables to select split variables and cut points, and apply a simple Bonferroni rule to declare whether a node worths splitting or not. Through simulations, we also demonstrate that the proposed method is unbiased and effective in selecting informative split variables. Our proposed method can be applied across various fields to provide a flexible and robust framework for analyzing rank data and understanding how various factors affect individual judgments on ranking. This can help improve the quality of products or services and assist with informed decision making.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2023-05-09
DOI: 10.3390/make5020027
Issue No: Vol. 5, No. 2 (2023)
- MAKE, Vol. 5, Pages 460-472: A Multi-Input Machine Learning Approach to
Classifying Sex Trafficking from Online Escort Advertisements
Authors: Lucia Summers, Alyssa N. Shallenberger, John Cruz, Lawrence V. Fulton
First page: 460
Abstract: Sex trafficking victims are often advertised through online escort sites. These ads can be publicly accessed, but law enforcement lacks the resources to comb through hundreds of ads to identify those that may feature sex-trafficked individuals. The purpose of this study was to implement and test multi-input, deep learning (DL) binary classification models to predict the probability of an online escort ad being associated with sex trafficking (ST) activity and aid in the detection and investigation of ST. Data from 12,350 scraped and classified ads were split into training and test sets (80% and 20%, respectively). Multi-input models that included recurrent neural networks (RNN) for text classification, convolutional neural networks (CNN, specifically EfficientNetB6 or ENET) for image/emoji classification, and neural networks (NN) for feature classification were trained and used to classify the 20% test set. The best-performing DL model included text and imagery inputs, resulting in an accuracy of 0.82 and an F1 score of 0.70. More importantly, the best classifier (RNN + ENET) correctly identified 14 of 14 sites that had classification probability estimates of 0.845 or greater (1.0 precision); precision was 96% for the multi-input model (NN + RNN + ENET) when only the ads associated with the highest positive classification probabilities (>0.90) were considered (n = 202 ads). The models developed could be productionalized and piloted with criminal investigators, as they could potentially increase their efficiency in identifying potential ST victims.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2023-05-10
DOI: 10.3390/make5020028
Issue No: Vol. 5, No. 2 (2023)
- MAKE, Vol. 5, Pages 473-490: Evaluating the Coverage and Depth of Latent
Dirichlet Allocation Topic Model in Comparison with Human Coding of
Qualitative Data: The Case of Education Research
Authors: Gaurav Nanda, Aparajita Jaiswal, Hugo Castellanos, Yuzhe Zhou, Alex Choi, Alejandra J. Magana
First page: 473
Abstract: Fields in the social sciences, such as education research, have started to expand the use of computer-based research methods to supplement traditional research approaches. Natural language processing techniques, such as topic modeling, may support qualitative data analysis by providing early categories that researchers may interpret and refine. This study contributes to this body of research and answers the following research questions: (RQ1) What is the relative coverage of the latent Dirichlet allocation (LDA) topic model and human coding in terms of the breadth of the topics/themes extracted from the text collection' (RQ2) What is the relative depth or level of detail among identified topics using LDA topic models and human coding approaches' A dataset of student reflections was qualitatively analyzed using LDA topic modeling and human coding approaches, and the results were compared. The findings suggest that topic models can provide reliable coverage and depth of themes present in a textual collection comparable to human coding but require manual interpretation of topics. The breadth and depth of human coding output is heavily dependent on the expertise of coders and the size of the collection; these factors are better handled in the topic modeling approach.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2023-05-14
DOI: 10.3390/make5020029
Issue No: Vol. 5, No. 2 (2023)
- MAKE, Vol. 5, Pages 491-511: Biologically Inspired Self-Organizing
Computational Model to Mimic Infant Learning
Authors: Karthik Santhanaraj, Dinakaran Devaraj, Ramya MM, Joshuva Dhanraj, Kuppan Ramanathan
First page: 491
Abstract: Recent technological advancements have fostered human–robot coexistence in work and residential environments. The assistive robot must exhibit humane behavior and consistent care to become an integral part of the human habitat. Furthermore, the robot requires an adaptive unsupervised learning model to explore unfamiliar conditions and collaborate seamlessly. This paper introduces variants of the growing hierarchical self-organizing map (GHSOM)-based computational models for assistive robots, which constructs knowledge from unsupervised exploration-based learning. Traditional self-organizing map (SOM) algorithms have shortcomings, including finite neuron structure, user-defined parameters, and non-hierarchical adaptive architecture. The proposed models overcome these limitations and dynamically grow to form problem-dependent hierarchical feature clusters, thereby allowing associative learning and symbol grounding. Infants can learn from their surroundings through exploration and experience, developing new neuronal connections as they learn. They can also apply their prior knowledge to solve unfamiliar problems. With infant-like emergent behavior, the presented models can operate on different problems without modifications, producing new patterns not present in the input vectors and allowing interactive result visualization. The proposed models are applied to the color, handwritten digits clustering, finger identification, and image classification problems to evaluate their adaptiveness and infant-like knowledge building. The results show that the proposed models are the preferred generalized models for assistive robots.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2023-05-15
DOI: 10.3390/make5020030
Issue No: Vol. 5, No. 2 (2023)
- MAKE, Vol. 5, Pages 512-538: Alzheimer’s Disease Detection from
Fused PET and MRI Modalities Using an Ensemble Classifier
Authors: Amar Shukla, Rajeev Tiwari, Shamik Tiwari
First page: 512
Abstract: Alzheimer’s disease (AD) is an old-age disease that comes in different stages and directly affects the different regions of the brain. The research into the detection of AD and its stages has new advancements in terms of single-modality and multimodality approaches. However, sustainable techniques for the detection of AD and its stages still require a greater extent of research. In this study, a multimodal image-fusion method is initially proposed for the fusion of two different modalities, i.e., PET (Positron Emission Tomography) and MRI (Magnetic Resonance Imaging). Further, the features obtained from fused and non-fused biomarkers are passed to the ensemble classifier with a Random Forest-based feature selection strategy. Three classes of Alzheimer’s disease are used in this work, namely AD, MCI (Mild Cognitive Impairment) and CN (Cognitive Normal). In the resulting analysis, the Binary classifications, i.e., AD vs. CN and MCI vs. CN, attained an accuracy (Acc) of 99% in both cases. The class AD vs. MCI detection achieved an adequate accuracy (Acc) of 91%. Furthermore, the Multi Class classification, i.e., AD vs. MCI vs. CN, achieved 96% (Acc).
Citation: Machine Learning and Knowledge Extraction
PubDate: 2023-05-18
DOI: 10.3390/make5020031
Issue No: Vol. 5, No. 2 (2023)
- MAKE, Vol. 5, Pages 539-559: What about the Latent Space' The Need for
Latent Feature Saliency Detection in Deep Time Series Classification
Authors: Maresa Schröder, Alireza Zamanian, Narges Ahmidi
First page: 539
Abstract: Saliency methods are designed to provide explainability for deep image processing models by assigning feature-wise importance scores and thus detecting informative regions in the input images. Recently, these methods have been widely adapted to the time series domain, aiming to identify important temporal regions in a time series. This paper extends our former work on identifying the systematic failure of such methods in the time series domain to produce relevant results when informative patterns are based on underlying latent information rather than temporal regions. First, we both visually and quantitatively assess the quality of explanations provided by multiple state-of-the-art saliency methods, including Integrated Gradients, Deep-Lift, Kernel SHAP, and Lime using univariate simulated time series data with temporal or latent patterns. In addition, to emphasize the severity of the latent feature saliency detection problem, we also run experiments on a real-world predictive maintenance dataset with known latent patterns. We identify Integrated Gradients, Deep-Lift, and the input-cell attention mechanism as potential candidates for refinement to yield latent saliency scores. Finally, we provide recommendations on using saliency methods for time series classification and suggest a guideline for developing latent saliency methods for time series.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2023-05-18
DOI: 10.3390/make5020032
Issue No: Vol. 5, No. 2 (2023)
- MAKE, Vol. 5, Pages 29-42: Detecting Arabic Cyberbullying Tweets Using
Machine Learning
Authors: Alanoud Mohammed Alduailaj, Aymen Belghith
First page: 29
Abstract: The advancement of technology has paved the way for a new type of bullying, which often leads to negative stigma in the social setting. Cyberbullying is a cybercrime wherein one individual becomes the target of harassment and hatred. It has recently become more prevalent due to a rise in the usage of social media platforms, and, in some severe situations, it has even led to victims’ suicides. In the literature, several cyberbullying detection methods are proposed, but they are mainly focused on word-based data and user account attributes. Furthermore, most of them are related to the English language. Meanwhile, only a few papers have studied cyberbullying detection in Arabic social media platforms. This paper, therefore, aims to use machine learning in the Arabic language for automatic cyberbullying detection. The proposed mechanism identifies cyberbullying using the Support Vector Machine (SVM) classifier algorithm by using a real dataset obtained from YouTube and Twitter to train and test the classifier. Moreover, we include the Farasa tool to overcome text limitations and improve the detection of bullying attacks.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2023-01-05
DOI: 10.3390/make5010003
Issue No: Vol. 5, No. 1 (2023)
- MAKE, Vol. 5, Pages 43-58: IPPT4KRL: Iterative Post-Processing Transfer
for Knowledge Representation Learning
Authors: Weihang Zhang, Ovidiu Șerban, Jiahao Sun, Yike Guo
First page: 43
Abstract: Knowledge Graphs (KGs), a structural way to model human knowledge, have been a critical component of many artificial intelligence applications. Many KG-based tasks are built using knowledge representation learning, which embeds KG entities and relations into a low-dimensional semantic space. However, the quality of representation learning is often limited by the heterogeneity and sparsity of real-world KGs. Multi-KG representation learning, which utilizes KGs from different sources collaboratively, presents one promising solution. In this paper, we propose a simple, but effective iterative method that post-processes pre-trained knowledge graph embedding (IPPT4KRL) on individual KGs to maximize the knowledge transfer from another KG when a small portion of alignment information is introduced. Specifically, additional triples are iteratively included in the post-processing based on their adjacencies to the cross-KG alignments to refine the pre-trained embedding space of individual KGs. We also provide the benchmarking results of existing multi-KG representation learning methods on several generated and well-known datasets. The empirical results of the link prediction task on these datasets show that the proposed IPPT4KRL method achieved comparable and even superior results when compared against more complex methods in multi-KG representation learning.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2023-01-06
DOI: 10.3390/make5010004
Issue No: Vol. 5, No. 1 (2023)
- MAKE, Vol. 5, Pages 59-77: Learning Sentence-Level Representations with
Predictive Coding
Authors: Vladimir Araujo, Marie-Francine Moens, Alvaro Soto
First page: 59
Abstract: Learning sentence representations is an essential and challenging topic in the deep learning and natural language processing communities. Recent methods pre-train big models on a massive text corpus, focusing mainly on learning the representation of contextualized words. As a result, these models cannot generate informative sentence embeddings since they do not explicitly exploit the structure and discourse relationships existing in contiguous sentences. Drawing inspiration from human language processing, this work explores how to improve sentence-level representations of pre-trained models by borrowing ideas from predictive coding theory. Specifically, we extend BERT-style models with bottom-up and top-down computation to predict future sentences in latent space at each intermediate layer in the networks. We conduct extensive experimentation with various benchmarks for the English and Spanish languages, designed to assess sentence- and discourse-level representations and pragmatics-focused assessments. Our results show that our approach improves sentence representations consistently for both languages. Furthermore, the experiments also indicate that our models capture discourse and pragmatics knowledge. In addition, to validate the proposed method, we carried out an ablation study and a qualitative study with which we verified that the predictive mechanism helps to improve the quality of the representations.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2023-01-09
DOI: 10.3390/make5010005
Issue No: Vol. 5, No. 1 (2023)
- MAKE, Vol. 5, Pages 78-108: XAIR: A Systematic Metareview of Explainable
AI (XAI) Aligned to the Software Development Process
Authors: Tobias Clement, Nils Kemmerzell, Mohamed Abdelaal, Michael Amberg
First page: 78
Abstract: Currently, explainability represents a major barrier that Artificial Intelligence (AI) is facing in regard to its practical implementation in various application domains. To combat the lack of understanding of AI-based systems, Explainable AI (XAI) aims to make black-box AI models more transparent and comprehensible for humans. Fortunately, plenty of XAI methods have been introduced to tackle the explainability problem from different perspectives. However, due to the vast search space, it is challenging for ML practitioners and data scientists to start with the development of XAI software and to optimally select the most suitable XAI methods. To tackle this challenge, we introduce XAIR, a novel systematic metareview of the most promising XAI methods and tools. XAIR differentiates itself from existing reviews by aligning its results to the five steps of the software development process, including requirement analysis, design, implementation, evaluation, and deployment. Through this mapping, we aim to create a better understanding of the individual steps of developing XAI software and to foster the creation of real-world AI applications that incorporate explainability. Finally, we conclude with highlighting new directions for future research.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2023-01-11
DOI: 10.3390/make5010006
Issue No: Vol. 5, No. 1 (2023)
- MAKE, Vol. 5, Pages 109-127: E2H Distance-Weighted Minimum Reference Set
for Numerical and Categorical Mixture Data and a Bayesian Swap Feature
Selection Algorithm
Authors: Yuto Omae, Masaya Mori
First page: 109
Abstract: Generally, when developing classification models using supervised learning methods (e.g., support vector machine, neural network, and decision tree), feature selection, as a pre-processing step, is essential to reduce calculation costs and improve the generalization scores. In this regard, the minimum reference set (MRS), which is a feature selection algorithm, can be used. The original MRS considers a feature subset as effective if it leads to the correct classification of all samples by using the 1-nearest neighbor algorithm based on small samples. However, the original MRS is only applicable to numerical features, and the distances between different classes cannot be considered. Therefore, herein, we propose a novel feature subset evaluation algorithm, referred to as the “E2H distance-weighted MRS,” which can be used for a mixture of numerical and categorical features and considers the distances between different classes in the evaluation. Moreover, a Bayesian swap feature selection algorithm, which is used to identify an effective feature subset, is also proposed. The effectiveness of the proposed methods is verified based on experiments conducted using artificially generated data comprising a mixture of numerical and categorical features.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2023-01-11
DOI: 10.3390/make5010007
Issue No: Vol. 5, No. 1 (2023)
- MAKE, Vol. 5, Pages 128-143: Detection of Temporal Shifts in Semantics
Using Local Graph Clustering
Authors: Neil Hwang, Shirshendu Chatterjee, Yanming Di, Sharmodeep Bhattacharyya
First page: 128
Abstract: Many changes in our digital corpus have been brought about by the interplay between rapid advances in digital communication and the current environment characterized by pandemics, political polarization, and social unrest. One such change is the pace with which new words enter the mass vocabulary and the frequency at which meanings, perceptions, and interpretations of existing expressions change. The current state-of-the-art algorithms do not allow for an intuitive and rigorous detection of these changes in word meanings over time. We propose a dynamic graph-theoretic approach to inferring the semantics of words and phrases (“terms”) and detecting temporal shifts. Our approach represents each term as a stochastic time-evolving set of contextual words and is a count-based distributional semantic model in nature. We use local clustering techniques to assess the structural changes in a given word’s contextual words. We demonstrate the efficacy of our method by investigating the changes in the semantics of the phrase “Chinavirus”. We conclude that the term took on a much more pejorative meaning when the White House used the term in the second half of March 2020, although the effect appears to have been temporary. We make both the dataset and the code used to generate this paper’s results available.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2023-01-13
DOI: 10.3390/make5010008
Issue No: Vol. 5, No. 1 (2023)
- MAKE, Vol. 5, Pages 144-168: On Deceiving Malware Classification with
Section Injection
Authors: Adeilson Antonio da Silva, Mauricio Pamplona Segundo
First page: 144
Abstract: We investigate how to modify executable files to deceive malware classification systems. This work’s main contribution is a methodology to inject bytes across a malware file randomly and use it both as an attack to decrease classification accuracy but also as a defensive method, augmenting the data available for training. It respects the operating system file format to make sure the malware will still execute after our injection and will not change its behavior. We reproduced five state-of-the-art malware classification approaches to evaluate our injection scheme: one based on Global Image Descriptor (GIST) + K-Nearest-Neighbors (KNN), three Convolutional Neural Network (CNN) variations and one Gated CNN. We performed our experiments on a public dataset with 9339 malware samples from 25 different families. Our results show that a mere increase of 7% in the malware size causes an accuracy drop between 25% and 40% for malware family classification. They show that an automatic malware classification system may not be as trustworthy as initially reported in the literature. We also evaluate using modified malware alongside the original ones to increase networks robustness against the mentioned attacks. The results show that a combination of reordering malware sections and injecting random data can improve the overall performance of the classification. All the code is publicly available.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2023-01-16
DOI: 10.3390/make5010009
Issue No: Vol. 5, No. 1 (2023)
- MAKE, Vol. 5, Pages 169-170: Explainable Machine Learning
Authors: Jochen Garcke, Ribana Roscher
First page: 169
Abstract: Machine learning methods are widely used in commercial applications and in many scientific areas [...]
Citation: Machine Learning and Knowledge Extraction
PubDate: 2023-01-17
DOI: 10.3390/make5010010
Issue No: Vol. 5, No. 1 (2023)
- MAKE, Vol. 5, Pages 171-172: Acknowledgment to the Reviewers of Machine
Learning and Knowledge Extraction in 2022
Authors: Machine Learning; Knowledge Extraction Editorial Office
First page: 171
Abstract: High-quality academic publishing is built on rigorous peer review [...]
Citation: Machine Learning and Knowledge Extraction
PubDate: 2023-01-18
DOI: 10.3390/make5010011
Issue No: Vol. 5, No. 1 (2023)
- MAKE, Vol. 5, Pages 173-174: Special Issue “Selected Papers from
CD-MAKE 2020 and ARES 2020”
Authors: Edgar R. Weippl, Andreas Holzinger, Peter Kieseberg
First page: 173
Abstract: In the current era of rapid technological advancement, machine learning (ML) is quickly becoming a dominant force in the development of smart environments [...]
Citation: Machine Learning and Knowledge Extraction
PubDate: 2023-01-20
DOI: 10.3390/make5010012
Issue No: Vol. 5, No. 1 (2023)
- MAKE, Vol. 5, Pages 175-198: Machine Learning and Prediction of Infectious
Diseases: A Systematic Review
Authors: Omar Enzo Santangelo, Vito Gentile, Stefano Pizzo, Domiziana Giordano, Fabrizio Cedrone
First page: 175
Abstract: The aim of the study is to show whether it is possible to predict infectious disease outbreaks early, by using machine learning. This study was carried out following the guidelines of the Cochrane Collaboration and the meta-analysis of observational studies in epidemiology and the preferred reporting items for systematic reviews and meta-analyses. The suitable bibliography on PubMed/Medline and Scopus was searched by combining text, words, and titles on medical topics. At the end of the search, this systematic review contained 75 records. The studies analyzed in this systematic review demonstrate that it is possible to predict the incidence and trends of some infectious diseases; by combining several techniques and types of machine learning, it is possible to obtain accurate and plausible results.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2023-02-01
DOI: 10.3390/make5010013
Issue No: Vol. 5, No. 1 (2023)
- MAKE, Vol. 5, Pages 199-236: InvMap and Witness Simplicial Variational
Auto-Encoders
Authors: Aniss Aiman Medbouhi, Vladislav Polianskii, Anastasia Varava, Danica Kragic
First page: 199
Abstract: Variational auto-encoders (VAEs) are deep generative models used for unsupervised learning, however their standard version is not topology-aware in practice since the data topology may not be taken into consideration. In this paper, we propose two different approaches with the aim to preserve the topological structure between the input space and the latent representation of a VAE. Firstly, we introduce InvMap-VAE as a way to turn any dimensionality reduction technique, given an embedding it produces, into a generative model within a VAE framework providing an inverse mapping into original space. Secondly, we propose the Witness Simplicial VAE as an extension of the simplicial auto-encoder to the variational setup using a witness complex for computing the simplicial regularization, and we motivate this method theoretically using tools from algebraic topology. The Witness Simplicial VAE is independent of any dimensionality reduction technique and together with its extension, Isolandmarks Witness Simplicial VAE, preserves the persistent Betti numbers of a dataset better than a standard VAE.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2023-02-05
DOI: 10.3390/make5010014
Issue No: Vol. 5, No. 1 (2023)
- MAKE, Vol. 5, Pages 237-251: Can Principal Component Analysis Be Used to
Explore the Relationship of Rowing Kinematics and Force Production in
Elite Rowers during a Step Test' A Pilot Study
Authors: Matt Jensen, Trent Stellingwerff, Courtney Pollock, James Wakeling, Marc Klimstra
First page: 237
Abstract: Investigating the relationship between the movement patterns of multiple limb segments during the rowing stroke on the resulting force production in elite rowers can provide foundational insight into optimal technique. It can also highlight potential mechanisms of injury and performance improvement. The purpose of this study was to conduct a kinematic analysis of the rowing stroke together with force production during a step test in elite national-team heavyweight men to evaluate the fundamental patterns that contribute to expert performance. Twelve elite heavyweight male rowers performed a step test on a row-perfect sliding ergometer [5 × 1 min with 1 min rest at set stroke rates (20, 24, 28, 32, 36)]. Joint angle displacement and velocity of the hip, knee and elbow were measured with electrogoniometers, and force was measured with a tension/compression force transducer in line with the handle. To explore interactions between kinematic patterns and stroke performance variables, joint angular velocities of the hip, knee and elbow were entered into principal component analysis (PCA) and separate ANCOVAs were run for each performance variable (peak force, impulse, split time) with dependent variables, and the kinematic loading scores (Kpc,ls) as covariates with athlete/stroke rate as fixed factors. The results suggested that rowers’ kinematic patterns respond differently across varying stroke rates. The first seven PCs accounted for 79.5% (PC1 [26.4%], PC2 [14.6%], PC3 [11.3%], PC4 [8.4%], PC5 [7.5%], PC6 [6.5%], PC7 [4.8%]) of the variances in the signal. The PCs contributing significantly (p ≤ 0.05) to performance metrics based on PC loading scores from an ANCOVA were (PC1, PC2, PC6) for split time, (PC3, PC4, PC5, PC6) for impulse, and (PC1, PC6, PC7) for peak force. The significant PCs for each performance measure were used to reconstruct the kinematic patterns for split time, impulse and peak force separately. Overall, PCA was able to differentiate between rowers and stroke rates, and revealed features of the rowing-stroke technique correlated with measures of performance that may highlight meaningful technique-optimization strategies. PCA could be used to provide insight into differences in kinematic strategies that could result in suboptimal performance, potential asymmetries or to determine how well a desired technique change has been accomplished by group and/or individual athletes.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2023-02-17
DOI: 10.3390/make5010015
Issue No: Vol. 5, No. 1 (2023)
- MAKE, Vol. 5, Pages 252-268: A Novel Pipeline Age Evaluation: Considering
Overall Condition Index and Neural Network Based on Measured Data
Authors: Hassan Noroznia, Majid Gandomkar, Javad Nikoukar, Ali Aranizadeh, Mirpouya Mirmozaffari
First page: 252
Abstract: Today, the chemical corrosion of metals is one of the main problems of large productions, especially in the oil and gas industries. Due to massive downtime connected to corrosion failures, pipeline corrosion is a central issue in many oil and gas industries. Therefore, the determination of the corrosion progress of oil and gas pipelines is crucial for monitoring the reliability and alleviation of failures that can positively impact health, safety, and the environment. Gas transmission and distribution pipes and other structures buried (or immersed) in an electrolyte, by the existing conditions and due to the metallurgical structure, are corroded. After some time, this disrupts an active system and process by causing damage. The worst corrosion for metals implanted in the soil is in areas where electrical currents are lost. Therefore, cathodic protection (CP) is the most effective method to prevent the corrosion of structures buried in the soil. Our aim in this paper is first to investigate the effect of stray currents on failure rate using the condition index, and then to estimate the remaining useful life of CP gas pipelines using an artificial neural network (ANN). Predicting future values using previous data based on the time series feature is also possible. Therefore, this paper first uses the general equipment condition monitoring method to detect failures. The time series model of data is then measured and operated by neural networks. Finally, the amount of failure over time is determined.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2023-02-20
DOI: 10.3390/make5010016
Issue No: Vol. 5, No. 1 (2023)
- MAKE, Vol. 5, Pages 269-286: Painting the Black Box White: Experimental
Findings from Applying XAI to an ECG Reading Setting
Authors: Federico Cabitza, Andrea Campagner, Chiara Natali, Enea Parimbelli, Luca Ronzio, Matteo Cameli
First page: 269
Abstract: The emergence of black-box, subsymbolic, and statistical AI systems has motivated a rapid increase in the interest regarding explainable AI (XAI), which encompasses both inherently explainable techniques, as well as approaches to make black-box AI systems explainable to human decision makers. Rather than always making black boxes transparent, these approaches are at risk of painting the black boxes white, thus failing to provide a level of transparency that would increase the system’s usability and comprehensibility, or even at risk of generating new errors (i.e., white-box paradox). To address these usability-related issues, in this work we focus on the cognitive dimension of users’ perception of explanations and XAI systems. We investigated these perceptions in light of their relationship with users’ characteristics (e.g., expertise) through a questionnaire-based user study involved 44 cardiology residents and specialists in an AI-supported ECG reading task. Our results point to the relevance and correlation of the dimensions of trust, perceived quality of explanations, and tendency to defer the decision process to automation (i.e., technology dominance). This contribution calls for the evaluation of AI-based support systems from a human–AI interaction-oriented perspective, laying the ground for further investigation of XAI and its effects on decision making and user experience.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2023-03-08
DOI: 10.3390/make5010017
Issue No: Vol. 5, No. 1 (2023)
- MAKE, Vol. 5, Pages 287-303: Skew Class-Balanced Re-Weighting for Unbiased
Scene Graph Generation
Authors: Haeyong Kang, Chang D. Yoo
First page: 287
Abstract: An unbiased scene graph generation (SGG) algorithm referred to as Skew Class-Balanced Re-Weighting (SCR) is proposed for considering the unbiased predicate prediction caused by the long-tailed distribution. The prior works focus mainly on alleviating the deteriorating performances of the minority predicate predictions, showing drastic dropping recall scores, i.e., losing the majority predicate performances. It has not yet correctly analyzed the trade-off between majority and minority predicate performances in the limited SGG datasets. In this paper, to alleviate the issue, the Skew Class-Balanced Re-Weighting (SCR) loss function is considered for the unbiased SGG models. Leveraged by the skewness of biased predicate predictions, the SCR estimates the target predicate weight coefficient and then re-weights more to the biased predicates for better trading-off between the majority predicates and the minority ones. Extensive experiments conducted on the standard Visual Genome dataset and Open Image V4 and V6 show the performances and generality of the SCR with the traditional SGG models.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2023-03-10
DOI: 10.3390/make5010018
Issue No: Vol. 5, No. 1 (2023)
- MAKE, Vol. 5, Pages 304-329: A Survey on GAN Techniques for Data
Augmentation to Address the Imbalanced Data Issues in Credit Card Fraud
Detection
Authors: Emilija Strelcenia, Simant Prakoonwit
First page: 304
Abstract: Data augmentation is an important procedure in deep learning. GAN-based data augmentation can be utilized in many domains. For instance, in the credit card fraud domain, the imbalanced dataset problem is a major one as the number of credit card fraud cases is in the minority compared to legal payments. On the other hand, generative techniques are considered effective ways to rebalance the imbalanced class issue, as these techniques balance both minority and majority classes before the training. In a more recent period, Generative Adversarial Networks (GANs) are considered one of the most popular data generative techniques as they are used in big data settings. This research aims to present a survey on data augmentation using various GAN variants in the credit card fraud detection domain. In this survey, we offer a comprehensive summary of several peer-reviewed research papers on GAN synthetic generation techniques for fraud detection in the financial sector. In addition, this survey includes various solutions proposed by different researchers to balance imbalanced classes. In the end, this work concludes by pointing out the limitations of the most recent research articles and future research issues, and proposes solutions to address these problems.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2023-03-11
DOI: 10.3390/make5010019
Issue No: Vol. 5, No. 1 (2023)
- MAKE, Vol. 5, Pages 330-345: Human Action Recognition-Based IoT Services
for Emergency Response Management
Authors: Talal H. Noor
First page: 330
Abstract: Emergency incidents can appear anytime and any place, which makes it very challenging for emergency medical services practitioners to predict the location and the time of such emergencies. The dynamic nature of the appearance of emergency incidents can cause delays in emergency medical services, which can sometimes lead to vital injury complications or even death, in some cases. The delay of emergency medical services may occur as a result of a call that was made too late or because no one was present to make the call. With the emergence of smart cities and promising technologies, such as the Internet of Things (IoT) and computer vision techniques, such issues can be tackled. This article proposes a human action recognition-based IoT services architecture for emergency response management. In particular, the architecture exploits IoT devices (e.g., surveillance cameras) that are distributed in public areas to detect emergency incidents, make a request for the nearest emergency medical services, and send emergency location information. Moreover, this article proposes an emergency incidents detection model, based on human action recognition and object tracking, using image processing and classifying the collected images, based on action modeling. The primary notion of the proposed model is to classify human activity, whether it is an emergency incident or other daily activities, using a Convolutional Neural Network (CNN) and Support Vector Machine (SVM). To demonstrate the feasibility of the proposed emergency detection model, several experiments were conducted using the UR fall detection dataset, which consists of emergency and other daily activities footage. The results of the conducted experiments were promising, with the proposed model scoring 0.99, 0.97, 0.97, and 0.98 in terms of sensitivity, specificity, precision, and accuracy, respectively.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2023-03-13
DOI: 10.3390/make5010020
Issue No: Vol. 5, No. 1 (2023)
- MAKE, Vol. 5, Pages 1-13: Multimodal AutoML via Representation Evolution
Authors: Blaž Škrlj, Matej Bevec, Nada Lavrač
First page: 1
Abstract: With the increasing amounts of available data, learning simultaneously from different types of inputs is becoming necessary to obtain robust and well-performing models. With the advent of representation learning in recent years, lower-dimensional vector-based representations have become available for both images and texts, while automating simultaneous learning from multiple modalities remains a challenging problem. This paper presents an AutoML (automated machine learning) approach to automated machine learning model configuration identification for data composed of two modalities: texts and images. The approach is based on the idea of representation evolution, the process of automatically amplifying heterogeneous representations across several modalities, optimized jointly with a collection of fast, well-regularized linear models. The proposed approach is benchmarked against 11 unimodal and multimodal (texts and images) approaches on four real-life benchmark datasets from different domains. It achieves competitive performance with minimal human effort and low computing requirements, enabling learning from multiple modalities in automated manner for a wider community of researchers.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2022-12-23
DOI: 10.3390/make5010001
Issue No: Vol. 5, No. 1 (2022)
- MAKE, Vol. 5, Pages 14-28: Synthetic Data Generation for Visual Detection
of Flattened PET Bottles
Authors: Vitālijs Feščenko, Jānis Ārents, Roberts Kadiķis
First page: 14
Abstract: Polyethylene terephthalate (PET) bottle recycling is a highly automated task; however, manual quality control is required due to inefficiencies of the process. In this paper, we explore automation of the quality control sub-task, namely visual bottle detection, using convolutional neural network (CNN)-based methods and synthetic generation of labelled training data. We propose a synthetic generation pipeline tailored for transparent and crushed PET bottle detection; however, it can also be applied to undeformed bottles if the viewpoint is set from above. We conduct various experiments on CNNs to compare the quality of real and synthetic data, show that synthetic data can reduce the amount of real data required and experiment with the combination of both datasets in multiple ways to obtain the best performance.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2022-12-29
DOI: 10.3390/make5010002
Issue No: Vol. 5, No. 1 (2022)
- MAKE, Vol. 4, Pages 814-826: Artificial Intelligence Methods for
Identifying and Localizing Abnormal Parathyroid Glands: A Review Study
Authors: Ioannis D. Apostolopoulos, Nikolaos I. Papandrianos, Elpiniki I. Papageorgiou, Dimitris J. Apostolopoulos
First page: 814
Abstract: Background: Recent advances in Artificial Intelligence (AI) algorithms, and specifically Deep Learning (DL) methods, demonstrate substantial performance in detecting and classifying medical images. Recent clinical studies have reported novel optical technologies which enhance the localization or assess the viability of Parathyroid Glands (PG) during surgery, or preoperatively. These technologies could become complementary to the surgeon’s eyes and may improve surgical outcomes in thyroidectomy and parathyroidectomy. Methods: The study explores and reports the use of AI methods for identifying and localizing PGs, Primary Hyperparathyroidism (PHPT), Parathyroid Adenoma (PTA), and Multiglandular Disease (MGD). Results: The review identified 13 publications that employ Machine Learning and DL methods for preoperative and operative implementations. Conclusions: AI can aid in PG, PHPT, PTA, and MGD detection, as well as PG abnormality discrimination, both during surgery and non-invasively.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2022-09-21
DOI: 10.3390/make4040040
Issue No: Vol. 4, No. 4 (2022)
- MAKE, Vol. 4, Pages 827-838: Comparison of Imputation Methods for Missing
Rate of Perceived Exertion Data in Rugby
Authors: Amarah Epp-Stobbe, Ming-Chang Tsai, Marc Klimstra
First page: 827
Abstract: Rate of perceived exertion (RPE) is used to calculate athlete load. Incomplete load data, due to missing athlete-reported RPE, can increase injury risk. The current standard for missing RPE imputation is daily team mean substitution. However, RPE reflects an individual’s effort; group mean substitution may be suboptimal. This investigation assessed an ideal method for imputing RPE. A total of 987 datasets were collected from women’s rugby sevens competitions. Daily team mean substitution, k-nearest neighbours, random forest, support vector machine, neural network, linear, stepwise, lasso, ridge, and elastic net regression models were assessed at different missingness levels. Statistical equivalence of true and imputed scores by model were evaluated. An ANOVA of accuracy by model and missingness was completed. While all models were equivalent to the true RPE, differences by model existed. Daily team mean substitution was the poorest performing model, and random forest, the best. Accuracy was low in all models, affirming RPE as multifaceted and requiring quantification of potentially overlapping factors. While group mean substitution is discouraged, practitioners are recommended to scrutinize any imputation method relating to athlete load.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2022-09-23
DOI: 10.3390/make4040041
Issue No: Vol. 4, No. 4 (2022)
- MAKE, Vol. 4, Pages 839-851: On the Application of Artificial Neural
Network for Classification of Incipient Faults in Dissolved Gas Analysis
of Power Transformers
Authors: Thango
First page: 839
Abstract: Oil-submerged transformer is one of the inherent instruments in the South African power system. Transformer malfunction or impairment may interpose the operation of the electric power distribution and transmission system, coupled with liability for high overhaul costs. Hence, recognition of inchoate faults in an oil-submerged transformer is indispensable and it has turned into an intriguing subject of interest by utility owners and transformer manufacturers. This work proposes a hybrid implementation of a multi-layer artificial neural network (MLANN) and IEC 60599:2022 gas ratio method in identifying inchoate faults in mineral oil-based submerged transformers by employing the dissolved gas analysis (DGA) method. DGA is a staunch practice to discover inchoate faults as it furnishes comprehensive information in examining the transformer state. In current work, MLANN was established to pigeonhole seven fault types of transformer states predicated on the three International Electrotechnical Commission (IEC) combustible gas ratios. The designs enmesh the development of numerous MLANN algorithms and picking networks with the optimum performance. The gas ratios are in accordance with the IEC 60599:2022 standard whilst an empirical databank comprised of 100 datasets was used in the training and testing activities. The designated MLANN design produces an overall correlation coefficient of 0.998 in the categorization of transformer state with reference to the combustible gas produced.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2022-09-26
DOI: 10.3390/make4040042
Issue No: Vol. 4, No. 4 (2022)
- MAKE, Vol. 4, Pages 852-864: Automatic Extraction of Medication
Information from Cylindrically Distorted Pill Bottle Labels
Authors: Kseniia Gromova, Vinayak Elangovan
First page: 852
Abstract: Patient compliance with prescribed medication regimens is critical for maintaining health and managing disease and illness. To encourage patient compliance, multiple aids, like automatic pill dispensers, pill organizers, and various reminder applications, have been developed to help people adhere to their medication regimens. However, when utilizing these aids, the user or patient must manually enter their medication information and schedule. This process is time-consuming and often prone to error. For example, elderly patients may have difficulty reading medication information on the bottle due to decreased eyesight, leading them to enter medication information incorrectly. This study explored methods for extracting pertinent information from cylindrically distorted prescription drug labels using Machine Learning and Computer Vision techniques. This study found that Deep Convolutional Neural Networks (DCNN) performed better than other techniques in identifying label key points under different lighting conditions and various backgrounds. This method achieved a percentage of Correct Key points PCK @ 0.03 of 97%. These key points were then used to correct the cylindrical distortion. Next, the multiple dewarped label images were stitched together and processed by an Optical Character Recognition (OCR) engine. Pertinent information, such as patient name, drug name, drug strength, and directions of use, were extracted from the recognized text using Natural Language Processing (NLP) techniques. The system created in this study can be used to improve patient health and compliance by creating an accurate medication schedule.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2022-09-27
DOI: 10.3390/make4040043
Issue No: Vol. 4, No. 4 (2022)
- MAKE, Vol. 4, Pages 865-887: Entropic Statistics: Concept, Estimation, and
Application in Machine Learning and Knowledge Extraction
Authors: Jialin Zhang
First page: 865
Abstract: The demands for machine learning and knowledge extraction methods have been booming due to the unprecedented surge in data volume and data quality. Nevertheless, challenges arise amid the emerging data complexity as significant chunks of information and knowledge lie within the non-ordinal realm of data. To address the challenges, researchers developed considerable machine learning and knowledge extraction methods regarding various domain-specific challenges. To characterize and extract information from non-ordinal data, all the developed methods pointed to the subject of Information Theory, established following Shannon’s landmark paper in 1948. This article reviews recent developments in entropic statistics, including estimation of Shannon’s entropy and its functionals (such as mutual information and Kullback–Leibler divergence), concepts of entropic basis, generalized Shannon’s entropy (and its functionals), and their estimations and potential applications in machine learning and knowledge extraction. With the knowledge of recent development in entropic statistics, researchers can customize existing machine learning and knowledge extraction methods for better performance or develop new approaches to address emerging domain-specific challenges.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2022-09-30
DOI: 10.3390/make4040044
Issue No: Vol. 4, No. 4 (2022)
- MAKE, Vol. 4, Pages 888-911: How Do Deep-Learning Framework Versions
Affect the Reproducibility of Neural Network Models'
Authors: Mostafa Shahriari, Rudolf Ramler, Lukas Fischer
First page: 888
Abstract: In the last decade, industry’s demand for deep learning (DL) has increased due to its high performance in complex scenarios. Due to the DL method’s complexity, experts and non-experts rely on blackbox software packages such as Tensorflow and Pytorch. The frameworks are constantly improving, and new versions are released frequently. As a natural process in software development, the released versions contain improvements/changes in the methods and their implementation. Moreover, versions may be bug-polluted, leading to the model performance decreasing or stopping the model from working. The aforementioned changes in implementation can lead to variance in obtained results. This work investigates the effect of implementation changes in different major releases of these frameworks on the model performance. We perform our study using a variety of standard datasets. Our study shows that users should consider that changing the framework version can affect the model performance. Moreover, they should consider the possibility of a bug-polluted version before starting to debug source code that had an excellent performance before a version change. This also shows the importance of using virtual environments, such as Docker, when delivering a software product to clients.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2022-10-05
DOI: 10.3390/make4040045
Issue No: Vol. 4, No. 4 (2022)
- MAKE, Vol. 4, Pages 912-923: Prospective Neural Network Model for Seismic
Precursory Signal Detection in Geomagnetic Field Records
Authors: Laura Petrescu, Iren-Adelina Moldovan
First page: 912
Abstract: We designed a convolutional neural network application to detect seismic precursors in geomagnetic field records. Earthquakes are among the most destructive natural hazards on Earth, yet their short-term forecasting has not been achieved. Stress loading in dry rocks can generate electric currents that cause short-term changes to the geomagnetic field, yielding theoretically detectable pre-earthquake electromagnetic emissions. We propose a CNN model that scans windows of geomagnetic data streams and self-updates using nearby earthquakes as labels, under strict detectability criteria. We show how this model can be applied in three key seismotectonic settings, where geomagnetic observatories are optimally located in high-seismicity-rate epicentral areas. CNNs require large datasets to be able to accurately label seismic precursors, so we expect the model to improve as more data become available with time. At present, there is no synthetic data generator for this kind of application, so artificial data augmentation is not yet possible. However, this deep learning model serves to illustrate its potential usage in earthquake forecasting in a systematic and unbiased way. Our method can be prospectively applied to any kind of three-component dataset that may be physically connected to seismogenic processes at a given depth.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2022-10-07
DOI: 10.3390/make4040046
Issue No: Vol. 4, No. 4 (2022)
- MAKE, Vol. 4, Pages 924-953: Actionable Explainable AI (AxAI): A Practical
Example with Aggregation Functions for Adaptive Classification and Textual
Explanations for Interpretable Machine Learning
Authors: Anna Saranti, Miroslav Hudec, Erika Mináriková, Zdenko Takác̆, Udo Großschedl, Christoph Koch, Bastian Pfeifer, Alessa Angerschmid, Andreas Holzinger
First page: 924
Abstract: In many domains of our daily life (e.g., agriculture, forestry, health, etc.), both laymen and experts need to classify entities into two binary classes (yes/no, good/bad, sufficient/insufficient, benign/malign, etc.). For many entities, this decision is difficult and we need another class called “maybe”, which contains a corresponding quantifiable tendency toward one of these two opposites. Human domain experts are often able to mark any entity, place it in a different class and adjust the position of the slope in the class. Moreover, they can often explain the classification space linguistically—depending on their individual domain experience and previous knowledge. We consider this human-in-the-loop extremely important and call our approach actionable explainable AI. Consequently, the parameters of the functions are adapted to these requirements and the solution is explained to the domain experts accordingly. Specifically, this paper contains three novelties going beyond the state-of-the-art: (1) A novel method for detecting the appropriate parameter range for the averaging function to treat the slope in the “maybe” class, along with a proposal for a better generalisation than the existing solution. (2) the insight that for a given problem, the family of t-norms and t-conorms covering the whole range of nilpotency is suitable because we need a clear “no” or “yes” not only for the borderline cases. Consequently, we adopted the Schweizer–Sklar family of t-norms or t-conorms in ordinal sums. (3) A new fuzzy quasi-dissimilarity function for classification into three classes: Main difference, irrelevant difference and partial difference. We conducted all of our experiments with real-world datasets.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2022-10-27
DOI: 10.3390/make4040047
Issue No: Vol. 4, No. 4 (2022)
- MAKE, Vol. 4, Pages 954-967: Lottery Ticket Structured Node Pruning for
Tabular Datasets
Authors: Ryan Bluteau, Robin Gras, Zachary Innes, Mitchel Paulin
First page: 954
Abstract: This paper experiments with well known pruning approaches, iterative and one-shot, and presents a new approach to lottery ticket pruning applied to tabular neural networks based on iterative pruning. Our contribution is a standard model for comparison in terms of speed and performance for tabular datasets that often do not get optimized through research. We show leading results in several tabular datasets that can compete with ensemble approaches. We tested on a wide range of datasets with a general improvement over the original (already leading) model in 6 of 8 datasets tested in terms of F1/RMSE. This includes a total reduction of over 85% of nodes with the additional ability to prune over 98% of nodes with minimal affect to accuracy. The new iterative approach we present will first optimize for lottery ticket quality by selecting an optimal architecture size and weights, then apply the iterative pruning strategy. The new iterative approach shows minimal degradation in accuracy compared to the original iterative approach, but it is capable of pruning models much smaller due to optimal weight pre-selection. Training and inference time improved over 50% and 10%, respectively, and up to 90% and 35%, respectively, for large datasets.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2022-10-28
DOI: 10.3390/make4040048
Issue No: Vol. 4, No. 4 (2022)
- MAKE, Vol. 4, Pages 968-993: FeaSel-Net: A Recursive Feature Selection
Callback in Neural Networks
Authors: Felix Fischer, Alexander Birk, Peter Somers, Karsten Frenner, Cristina Tarín, Alois Herkommer
First page: 968
Abstract: Selecting only the relevant subsets from all gathered data has never been as challenging as it is in these times of big data and sensor fusion. Multiple complementary methods have emerged for the observation of similar phenomena; oftentimes, many of these techniques are superimposed in order to make the best possible decisions. A pathologist, for example, uses microscopic and spectroscopic techniques to discriminate between healthy and cancerous tissue. Especially in the field of spectroscopy in medicine, an immense number of frequencies are recorded and appropriately sized datasets are rarely acquired due to the time-intensive measurements and the lack of patients. In order to cope with the curse of dimensionality in machine learning, it is necessary to reduce the overhead from irrelevant or redundant features. In this article, we propose a feature selection callback algorithm (FeaSel-Net) that can be embedded in deep neural networks. It recursively prunes the input nodes after the optimizer in the neural network achieves satisfying results. We demonstrate the performance of the feature selection algorithm on different publicly available datasets and compare it to existing feature selection methods. Our algorithm combines the advantages of neural networks’ nonlinear learning ability and the embedding of the feature selection algorithm into the actual classifier optimization.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2022-10-31
DOI: 10.3390/make4040049
Issue No: Vol. 4, No. 4 (2022)
- MAKE, Vol. 4, Pages 994-1010: Semantic Interactive Learning for Text
Classification: A Constructive Approach for Contextual Interactions
Authors: Sebastian Kiefer, Mareike Hoffmann, Ute Schmid
First page: 994
Abstract: Interactive Machine Learning (IML) can enable intelligent systems to interactively learn from their end-users, and is quickly becoming more and more relevant to many application domains. Although it places the human in the loop, interactions are mostly performed via mutual explanations that miss contextual information. Furthermore, current model-agnostic IML strategies such as CAIPI are limited to ’destructive’ feedback, meaning that they solely allow an expert to prevent a learner from using irrelevant features. In this work, we propose a novel interaction framework called Semantic Interactive Learning for the domain of document classification, located at the intersection between Natural Language Processing (NLP) and Machine Learning (ML). We frame the problem of incorporating constructive and contextual feedback into the learner as a task involving finding an architecture that enables more semantic alignment between humans and machines while at the same time helping to maintain the statistical characteristics of the input domain when generating user-defined counterexamples based on meaningful corrections. Therefore, we introduce a technique called SemanticPush that is effective for translating conceptual corrections of humans to non-extrapolating training examples such that the learner’s reasoning is pushed towards the desired behavior. Through several experiments we show how our method compares to CAIPI, a state of the art IML strategy, in terms of Predictive Performance and Local Explanation Quality in downstream multi-class classification tasks. Especially in the early stages of interactions, our proposed method clearly outperforms CAIPI while allowing for contextual interpretation and intervention. Overall, SemanticPush stands out with regard to data efficiency, as it requires fewer queries from the pool dataset to achieve high accuracy.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2022-11-13
DOI: 10.3390/make4040050
Issue No: Vol. 4, No. 4 (2022)
- MAKE, Vol. 4, Pages 1011-1023: Evidence-Based Regularization for Neural
Networks
Authors: Giuseppe Nuti, Andreea-Ingrid Cross, Philipp Rindler
First page: 1011
Abstract: Numerous approaches address over-fitting in neural networks: by imposing a penalty on the parameters of the network (L1, L2, etc.); by changing the network stochastically (drop-out, Gaussian noise, etc.); or by transforming the input data (batch normalization, etc.). In contrast, we aim to ensure that a minimum amount of supporting evidence is present when fitting the model parameters to the training data. This, at the single neuron level, is equivalent to ensuring that both sides of the separating hyperplane (for a standard artificial neuron) have a minimum number of data points, noting that these points need not belong to the same class for the inner layers. We firstly benchmark the results of this approach on the standard Fashion-MINST dataset, comparing it to various regularization techniques. Interestingly, we note that by nudging each neuron to divide, at least in part, its input data, the resulting networks make use of each neuron, avoiding a hyperplane completely on one side of its input data (which is equivalent to a constant into the next layers). To illustrate this point, we study the prevalence of saturated nodes throughout training, showing that neurons are activated more frequently and earlier in training when using this regularization approach. A direct consequence of the improved neuron activation is that deep networks are now easier to train. This is crucially important when the network topology is not known a priori and fitting often remains stuck in a suboptimal local minima. We demonstrate this property by training a network of increasing depth (and constant width); most regularization approaches will result in increasingly frequent training failures (over different random seeds), whilst the proposed evidence-based regularization significantly outperforms in its ability to train deep networks.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2022-11-15
DOI: 10.3390/make4040051
Issue No: Vol. 4, No. 4 (2022)
- MAKE, Vol. 4, Pages 1024-1041: A Morphological Post-Processing Approach
for Overlapped Segmentation of Bacterial Cell Images
Authors: Dilanga Abeyrathna, Shailabh Rauniyar, Rajesh K. Sani, Pei-Chi Huang
First page: 1024
Abstract: Scanning electron microscopy (SEM) techniques have been extensively performed to image and study bacterial cells with high-resolution images. Bacterial image segmentation in SEM images is an essential task to distinguish an object of interest and its specific region. These segmentation results can then be used to retrieve quantitative measures (e.g., cell length, area, cell density) for the accurate decision-making process of obtaining cellular objects. However, the complexity of the bacterial segmentation task is a barrier, as the intensity and texture of foreground and background are similar, and also, most clustered bacterial cells in images are partially overlapping with each other. The traditional approaches for identifying cell regions in microscopy images are labor intensive and heavily dependent on the professional knowledge of researchers. To mitigate the aforementioned challenges, in this study, we tested a U-Net-based semantic segmentation architecture followed by a post-processing step of morphological over-segmentation resolution to achieve accurate cell segmentation of SEM-acquired images of bacterial cells grown in a rotary culture system. The approach showed an 89.52% Dice similarity score on bacterial cell segmentation with lower segmentation error rates, validated over several cell overlapping object segmentation approaches with significant performance improvement.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2022-11-17
DOI: 10.3390/make4040052
Issue No: Vol. 4, No. 4 (2022)
- MAKE, Vol. 4, Pages 1042-1064: On the Dimensionality and Utility of
Convolutional Autoencoder’s Latent Space Trained with
Topology-Preserving Spectral EEG Head-Maps
Authors: Arjun Vinayak Chikkankod, Luca Longo
First page: 1042
Abstract: Electroencephalography (EEG) signals can be analyzed in the temporal, spatial, or frequency domains. Noise and artifacts during the data acquisition phase contaminate these signals adding difficulties in their analysis. Techniques such as Independent Component Analysis (ICA) require human intervention to remove noise and artifacts. Autoencoders have automatized artifact detection and removal by representing inputs in a lower dimensional latent space. However, little research is devoted to understanding the minimum dimension of such latent space that allows meaningful input reconstruction. Person-specific convolutional autoencoders are designed by manipulating the size of their latent space. A sliding window technique with overlapping is employed to segment varied-sized windows. Five topographic head-maps are formed in the frequency domain for each window. The latent space of autoencoders is assessed using the input reconstruction capacity and classification utility. Findings indicate that the minimal latent space dimension is 25% of the size of the topographic maps for achieving maximum reconstruction capacity and maximizing classification accuracy, which is achieved with a window length of at least 1 s and a shift of 125 ms, using the 128 Hz sampling rate. This research contributes to the body of knowledge with an architectural pipeline for eliminating redundant EEG data while preserving relevant features with deep autoencoders.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2022-11-18
DOI: 10.3390/make4040053
Issue No: Vol. 4, No. 4 (2022)
- MAKE, Vol. 4, Pages 1065-1087: A New Rough Set Classifier for Numerical
Data Based on Reflexive and Antisymmetric Relations
Authors: Yoshie Ishii, Koki Iwao, Tsuguki Kinoshita
First page: 1065
Abstract: The grade-added rough set (GRS) approach is an extension of the rough set theory proposed by Pawlak to deal with numerical data. However, the GRS has problems with overtraining, unclassified and unnatural results. In this study, we propose a new approach called the directional neighborhood rough set (DNRS) approach to solve the problems of the GRS. The information granules in the DNRS are based on reflexive and antisymmetric relations. Following these relations, new lower and upper approximations are defined. Based on these definitions, we developed a classifier with a three-step algorithm, including DN-lower approximation classification, DN-upper approximation classification, and exceptional processing. Three experiments were conducted using the University of California Irvine (UCI)’s machine learning dataset to demonstrate the effect of each step in the DNRS model, overcoming the problems of the GRS, and achieving more accurate classifiers. The results showed that when the number of dimensions is reduced and both the lower and upper approximation algorithms are used, the DNRS model is more efficient than when the number of dimensions is large. Additionally, it was shown that the DNRS solves the problems of the GRS and the DNRS model is as accurate as existing classifiers.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2022-11-18
DOI: 10.3390/make4040054
Issue No: Vol. 4, No. 4 (2022)
- MAKE, Vol. 4, Pages 1088-1106: SGD-Based Cascade Scheme for Higher Degrees
Wiener Polynomial Approximation of Large Biomedical Datasets
Authors: Ivan Izonin, Roman Tkachenko, Rostyslav Holoven, Kyrylo Yemets, Myroslav Havryliuk, Shishir Kumar Shandilya
First page: 1088
Abstract: The modern development of the biomedical engineering area is accompanied by the availability of large volumes of data with a non-linear response surface. The effective analysis of such data requires the development of new, more productive machine learning methods. This paper proposes a cascade ensemble that combines the advantages of using a high-order Wiener polynomial and Stochastic Gradient Descent algorithm while eliminating their disadvantages to ensure a high accuracy of the approximation of such data with a satisfactory training time. The work presents flow charts of the learning algorithms and the application of the developed ensemble scheme, and all the steps are described in detail. The simulation was carried out based on a real-world dataset. Procedures for the proposed model tuning have been performed. The high accuracy of the approximation based on the developed ensemble scheme was established experimentally. The possibility of an implicit approximation by high orders of the Wiener polynomial with a slight increase in the number of its members is shown. It ensures a low training time for the proposed method during the analysis of large datasets, which provides the possibility of its practical use in the biomedical engineering area.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2022-11-21
DOI: 10.3390/make4040055
Issue No: Vol. 4, No. 4 (2022)
- MAKE, Vol. 4, Pages 1107-1123: Ontology Completion with Graph-Based
Machine Learning: A Comprehensive Evaluation
Authors: Sebastian Mežnar, Matej Bevec, Nada Lavrač, Blaž Škrlj
First page: 1107
Abstract: Increasing quantities of semantic resources offer a wealth of human knowledge, but their growth also increases the probability of wrong knowledge base entries. The development of approaches that identify potentially spurious parts of a given knowledge base is therefore highly relevant. We propose an approach for ontology completion that transforms an ontology into a graph and recommends missing edges using structure-only link analysis methods. By systematically evaluating thirteen methods (some for knowledge graphs) on eight different semantic resources, including Gene Ontology, Food Ontology, Marine Ontology, and similar ontologies, we demonstrate that a structure-only link analysis can offer a scalable and computationally efficient ontology completion approach for a subset of analyzed data sets. To the best of our knowledge, this is currently the most extensive systematic study of the applicability of different types of link analysis methods across semantic resources from different domains. It demonstrates that by considering symbolic node embeddings, explanations of the predictions (links) can be obtained, making this branch of methods potentially more valuable than black-box methods.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2022-12-01
DOI: 10.3390/make4040056
Issue No: Vol. 4, No. 4 (2022)
- MAKE, Vol. 4, Pages 1124-1135: An Explainable Deep Learning Framework for
Detecting and Localising Smoke and Fire Incidents: Evaluation of
Grad-CAM++ and LIME
Authors: Ioannis D. Apostolopoulos, Ifigeneia Athanasoula, Mpesi Tzani, Peter P. Groumpos
First page: 1124
Abstract: Climate change is expected to increase fire events and activity with multiple impacts on human lives. Large grids of forest and city monitoring devices can assist in incident detection, accelerating human intervention in extinguishing fires before they get out of control. Artificial Intelligence promises to automate the detection of fire-related incidents. This study enrols 53,585 fire/smoke and normal images and benchmarks seventeen state-of-the-art Convolutional Neural Networks for distinguishing between the two classes. The Xception network proves to be superior to the rest of the CNNs, obtaining very high accuracy. Grad-CAM++ and LIME algorithms improve the post hoc explainability of Xception and verify that it is learning features found in the critical locations of the image. Both methods agree on the suggested locations, strengthening the abovementioned outcome.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2022-12-06
DOI: 10.3390/make4040057
Issue No: Vol. 4, No. 4 (2022)
- MAKE, Vol. 4, Pages 580-590: Real Quadratic-Form-Based Graph Pooling for
Graph Neural Networks
Authors: Youfa Liu, Guo Chen
First page: 580
Abstract: Graph neural networks (GNNs) have developed rapidly in recent years because they can work over non-Euclidean data and possess promising prediction power in many real-word applications. The graph classification problem is one of the central problems in graph neural networks, and aims to predict the label of a graph with the help of training graph neural networks over graph-structural datasets. The graph pooling scheme is an important part of graph neural networks for the graph classification objective. Previous works typically focus on using the graph pooling scheme in a linear manner. In this paper, we propose the real quadratic-form-based graph pooling framework for graph neural networks in graph classification. The quadratic form can capture a pairwise relationship, which brings a stronger expressive power than existing linear forms. Experiments on benchmarks verify the effectiveness of the proposed graph pooling scheme based on the quadratic form in graph classification tasks.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2022-06-21
DOI: 10.3390/make4030027
Issue No: Vol. 4, No. 3 (2022)
- MAKE, Vol. 4, Pages 591-620: Certifiable Unlearning Pipelines for Logistic
Regression: An Experimental Study
Authors: Ananth Mahadevan, Michael Mathioudakis
First page: 591
Abstract: Machine unlearning is the task of updating machine learning (ML) models after a subset of the training data they were trained on is deleted. Methods for the task are desired to combine effectiveness and efficiency (i.e., they should effectively “unlearn” deleted data, but in a way that does not require excessive computational effort (e.g., a full retraining) for a small amount of deletions). Such a combination is typically achieved by tolerating some amount of approximation in the unlearning. In addition, laws and regulations in the spirit of “the right to be forgotten” have given rise to requirements for certifiability (i.e., the ability to demonstrate that the deleted data has indeed been unlearned by the ML model). In this paper, we present an experimental study of the three state-of-the-art approximate unlearning methods for logistic regression and demonstrate the trade-offs between efficiency, effectiveness and certifiability offered by each method. In implementing this study, we extend some of the existing works and describe a common unlearning pipeline to compare and evaluate the unlearning methods on six real-world datasets and a variety of settings. We provide insights into the effect of the quantity and distribution of the deleted data on ML models and the performance of each unlearning method in different settings. We also propose a practical online strategy to determine when the accumulated error from approximate unlearning is large enough to warrant a full retraining of the ML model.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2022-06-22
DOI: 10.3390/make4030028
Issue No: Vol. 4, No. 3 (2022)
- MAKE, Vol. 4, Pages 621-640: Semantic Image Segmentation Using Scant Pixel
Annotations
Authors: Adithi D. Chakravarthy, Dilanga Abeyrathna, Mahadevan Subramaniam, Parvathi Chundi, Venkataramana Gadhamshetty
First page: 621
Abstract: The success of deep networks for the semantic segmentation of images is limited by the availability of annotated training data. The manual annotation of images for segmentation is a tedious and time-consuming task that often requires sophisticated users with significant domain expertise to create high-quality annotations over hundreds of images. In this paper, we propose the segmentation with scant pixel annotations (SSPA) approach to generate high-performing segmentation models using a scant set of expert annotated images. The models are generated by training them on images with automatically generated pseudo-labels along with a scant set of expert annotated images selected using an entropy-based algorithm. For each chosen image, experts are directed to assign labels to a particular group of pixels, while a set of replacement rules that leverage the patterns learned by the model is used to automatically assign labels to the remaining pixels. The SSPA approach integrates active learning and semi-supervised learning with pseudo-labels, where expert annotations are not essential but generated on demand. Extensive experiments on bio-medical and biofilm datasets show that the SSPA approach achieves state-of-the-art performance with less than 5% cumulative annotation of the pixels of the training data by the experts.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2022-07-01
DOI: 10.3390/make4030029
Issue No: Vol. 4, No. 3 (2022)
- MAKE, Vol. 4, Pages 641-664: Do We Need a Specific Corpus and Multiple
High-Performance GPUs for Training the BERT Model' An Experiment on
COVID-19 Dataset
Authors: Nontakan Nuntachit, Prompong Sugunnasil
First page: 641
Abstract: The COVID-19 pandemic has impacted daily lives around the globe. Since 2019, the amount of literature focusing on COVID-19 has risen exponentially. However, it is almost impossible for humans to read all of the studies and classify them. This article proposes a method of making an unsupervised model called a zero-shot classification model, based on the pre-trained BERT model. We used the CORD-19 dataset in conjunction with the LitCovid database to construct new vocabulary and prepare the test dataset. For NLI downstream task, we used three corpora: SNLI, MultiNLI, and MedNLI. We significantly reduced the training time by 98.2639% to build a task-specific machine learning model, using only one Nvidia Tesla V100. The final model can run faster and use fewer resources than its comparators. It has an accuracy of 27.84%, which is lower than the best-achieved accuracy by 6.73%, but it is comparable. Finally, we identified that the tokenizer and vocabulary more specific to COVID-19 could not outperform the generalized ones. Additionally, it was found that BART architecture affects the classification results.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2022-07-04
DOI: 10.3390/make4030030
Issue No: Vol. 4, No. 3 (2022)
- MAKE, Vol. 4, Pages 665-687: Improving Deep Learning for Maritime Remote
Sensing through Data Augmentation and Latent Space
Authors: Daniel Sobien, Erik Higgins, Justin Krometis, Justin Kauffman, Laura Freeman
First page: 665
Abstract: Training deep learning models requires having the right data for the problem and understanding both your data and the models’ performance on that data. Training deep learning models is difficult when data are limited, so in this paper, we seek to answer the following question: how can we train a deep learning model to increase its performance on a targeted area with limited data' We do this by applying rotation data augmentations to a simulated synthetic aperture radar (SAR) image dataset. We use the Uniform Manifold Approximation and Projection (UMAP) dimensionality reduction technique to understand the effects of augmentations on the data in latent space. Using this latent space representation, we can understand the data and choose specific training samples aimed at boosting model performance in targeted under-performing regions without the need to increase training set sizes. Results show that using latent space to choose training data significantly improves model performance in some cases; however, there are other cases where no improvements are made. We show that linking patterns in latent space is a possible predictor of model performance, but results require some experimentation and domain knowledge to determine the best options.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2022-07-07
DOI: 10.3390/make4030031
Issue No: Vol. 4, No. 3 (2022)
- MAKE, Vol. 4, Pages 688-699: Input/Output Variables Selection in Data
Envelopment Analysis: A Shannon Entropy Approach
Authors: Pejman Peykani, Fatemeh Sadat Seyed Esmaeili, Mirpouya Mirmozaffari, Armin Jabbarzadeh, Mohammad Khamechian
First page: 688
Abstract: The purpose of this study is to provide an efficient method for the selection of input–output indicators in the data envelopment analysis (DEA) approach, in order to improve the discriminatory power of the DEA method in the evaluation process and performance analysis of homogeneous decision-making units (DMUs) in the presence of negative values and data. For this purpose, the Shannon entropy technique is used as one of the most important methods for determining the weight of indicators. Moreover, due to the presence of negative data in some indicators, the range directional measure (RDM) model is used as the basic model of the research. Finally, to demonstrate the applicability of the proposed approach, the food and beverage industry has been selected from the Tehran stock exchange (TSE) as a case study, and data related to 15 stocks have been extracted from this industry. The numerical and experimental results indicate the efficacy of the hybrid data envelopment analysis–Shannon entropy (DEASE) approach to evaluate stocks under negative data. Furthermore, the discriminatory power of the proposed DEASE approach is greater than that of a classical DEA model.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2022-07-14
DOI: 10.3390/make4030032
Issue No: Vol. 4, No. 3 (2022)
- MAKE, Vol. 4, Pages 700-714: Data Mining Algorithms for Operating Pressure
Forecasting of Crude Oil Distribution Pipelines to Identify Potential
Blockages
Authors: Agus Santoso, Fransisco Danang Wijaya, Noor Akhmad Setiawan, Joko Waluyo
First page: 700
Abstract: The implementation of data mining has become very popular in many fields recently, including in the petroleum industry. It is widely used to help in decision-making processes in order to minimize oil losses during operations. One of the major causes of loss is oil flow blockages during transport to the gathering facility, known as the congeal phenomenon. To overcome this situation, real-time surveillance is used to monitor the oil flow condition inside pipes. However, this system is not able to forecast the pipeline pressure on the next several days. The objective of this study is to forecast the pressure several days in advance using real-time pressure data, as well as external factor data recorded by nearby weather stations, such as ambient temperature and precipitation. Three machine learning algorithms—multi-layer perceptron (MLP), long short-term memory (LSTM), and nonlinear autoregressive exogenous model (NARX)—are evaluated and compared with each other using standard regression evaluation metrics, including a steady-state model. As a result, with proper hyperparameters, in the proposed method of NARX with MLP as a regressor, the NARX algorithm showed the best performance among the evaluated algorithms, indicated by the highest values of R2 and lowest values of RMSE. This algorithm is capable of forecasting the pressure with high correlation to actual field data. By forecasting the pressure several days ahead, system owners may take pre-emptive actions to prevent congealing.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2022-07-21
DOI: 10.3390/make4030033
Issue No: Vol. 4, No. 3 (2022)
- MAKE, Vol. 4, Pages 715-737: VLA-SMILES: Variable-Length-Array SMILES
Descriptors in Neural Network-Based QSAR Modeling
Authors: Antonina L. Nazarova, Aiichiro Nakano
First page: 715
Abstract: Machine learning represents a milestone in data-driven research, including material informatics, robotics, and computer-aided drug discovery. With the continuously growing virtual and synthetically available chemical space, efficient and robust quantitative structure–activity relationship (QSAR) methods are required to uncover molecules with desired properties. Herein, we propose variable-length-array SMILES-based (VLA-SMILES) structural descriptors that expand conventional SMILES descriptors widely used in machine learning. This structural representation extends the family of numerically coded SMILES, particularly binary SMILES, to expedite the discovery of new deep learning QSAR models with high predictive ability. VLA-SMILES descriptors were shown to speed up the training of QSAR models based on multilayer perceptron (MLP) with optimized backpropagation (ATransformedBP), resilient propagation (iRPROP‒), and Adam optimization learning algorithms featuring rational train–test splitting, while improving the predictive ability toward the more compute-intensive binary SMILES representation format. All the tested MLPs under the same length-array-based SMILES descriptors showed similar predictive ability and convergence rate of training in combination with the considered learning procedures. Validation with the Kennard–Stone train–test splitting based on the structural descriptor similarity metrics was found more effective than the partitioning with the ranking by activity based on biological activity values metrics for the entire set of VLA-SMILES featured QSAR. Robustness and the predictive ability of MLP models based on VLA-SMILES were assessed via the method of QSAR parametric model validation. In addition, the method of the statistical H0 hypothesis testing of the linear regression between real and observed activities based on the F2,n−2 -criteria was used for predictability estimation among VLA-SMILES featured QSAR-MLPs (with n being the volume of the testing set). Both approaches of QSAR parametric model validation and statistical hypothesis testing were found to correlate when used for the quantitative evaluation of predictabilities of the designed QSAR models with VLA-SMILES descriptors.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2022-08-05
DOI: 10.3390/make4030034
Issue No: Vol. 4, No. 3 (2022)
- MAKE, Vol. 4, Pages 738-752: Deep Leaning Based Frequency-Aware Single
Image Deraining by Extracting Knowledge from Rain and Background
Authors: Yuhong He, Tao Zeng, Ye Xiong, Jialu Li, Haoran Wei
First page: 738
Abstract: Due to the requirement of video surveillance, machine learning-based single image deraining has become a research hotspot in recent years. In order to efficiently obtain rain removal images that contain more detailed information, this paper proposed a novel frequency-aware single image deraining network via the separation of rain and background. For the rainy images, most of the background key information belongs to the low-frequency components, while the high-frequency components are mixed by background image details and rain streaks. This paper attempted to decouple background image details from high frequency components under the guidance of the restored low frequency components. Compared with existing approaches, the proposed network has three major contributions. (1) A residual dense network based on Discrete Wavelet Transform (DWT) was proposed to study the rainy image background information. (2) The frequency channel attention module was introduced into the adaptive decoupling of high-frequency image detail signals. (3) A fusion module was introduced that contains the attention mechanism to make full use of the multi receptive fields information using a two-branch structure, using the context information in a large area. The proposed approach was evaluated using several representative datasets. Experimental results shows this proposed approach outperforms other state-of-the-art deraining algorithms.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2022-08-16
DOI: 10.3390/make4030035
Issue No: Vol. 4, No. 3 (2022)
- MAKE, Vol. 4, Pages 753-767: Live Fish Species Classification in
Underwater Images by Using Convolutional Neural Networks Based on
Incremental Learning with Knowledge Distillation Loss
Authors: Abdelouahid Ben Tamou, Abdesslam Benzinou, Kamal Nasreddine
First page: 753
Abstract: Nowadays, underwater video systems are largely used by marine ecologists to study the biodiversity in underwater environments. These systems are non-destructive, do not perturb the environment and generate a large amount of visual data usable at any time. However, automatic video analysis requires efficient techniques of image processing due to the poor quality of underwater images and the challenging underwater environment. In this paper, we address live reef fish species classification in an unconstrained underwater environment. We propose using a deep Convolutional Neural Network (CNN) and training this network by using a new strategy based on incremental learning. This training strategy consists of training the CNN progressively by focusing at first on learning the difficult species well and then gradually learning the new species incrementally using knowledge distillation loss while keeping the high performances of the old species already learned. The proposed approach yields an accuracy of 81.83% on the LifeClef 2015 Fish benchmark dataset.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2022-08-22
DOI: 10.3390/make4030036
Issue No: Vol. 4, No. 3 (2022)
- MAKE, Vol. 4, Pages 768-778: Investigating Machine Learning Applications
in the Prediction of Occupational Injuries in South African National Parks
Authors: Martha Chadyiwa, Juliana Kagura, Aimee Stewart
First page: 768
Abstract: There is a need to predict occupational injuries in South African National Parks for the purpose of implementing targeted interventions or preventive measures. Machine-learning models have the capability of predicting injuries such that the employees that are at risk of experiencing occupational injuries can be identified. Support Vector Machines (SVMs), k Nearest Neighbours (k-NN), XGB classifier and Deep Neural Networks were applied and overall performance was compared to the accuracy of baseline models that always predict low extremity injuries. Data extracted from the Department of Employment and Labour’s Compensation Fund was used for training the models. SVMs had the best performance in predicting between low extremity injuries and injuries in the torso and hands regions. However, the overall accuracy was 56%, which was slightly above the baseline and below findings from similar previous research that reported a minimum of 62%. Gender was the only feature with an importance score significantly greater than zero. There is a need to use more features related to work conditions and which acknowledge the importance of environment in order to improve the accuracy of the predictions of the models. Furthermore, more types of injuries, and employees that have not experienced any injuries, should be included in future studies.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2022-08-22
DOI: 10.3390/make4030037
Issue No: Vol. 4, No. 3 (2022)
- MAKE, Vol. 4, Pages 779-802: Factorizable Joint Shift in Multinomial
Classification
Authors: Dirk Tasche
First page: 779
Abstract: Factorizable joint shift (FJS) was recently proposed as a type of dataset shift for which the complete characteristics can be estimated from feature data observations on the test dataset by a method called Joint Importance Aligning. For the multinomial (multiclass) classification setting, we derive a representation of factorizable joint shift in terms of the source (training) distribution, the target (test) prior class probabilities and the target marginal distribution of the features. On the basis of this result, we propose alternatives to joint importance aligning and, at the same time, point out that factorizable joint shift is not fully identifiable if no class label information on the test dataset is available and no additional assumptions are made. Other results of the paper include correction formulae for the posterior class probabilities both under general dataset shift and factorizable joint shift. In addition, we investigate the consequences of assuming factorizable joint shift for the bias caused by sample selection.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2022-09-10
DOI: 10.3390/make4030038
Issue No: Vol. 4, No. 3 (2022)
- MAKE, Vol. 4, Pages 803-812: Sensor Fusion for Occupancy Estimation: A
Study Using Multiple Lecture Rooms in a Complex Building
Authors: Roussel, Böhm, Neis
First page: 803
Abstract: This paper uses various machine learning methods which explore the combination of multiple sensors for quality improvement. It is known that a reliable occupancy estimation can help in many different cases and applications. For the containment of the SARS-CoV-2 virus, in particular, room occupancy is a major factor. The estimation can benefit visitor management systems in real time, but can also be predictive of room reservation strategies. By using different terminal and non-terminal sensors in different premises of varying sizes, this paper aims to estimate room occupancy. In the process, the proposed models are trained with different combinations of rooms in training and testing datasets to examine distinctions in the infrastructure of the considered building. The results indicate that the estimation benefits from a combination of different sensors. Additionally, it is found that a model should be trained with data from every room in a building and cannot be transferred to other rooms.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2022-09-16
DOI: 10.3390/make4030039
Issue No: Vol. 4, No. 3 (2022)
- MAKE, Vol. 4, Pages 316-349: Counterfactual Models for Fair and Adequate
Explanations
Authors: Nicholas Asher, Lucas De Lara, Soumya Paul, Chris Russell
First page: 316
Abstract: Recent efforts have uncovered various methods for providing explanations that can help interpret the behavior of machine learning programs. Exact explanations with a rigorous logical foundation provide valid and complete explanations, but they have an epistemological problem: they are often too complex for humans to understand and too expensive to compute even with automated reasoning methods. Interpretability requires good explanations that humans can grasp and can compute. We take an important step toward specifying what good explanations are by analyzing the epistemically accessible and pragmatic aspects of explanations. We characterize sufficiently good, or fair and adequate, explanations in terms of counterfactuals and what we call the conundra of the explainee, the agent that requested the explanation. We provide a correspondence between logical and mathematical formulations for counterfactuals to examine the partiality of counterfactual explanations that can hide biases; we define fair and adequate explanations in such a setting. We provide formal results about the algorithmic complexity of fair and adequate explanations. We then detail two sophisticated counterfactual models, one based on causal graphs, and one based on transport theories. We show transport based models have several theoretical advantages over the competition as explanation frameworks for machine learning algorithms.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2022-03-31
DOI: 10.3390/make4020014
Issue No: Vol. 4, No. 2 (2022)
- MAKE, Vol. 4, Pages 350-370: An Attention-Based ConvLSTM Autoencoder with
Dynamic Thresholding for Unsupervised Anomaly Detection in Multivariate
Time Series
Authors: Tareq Tayeh, Sulaiman Aburakhia, Ryan Myers, Abdallah Shami
First page: 350
Abstract: As a substantial amount of multivariate time series data is being produced by the complex systems in smart manufacturing (SM), improved anomaly detection frameworks are needed to reduce the operational risks and the monitoring burden placed on the system operators. However, building such frameworks is challenging, as a sufficiently large amount of defective training data is often not available and frameworks are required to capture both the temporal and contextual dependencies across different time steps while being robust to noise. In this paper, we propose an unsupervised Attention-Based Convolutional Long Short-Term Memory (ConvLSTM) Autoencoder with Dynamic Thresholding (ACLAE-DT) framework for anomaly detection and diagnosis in multivariate time series. The framework starts by pre-processing and enriching the data, before constructing feature images to characterize the system statuses across different time steps by capturing the inter-correlations between pairs of time series. Afterwards, the constructed feature images are fed into an attention-based ConvLSTM autoencoder, which aims to encode the constructed feature images and capture the temporal behavior, followed by decoding the compressed knowledge representation to reconstruct the feature images’ input. The reconstruction errors are then computed and subjected to a statistical-based, dynamic thresholding mechanism to detect and diagnose the anomalies. Evaluation results conducted on real-life manufacturing data demonstrate the performance strengths of the proposed approach over state-of-the-art methods under different experimental settings.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2022-04-02
DOI: 10.3390/make4020015
Issue No: Vol. 4, No. 2 (2022)
- MAKE, Vol. 4, Pages 371-396: VloGraph: A Virtual Knowledge Graph Framework
for Distributed Security Log Analysis
Authors: Kabul Kurniawan, Andreas Ekelhart, Elmar Kiesling, Dietmar Winkler, Gerald Quirchmayr, A Min Tjoa
First page: 371
Abstract: The integration of heterogeneous and weakly linked log data poses a major challenge in many log-analytic applications. Knowledge graphs (KGs) can facilitate such integration by providing a versatile representation that can interlink objects of interest and enrich log events with background knowledge. Furthermore, graph-pattern based query languages, such as SPARQL, can support rich log analyses by leveraging semantic relationships between objects in heterogeneous log streams. Constructing, materializing, and maintaining centralized log knowledge graphs, however, poses significant challenges. To tackle this issue, we propose VloGraph—a distributed and virtualized alternative to centralized log knowledge graph construction. The proposed approach does not involve any a priori parsing, aggregation, and processing of log data, but dynamically constructs a virtual log KG from heterogeneous raw log sources across multiple hosts. To explore the feasibility of this approach, we developed a prototype and demonstrate its applicability to three scenarios. Furthermore, we evaluate the approach in various experimental settings with multiple heterogeneous log sources and machines; the encouraging results from this evaluation suggest that the approach can enable efficient graph-based ad-hoc log analyses in federated settings.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2022-04-11
DOI: 10.3390/make4020016
Issue No: Vol. 4, No. 2 (2022)
- MAKE, Vol. 4, Pages 397-417: Missing Data Estimation in Temporal
Multilayer Position-Aware Graph Neural Network (TMP-GNN)
Authors: Bahareh Najafi, Saeedeh Parsaeefard, Alberto Leon-Garcia
First page: 397
Abstract: GNNs have been proven to perform highly effectively in various node-level, edge-level, and graph-level prediction tasks in several domains. Existing approaches mainly focus on static graphs. However, many graphs change over time and their edge may disappear, or the node/edge attribute may alter from one time to the other. It is essential to consider such evolution in the representation learning of nodes in time-varying graphs. In this paper, we propose a Temporal Multilayer Position-Aware Graph Neural Network (TMP-GNN), a node embedding approach for dynamic graphs that incorporates the interdependence of temporal relations into embedding computation. We evaluate the performance of TMP-GNN on two different representations of temporal multilayered graphs. The performance is assessed against the most popular GNNs on a node-level prediction task. Then, we incorporate TMP-GNN into a deep learning framework to estimate missing data and compare the performance with their corresponding competent GNNs from our former experiment, and a baseline method. Experimental results on four real-world datasets yield up to 58% lower ROCAUC for the pair-wise node classification task, and 96% lower MAE in missing feature estimation, particularly for graphs with a relatively high number of nodes and lower mean degree of connectivity.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2022-04-30
DOI: 10.3390/make4020017
Issue No: Vol. 4, No. 2 (2022)
- MAKE, Vol. 4, Pages 418-431: Estimating the Best Time to View Cherry
Blossoms Using Time-Series Forecasting Method
Authors: Tomonari Horikawa, Munenori Takahashi, Masaki Endo, Shigeyoshi Ohno, Masaharu Hirota, Hiroshi Ishikawa
First page: 418
Abstract: In recent years, tourist information collection using the internet has become common. Tourists are increasingly using internet resources to obtain tourist information. Social network service (SNS) users share tourist information of various kinds. Twitter, one SNS, has been used for many studies. We are pursuing research supporting a method using Twitter to help tourists obtain information: estimates of the best time to view cherry blossoms. Earlier studies have proposed a low-cost moving average method using geotagged tweets related to location information. Geotagged tweets are helpful as social sensors for real-time estimation and for the acquisition of local tourist information because the information can reflect real-world situations. Earlier studies have used weighted moving averages, indicating that a person can estimate the best time to view cherry blossoms in each prefecture. This study proposes a time-series prediction method using SNS data and machine learning as a new method for estimating the best times for viewing for a certain period. Combining the time-series forecasting method and the low-cost moving average method yields an estimate of the best time to view cherry blossoms. This report describes results confirming the usefulness of the proposed method by experimentation with estimation of the best time to view beautiful cherry blossoms in each prefecture and municipality.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2022-04-30
DOI: 10.3390/make4020018
Issue No: Vol. 4, No. 2 (2022)
- MAKE, Vol. 4, Pages 432-445: Knowledgebra: An Algebraic Learning Framework
for Knowledge Graph
Authors: Tong Yang, Yifei Wang, Long Sha, Jan Engelbrecht, Pengyu Hong
First page: 432
Abstract: Knowledge graph (KG) representation learning aims to encode entities and relations into dense continuous vector spaces such that knowledge contained in a dataset could be consistently represented. Dense embeddings trained from KG datasets benefit a variety of downstream tasks such as KG completion and link prediction. However, existing KG embedding methods fell short to provide a systematic solution for the global consistency of knowledge representation. We developed a mathematical language for KG based on an observation of their inherent algebraic structure, which we termed as Knowledgebra. By analyzing five distinct algebraic properties, we proved that the semigroup is the most reasonable algebraic structure for the relation embedding of a general knowledge graph. We implemented an instantiation model, SemE, using simple matrix semigroups, which exhibits state-of-the-art performance on standard datasets. Moreover, we proposed a regularization-based method to integrate chain-like logic rules derived from human knowledge into embedding training, which further demonstrates the power of the developed language. As far as we know, by applying abstract algebra in statistical learning, this work develops the first formal language for general knowledge graphs, and also sheds light on the problem of neural-symbolic integration from an algebraic perspective.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2022-05-05
DOI: 10.3390/make4020019
Issue No: Vol. 4, No. 2 (2022)
- MAKE, Vol. 4, Pages 446-473: Machine Learning in Disaster Management:
Recent Developments in Methods and Applications
Authors: Vasileios Linardos, Maria Drakaki, Panagiotis Tzionas, Yannis L. Karnavas
First page: 446
Abstract: Recent years include the world’s hottest year, while they have been marked mainly, besides the COVID-19 pandemic, by climate-related disasters, based on data collected by the Emergency Events Database (EM-DAT). Besides the human losses, disasters cause significant and often catastrophic socioeconomic impacts, including economic losses. Recent developments in artificial intelligence (AI) and especially in machine learning (ML) and deep learning (DL) have been used to better cope with the severe and often catastrophic impacts of disasters. This paper aims to provide an overview of the research studies, presented since 2017, focusing on ML and DL developed methods for disaster management. In particular, focus has been given on studies in the areas of disaster and hazard prediction, risk and vulnerability assessment, disaster detection, early warning systems, disaster monitoring, damage assessment and post-disaster response as well as cases studies. Furthermore, some recently developed ML and DL applications for disaster management have been analyzed. A discussion of the findings is provided as well as directions for further research.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2022-05-07
DOI: 10.3390/make4020020
Issue No: Vol. 4, No. 2 (2022)
- MAKE, Vol. 4, Pages 474-487: The Case of Aspect in Sentiment Analysis:
Seeking Attention or Co-Dependency'
Authors: Anastazia Žunić, Padraig Corcoran, Irena Spasić
First page: 474
Abstract: (1) Background: Aspect-based sentiment analysis (SA) is a natural language processing task, the aim of which is to classify the sentiment associated with a specific aspect of a written text. The performance of SA methods applied to texts related to health and well-being lags behind that of other domains. (2) Methods: In this study, we present an approach to aspect-based SA of drug reviews. Specifically, we analysed signs and symptoms, which were extracted automatically using the Unified Medical Language System. This information was then passed onto the BERT language model, which was extended by two layers to fine-tune the model for aspect-based SA. The interpretability of the model was analysed using an axiomatic attribution method. We performed a correlation analysis between the attribution scores and syntactic dependencies. (3) Results: Our fine-tuned model achieved accuracy of approximately 95% on a well-balanced test set. It outperformed our previous approach, which used syntactic information to guide the operation of a neural network and achieved an accuracy of approximately 82%. (4) Conclusions: We demonstrated that a BERT-based model of SA overcomes the negative bias associated with health-related aspects and closes the performance gap against the state-of-the-art in other domains.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2022-05-13
DOI: 10.3390/make4020021
Issue No: Vol. 4, No. 2 (2022)
- MAKE, Vol. 4, Pages 488-501: TabFairGAN: Fair Tabular Data Generation with
Generative Adversarial Networks
Authors: Amirarsalan Rajabi, Ozlem Ozmen Garibay
First page: 488
Abstract: With the increasing reliance on automated decision making, the issue of algorithmic fairness has gained increasing importance. In this paper, we propose a Generative Adversarial Network for tabular data generation. The model includes two phases of training. In the first phase, the model is trained to accurately generate synthetic data similar to the reference dataset. In the second phase we modify the value function to add fairness constraint, and continue training the network to generate data that is both accurate and fair. We test our results in both cases of unconstrained, and constrained fair data generation. We show that using a fairly simple architecture and applying quantile transformation of numerical attributes the model achieves promising performance. In the unconstrained case, i.e., when the model is only trained in the first phase and is only meant to generate accurate data following the same joint probability distribution of the real data, the results show that the model beats the state-of-the-art GANs proposed in the literature to produce synthetic tabular data. Furthermore, in the constrained case in which the first phase of training is followed by the second phase, we train the network and test it on four datasets studied in the fairness literature and compare our results with another state-of-the-art pre-processing method, and present the promising results that it achieves. Comparing to other studies utilizing GANs for fair data generation, our model is comparably more stable by using only one critic, and also by avoiding major problems of original GAN model, such as mode-dropping and non-convergence.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2022-05-16
DOI: 10.3390/make4020022
Issue No: Vol. 4, No. 2 (2022)
- MAKE, Vol. 4, Pages 502-518: Machine and Deep Learning Applications to
Mouse Dynamics for Continuous User Authentication
Authors: Nyle Siddiqui, Rushit Dave, Mounika Vanamala, Naeem Seliya
First page: 502
Abstract: Static authentication methods, like passwords, grow increasingly weak with advancements in technology and attack strategies. Continuous authentication has been proposed as a solution, in which users who have gained access to an account are still monitored in order to continuously verify that the user is not an imposter who had access to the user credentials. Mouse dynamics is the behavior of a user’s mouse movements and is a biometric that has shown great promise for continuous authentication schemes. This article builds upon our previous published work by evaluating our dataset of 40 users using three machine learning and three deep learning algorithms. Two evaluation scenarios are considered: binary classifiers are used for user authentication, with the top performer being a 1-dimensional convolutional neural network (1D-CNN) with a peak average test accuracy of 85.73% across the top-10 users. Multi-class classification is also examined using an artificial neural network (ANN) which reaches an astounding peak accuracy of 92.48%, the highest accuracy we have seen for any classifier on this dataset.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2022-05-19
DOI: 10.3390/make4020023
Issue No: Vol. 4, No. 2 (2022)
- MAKE, Vol. 4, Pages 519-542: Quality Criteria and Method of Synthesis for
Adversarial Attack-Resistant Classifiers
Authors: Anastasia Gurina, Vladimir Eliseev
First page: 519
Abstract: The actual problem of adversarial attacks on classifiers, mainly implemented using deep neural networks, is considered. This problem is analyzed with a generalization to the case of any classifiers synthesized by machine learning methods. The imperfection of generally accepted criteria for assessing the quality of classifiers, including those used to confirm the effectiveness of protection measures against adversarial attacks, is noted. The reason for the appearance of adversarial examples and other errors of classifiers based on machine learning is investigated. A method for modeling adversarial attacks with a demonstration of the main effects observed during the attack is proposed. It is noted that it is necessary to develop quality criteria for classifiers in terms of potential susceptibility to adversarial attacks. To assess resistance to adversarial attacks, it is proposed to use the multidimensional EDCAP criterion (Excess, Deficit, Coating, Approx, Pref). We also propose a method for synthesizing a new EnAE (Ensemble of Auto-Encoders) multiclass classifier based on an ensemble of quality-controlled one-class classifiers according to EDCAP criteria. The EnAE classification algorithm implements a hard voting approach and can detect anomalous inputs. The proposed criterion, synthesis method and classifier are tested on several data sets with a medium dimension of the feature space.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2022-06-05
DOI: 10.3390/make4020024
Issue No: Vol. 4, No. 2 (2022)
- MAKE, Vol. 4, Pages 542-555: Benefits from Variational Regularization in
Language Models
Authors: Cornelia Ferner, Stefan Wegenkittl
First page: 542
Abstract: Representations from common pre-trained language models have been shown to suffer from the degeneration problem, i.e., they occupy a narrow cone in latent space. This problem can be addressed by enforcing isotropy in latent space. In analogy with variational autoencoders, we suggest applying a token-level variational loss to a Transformer architecture and optimizing the standard deviation of the prior distribution in the loss function as the model parameter to increase isotropy. The resulting latent space is complete and interpretable: any given point is a valid embedding and can be decoded into text again. This allows for text manipulations such as paraphrase generation directly in latent space. Surprisingly, features extracted at the sentence level also show competitive results on benchmark classification tasks.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2022-06-09
DOI: 10.3390/make4020025
Issue No: Vol. 4, No. 2 (2022)
- MAKE, Vol. 4, Pages 556-579: Fairness and Explanation in AI-Informed
Decision Making
Authors: Alessa Angerschmid, Jianlong Zhou, Kevin Theuermann, Fang Chen, Andreas Holzinger
First page: 556
Abstract: AI-assisted decision-making that impacts individuals raises critical questions about transparency and fairness in artificial intelligence (AI). Much research has highlighted the reciprocal relationships between the transparency/explanation and fairness in AI-assisted decision-making. Thus, considering their impact on user trust or perceived fairness simultaneously benefits responsible use of socio-technical AI systems, but currently receives little attention. In this paper, we investigate the effects of AI explanations and fairness on human-AI trust and perceived fairness, respectively, in specific AI-based decision-making scenarios. A user study simulating AI-assisted decision-making in two health insurance and medical treatment decision-making scenarios provided important insights. Due to the global pandemic and restrictions thereof, the user studies were conducted as online surveys. From the participant’s trust perspective, fairness was found to affect user trust only under the condition of a low fairness level, with the low fairness level reducing user trust. However, adding explanations helped users increase their trust in AI-assisted decision-making. From the perspective of perceived fairness, our work found that low levels of introduced fairness decreased users’ perceptions of fairness, while high levels of introduced fairness increased users’ perceptions of fairness. The addition of explanations definitely increased the perception of fairness. Furthermore, we found that application scenarios influenced trust and perceptions of fairness. The results show that the use of AI explanations and fairness statements in AI applications is complex: we need to consider not only the type of explanations and the degree of fairness introduced, but also the scenarios in which AI-assisted decision-making is used.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2022-06-16
DOI: 10.3390/make4020026
Issue No: Vol. 4, No. 2 (2022)
- MAKE, Vol. 4, Pages 22-41: A Transfer Learning Evaluation of Deep Neural
Networks for Image Classification
Authors: Nermeen Abou Baker, Nico Zengeler, Uwe Handmann
First page: 22
Abstract: Transfer learning is a machine learning technique that uses previously acquired knowledge from a source domain to enhance learning in a target domain by reusing learned weights. This technique is ubiquitous because of its great advantages in achieving high performance while saving training time, memory, and effort in network design. In this paper, we investigate how to select the best pre-trained model that meets the target domain requirements for image classification tasks. In our study, we refined the output layers and general network parameters to apply the knowledge of eleven image processing models, pre-trained on ImageNet, to five different target domain datasets. We measured the accuracy, accuracy density, training time, and model size to evaluate the pre-trained models both in training sessions in one episode and with ten episodes.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2022-01-14
DOI: 10.3390/make4010002
Issue No: Vol. 4, No. 1 (2022)
- MAKE, Vol. 4, Pages 42-65: NER in Archival Finding Aids: Extended
Authors: Luís Filipe da Costa Cunha, José Carlos Ramalho
First page: 42
Abstract: The amount of information preserved in Portuguese archives has increased over the years. These documents represent a national heritage of high importance, as they portray the country’s history. Currently, most Portuguese archives have made their finding aids available to the public in digital format, however, these data do not have any annotation, so it is not always easy to analyze their content. In this work, Named Entity Recognition solutions were created that allow the identification and classification of several named entities from the archival finding aids. These named entities translate into crucial information about their context and, with high confidence results, they can be used for several purposes, for example, the creation of smart browsing tools by using entity linking and record linking techniques. In order to achieve high result scores, we annotated several corpora to train our own Machine Learning algorithms in this context domain. We also used different architectures, such as CNNs, LSTMs, and Maximum Entropy models. Finally, all the created datasets and ML models were made available to the public with a developed web platform, NER@DI.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2022-01-17
DOI: 10.3390/make4010003
Issue No: Vol. 4, No. 1 (2022)
- MAKE, Vol. 4, Pages 66-102: A Survey of Near-Data Processing Architectures
for Neural Networks
Authors: Mehdi Hassanpour, Marc Riera, Antonio González
First page: 66
Abstract: Data-intensive workloads and applications, such as machine learning (ML), are fundamentally limited by traditional computing systems based on the von-Neumann architecture. As data movement operations and energy consumption become key bottlenecks in the design of computing systems, the interest in unconventional approaches such as Near-Data Processing (NDP), machine learning, and especially neural network (NN)-based accelerators has grown significantly. Emerging memory technologies, such as ReRAM and 3D-stacked, are promising for efficiently architecting NDP-based accelerators for NN due to their capabilities to work as both high-density/low-energy storage and in/near-memory computation/search engine. In this paper, we present a survey of techniques for designing NDP architectures for NN. By classifying the techniques based on the memory technology employed, we underscore their similarities and differences. Finally, we discuss open challenges and future perspectives that need to be explored in order to improve and extend the adoption of NDP architectures for future computing platforms. This paper will be valuable for computer architects, chip designers, and researchers in the area of machine learning.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2022-01-17
DOI: 10.3390/make4010004
Issue No: Vol. 4, No. 1 (2022)
- MAKE, Vol. 4, Pages 103-104: Acknowledgment to Reviewers of Machine
Learning and Knowledge Extraction in 2021
Authors: Machine Learning; Knowledge Extraction Editorial Office
First page: 103
Abstract: Rigorous peer-reviews are the basis of high-quality academic publishing [...]
Citation: Machine Learning and Knowledge Extraction
PubDate: 2022-01-28
DOI: 10.3390/make4010005
Issue No: Vol. 4, No. 1 (2022)
- MAKE, Vol. 4, Pages 105-130: Machine Learning Based Restaurant Sales
Forecasting
Authors: Austin Schmidt, Md Wasi Ul Kabir, Md Tamjidul Hoque
First page: 105
Abstract: To encourage proper employee scheduling for managing crew load, restaurants need accurate sales forecasting. This paper proposes a case study on many machine learning (ML) models using real-world sales data from a mid-sized restaurant. Trendy recurrent neural network (RNN) models are included for direct comparison to many methods. To test the effects of trend and seasonality, we generate three different datasets to train our models with and to compare our results. To aid in forecasting, we engineer many features and demonstrate good methods to select an optimal sub-set of highly correlated features. We compare the models based on their performance for forecasting time steps of one-day and one-week over a curated test dataset. The best results seen in one-day forecasting come from linear models with a sMAPE of only 19.6%. Two RNN models, LSTM and TFT, and ensemble models also performed well with errors less than 20%. When forecasting one-week, non-RNN models performed poorly, giving results worse than 20% error. RNN models extended better with good sMAPE scores giving 19.5% in the best result. The RNN models performed worse overall on datasets with trend and seasonality removed, however many simpler ML models performed well when linearly separating each training instance.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2022-01-30
DOI: 10.3390/make4010006
Issue No: Vol. 4, No. 1 (2022)
- MAKE, Vol. 4, Pages 131-149: A Novel Framework for Fast Feature Selection
Based on Multi-Stage Correlation Measures
Authors: Ivan-Alejandro Garcia-Ramirez, Arturo Calderon-Mora, Andres Mendez-Vazquez, Susana Ortega-Cisneros, Ivan Reyes-Amezcua
First page: 131
Abstract: Datasets with thousands of features represent a challenge for many of the existing learning methods because of the well known curse of dimensionality. Not only that, but the presence of irrelevant and redundant features on any dataset can degrade the performance of any model where training and inference is attempted. In addition, in large datasets, the manual management of features tends to be impractical. Therefore, the increasing interest of developing frameworks for the automatic discovery and removal of useless features through the literature of Machine Learning. This is the reason why, in this paper, we propose a novel framework for selecting relevant features in supervised datasets based on a cascade of methods where speed and precision are in mind. This framework consists of a novel combination of Approximated and Simulate Annealing versions of the Maximal Information Coefficient (MIC) to generalize the simple linear relation between features. This process is performed in a series of steps by applying the MIC algorithms and cutoff strategies to remove irrelevant and redundant features. The framework is also designed to achieve a balance between accuracy and speed. To test the performance of the proposed framework, a series of experiments are conducted on a large battery of datasets from SPECTF Heart to Sonar data. The results show the balance of accuracy and speed that the proposed framework can achieve.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2022-02-08
DOI: 10.3390/make4010007
Issue No: Vol. 4, No. 1 (2022)
- MAKE, Vol. 4, Pages 150-171: Explainable Machine Learning Reveals
Capabilities, Redundancy, and Limitations of a Geospatial Air Quality
Benchmark Dataset
Authors: Scarlet Stadtler, Clara Betancourt, Ribana Roscher
First page: 150
Abstract: Air quality is relevant to society because it poses environmental risks to humans and nature. We use explainable machine learning in air quality research by analyzing model predictions in relation to the underlying training data. The data originate from worldwide ozone observations, paired with geospatial data. We use two different architectures: a neural network and a random forest trained on various geospatial data to predict multi-year averages of the air pollutant ozone. To understand how both models function, we explain how they represent the training data and derive their predictions. By focusing on inaccurate predictions and explaining why these predictions fail, we can (i) identify underrepresented samples, (ii) flag unexpected inaccurate predictions, and (iii) point to training samples irrelevant for predictions on the test set. Based on the underrepresented samples, we suggest where to build new measurement stations. We also show which training samples do not substantially contribute to the model performance. This study demonstrates the application of explainable machine learning beyond simply explaining the trained model.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2022-02-11
DOI: 10.3390/make4010008
Issue No: Vol. 4, No. 1 (2022)
- MAKE, Vol. 4, Pages 172-221: Hierarchical Reinforcement Learning: A Survey
and Open Research Challenges
Authors: Matthias Hutsebaut-Buysse, Kevin Mets, Steven Latré
First page: 172
Abstract: Reinforcement learning (RL) allows an agent to solve sequential decision-making problems by interacting with an environment in a trial-and-error fashion. When these environments are very complex, pure random exploration of possible solutions often fails, or is very sample inefficient, requiring an unreasonable amount of interaction with the environment. Hierarchical reinforcement learning (HRL) utilizes forms of temporal- and state-abstractions in order to tackle these challenges, while simultaneously paving the road for behavior reuse and increased interpretability of RL systems. In this survey paper we first introduce a selection of problem-specific approaches, which provided insight in how to utilize often handcrafted abstractions in specific task settings. We then introduce the Options framework, which provides a more generic approach, allowing abstractions to be discovered and learned semi-automatically. Afterwards we introduce the goal-conditional approach, which allows sub-behaviors to be embedded in a continuous space. In order to further advance the development of HRL agents, capable of simultaneously learning abstractions and how to use them, solely from interaction with complex high dimensional environments, we also identify a set of promising research directions.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2022-02-17
DOI: 10.3390/make4010009
Issue No: Vol. 4, No. 1 (2022)
- MAKE, Vol. 4, Pages 222-239: An Analysis of Cholesteric Spherical
Reflector Identifiers for Object Authenticity Verification
Authors: Arenas, Demirci, Lenzini
First page: 222
Abstract: Arrays of Cholesteric Spherical Reflectors (CSRs), microscopic cholesteric liquid crystals in a spherical shape, have been argued to become a game-changing technology in anti-counterfeiting. Used to build identifiable tags or coating, called CSR IDs, they can supply objects with unclonable fingerprint-like characteristics, making it possible to authenticate objects. In a previous study, we have shown how to extract minutiæ from CSR IDs. In this journal version, we build on that previous research, consolidate the methodology, and test it over CSR IDs obtained by different production processes. We measure the robustness and reliability of our procedure on large and variegate sets of CSR IDs’ images taken with a professional microscope (Laboratory Data set) and with a microscope that could be used in a realistic scenario (Realistic Data set). We measure intra-distance and interdistance, proving that we can distinguish images coming from the same CSR ID from images of different CSR IDs. However, without surprise, images in Laboratory Data set have an intra-distance that on average is less, and with less variance, than the intra-distance between responses from Realistic Data set. With this evidence, we discuss a few requirements for an anti-counterfeiting technology based on CSRs.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2022-02-24
DOI: 10.3390/make4010010
Issue No: Vol. 4, No. 1 (2022)
- MAKE, Vol. 4, Pages 240-253: Developing a Novel Fair-Loan Classifier
through a Multi-Sensitive Debiasing Pipeline: DualFair
Authors: Arashdeep Singh, Jashandeep Singh, Ariba Khan, Amar Gupta
First page: 240
Abstract: Machine learning (ML) models are increasingly being used for high-stake applications that can greatly impact people’s lives. Sometimes, these models can be biased toward certain social groups on the basis of race, gender, or ethnicity. Many prior works have attempted to mitigate this “model discrimination” by updating the training data (pre-processing), altering the model learning process (in-processing), or manipulating the model output (post-processing). However, more work can be done in extending this situation to intersectional fairness, where we consider multiple sensitive parameters (e.g., race) and sensitive options (e.g., black or white), thus allowing for greater real-world usability. Prior work in fairness has also suffered from an accuracy–fairness trade-off that prevents both accuracy and fairness from being high. Moreover, the previous literature has not clearly presented holistic fairness metrics that work with intersectional fairness. In this paper, we address all three of these problems by (a) creating a bias mitigation technique called DualFair and (b) developing a new fairness metric (i.e., AWI, a measure of bias of an algorithm based upon inconsistent counterfactual predictions) that can handle intersectional fairness. Lastly, we test our novel mitigation method using a comprehensive U.S. mortgage lending dataset and show that our classifier, or fair loan predictor, obtains relatively high fairness and accuracy metrics.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2022-03-12
DOI: 10.3390/make4010011
Issue No: Vol. 4, No. 1 (2022)
- MAKE, Vol. 4, Pages 254-275: Comparison of Text Mining Models for Food and
Dietary Constituent Named-Entity Recognition
Authors: Nadeesha Perera, Thi Thuy Linh Nguyen, Matthias Dehmer, Frank Emmert-Streib
First page: 254
Abstract: Biomedical Named-Entity Recognition (BioNER) has become an essential part of text mining due to the continuously increasing digital archives of biological and medical articles. While there are many well-performing BioNER tools for entities such as genes, proteins, diseases or species, there is very little research into food and dietary constituent named-entity recognition. For this reason, in this paper, we study seven BioNER models for food and dietary constituents recognition. Specifically, we study a dictionary-based model, a conditional random fields (CRF) model and a new hybrid model, called FooDCoNER (Food and Dietary Constituents Named-Entity Recognition), which we introduce combining the former two models. In addition, we study deep language models including BERT, BioBERT, RoBERTa and ELECTRA. As a result, we find that FooDCoNER does not only lead to the overall best results, comparable with the deep language models, but FooDCoNER is also much more efficient with respect to run time and sample size requirements of the training data. The latter has been identified via the study of learning curves. Overall, our results not only provide a new tool for food and dietary constituent NER but also shed light on the difference between classical machine learning models and recent deep language models.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2022-03-16
DOI: 10.3390/make4010012
Issue No: Vol. 4, No. 1 (2022)
- MAKE, Vol. 4, Pages 276-315: Robust Reinforcement Learning: A Review of
Foundations and Recent Advances
Authors: Janosch Moos, Kay Hansel, Hany Abdulsamad, Svenja Stark, Debora Clever, Jan Peters
First page: 276
Abstract: Reinforcement learning (RL) has become a highly successful framework for learning in Markov decision processes (MDP). Due to the adoption of RL in realistic and complex environments, solution robustness becomes an increasingly important aspect of RL deployment. Nevertheless, current RL algorithms struggle with robustness to uncertainty, disturbances, or structural changes in the environment. We survey the literature on robust approaches to reinforcement learning and categorize these methods in four different ways: (i) Transition robust designs account for uncertainties in the system dynamics by manipulating the transition probabilities between states; (ii) Disturbance robust designs leverage external forces to model uncertainty in the system behavior; (iii) Action robust designs redirect transitions of the system by corrupting an agent’s output; (iv) Observation robust designs exploit or distort the perceived system state of the policy. Each of these robust designs alters a different aspect of the MDP. Additionally, we address the connection of robustness to the risk-based and entropy-regularized RL formulations. The resulting survey covers all fundamental concepts underlying the approaches to robust reinforcement learning and their recent advances.
Citation: Machine Learning and Knowledge Extraction
PubDate: 2022-03-19
DOI: 10.3390/make4010013
Issue No: Vol. 4, No. 1 (2022)
