Adversarial Training has proved to be an effective training paradigm to enforce robustness against adversarial examples in modern neural network architectures. Despite many efforts, explanations of the foundational principles underpinning the effectiveness of Adversarial Training are limited and far from being widely accepted by the Deep Learning community. In this paper, we describe surprising properties of adversarially-trained models, shedding light on mechanisms through which robustness against adversarial attacks is implemented. Moreover, we highlight limitations and failure modes affecting these models that were not discussed by prior works. We conduct extensive analyses on a wide range of architectures and datasets, performing a deep comparison between robust and natural models.

Anomaly Detection is an unsupervised learning task aimed at detecting anomalous behaviors with respect to historical data. In particular, multivariate Anomaly Detection has an important role in many applications thanks to the capability of summarizing the status of a complex system or observed phenomenon with a single indicator (typically called ‘anomaly score’) and thanks to the unsupervised nature of the task that does not require human tagging. The Isolation Forest is one of the most commonly adopted algorithms in the field of Anomaly Detection due to its proven effectiveness and low computational complexity. A major problem affecting Isolation Forest is represented by the lack of interpretability, an effect of the inherent randomness governing the splits performed by the Isolation Trees, the building blocks of the Isolation Forest. In this paper, we propose effective yet computationally inexpensive methods to define feature importance scores at both global and local levels for the Isolation Forest. Moreover, we define a procedure to perform unsupervised feature selection for Anomaly Detection problems based on our interpretability method. Such a procedure also serves the purpose of tackling the challenging task of feature importance evaluation in unsupervised anomaly detection. We assess the performance on several synthetic and real-world datasets, including comparisons against state-of-the-art interpretability techniques, and make the code publicly available to enhance reproducibility and foster research in the field.

Learning useful representations of complex data has been the subject of extensive research for many years. With the diffusion of Deep Neural Networks, Variational Autoencoders have gained lots of attention since they provide an explicit model of the data distribution based on an encoder/decoder architecture which is able to both generate images and encode them in a low-dimensional subspace. However, the latent space is not easily interpretable and the generation capabilities show some limitations since images typically look blurry and lack details. In this paper, we propose the Introspective Variational Classifier (IntroVAC), a model that learns interpretable latent subspaces by exploiting information from an additional label and provides improved image quality thanks to an adversarial training strategy.We show that IntroVAC is able to learn meaningful directions in the latent space enabling fine-grained manipulation of image attributes. We validate our approach on the CelebA dataset.

Recent results show that features of adversarially trained networks for classification, in addition to being robust, enable desirable properties such as invertibility. The latter property may seem counter-intuitive as it is widely accepted by the community that classification models should only capture the minimal information (features) required for the task. Motivated by this discrepancy, we investigate the dual relationship between Adversarial Training and Information Theory. We show that the Adversarial Training can improve linear transferability to new tasks, from which arises a new trade-off between transferability of representations and accuracy on the source task. We validate our results employing robust networks trained on CIFAR-10, CIFAR-100 and ImageNet on several datasets. Moreover, we show that Adversarial Training reduces Fisher information of representations about the input and of the weights about the task, and we provide a theoretical argument which explains the invertibility of deterministic networks without violating the principle of minimality. Finally, we leverage our theoretical insights to remarkably improve the quality of reconstructed images through inversion.

The last decade has seen tremendous improvements in technologies for Type 1 Diabetes (T1D) management, in particular the so-called artificial pancreas (AP), a wearable closed-loop device modulating insulin injection based on glucose sensor readings. Unluckily, the AP actuator, an insulin pump, is subject to failures, with potentially serious consequences for subject safety. This calls for the development of advanced monitoring systems, leveraging the unprecedented data availability. This paper tackles for the first time the problem of automatically detecting pump faults with multidimensional data-driven anomaly detection (AD) methodologies. The approach allows to avoid the subtask of identifying a physiological model, typical of model-based approaches. Furthermore, we employ unsupervised methods, removing the need of labeled data for training, hardly available in practice. The adopted data-driven AD methods are local outlier factor, connectivity-based outlier factor, and isolation forest. Moreover, we propose a modification of these methods to cope with the dynamic nature of the underlying problem. The algorithms were tuned and tested on: 1) two-synthetic 100-patients' data set, of one-month data each, generated using the ``UVA/Padova T1D Simulator,'' a large-scale nonlinear computer simulator of T1D subject physiology, largely adopted in AP research and accepted by the American Food and Drug Administration as a replacement of preclinical animal trials for AP and 2) a real 7-patients' data set consisting of one month in free-living conditions. The satisfactory accuracy of the proposed approach paves the way to the embedding of these methodologies in AP systems or their deployment in remote monitoring systems.

In many real-world applications of Machine Learning it is of paramount importance not only to provide accurate predictions, but also to ensure certain levels of robustness. Adversarial Training is a training procedure aiming at providing models that are robust to worst-case perturbations around predefined points. Unfortunately, one of the main issues in adversarial training is that robustness w.r.t. gradient-based attackers is always achieved at the cost of prediction accuracy. In this paper, a new algorithm, called Wasserstein Projected Gradient Descent (WPGD), for adversarial training is proposed. WPGD provides a simple way to obtain cost-sensitive robustness, resulting in a finer control of the robustness-accuracy trade-off. Moreover, WPGD solves an optimal transport problem on the output space of the network and it can efficiently discover directions where robustness is required, allowing to control the directional trade-off between accuracy and robustness. The proposed WPGD is validated in this work on image recognition tasks with different benchmark datasets and architectures. Moreover, real world-like datasets are often unbalanced: this paper shows that when dealing with such type of datasets, the performance of adversarial training are mainly affected in term of standard accuracy.

Gesture Recognition has a prominent importance in smart environment and home automation. Thanks to the availability of Machine Learning approaches it is possible for users to define gestures that can be associated with commands for the smart environment. In this paper we propose a Random Forest-based approach for Gesture Recognition of hand movements starting from wireless wearable motion capture data. In the presented approach, we evaluate different feature extraction procedures to handle gestures and data with different duration. To enhance reproducibility of our results and to foster research in the Gesture Recognition area, we share the dataset that we have collected and exploited for the present work.

In this paper we present a novel algorithm to solve the robot kinematic structure identification problem. Given a time series of data, typically obtained processing a set of visual observations, the proposed approach identifies the ordered sequence of links associated to the kinematic chain, the joint type interconnecting each couple of consecutive links, and the input signal influencing the relative motion. Compared to the state of the art, the proposed algorithm has reduced computational costs, and is able to identify also the joints' type sequence. 

On-line and phone banking frauds are responsible for millions of dollars loss every year. In this work, we propose a Machine Learning-based Decision Support System to automatically associate a risk factor to each transaction performed through an on-line/mobile banking system. The proposed approach has a hierarchical architecture: First, an unsupervised Machine Learning module is used to detect abnormal patterns or wrongly labeled transactions; then, a supervised module provides a risk factor for the transactions that were not marked as anomalies in the previous step. Our solution exploits personal and historical information about the user, statistics that describe online traffic generated on the online/mobile banking system, and features extracted from motives of the transactions. The proposed approach deals with dataset unbalancing effectively. Moreover, it has been validated on a large database of transactions and on-line traffic provided by an industrial partner.

Subjects affected by Type I Diabetes (T1D) are constantly confronted with the complicated problem of administering themselves an adequate amount of insulin, so as to keep their blood-glucose concentration in a nearly physiological range. Recently, powerful technological tools have been developed to better face this challenge, in particular the so-called Artificial Pancreas (AP). Unluckily, the AP actuator, an insulin pump, is subject to faults, with potential serious consequences for subjects' safety. This calls for the development of advanced fault detection (FD) methods, leveraging the unprecedented data availability in this application. In this paper we tackle the problem of detecting insulin pump malfunctioning using a model-free approach, so that the complex sub-task of identifying a model of patients physiology is avoided. Moreover, we employed unsupervised methods since labeled data are hardly available in practice. The adopted data-driven Anomaly Detection (AD) methods are Local Outlier Factor and Connectivity-based Outlier Factor. The methods are applied on a feature set able to account for the physiological dynamics of T1D patients. The proposed algorithms are tested on a synthetic dataset, generated using the “UVA/Padova Type 1 Diabetic Simulator”, an accurate nonlinear computer simulator of the T1D subject physiology. Both methods show precision ~75% and recall ~60%• The described approach is suitable both for embedding in medical devices, such as the AP, and implementation in cloud-based remote monitoring systems.

Smart production monitoring is a crucial activity in advanced manufacturing for quality, control and maintenance purposes. Advanced Monitoring Systems aim to detect anomalies and trends; anomalies are data patterns that have different data characteristics from normal instances, while trends are tendencies of production to move in a particular direction over time. In this work, we compare state-of-the-art ML approaches (ABOD, LOF, onlinePCA and osPCA) to detect outliers and events in high-dimensional monitoring problems. The compared anomaly detection strategies have been tested on a real industrial dataset related to a Semiconductor Manufacturing Etching process

The diffusion in Power Systems of distributed renewable energy resources, electric vehicles and controllable loads has made advanced monitoring systems fundamental to cope with the consequent disturbances in power flows; advanced monitoring systems can be employed for Anomaly Detection, Root Cause Analysis and Control purposes.

Several Machine Learning-based approaches have been developed in the past recent years to detect if a power system is running under anomalous conditions and, eventually, to classify such situation with respect to known problems.

One of the aspects that makes Power Systems challenging to be tackled, is that the monitoring has to be performed on streams of data that have a time series evolution; this issue is generally tackled by performing a features extraction procedure before the classication phase. The features extraction phase consists of translating the informative content of time series data into scalar quantities: such procedure may be a time-consuming step that requires the involvement of process experts to avoid loss of information in the making; moreover, extracted features designed to capture certain behaviors of the system, may not be informative under unseen conditions leading to poor monitoring performances.

A different type of data-driven approaches, that will be reviewed in this chapter, allow to perform classication directly on the raw time series data, avoiding the features extraction phase: among these approaches, Dynamic Time Warping and Symbolic-based methodologies have been widely applied in many application areas.

In the following, pros and cons of each approach will be discussed and practical implementation guidelines will be provided.

In the context of activity recognition, wearable devices arenowadays the preferable hardware thanks to their usability, user expe-rience and performances; at the same time, these devices present limi-tations in terms of computational capability and memory, which forcethe algorithm design to be at the same time ecient and simple. Inthis work, we adopt Symbolic Aggregate Approximation (SAX), a sym-bolic approach for information retrieval in time series data that allowsdimensionality and numerosity reduction; SAX is employed here, in com-bination with 1-Nearest Neighbor classier, to identify activity phases incontinuous repetitive activities from inertial time-series data. The pro-posed approach is validated on a public activity recognition dataset.