Publications Lists     [ all BibTeX ]  

Anomaly Detection is an unsupervised learning task aimed at detecting anomalous behaviors with respect to historical data. In particular, multivariate Anomaly Detection has an important role in many applications thanks to the capability of summarizing the status of a complex system or observed phenomenon with a single indicator (typically called ‘anomaly score’) and thanks to the unsupervised nature of the task that does not require human tagging. The Isolation Forest is one of the most commonly adopted algorithms in the field of Anomaly Detection due to its proven effectiveness and low computational complexity. A major problem affecting Isolation Forest is represented by the lack of interpretability, an effect of the inherent randomness governing the splits performed by the Isolation Trees, the building blocks of the Isolation Forest. In this paper, we propose effective yet computationally inexpensive methods to define feature importance scores at both global and local levels for the Isolation Forest. Moreover, we define a procedure to perform unsupervised feature selection for Anomaly Detection problems based on our interpretability method. Such a procedure also serves the purpose of tackling the challenging task of feature importance evaluation in unsupervised anomaly detection. We assess the performance on several synthetic and real-world datasets, including comparisons against state-of-the-art interpretability techniques, and make the code publicly available to enhance reproducibility and foster research in the field.

Data-driven Anomaly Detection approaches have received increasing attention in  many other application areas in the past fewyears as a tool for monitoring of complex systems in addition to classical univariatecontrol charts. Tree-based approaches have proven to be particularly effective when dealing with high-dimensional Anomaly Detection problems and with underlying non-gaussian data distributions. The most popular approach in this family is the Isolation Forest, which is currently one of the mostpopular choices for scientists and practitioners when dealing with Anomaly Detection tasks. The Isolation Forest represents a seminal algorithm upon which many extended approaches have been presented in the past years aiming at improving the original method or at dealing with peculiar applicationscenarios. In this work, we revise some of the most popular and powerful Tree-based approaches to Anomaly Detection (extensions of the Isolation Forest and other approaches), considering both batch and streaming datascenarios. This work will review several relevant aspects of the methods, like computational costs and interpretability traits. To help practitioners, we also report available relevant libraries and open implementations and we review real-world industrial applications of the considered approaches.

The monitoring of rotating machinery is an essential task in today's production processes. Currently, several machine learning and deep learning-based modules have achieved excellent results in fault detection and diagnosis. Nevertheless, to further increase user adoption and diffusion of such technologies, users and human experts must be provided with explanations and insights by the modules. Another issue is related, in most cases, with the unavailability of labeled historical data that makes the use of supervised models unfeasible. Therefore, a new approach for fault detection and diagnosis in rotating machinery is here proposed. The methodology consists of three parts: feature extraction, fault detection and fault diagnosis. In the first part, the vibration features in the time and frequency domains are extracted. Secondly, in the fault detection, the presence of fault is verified in an unsupervised manner based on anomaly detection algorithms. The modularity of the methodology allows different algorithms to be implemented. Finally, in fault diagnosis, Shapley Additive Explanations (SHAP), a technique to interpret black-box models, is used. Through the feature importance ranking obtained by the model explainability, the fault diagnosis is performed. Two tools for diagnosis are proposed, namely: unsupervised classification and root cause analysis. The effectiveness of the proposed approach is shown on three datasets containing different mechanical faults in rotating machinery. The study also presents a comparison between models used in machine learning explainability: SHAP and Local Depth-based Feature Importance for the Isolation Forest (Local- DIFFI). Lastly, an analysis of several state-of-art anomaly detection algorithms in rotating machinery is included.

Learning useful representations of complex data has been the subject of extensive research for many years. With the diffusion of Deep Neural Networks, Variational Autoencoders have gained lots of attention since they provide an explicit model of the data distribution based on an encoder/decoder architecture which is able to both generate images and encode them in a low-dimensional subspace. However, the latent space is not easily interpretable and the generation capabilities show some limitations since images typically look blurry and lack details. In this paper, we propose the Introspective Variational Classifier (IntroVAC), a model that learns interpretable latent subspaces by exploiting information from an additional label and provides improved image quality thanks to an adversarial training strategy.We show that IntroVAC is able to learn meaningful directions in the latent space enabling fine-grained manipulation of image attributes. We validate our approach on the CelebA dataset.

LEarning TO Rank (LETOR) algorithms are usually trained on annotated corpora where a single relevance label is assigned to each available document-topic pair. Within the Cranfield framework, relevance labels result from merging either multiple expertly curated or crowdsourced human assessments. In this paper, we explore how to train LETOR models with relevance judgments distributions (either real or synthetically generated) assigned to document-topic pairs instead of single-valued relevance labels. We propose five new probabilistic loss functions to deal with the higher expressive power provided by relevance judgments distributions and show how they can be applied both to neural and gradient boosting machine (GBM) architectures. Moreover, we show how training a LETOR model on a sampled version of the relevance judgments from certain probability distributions can improve its performance when relying either on traditional or probabilistic loss functions. Finally, we validate our hypothesis on real-world crowdsourced relevance judgments distributions. Overall, we observe that relying on relevance judgments distributions to train different LETOR models can boost their performance and even outperform strong baselines such as LambdaMART on several test collections.

The diffusion of the Industry 4.0 paradigm lead to the creation and collection of huge manufacturing datasets; such datasets contain for example measurements coming from physical sensors located in different equipment or even in different productive manufacturing organizations. Such large and heterogeneous datasets represent a challenge when aiming for developing data-driven approaches like Anomaly Detection or Predictive Maintenance. In this work we present a new approach for performing Anomaly Detection that is able to handle heterogeneous data coming from different equipment, work centers or production sites.

Recent results show that features of adversarially trained networks for classification, in addition to being robust, enable desirable properties such as invertibility. The latter property may seem counter-intuitive as it is widely accepted by the community that classification models should only capture the minimal information (features) required for the task. Motivated by this discrepancy, we investigate the dual relationship between Adversarial Training and Information Theory. We show that the Adversarial Training can improve linear transferability to new tasks, from which arises a new trade-off between transferability of representations and accuracy on the source task. We validate our results employing robust networks trained on CIFAR-10, CIFAR-100 and ImageNet on several datasets. Moreover, we show that Adversarial Training reduces Fisher information of representations about the input and of the weights about the task, and we provide a theoretical argument which explains the invertibility of deterministic networks without violating the principle of minimality. Finally, we leverage our theoretical insights to remarkably improve the quality of reconstructed images through inversion.

Deep Learning approaches have revolutionized in the past decade the field of Computer Vision and, as a consequence, they are having a major impact in Industry 4.0 applications like automatic defect classification. Nevertheless, additional data, beside the image/video itself, is typically never exploited in a defect classification module: this aspect, given the abundance of data in data-intensive manufacturing environments (like semiconductor manufacturing) represents a missed opportunity. In this work we present a use case related to Scanning Electron Microscope (SEM) images where we exploit a  Bayesian approach to improve defect classification. We validate our approach on a real-world case study and by employing modern Deep Learning architectures for classification.

Visual inspection has recently gained increasing importance in the manufacturing industry and is often addressed by means of learning methodologies applied to data obtained from specific lighting and camera system setups. The industrial scenario becomes particularly challenging when the inspection regards reflective objects, which may affect both the data acquisition and the classification decision process, thus limiting the overall performance. In this context, we observe that the dynamics of the reflected light is the key aspect to characterize these surfaces and needs to be accurately exploited to improve the performances of the learning algorithms. To this aim, we propose a combined model-based and data-driven approach designed to detect defects on the reflective surfaces of industrial products, captured as video sequences under coaxial structured illumination. Specifically, a tunable spatial-temporal descriptor of the evolution of the reflected light (Dynamic Evolution of the Light, DEL) is designed and employed within a Hybrid Learning (HL) framework, where the learning process of a Convolutional Neural Network (CNN) is driven by the model-based descriptor. This approach is also extended by adopting the similar in nature descriptor Dynamic Image. The proposed HL solutions are validated against a whole spectrum of state-of-the-art learning procedures and different descriptors. Experiments run on a dataset coming from an actual industrial scenario confirm the ability of DEL to accurately characterize reflective surfaces and the validity of the HL method, which shows remarkably better performance in fault detection even with respect to modern 3D- CNNs with comparable computational effort.

Particle accelerators are complex systems composed of multiple subsystems that must work together to produce high quality beams employed for physics experiments. A fault or an anomalous behaviour in one of such subsystems can lead to expensive downtime for the whole facility. Thus, it is of paramount importance to be able to promptly detect anomalies.Given the vast amount of streaming data generated by accelerator field sensors, Machine Learning (ML)-based tools are promising candidates for efficient monitoring of such systems: an approach based on unsupervised ML techniques exploiting the data from a Radio Frequency tuning system is here proposed. Feature importance is exploited to guide the definition of the optimal windowing for feature extraction. The proposed approach is here validated on real-world data related to the ALPI accelerator at Legnaro National Laboratories in Italy.

The advent of the Industrial Internet of Things (IIoT) has led to the availability of huge amounts of data, that can be used to train advanced Machine Learning algorithms to perform tasks such as Anomaly Detection, Fault Classification and Predictive Maintenance. Even though not all pieces of equipment are equipped with sensors yet, usually most of them are already capable of logging warnings and alarms occurring during operation. Turning this data, which is easy to collect, into meaningful information about the health state of machinery can have a disruptive impact on the improvement of efficiency and up-time. The provided dataset consists of a sequence of alarms logged by packaging equipment in an industrial environment. The collection includes data logged by 20 machines, deployed in different plants around the world, from 2019-02-21 to 2020-06-17. There are 154 distinct alarm codes, whose distribution is highly unbalanced. This data can be used to address the following tasks: 

1. Next alarm forecasting: this problem can be framed as a supervised multi-class classification task, or a binary classification task when a specific alarm code is considered.
2. Predicting alarms occurring in a future time frame: here the goal is to forecast the occurrence of certain alarm types in a future time window. Since many alarms can occur, this is a supervised multi-label classification.
3. Future alarm sequence prediction: here the goal is predicting an ordered sequence of future alarms, in a sequence-to-sequence forecasting scenario.
4. Anomaly Detection: the task is to detect abnormal equipment conditions, based on the pattern of alarms sequence. This task can be either unsupervised, if only the input sequence is considered, or supervised if future alarms are taken into account to assess whether or not there is an anomaly.

 All of the above tasks can also be studied from a continual learning perspective. Indeed, information about the serial code of the specific piece of equipment can be used to train the model; however, a scalable model should also be easy to apply to new machines, without the need of a new training from scratch. The collection and release of this dataset has been supported by the Regione Veneto project PreMANI (MANIFATTURA PREDITTIVA: progettazione, sviluppo e implementazione di soluzioni di Digital Manufacturing per la previsione della Qualita e la Manutenzione Intelligente - PREDICTIVE MAINTENANCE: design, development and implementation of Digital Manufacturing solutions for the intelligent quality and maintenance systems).

The last decade has seen tremendous improvements in technologies for Type 1 Diabetes (T1D) management, in particular the so-called artificial pancreas (AP), a wearable closed-loop device modulating insulin injection based on glucose sensor readings. Unluckily, the AP actuator, an insulin pump, is subject to failures, with potentially serious consequences for subject safety. This calls for the development of advanced monitoring systems, leveraging the unprecedented data availability. This paper tackles for the first time the problem of automatically detecting pump faults with multidimensional data-driven anomaly detection (AD) methodologies. The approach allows to avoid the subtask of identifying a physiological model, typical of model-based approaches. Furthermore, we employ unsupervised methods, removing the need of labeled data for training, hardly available in practice. The adopted data-driven AD methods are local outlier factor, connectivity-based outlier factor, and isolation forest. Moreover, we propose a modification of these methods to cope with the dynamic nature of the underlying problem. The algorithms were tuned and tested on: 1) two-synthetic 100-patients' data set, of one-month data each, generated using the ``UVA/Padova T1D Simulator,'' a large-scale nonlinear computer simulator of T1D subject physiology, largely adopted in AP research and accepted by the American Food and Drug Administration as a replacement of preclinical animal trials for AP and 2) a real 7-patients' data set consisting of one month in free-living conditions. The satisfactory accuracy of the proposed approach paves the way to the embedding of these methodologies in AP systems or their deployment in remote monitoring systems.

In many real-world applications of Machine Learning it is of paramount importance not only to provide accurate predictions, but also to ensure certain levels of robustness. Adversarial Training is a training procedure aiming at providing models that are robust to worst-case perturbations around predefined points. Unfortunately, one of the main issues in adversarial training is that robustness w.r.t. gradient-based attackers is always achieved at the cost of prediction accuracy. In this paper, a new algorithm, called Wasserstein Projected Gradient Descent (WPGD), for adversarial training is proposed. WPGD provides a simple way to obtain cost-sensitive robustness, resulting in a finer control of the robustness-accuracy trade-off. Moreover, WPGD solves an optimal transport problem on the output space of the network and it can efficiently discover directions where robustness is required, allowing to control the directional trade-off between accuracy and robustness. The proposed WPGD is validated in this work on image recognition tasks with different benchmark datasets and architectures. Moreover, real world-like datasets are often unbalanced: this paper shows that when dealing with such type of datasets, the performance of adversarial training are mainly affected in term of standard accuracy.

One of the main challenges in developing Machine Learning-based solutions for Semiconductor Manu-facturing is the high number of machines in the production and their differences, even when consideringchambers of the same machine; this poses a challenge in the scalability of Machine Learning-based so-lutions in this context, since the development of chamber-specific models for all equipment in the fab isunsustainable. In this work, we present a domain adaptation approach for Virtual Metrology (VM), one ofthe most successful Machine Learning-based technology in this context. The approach provides a commonVM model for two identical-in-design chambers whose data follow different distributions. The approach isbased on Domain-Adversarial Neural Networks and it has the merit of exploiting raw trace data, avoidingthe loss of information that typically affects VM modules based on features. The effectiveness of theapproach is demonstrated on real-world Etching.

To increase the classification, the rate of prediction based on existing models requires additional technique or in this case optimizing the model. Hyperparameter tuning is an optimization technique that evaluates and adjusts the free parameters that define the behaviour of classifiers. Data sets were classified practical with classifiers like SVM, k-NN, ANN and DA. To further improve the design efficiency, the secondary optimization level called hyperparameter tuning will be further investigated. DA, SVM, k-NN, decision tree (Tree), logistic regression (LR), random forest tree (RF) and neural network (NN) are evaluated. The k-NN provided 96.47% of the test sets with the best reliability in classifications. Bayesian optimization has been used to refine the hyperparameter; hence, standardize Euclidean distance metric with a k value of one is the ideal hyperparameters which could achieve classification performance of 97.16%.

This chapter demonstrates the pipeline from data collection until classifier models that achieve the best possible model in identifying the disparity between hunger states. The pre-processing segment describes the features of the data sets obtained by means of image processing. The method includes the simple moving average (SMA), downsizing factors, dynamic time warping (DTW) and clustering by the k-means method. This is to rationally assign the necessary significant information from the data collected and processed the images captured for demand feeder and fish motion as a synthesis for anticipating the state of fish starvation. The selection of features in this study takes place via the boxplot analysis and the principal component analysis (PCA) on dimensionality reduction. Finally, the validation of the hunger state will be addressed by comparing machine learning (ML) classifiers, namely the discriminant analysis (DA), support vector machine (SVM) and k-nearest neighbour (k-NN). The outcome in this chapter will validate the features from image processing as a tool for identifying the behavioural changes of the fish in school size.

Machine Learning-based Anomaly Detection (AD) approaches are efficient tools to monitor complexprocesses. One of the advantages of such approaches is that they provide a unique anomaly indicator,a quantitative index that captures the degree of ’outlierness’ of the process at hand considering possiblyhundreds or more variables at the same time, the typical scenario in semiconductor manufacturing. Oneof the drawback of such approaches is that Root Cause Analysis is not guided by the system itself. Inthis work, we show the effectiveness of a method, called DIFFI, to equip Isolation Forest, one of themost popular AD algorithms, with interpretability traits that can help corrective actions and knowledgeunderstanding. Such approach is validated on real world semiconductor manufacturing data related to aChemical Vapor Deposition process.

This book highlights the fundamental association between aquaculture and engineering in classifying fish hunger behaviour by means of machine learning techniques. Understanding the underlying factors that affect fish growth is essential, since they have implications for higher productivity in fish farms. Computer vision and machine learning techniques make it possible to quantify the subjective perception of hunger behaviour and so allow food to be provided as necessary. The book analyses the conceptual framework of motion tracking, feeding schedule and prediction classifiers in order to classify the hunger state, and proposes a system comprising an automated feeder system, image-processing module, as well as machine learning classifiers. Furthermore, the system substitutes conventional, complex modelling techniques with a robust, artificial intelligence approach. The findings presented are of interest to researchers, fish farmers, and aquaculture technologist wanting to gain insights into the productivity of fish and fish behaviour.

This chapter highlights the findings of the developmental monitoring systems for swimming pattern or motion analysis with regard to feeding behaviour. A benchmark for examining the framework on how scientists control fish in animal variable function factors was gathered and referred to gauge the adequate design in constructing a viable device. The validation of image processing and automated demand feeder to determine the results will also be considered, as a validation aspect between the system of tracking and the behaviour of the Lates calcarifer where the pixel intensity will be extracted as the features. The results of this chapter will enable the reader on the development of an integrated feeder scheme that consolidates surveillance scheme to identify the feeding behaviour and relation towards the specific growth rate (SGR).

Solutions for the quality control of metallic surfaces are proposed. Specifically, we study a deflectometric apparatus based on coaxial structured light and the related algorithmic procedure, which is able to detect the faulty surface of a sample captured by a video sequence. First, by considering the metallic surface a dynamic scene illuminated under different light conditions, we develop the descriptor residuals of linear evolution of light (RLEL) that extracts the defectiveness information starting from the movement of the object without explicitly considering the physical characteristics of the light structure. Then, leveraging on RLEL, we present a hybrid learning (HL) technique capable of overcoming the data-driven approach used in classic deep learning (DL). By exploiting a multisteps training process, we combine the model-based descriptor RLEL and a classical data-driven convolutional neural network (CNN) to obtain an unconventional gray-box CNN, which exceeds the performance of popular DL solutions such as 3-D-inception and 3-D-residual DL networks. Remarkably, HL also shows its validity in comparing the performance of the same network structures trained not in a hybrid way, namely without the injection of the model-based information given by RLEL.

Measuring systems are becoming increasingly sophisticated in order to tackle the challenges of modern industrial problems. In particular, the Multiphase Flow Meter (MPFM) combines different sensors and data fusion techniques to estimate quantities that are difficult to be measured like the water or gas content of a multiphase flow, coming from an oil well. The evaluation of the flow composition is essential for the well productivity prediction and management, and for this reason, the quantification of the meter measurement quality is crucial. While instrument complexity is increasing, demands for confidence levels in the provided measures are becoming increasingly more common. In this work, we propose an Anomaly Detection approach, based on unsupervised Machine Learning algorithms, that enables the metrology system to detect outliers and to provide a statistical level of confidence in the measures. The proposed approach, called AD4MPFM (Anomaly Detection for Multiphase Flow Meters), is designed for embedded implementation and for multivariate time-series data streams. The approach is validated both on real and synthetic data.

In knowledge-based systems, besides obtaining good output prediction accuracy, it is crucial to understand the subset of input variables that have most influence on the output, with the goal of gaining deeper insight into the underlying process. These requirements call for logistic model estimation techniques that provide a sparse solution, i.e., where coefficients associated with non-important variables are set to zero. In this work we compare the performance of two methods: the first one is based on the well known Least Absolute Shrinkage and Selection Operator (LASSO) which involves regularization with an ?1 norm; the second one is the Relevance Vector Machine (RVM) which is based on a Bayesian implementation of the linear logistic model. The two methods are extensively compared in this paper, on real and simulated datasets. Results show that, in general, the two approaches are comparable in terms of prediction performance. RVM outperforms the LASSO both in term of structure recovery (estimation of the correct non-zero model coefficients) and prediction accuracy when the dimensionality of the data tends to increase. However, LASSO shows comparable performance to RVM when the dimensionality of the data is much higher than number of samples that is p>>n 

The identification of relevant parameters that could describe the state of fish hunger is vital for ensuring the appropriate allocation of food to the fish. The establishment of these relevant parameters is non-trivial, particularly when developing an automated demand feeder system. The present inquiry is being undertaken to determine the hunger state of Lates calcarifer. For data collection, a video analysis system is used, and the video was taken all day, where the fish was fed by an automatic feeding system. Sixteen characteristics of the raw data set have been extracted through feature engineering for 0.5 min, 1.0 min, 1.5 min and 2.0 min, respectively, in accordance with the mean, peak, minimum and variability of each of the different time window scales. Furthermore, the features extracted have been evaluated through principal component analysis (PCA) both for dimension reduction and PCA with varimax rotation. The details were then categorized using support vector machine (SVM), K-NN and random forest tree (RF) classifiers. The best identification accuracy was shown with eight described features in the varimax-based PCA. The forecast results based on the K-NN model built on selected data characteristics showed a level of 96.5% indicating that the characteristics analysed were crucial to classifying the actions of hunger among fisheries.

We propose a Deep Learning (DL)-based approach for production performance forecasting in fresh products packaging. On the one hand, this is a very demanding scenario where high throughput is mandatory; on the other, due to strict hygiene requirements, unexpected downtime caused by packaging machines can lead to huge product waste. Thus, our aim is predicting future values of key performance indexes such as Machine Mechanical Efficiency (MME) and Overall Equipment Effectiveness (OEE). We address this problem by leveraging DL-based approaches and historical production performance data related to measurements, warnings and alarms. Different architectures and prediction horizons are analyzed and compared to identify the most robust and effective solutions. We provide experimental results on a real industrial case, showing advantages with respect to current policies implemented by the industrial partner both in terms of forecasting accuracy and maintenance costs. The proposed architecture is shown to be effective on a real case study and it enables the development of predictive services in the area of Predictive Maintenance and Quality Monitoring for packaging equipment providers.

In modern manufacturing scenarios, detecting anomalies in production systems is pivotal to keep high-quality standards and reduce costs. Even in the Industry 4.0 context, realworld monitoring systems are often simple and based on the use of multiple univariate control charts. Data-driven technologies offer a whole range of tools to perform multivariate data analysis that allow to implement more effective monitoring procedures. However, when dealing with complex data, common data-driven methods cannot be directly used, and a feature extraction phase must be employed. Feature extraction is a particularly critical operation, especially in anomaly detection tasks, and it is generally associated with information loss and low scalability. In this paper we consider the task of Anomaly Detection with twodimensional, image-like input data, by adopting a Deep Learningbased monitoring procedure, that makes use of convolutional autoencoders. The procedure is tested on real Optical Emission Spectroscopy data, typical of semiconductor manufacturing. The results show that the proposed approach outperforms classical feature extraction procedures.

Gesture Recognition has a prominent importance in smart environment and home automation. Thanks to the availability of Machine Learning approaches it is possible for users to define gestures that can be associated with commands for the smart environment. In this paper we propose a Random Forest-based approach for Gesture Recognition of hand movements starting from wireless wearable motion capture data. In the presented approach, we evaluate different feature extraction procedures to handle gestures and data with different duration. To enhance reproducibility of our results and to foster research in the Gesture Recognition area, we share the dataset that we have collected and exploited for the present work.

Two methods based on Regression are presented to determine the moisture content of items, e.g. clothes and the like, which are introduced in a household laundry dryer appliance. The aim of this work is to develop Soft Sensors (SS) for a household Heat Pump Washer-Dryer (WD-HP) to provide an estimation of the desired signal (the laundry moisture during drying) avoiding the use of additional physical sensors with the goal of improving the current performance in terms of precision and energy consumption of the automatic drying cycle end and using the machine equipment already available. On an algorithmic point of view, the SS developed in this work exploits regularization methods and Genetic Programming for Symbolic Regression in order to find suitable models for the purpose at hand. Proposed approaches have been tested on real data provided by an industrial partner.

Industry 4.0 encapsulates methods, technologies, and procedures that transform data into informed decisions and added value inan industrial context. In this regard, technologies such as Virtual Metrology or Soft Sensing have gained much interest in thelast two decades due to their ability to provide valuable knowledge for production purposes at limited added expense. However,these technologies have struggled to achieve wide-scale industrial adoption, largely due to the challenges associated with handlingcomplex data structures and the feature extraction phase of model building. This phase is generally hand-engineered and basedon specific domain knowledge, making it time consuming, difficult to automate, and prone to loss of information, thus ultimatelylimiting portability. Moreover, in the presence of complex data structures, such as 2-dimensional input data, there are no establishedprocedures for feature extraction. In this paper, we present a Deep Learning approach for Virtual Metrology, called DeepVM,that exploits semi-supervised feature extraction based on Convolutional Autoencoders. The proposed approach is demonstratedusing a real world semiconductor manufacturing dataset where the Virtual Metrology input data is 2-dimensional Optical EmissionSpectrometry data. The feature extraction method is tested with different types of state-of-the-art autoencoder.

In the past recent years, Machine Learning methodologies have been applied in countless application areas. In particular, they play a key role in enabling Industry4.0. However, one of the main obstacles to the diffusion of Machine Learning-based applications is related to the lack of interpretability of most of these methods. In this work, we propose an approach for defining a ‘feature importance’ in Anomaly Detection problems. Anomaly Detection is an important Machine Learning task that has an enormous applicability in industrial scenarios. Indeed, it is extremely relevant for the purpose of quality monitoring. Moreover, it is often the first step towards the design of a Machine Learning based smart monitoring solutions because Anomaly Detection can be implemented without the need of labelled data. The proposed feature importance evaluation approach is designed for Isolation Forest, one of the most commonly used algorithm for Anomaly Detection. The efficacy of the proposed method is tested on synthetic and real industrial datasets.

In an automated demand feeder system, underlining the parameters that contribute to fish hunger is crucial in order to facilitate an optimised food allocation to the fish. The present investigation is carried out to classify the hunger state of Lates calcarifer. A video surveillance technique is employed for data collection. The video was taken throughout the daytime, and the fish were fed through an automated feeding system. It was demonstrated through this investigation that the use of such automated system does contribute towards a higher specific growth rate percentage of body weight as well as the total length by approximately 26.00% and 15.00%, respectively against the conventional time-based method. Sixteen features were feature engineered from the raw dataset into window sizes ranging from 0.5?min, 1.0?min, 1.5?min and 2.0?min, respectively coupled with the mean, maximum, minimum and variance for each of the distinctive temporal window sizes. In addition, the extracted features were analysed through Principal Component Analysis (PCA) for dimensionality reduction as well as PCA with varimax rotation. The data were then classified using a Support Vector Machine (SVM), k-Nearest Neighbor (k-NN) and Random Forest Tree models. It was demonstrated that the varimax based PCA yielded the highest classification accuracy with eight identified features. The prediction results based of the developed k-NN model on the selected features on the test data exhibited a classification rate of 96.5% was achieved suggesting that the features examined are non-trivial in classifying the fish hunger behaviour.

Embedding household appliances with smart capabilities is becoming common practice among major fabric-care producers that seek competitiveness on the market by providing more efficient and easy-to-use products. In Vertical Axis Washing Machines (VA-WM), knowing the laundry composition is fundamental to setting the washing cycle properly with positive impact both on energy/water consumption and on washing performance. An indication of the load typology composition (cotton, silk, etc.) is typically provided by the user through a physical selector that, unfortunately, is often placed by the user on the most general setting due to the discomfort of manually changing configurations. An automated mechanism to determine such key information would thus provide increased user experience, better washing performance, and reduced consumption; for this reason, we present here a data-driven soft sensor that exploits physical measurements already available on board a commercial VA-WM to provide an estimate of the load typology through a machine-learning-based statistical model of the process. The proposed method is able to work in a resource-constrained environment such as the firmware of a VA-WM.

The aim is to develop soft sensors (SSs) to provide an estimation of the laundry moisture of clothes introduced in a household Heat Pump Washer–Dryer (WD-HP) appliance. The developed SS represents a cost-effective alternative to physical sensors, and it aims at improving the WD-HP performance in terms of drying process efficiency of the automatic drying cycle. To this end, we make use of appropriate Machine Learning models, which are derived by means of Regularization and Symbolic Regression methods. These methods connect easy-to-measure variables with the laundry moisture content, which is a difficult and costly to measure variable. Thanks to the use of SSs, the laundry moisture estimation during the drying process is effectively available. The proposed models have been tested by exploiting real data through an experimental test campaign on household drying machines.

Emotion Classification (EC) aims at assigning an emotion label to a textual document with two inputs – a set of emotion labels (e.g. anger, joy, sadness) and a document collection. The best performing approaches for EC are dictionary-based and suffer from two main limitations: (i) the out-of-vocabulary (OOV) keywords problem and (ii) they cannot be used across heterogeneous domains. In this work, we propose a way to overcome these limitations with a supervised approach based on TF-IDF indexing and Multinomial Linear Regression with Elastic-Net regularization to extract an emotion lexicon and classify short documents from diversified domains. We compare the proposed approach to state-of-the-art methods for document representation and classification by running an extensive experimental study on two shared and heterogeneous data sets.

Exploiting the huge amount of data collected by industries is definitely one of the main challenges of the so-called Big Data era. In this sense, Machine Learning has gained growing attention in the scientific community, as it allows to extract valuable information by means of statistical predictive models trained on historical process data. In Semiconductor Manufacturing, one of the most extensively employed data-driven applications is Virtual Metrology, where a costly or unmeasurable variable is estimated by means of cheap and easy to obtain measures that are already available in the system. Often, these measures are multi-dimensional, so traditional Machine Learning algorithms cannot handle them directly. Instead, they require feature extraction, that is a preliminary step where relevant information is extracted from raw data and converted into a design matrix. Features are often hand-engineered and based on specific domain knowledge. Moreover, they may be difficult to scale and prone to information loss, affecting the effectiveness and maintainability of machine learning procedures. In this paper, we present a Deep Learning method for semi-supervised feature extraction based on Convolutional Autoencoders that is able to overcome the aforementioned problems. The proposed method is tested on a real dataset for Etch rate estimation. Optical Emission Spectrometry data, that exhibit a complex bi-dimensional time and wavelength evolution, are used as input.

In semiconductor manufacturing, metrology is generally a high cost, non-value added operation that impacts significantly on cycle time. As such, reducing wafer metrology continues to be a major target in semiconductor manufacturing efficiency initiatives. Data-driven spatial dynamic sampling methodologies are here compared. Such strategies aim at minimizing the number of sites that need to be measured across a wafer surface while maintaining an acceptable level of wafer profile reconstruction accuracy. The Spatial Dynamic Sampling approaches are based on analyzing historical metrology data to determine, from a set of candidate wafer sites, the minimum set of sites that need to be monitored in order to reconstruct the full wafer profile using statistical regression techniques. Spatial Dynamic sampling is then implemented in various strategies that guarantee coverage of all the possible sites in a given set of process iteration. In this way, the risk of not detecting previously unseen process behavior is mitigated. In this work, we demonstrate the efficacy of spatial dynamic sampling methodologies using both simulation studies and metrology data from a semiconductor manufacturing process.

On-line and phone banking frauds are responsible for millions of dollars loss every year. In this work, we propose a Machine Learning-based Decision Support System to automatically associate a risk factor to each transaction performed through an on-line/mobile banking system. The proposed approach has a hierarchical architecture: First, an unsupervised Machine Learning module is used to detect abnormal patterns or wrongly labeled transactions; then, a supervised module provides a risk factor for the transactions that were not marked as anomalies in the previous step. Our solution exploits personal and historical information about the user, statistics that describe online traffic generated on the online/mobile banking system, and features extracted from motives of the transactions. The proposed approach deals with dataset unbalancing effectively. Moreover, it has been validated on a large database of transactions and on-line traffic provided by an industrial partner.

In semiconductor manufacturing, metrology is generally a high cost nonvalue-added operation that significantly impacts on cycle time. As such, reducing wafer metrology continues to be a major target in semiconductor manufacturing efficiency initiatives. A novel data-driven spatial dynamic sampling methodology is presented that minimizes the number of sites that need to be measured across a wafer surface while maintaining an acceptable level of wafer profile reconstruction accuracy. The methodology is based on analyzing historical metrology data using forward selection component analysis (FSCA) to determine, from a set of candidate wafer sites, the minimum set of sites that need to be monitored in order to reconstruct the full wafer profile using statistical regression techniques. Dynamic sampling is then implemented by clustering unmeasured sites in accordance with their similarity to the FSCA selected sites and temporally selecting a different sample from each cluster. In this way, the risk of not detecting previously unseen process behavior is mitigated. We demonstrate the efficacy of the proposed methodology using both simulation studies and metrology data from a semiconductor manufacturing process.

Fabric care home appliances are pervasive inhouses worldwide and manufactures are constantly working for improving product performance, efficiency, and usability. From a manufacturing perspective, increase of performancehas to be attained while minimizing the increase of production costs. In this context, a Soft Sensor for estimating the clothes load weight in a horizontal axis household washing machines ishere presented. The proposed Soft Sensor is based on Machine Learning approaches. Several methodologies, both time-seriesand feature-based, are employed and compared. The approach has been tested on real world data on commercial household washing machines.

Subjects affected by Type I Diabetes (T1D) are constantly confronted with the complicated problem of administering themselves an adequate amount of insulin, so as to keep their blood-glucose concentration in a nearly physiological range. Recently, powerful technological tools have been developed to better face this challenge, in particular the so-called Artificial Pancreas (AP). Unluckily, the AP actuator, an insulin pump, is subject to faults, with potential serious consequences for subjects' safety. This calls for the development of advanced fault detection (FD) methods, leveraging the unprecedented data availability in this application. In this paper we tackle the problem of detecting insulin pump malfunctioning using a model-free approach, so that the complex sub-task of identifying a model of patients physiology is avoided. Moreover, we employed unsupervised methods since labeled data are hardly available in practice. The adopted data-driven Anomaly Detection (AD) methods are Local Outlier Factor and Connectivity-based Outlier Factor. The methods are applied on a feature set able to account for the physiological dynamics of T1D patients. The proposed algorithms are tested on a synthetic dataset, generated using the “UVA/Padova Type 1 Diabetic Simulator”, an accurate nonlinear computer simulator of the T1D subject physiology. Both methods show precision ~75% and recall ~60%• The described approach is suitable both for embedding in medical devices, such as the AP, and implementation in cloud-based remote monitoring systems.

In laundry treatment appliances, the weight ofthe laundry loaded by the user inside the drum dramaticallyaffects the operating behavior. Therefore, it is important toobtain a good estimate of the said quantity in order tocorrectly configure the machine before the washing/dryingstarts. In Vertical Axis Washing Machinesthe laundry weightis computed by exploiting the quantity of water absorbed bythe clothes. However, such approach does not grant accurateresults because the water absorption depends on the clothesfabric. For this reason, we propose a Soft Sensing approachfor weight estimation that exploits the information obtainedfrom physical sensors available on board without added costs.Data-driven Soft Sensors are developed, where, using MachineLearning techniques, a statistical model of the phenomenon ofinterest is created from a set of sample data.

The laundry weight of the loaded in the drum of a laundry treatment machine is an important piece of information; laundry weight can be used to set various washing/drying cycle parameters and to optimize performances and efficiency. Unfortunately, dedicated weight sensors cannot be included in consumer laundry equipment given the related costs. For this reason, we present in this work a soft sensor approach for estimating laundry weight based on sensors already in place in a laundry treatment equipment; in particular, we consider here a heat pump tumble dryer as case study. The proposed soft sensor is based on regularization, a popular approach in Machine Learning to provide models without overfitting the training data. Different studies are provided in this work, by considering different constrains on timing and complexity of the Soft Sensor solution. The developed Soft Sensors have been tested on laboratory data provided by an industrial partner.

The goal of Aspect-Based Sentiment Analysis is to identify opinions regarding specific targets and the corresponding sentiment polarity in a document. The proposed approach is designed for real-world scenarios, where the amount of available information and annotated data is often too limited to train supervised models. We focus on the two core tasks of Aspect-Based Sentiment Analysis: aspect and sentiment polarity classification. The first task – which consists in the identification of the opinion targets in a document – is tackled by means of a weakly-supervised technique based on Non-negative Matrix Factorization. This strategy allows users to easily embed some a priori domain knowledge by means of short seed terms lists. Experimental results on publicly available data sets related to online reviews suggest that the proposed approach is very flexible and can be easily adapted to different languages and domains.

Smart production monitoring is a crucial activity in advanced manufacturing for quality, control and maintenance purposes. Advanced Monitoring Systems aim to detect anomalies and trends; anomalies are data patterns that have different data characteristics from normal instances, while trends are tendencies of production to move in a particular direction over time. In this work, we compare state-of-the-art ML approaches (ABOD, LOF, onlinePCA and osPCA) to detect outliers and events in high-dimensional monitoring problems. The compared anomaly detection strategies have been tested on a real industrial dataset related to a Semiconductor Manufacturing Etching process

Advanced Monitoring Systems are fundamental in advanced manufacturing for control, quality and maintenance purposes. Nowadays, with the increasing availability of data in production and equipment, the need for high-dimensional Anomaly Detection techniques is thriving; anomalies are data patterns that have different data characteristics from normal production instances and that may be associated with faults or drifts in production. Tools for dealing with high-dimensional monitoring problems are provided by Machine Learning: in this paper, we test the performance of a state-of-the-art anomaly detection technique, called Isolation Forest, on a real industrial dataset related to Etching, one of the most important semiconductor manufacturing process. The monitoring has been performed exploiting Optical Spectroscopy Data.

Virtual Metrology is one of the most prominentAdvanced Process Control applications in SemiconductorManufacturing. The goal of Virtual Metrology is to provideestimations of quantities that are important for production andto assess process quality, but are costly or impossible to bemeasured. Virtual Metrology solutions are based on MachineLearning approaches. The bottleneck of developing VirtualMetrology solutions is generally the feature extraction phase thatcan be time-consuming, and can deeply affect the estimationperformance. In particular, in presence of data with additionaldimensions, such as time, feature extraction is typicallyperformed by means of heuristic approaches that may pickfeatures with poor predictive capabilities. In this work, wepropose the usage of modern Deep Learning approaches tobypass manual feature extraction and to provide highperformanceautomatic Virtual Metrology modules. Theproposed methodology is tested on a real industrial datasetrelated to Etching. The dataset at hand contains OpticalEmission Spectroscopy data and it is paradigmatic of the featureextraction problem under examination.

The invention relates to a method for the determination of a laundry weight in a laundry treatment appliance comprising: Selecting a laundry program in the laundry treatment appliance; Starting the selected laundry program; Sensing a plurality of parameters indicating operating conditions of the laundry treatment appliance during the laundry program; and Predicting a weight of the laundry present within the laundry treatment appliance based on said plurality of parameters by means of a data-driven soft sensor.

The diffusion in Power Systems of distributed renewable energy resources, electric vehicles and controllable loads has made advanced monitoring systems fundamental to cope with the consequent disturbances in power flows; advanced monitoring systems can be employed for Anomaly Detection, Root Cause Analysis and Control purposes.

Several Machine Learning-based approaches have been developed in the past recent years to detect if a power system is running under anomalous conditions and, eventually, to classify such situation with respect to known problems.

One of the aspects that makes Power Systems challenging to be tackled, is that the monitoring has to be performed on streams of data that have a time series evolution; this issue is generally tackled by performing a features extraction procedure before the classication phase. The features extraction phase consists of translating the informative content of time series data into scalar quantities: such procedure may be a time-consuming step that requires the involvement of process experts to avoid loss of information in the making; moreover, extracted features designed to capture certain behaviors of the system, may not be informative under unseen conditions leading to poor monitoring performances.

A different type of data-driven approaches, that will be reviewed in this chapter, allow to perform classication directly on the raw time series data, avoiding the features extraction phase: among these approaches, Dynamic Time Warping and Symbolic-based methodologies have been widely applied in many application areas.

In the following, pros and cons of each approach will be discussed and practical implementation guidelines will be provided.

In this paper an approach to deal with Predictive Maintenance (PdM) problems with time-series data is discussed. PdM is a important approach to tackle maintenance and it is gaining an increasing attention in advanced manufacturing to minimize scrap materials, downtime, and associated costs. PdM approaches are generally based on Machine Learning tools that require the availability of historical process and maintenance data. Given the exponential growth in data logging in modern equipment, time series dataset are increasingly available in PdM applications. To exploit time series data for PdM, a functional learning methodology, namely Supervised Aggregative Feature Extraction (SAFE), is here employed on a semiconductor manufacturing maintenance problem.

In many applications, and especially thosewhere batch processes are involved, a target scalar outputof interest is often dependent on one or more time seriesof data. With the exponential growth in data logging inmodern industries such time series are increasingly availablefor statistical modeling in soft sensing applications. In orderto exploit time series data for predictive modelling, it isnecessary to summarise the information they contain as aset of features to use as model regressors. Typically thisis done in an unsupervised fashion using simple techniquessuch as computing statistical moments, principalcomponents or wavelet decompositions, often leading tosignificant information loss and hence suboptimal predictivemodels. In this paper, a functional learning paradigm isexploited in a supervised fashion to derive continuous,smooth estimates of time series data (yielding aggregatedlocal information), while simultaneously estimating a continuousshape function yielding optimal predictions. Theproposed Supervised Aggregative Feature Extraction (SAFE)methodology can be extended to support nonlinear predictivemodels by embedding the functional learning framework ina Reproducing Kernel Hilbert Spaces setting. SAFE has anumber of attractive features including closed form solutionand the ability to explicitly incorporate first and secondorder derivative information. Using simulation studies and apractical semiconductor manufacturing case study we highlightthe strengths of the new methodology with respect tostandard unsupervised feature extraction approaches.

A new Bayesian approach to linear system identification
has been proposed in a series of recent papers. The
main idea is to frame linear system identification as predictor
estimation in an infinite dimensional space, with the aid of
regularization/Bayesian techniques. This approach guarantees
the identification of stable predictors based on the prediction
error minimization. Unluckily, the stability of the predictors
does not guarantee the stability of the impulse response of
the system. In this paper we propose and compare various
techniques to guarantee that the final model identified following
this Bayesian approach is stable. First, we consider the socalled
“LMI - constraint” approach and adapt it to constrain
the eigenvalues of the estimated model within the unit circle.
A second possibility which is being considered is to add to the
“classic” Stable-Spline algorithm a penalty term, depending on
the maximum absolute value of the eigenvalue of the system.
This last technique has the advantage of being integrated
directly inside the pre-existing optimization problem and not to
simply post-process the estimated model to guarantee stability.
Finally, we considered a Monte Carlo Markov Chain approach
sampling in both the space of hyper-parameters and of impulse
responses. Simulations results comparing these techniques will
be provided.

In this paper a multiple classifier machine learning methodology for Predictive Maintenance (PdM) is presented. PdM is a prominent strategy for dealing with maintenance issues given the increasing need to minimize downtime and associated costs. One of the challenges with PdM is generating so called ’health factors’ or quantitative indicators of the status of a system associated with a given maintenance issue, and determining their relationship to operating costs and failure risk. The proposed PdM methodology allows dynamical decision rules to be adopted for maintenance management and can be used with high-dimensionaland censored data problems. This is achieved by training multiple classification modules with different prediction horizons to provide different performance trade-offs in terms of frequency of unexpected breaks and unexploited lifetime and then employing this information in an operating cost based maintenance decision system to minimise expected costs. The effectiveness of the methodology is demonstrated using a simulated example and a benchmark semiconductor manufacturing maintenance problem.

Slow release drugs must be manufactured to meettarget speci?cations with respect to dissolution curve pro?les.In this paper we consider the problem of identifying thedrivers of dissolution curve variability of a drug from historicalmanufacturing data. Several data sources are considered: rawmaterial parameters, coating data, loss on drying and pellet sizestatistics. The methodology employed is to develop predictivemodels using LASSO, a powerful machine learning algorithmfor regression with high-dimensional datasets. LASSO providessparse solutions facilitating the identi?cation of the most importantcauses of variability in the drug fabrication process.The proposed methodology is illustrated using manufacturingdata for a slow release drug.