Adversarial Training has proved to be an effective training paradigm to enforce robustness against adversarial examples in modern neural network architectures. Despite many efforts, explanations of the foundational principles underpinning the effectiveness of Adversarial Training are limited and far from being widely accepted by the Deep Learning community. In this paper, we describe surprising properties of adversarially-trained models, shedding light on mechanisms through which robustness against adversarial attacks is implemented. Moreover, we highlight limitations and failure modes affecting these models that were not discussed by prior works. We conduct extensive analyses on a wide range of architectures and datasets, performing a deep comparison between robust and natural models.

Anomaly Detection is an unsupervised learning task aimed at detecting anomalous behaviors with respect to historical data. In particular, multivariate Anomaly Detection has an important role in many applications thanks to the capability of summarizing the status of a complex system or observed phenomenon with a single indicator (typically called ‘anomaly score’) and thanks to the unsupervised nature of the task that does not require human tagging. The Isolation Forest is one of the most commonly adopted algorithms in the field of Anomaly Detection due to its proven effectiveness and low computational complexity. A major problem affecting Isolation Forest is represented by the lack of interpretability, an effect of the inherent randomness governing the splits performed by the Isolation Trees, the building blocks of the Isolation Forest. In this paper, we propose effective yet computationally inexpensive methods to define feature importance scores at both global and local levels for the Isolation Forest. Moreover, we define a procedure to perform unsupervised feature selection for Anomaly Detection problems based on our interpretability method. Such a procedure also serves the purpose of tackling the challenging task of feature importance evaluation in unsupervised anomaly detection. We assess the performance on several synthetic and real-world datasets, including comparisons against state-of-the-art interpretability techniques, and make the code publicly available to enhance reproducibility and foster research in the field.

Data-driven Anomaly Detection approaches have received increasing attention in  many other application areas in the past fewyears as a tool for monitoring of complex systems in addition to classical univariatecontrol charts. Tree-based approaches have proven to be particularly effective when dealing with high-dimensional Anomaly Detection problems and with underlying non-gaussian data distributions. The most popular approach in this family is the Isolation Forest, which is currently one of the mostpopular choices for scientists and practitioners when dealing with Anomaly Detection tasks. The Isolation Forest represents a seminal algorithm upon which many extended approaches have been presented in the past years aiming at improving the original method or at dealing with peculiar applicationscenarios. In this work, we revise some of the most popular and powerful Tree-based approaches to Anomaly Detection (extensions of the Isolation Forest and other approaches), considering both batch and streaming datascenarios. This work will review several relevant aspects of the methods, like computational costs and interpretability traits. To help practitioners, we also report available relevant libraries and open implementations and we review real-world industrial applications of the considered approaches.

The monitoring of rotating machinery is an essential task in today's production processes. Currently, several machine learning and deep learning-based modules have achieved excellent results in fault detection and diagnosis. Nevertheless, to further increase user adoption and diffusion of such technologies, users and human experts must be provided with explanations and insights by the modules. Another issue is related, in most cases, with the unavailability of labeled historical data that makes the use of supervised models unfeasible. Therefore, a new approach for fault detection and diagnosis in rotating machinery is here proposed. The methodology consists of three parts: feature extraction, fault detection and fault diagnosis. In the first part, the vibration features in the time and frequency domains are extracted. Secondly, in the fault detection, the presence of fault is verified in an unsupervised manner based on anomaly detection algorithms. The modularity of the methodology allows different algorithms to be implemented. Finally, in fault diagnosis, Shapley Additive Explanations (SHAP), a technique to interpret black-box models, is used. Through the feature importance ranking obtained by the model explainability, the fault diagnosis is performed. Two tools for diagnosis are proposed, namely: unsupervised classification and root cause analysis. The effectiveness of the proposed approach is shown on three datasets containing different mechanical faults in rotating machinery. The study also presents a comparison between models used in machine learning explainability: SHAP and Local Depth-based Feature Importance for the Isolation Forest (Local- DIFFI). Lastly, an analysis of several state-of-art anomaly detection algorithms in rotating machinery is included.

Learning useful representations of complex data has been the subject of extensive research for many years. With the diffusion of Deep Neural Networks, Variational Autoencoders have gained lots of attention since they provide an explicit model of the data distribution based on an encoder/decoder architecture which is able to both generate images and encode them in a low-dimensional subspace. However, the latent space is not easily interpretable and the generation capabilities show some limitations since images typically look blurry and lack details. In this paper, we propose the Introspective Variational Classifier (IntroVAC), a model that learns interpretable latent subspaces by exploiting information from an additional label and provides improved image quality thanks to an adversarial training strategy.We show that IntroVAC is able to learn meaningful directions in the latent space enabling fine-grained manipulation of image attributes. We validate our approach on the CelebA dataset.

LEarning TO Rank (LETOR) algorithms are usually trained on annotated corpora where a single relevance label is assigned to each available document-topic pair. Within the Cranfield framework, relevance labels result from merging either multiple expertly curated or crowdsourced human assessments. In this paper, we explore how to train LETOR models with relevance judgments distributions (either real or synthetically generated) assigned to document-topic pairs instead of single-valued relevance labels. We propose five new probabilistic loss functions to deal with the higher expressive power provided by relevance judgments distributions and show how they can be applied both to neural and gradient boosting machine (GBM) architectures. Moreover, we show how training a LETOR model on a sampled version of the relevance judgments from certain probability distributions can improve its performance when relying either on traditional or probabilistic loss functions. Finally, we validate our hypothesis on real-world crowdsourced relevance judgments distributions. Overall, we observe that relying on relevance judgments distributions to train different LETOR models can boost their performance and even outperform strong baselines such as LambdaMART on several test collections.

The diffusion of the Industry 4.0 paradigm lead to the creation and collection of huge manufacturing datasets; such datasets contain for example measurements coming from physical sensors located in different equipment or even in different productive manufacturing organizations. Such large and heterogeneous datasets represent a challenge when aiming for developing data-driven approaches like Anomaly Detection or Predictive Maintenance. In this work we present a new approach for performing Anomaly Detection that is able to handle heterogeneous data coming from different equipment, work centers or production sites.

Recent results show that features of adversarially trained networks for classification, in addition to being robust, enable desirable properties such as invertibility. The latter property may seem counter-intuitive as it is widely accepted by the community that classification models should only capture the minimal information (features) required for the task. Motivated by this discrepancy, we investigate the dual relationship between Adversarial Training and Information Theory. We show that the Adversarial Training can improve linear transferability to new tasks, from which arises a new trade-off between transferability of representations and accuracy on the source task. We validate our results employing robust networks trained on CIFAR-10, CIFAR-100 and ImageNet on several datasets. Moreover, we show that Adversarial Training reduces Fisher information of representations about the input and of the weights about the task, and we provide a theoretical argument which explains the invertibility of deterministic networks without violating the principle of minimality. Finally, we leverage our theoretical insights to remarkably improve the quality of reconstructed images through inversion.

Deep Learning approaches have revolutionized in the past decade the field of Computer Vision and, as a consequence, they are having a major impact in Industry 4.0 applications like automatic defect classification. Nevertheless, additional data, beside the image/video itself, is typically never exploited in a defect classification module: this aspect, given the abundance of data in data-intensive manufacturing environments (like semiconductor manufacturing) represents a missed opportunity. In this work we present a use case related to Scanning Electron Microscope (SEM) images where we exploit a  Bayesian approach to improve defect classification. We validate our approach on a real-world case study and by employing modern Deep Learning architectures for classification.

Particle accelerators are complex systems composed of multiple subsystems that must work together to produce high quality beams employed for physics experiments. A fault or an anomalous behaviour in one of such subsystems can lead to expensive downtime for the whole facility. Thus, it is of paramount importance to be able to promptly detect anomalies.Given the vast amount of streaming data generated by accelerator field sensors, Machine Learning (ML)-based tools are promising candidates for efficient monitoring of such systems: an approach based on unsupervised ML techniques exploiting the data from a Radio Frequency tuning system is here proposed. Feature importance is exploited to guide the definition of the optimal windowing for feature extraction. The proposed approach is here validated on real-world data related to the ALPI accelerator at Legnaro National Laboratories in Italy.

The advent of the Industrial Internet of Things (IIoT) has led to the availability of huge amounts of data, that can be used to train advanced Machine Learning algorithms to perform tasks such as Anomaly Detection, Fault Classification and Predictive Maintenance. Even though not all pieces of equipment are equipped with sensors yet, usually most of them are already capable of logging warnings and alarms occurring during operation. Turning this data, which is easy to collect, into meaningful information about the health state of machinery can have a disruptive impact on the improvement of efficiency and up-time. The provided dataset consists of a sequence of alarms logged by packaging equipment in an industrial environment. The collection includes data logged by 20 machines, deployed in different plants around the world, from 2019-02-21 to 2020-06-17. There are 154 distinct alarm codes, whose distribution is highly unbalanced. This data can be used to address the following tasks: 

1. Next alarm forecasting: this problem can be framed as a supervised multi-class classification task, or a binary classification task when a specific alarm code is considered.
2. Predicting alarms occurring in a future time frame: here the goal is to forecast the occurrence of certain alarm types in a future time window. Since many alarms can occur, this is a supervised multi-label classification.
3. Future alarm sequence prediction: here the goal is predicting an ordered sequence of future alarms, in a sequence-to-sequence forecasting scenario.
4. Anomaly Detection: the task is to detect abnormal equipment conditions, based on the pattern of alarms sequence. This task can be either unsupervised, if only the input sequence is considered, or supervised if future alarms are taken into account to assess whether or not there is an anomaly.

 All of the above tasks can also be studied from a continual learning perspective. Indeed, information about the serial code of the specific piece of equipment can be used to train the model; however, a scalable model should also be easy to apply to new machines, without the need of a new training from scratch. The collection and release of this dataset has been supported by the Regione Veneto project PreMANI (MANIFATTURA PREDITTIVA: progettazione, sviluppo e implementazione di soluzioni di Digital Manufacturing per la previsione della Qualita e la Manutenzione Intelligente - PREDICTIVE MAINTENANCE: design, development and implementation of Digital Manufacturing solutions for the intelligent quality and maintenance systems).

The last decade has seen tremendous improvements in technologies for Type 1 Diabetes (T1D) management, in particular the so-called artificial pancreas (AP), a wearable closed-loop device modulating insulin injection based on glucose sensor readings. Unluckily, the AP actuator, an insulin pump, is subject to failures, with potentially serious consequences for subject safety. This calls for the development of advanced monitoring systems, leveraging the unprecedented data availability. This paper tackles for the first time the problem of automatically detecting pump faults with multidimensional data-driven anomaly detection (AD) methodologies. The approach allows to avoid the subtask of identifying a physiological model, typical of model-based approaches. Furthermore, we employ unsupervised methods, removing the need of labeled data for training, hardly available in practice. The adopted data-driven AD methods are local outlier factor, connectivity-based outlier factor, and isolation forest. Moreover, we propose a modification of these methods to cope with the dynamic nature of the underlying problem. The algorithms were tuned and tested on: 1) two-synthetic 100-patients' data set, of one-month data each, generated using the ``UVA/Padova T1D Simulator,'' a large-scale nonlinear computer simulator of T1D subject physiology, largely adopted in AP research and accepted by the American Food and Drug Administration as a replacement of preclinical animal trials for AP and 2) a real 7-patients' data set consisting of one month in free-living conditions. The satisfactory accuracy of the proposed approach paves the way to the embedding of these methodologies in AP systems or their deployment in remote monitoring systems.

In many real-world applications of Machine Learning it is of paramount importance not only to provide accurate predictions, but also to ensure certain levels of robustness. Adversarial Training is a training procedure aiming at providing models that are robust to worst-case perturbations around predefined points. Unfortunately, one of the main issues in adversarial training is that robustness w.r.t. gradient-based attackers is always achieved at the cost of prediction accuracy. In this paper, a new algorithm, called Wasserstein Projected Gradient Descent (WPGD), for adversarial training is proposed. WPGD provides a simple way to obtain cost-sensitive robustness, resulting in a finer control of the robustness-accuracy trade-off. Moreover, WPGD solves an optimal transport problem on the output space of the network and it can efficiently discover directions where robustness is required, allowing to control the directional trade-off between accuracy and robustness. The proposed WPGD is validated in this work on image recognition tasks with different benchmark datasets and architectures. Moreover, real world-like datasets are often unbalanced: this paper shows that when dealing with such type of datasets, the performance of adversarial training are mainly affected in term of standard accuracy.

One of the main challenges in developing Machine Learning-based solutions for Semiconductor Manu-facturing is the high number of machines in the production and their differences, even when consideringchambers of the same machine; this poses a challenge in the scalability of Machine Learning-based so-lutions in this context, since the development of chamber-specific models for all equipment in the fab isunsustainable. In this work, we present a domain adaptation approach for Virtual Metrology (VM), one ofthe most successful Machine Learning-based technology in this context. The approach provides a commonVM model for two identical-in-design chambers whose data follow different distributions. The approach isbased on Domain-Adversarial Neural Networks and it has the merit of exploiting raw trace data, avoidingthe loss of information that typically affects VM modules based on features. The effectiveness of theapproach is demonstrated on real-world Etching.

Search Engines (SE) have been shown to perpetuate well-known gender stereotypes identified in psychology literature and to influence users accordingly. Similar biases were found encoded in Word Embeddings (WEs) learned from large online corpora. In this context, we propose the Gender Stereotype Reinforcement (GSR) measure, which quantifies the tendency of a SE to support gender stereotypes, leveraging gender-related information encoded in WEs.

Through the critical lens of construct validity, we validate the proposed measure on synthetic and real collections. Subsequently, we use GSR to compare widely-used Information Retrieval ranking algorithms, including lexical, semantic, and neural models. We check if and how ranking algorithms based on WEs inherit the biases of the underlying embeddings. We also consider the most common debiasing approaches for WEs proposed in the literature and test their impact in terms of GSR and common performance measures. To the best of our knowledge, GSR is the first specifically tailored measure for IR, capable of quantifying representational harms.

To increase the classification, the rate of prediction based on existing models requires additional technique or in this case optimizing the model. Hyperparameter tuning is an optimization technique that evaluates and adjusts the free parameters that define the behaviour of classifiers. Data sets were classified practical with classifiers like SVM, k-NN, ANN and DA. To further improve the design efficiency, the secondary optimization level called hyperparameter tuning will be further investigated. DA, SVM, k-NN, decision tree (Tree), logistic regression (LR), random forest tree (RF) and neural network (NN) are evaluated. The k-NN provided 96.47% of the test sets with the best reliability in classifications. Bayesian optimization has been used to refine the hyperparameter; hence, standardize Euclidean distance metric with a k value of one is the ideal hyperparameters which could achieve classification performance of 97.16%.

This chapter demonstrates the pipeline from data collection until classifier models that achieve the best possible model in identifying the disparity between hunger states. The pre-processing segment describes the features of the data sets obtained by means of image processing. The method includes the simple moving average (SMA), downsizing factors, dynamic time warping (DTW) and clustering by the k-means method. This is to rationally assign the necessary significant information from the data collected and processed the images captured for demand feeder and fish motion as a synthesis for anticipating the state of fish starvation. The selection of features in this study takes place via the boxplot analysis and the principal component analysis (PCA) on dimensionality reduction. Finally, the validation of the hunger state will be addressed by comparing machine learning (ML) classifiers, namely the discriminant analysis (DA), support vector machine (SVM) and k-nearest neighbour (k-NN). The outcome in this chapter will validate the features from image processing as a tool for identifying the behavioural changes of the fish in school size.

Machine Learning-based Anomaly Detection (AD) approaches are efficient tools to monitor complexprocesses. One of the advantages of such approaches is that they provide a unique anomaly indicator,a quantitative index that captures the degree of ’outlierness’ of the process at hand considering possiblyhundreds or more variables at the same time, the typical scenario in semiconductor manufacturing. Oneof the drawback of such approaches is that Root Cause Analysis is not guided by the system itself. Inthis work, we show the effectiveness of a method, called DIFFI, to equip Isolation Forest, one of themost popular AD algorithms, with interpretability traits that can help corrective actions and knowledgeunderstanding. Such approach is validated on real world semiconductor manufacturing data related to aChemical Vapor Deposition process.

This book highlights the fundamental association between aquaculture and engineering in classifying fish hunger behaviour by means of machine learning techniques. Understanding the underlying factors that affect fish growth is essential, since they have implications for higher productivity in fish farms. Computer vision and machine learning techniques make it possible to quantify the subjective perception of hunger behaviour and so allow food to be provided as necessary. The book analyses the conceptual framework of motion tracking, feeding schedule and prediction classifiers in order to classify the hunger state, and proposes a system comprising an automated feeder system, image-processing module, as well as machine learning classifiers. Furthermore, the system substitutes conventional, complex modelling techniques with a robust, artificial intelligence approach. The findings presented are of interest to researchers, fish farmers, and aquaculture technologist wanting to gain insights into the productivity of fish and fish behaviour.

This chapter highlights the findings of the developmental monitoring systems for swimming pattern or motion analysis with regard to feeding behaviour. A benchmark for examining the framework on how scientists control fish in animal variable function factors was gathered and referred to gauge the adequate design in constructing a viable device. The validation of image processing and automated demand feeder to determine the results will also be considered, as a validation aspect between the system of tracking and the behaviour of the Lates calcarifer where the pixel intensity will be extracted as the features. The results of this chapter will enable the reader on the development of an integrated feeder scheme that consolidates surveillance scheme to identify the feeding behaviour and relation towards the specific growth rate (SGR).

Measuring systems are becoming increasingly sophisticated in order to tackle the challenges of modern industrial problems. In particular, the Multiphase Flow Meter (MPFM) combines different sensors and data fusion techniques to estimate quantities that are difficult to be measured like the water or gas content of a multiphase flow, coming from an oil well. The evaluation of the flow composition is essential for the well productivity prediction and management, and for this reason, the quantification of the meter measurement quality is crucial. While instrument complexity is increasing, demands for confidence levels in the provided measures are becoming increasingly more common. In this work, we propose an Anomaly Detection approach, based on unsupervised Machine Learning algorithms, that enables the metrology system to detect outliers and to provide a statistical level of confidence in the measures. The proposed approach, called AD4MPFM (Anomaly Detection for Multiphase Flow Meters), is designed for embedded implementation and for multivariate time-series data streams. The approach is validated both on real and synthetic data.

In knowledge-based systems, besides obtaining good output prediction accuracy, it is crucial to understand the subset of input variables that have most influence on the output, with the goal of gaining deeper insight into the underlying process. These requirements call for logistic model estimation techniques that provide a sparse solution, i.e., where coefficients associated with non-important variables are set to zero. In this work we compare the performance of two methods: the first one is based on the well known Least Absolute Shrinkage and Selection Operator (LASSO) which involves regularization with an ?1 norm; the second one is the Relevance Vector Machine (RVM) which is based on a Bayesian implementation of the linear logistic model. The two methods are extensively compared in this paper, on real and simulated datasets. Results show that, in general, the two approaches are comparable in terms of prediction performance. RVM outperforms the LASSO both in term of structure recovery (estimation of the correct non-zero model coefficients) and prediction accuracy when the dimensionality of the data tends to increase. However, LASSO shows comparable performance to RVM when the dimensionality of the data is much higher than number of samples that is p>>n 

The identification of relevant parameters that could describe the state of fish hunger is vital for ensuring the appropriate allocation of food to the fish. The establishment of these relevant parameters is non-trivial, particularly when developing an automated demand feeder system. The present inquiry is being undertaken to determine the hunger state of Lates calcarifer. For data collection, a video analysis system is used, and the video was taken all day, where the fish was fed by an automatic feeding system. Sixteen characteristics of the raw data set have been extracted through feature engineering for 0.5 min, 1.0 min, 1.5 min and 2.0 min, respectively, in accordance with the mean, peak, minimum and variability of each of the different time window scales. Furthermore, the features extracted have been evaluated through principal component analysis (PCA) both for dimension reduction and PCA with varimax rotation. The details were then categorized using support vector machine (SVM), K-NN and random forest tree (RF) classifiers. The best identification accuracy was shown with eight described features in the varimax-based PCA. The forecast results based on the K-NN model built on selected data characteristics showed a level of 96.5% indicating that the characteristics analysed were crucial to classifying the actions of hunger among fisheries.

We propose a Deep Learning (DL)-based approach for production performance forecasting in fresh products packaging. On the one hand, this is a very demanding scenario where high throughput is mandatory; on the other, due to strict hygiene requirements, unexpected downtime caused by packaging machines can lead to huge product waste. Thus, our aim is predicting future values of key performance indexes such as Machine Mechanical Efficiency (MME) and Overall Equipment Effectiveness (OEE). We address this problem by leveraging DL-based approaches and historical production performance data related to measurements, warnings and alarms. Different architectures and prediction horizons are analyzed and compared to identify the most robust and effective solutions. We provide experimental results on a real industrial case, showing advantages with respect to current policies implemented by the industrial partner both in terms of forecasting accuracy and maintenance costs. The proposed architecture is shown to be effective on a real case study and it enables the development of predictive services in the area of Predictive Maintenance and Quality Monitoring for packaging equipment providers.

In modern manufacturing scenarios, detecting anomalies in production systems is pivotal to keep high-quality standards and reduce costs. Even in the Industry 4.0 context, realworld monitoring systems are often simple and based on the use of multiple univariate control charts. Data-driven technologies offer a whole range of tools to perform multivariate data analysis that allow to implement more effective monitoring procedures. However, when dealing with complex data, common data-driven methods cannot be directly used, and a feature extraction phase must be employed. Feature extraction is a particularly critical operation, especially in anomaly detection tasks, and it is generally associated with information loss and low scalability. In this paper we consider the task of Anomaly Detection with twodimensional, image-like input data, by adopting a Deep Learningbased monitoring procedure, that makes use of convolutional autoencoders. The procedure is tested on real Optical Emission Spectroscopy data, typical of semiconductor manufacturing. The results show that the proposed approach outperforms classical feature extraction procedures.

Gesture Recognition has a prominent importance in smart environment and home automation. Thanks to the availability of Machine Learning approaches it is possible for users to define gestures that can be associated with commands for the smart environment. In this paper we propose a Random Forest-based approach for Gesture Recognition of hand movements starting from wireless wearable motion capture data. In the presented approach, we evaluate different feature extraction procedures to handle gestures and data with different duration. To enhance reproducibility of our results and to foster research in the Gesture Recognition area, we share the dataset that we have collected and exploited for the present work.

Two methods based on Regression are presented to determine the moisture content of items, e.g. clothes and the like, which are introduced in a household laundry dryer appliance. The aim of this work is to develop Soft Sensors (SS) for a household Heat Pump Washer-Dryer (WD-HP) to provide an estimation of the desired signal (the laundry moisture during drying) avoiding the use of additional physical sensors with the goal of improving the current performance in terms of precision and energy consumption of the automatic drying cycle end and using the machine equipment already available. On an algorithmic point of view, the SS developed in this work exploits regularization methods and Genetic Programming for Symbolic Regression in order to find suitable models for the purpose at hand. Proposed approaches have been tested on real data provided by an industrial partner.

Industry 4.0 encapsulates methods, technologies, and procedures that transform data into informed decisions and added value inan industrial context. In this regard, technologies such as Virtual Metrology or Soft Sensing have gained much interest in thelast two decades due to their ability to provide valuable knowledge for production purposes at limited added expense. However,these technologies have struggled to achieve wide-scale industrial adoption, largely due to the challenges associated with handlingcomplex data structures and the feature extraction phase of model building. This phase is generally hand-engineered and basedon specific domain knowledge, making it time consuming, difficult to automate, and prone to loss of information, thus ultimatelylimiting portability. Moreover, in the presence of complex data structures, such as 2-dimensional input data, there are no establishedprocedures for feature extraction. In this paper, we present a Deep Learning approach for Virtual Metrology, called DeepVM,that exploits semi-supervised feature extraction based on Convolutional Autoencoders. The proposed approach is demonstratedusing a real world semiconductor manufacturing dataset where the Virtual Metrology input data is 2-dimensional Optical EmissionSpectrometry data. The feature extraction method is tested with different types of state-of-the-art autoencoder.

In the past recent years, Machine Learning methodologies have been applied in countless application areas. In particular, they play a key role in enabling Industry4.0. However, one of the main obstacles to the diffusion of Machine Learning-based applications is related to the lack of interpretability of most of these methods. In this work, we propose an approach for defining a ‘feature importance’ in Anomaly Detection problems. Anomaly Detection is an important Machine Learning task that has an enormous applicability in industrial scenarios. Indeed, it is extremely relevant for the purpose of quality monitoring. Moreover, it is often the first step towards the design of a Machine Learning based smart monitoring solutions because Anomaly Detection can be implemented without the need of labelled data. The proposed feature importance evaluation approach is designed for Isolation Forest, one of the most commonly used algorithm for Anomaly Detection. The efficacy of the proposed method is tested on synthetic and real industrial datasets.

In an automated demand feeder system, underlining the parameters that contribute to fish hunger is crucial in order to facilitate an optimised food allocation to the fish. The present investigation is carried out to classify the hunger state of Lates calcarifer. A video surveillance technique is employed for data collection. The video was taken throughout the daytime, and the fish were fed through an automated feeding system. It was demonstrated through this investigation that the use of such automated system does contribute towards a higher specific growth rate percentage of body weight as well as the total length by approximately 26.00% and 15.00%, respectively against the conventional time-based method. Sixteen features were feature engineered from the raw dataset into window sizes ranging from 0.5?min, 1.0?min, 1.5?min and 2.0?min, respectively coupled with the mean, maximum, minimum and variance for each of the distinctive temporal window sizes. In addition, the extracted features were analysed through Principal Component Analysis (PCA) for dimensionality reduction as well as PCA with varimax rotation. The data were then classified using a Support Vector Machine (SVM), k-Nearest Neighbor (k-NN) and Random Forest Tree models. It was demonstrated that the varimax based PCA yielded the highest classification accuracy with eight identified features. The prediction results based of the developed k-NN model on the selected features on the test data exhibited a classification rate of 96.5% was achieved suggesting that the features examined are non-trivial in classifying the fish hunger behaviour.

Embedding household appliances with smart capabilities is becoming common practice among major fabric-care producers that seek competitiveness on the market by providing more efficient and easy-to-use products. In Vertical Axis Washing Machines (VA-WM), knowing the laundry composition is fundamental to setting the washing cycle properly with positive impact both on energy/water consumption and on washing performance. An indication of the load typology composition (cotton, silk, etc.) is typically provided by the user through a physical selector that, unfortunately, is often placed by the user on the most general setting due to the discomfort of manually changing configurations. An automated mechanism to determine such key information would thus provide increased user experience, better washing performance, and reduced consumption; for this reason, we present here a data-driven soft sensor that exploits physical measurements already available on board a commercial VA-WM to provide an estimate of the load typology through a machine-learning-based statistical model of the process. The proposed method is able to work in a resource-constrained environment such as the firmware of a VA-WM.

The aim is to develop soft sensors (SSs) to provide an estimation of the laundry moisture of clothes introduced in a household Heat Pump Washer–Dryer (WD-HP) appliance. The developed SS represents a cost-effective alternative to physical sensors, and it aims at improving the WD-HP performance in terms of drying process efficiency of the automatic drying cycle. To this end, we make use of appropriate Machine Learning models, which are derived by means of Regularization and Symbolic Regression methods. These methods connect easy-to-measure variables with the laundry moisture content, which is a difficult and costly to measure variable. Thanks to the use of SSs, the laundry moisture estimation during the drying process is effectively available. The proposed models have been tested by exploiting real data through an experimental test campaign on household drying machines.

In this paper we present a novel algorithm to solve the robot kinematic structure identification problem. Given a time series of data, typically obtained processing a set of visual observations, the proposed approach identifies the ordered sequence of links associated to the kinematic chain, the joint type interconnecting each couple of consecutive links, and the input signal influencing the relative motion. Compared to the state of the art, the proposed algorithm has reduced computational costs, and is able to identify also the joints' type sequence. 

Emotion Classification (EC) aims at assigning an emotion label to a textual document with two inputs – a set of emotion labels (e.g. anger, joy, sadness) and a document collection. The best performing approaches for EC are dictionary-based and suffer from two main limitations: (i) the out-of-vocabulary (OOV) keywords problem and (ii) they cannot be used across heterogeneous domains. In this work, we propose a way to overcome these limitations with a supervised approach based on TF-IDF indexing and Multinomial Linear Regression with Elastic-Net regularization to extract an emotion lexicon and classify short documents from diversified domains. We compare the proposed approach to state-of-the-art methods for document representation and classification by running an extensive experimental study on two shared and heterogeneous data sets.

Exploiting the huge amount of data collected by industries is definitely one of the main challenges of the so-called Big Data era. In this sense, Machine Learning has gained growing attention in the scientific community, as it allows to extract valuable information by means of statistical predictive models trained on historical process data. In Semiconductor Manufacturing, one of the most extensively employed data-driven applications is Virtual Metrology, where a costly or unmeasurable variable is estimated by means of cheap and easy to obtain measures that are already available in the system. Often, these measures are multi-dimensional, so traditional Machine Learning algorithms cannot handle them directly. Instead, they require feature extraction, that is a preliminary step where relevant information is extracted from raw data and converted into a design matrix. Features are often hand-engineered and based on specific domain knowledge. Moreover, they may be difficult to scale and prone to information loss, affecting the effectiveness and maintainability of machine learning procedures. In this paper, we present a Deep Learning method for semi-supervised feature extraction based on Convolutional Autoencoders that is able to overcome the aforementioned problems. The proposed method is tested on a real dataset for Etch rate estimation. Optical Emission Spectrometry data, that exhibit a complex bi-dimensional time and wavelength evolution, are used as input.

In semiconductor manufacturing, metrology is generally a high cost, non-value added operation that impacts significantly on cycle time. As such, reducing wafer metrology continues to be a major target in semiconductor manufacturing efficiency initiatives. Data-driven spatial dynamic sampling methodologies are here compared. Such strategies aim at minimizing the number of sites that need to be measured across a wafer surface while maintaining an acceptable level of wafer profile reconstruction accuracy. The Spatial Dynamic Sampling approaches are based on analyzing historical metrology data to determine, from a set of candidate wafer sites, the minimum set of sites that need to be monitored in order to reconstruct the full wafer profile using statistical regression techniques. Spatial Dynamic sampling is then implemented in various strategies that guarantee coverage of all the possible sites in a given set of process iteration. In this way, the risk of not detecting previously unseen process behavior is mitigated. In this work, we demonstrate the efficacy of spatial dynamic sampling methodologies using both simulation studies and metrology data from a semiconductor manufacturing process.

On-line and phone banking frauds are responsible for millions of dollars loss every year. In this work, we propose a Machine Learning-based Decision Support System to automatically associate a risk factor to each transaction performed through an on-line/mobile banking system. The proposed approach has a hierarchical architecture: First, an unsupervised Machine Learning module is used to detect abnormal patterns or wrongly labeled transactions; then, a supervised module provides a risk factor for the transactions that were not marked as anomalies in the previous step. Our solution exploits personal and historical information about the user, statistics that describe online traffic generated on the online/mobile banking system, and features extracted from motives of the transactions. The proposed approach deals with dataset unbalancing effectively. Moreover, it has been validated on a large database of transactions and on-line traffic provided by an industrial partner.

In the context of Smart Monitoring and Fault Detection and Isolation in industrial systems, the aim of Predictive Maintenance technologies is to predict the happening of process or equipment faults. In order for a Predictive Maintenance technology to be effective, its predictions have to be both accurate and timely for taking strategic decisions on maintenance scheduling, in a cost-minimization perspective. A number of Predictive Maintenance technologies are based on the use of “health factors” quantitative indicators associated with the equipment wear that exhibit a monotone evolution. In real industrial environment, such indicators are usually affected by measurement noise and non-uniform sampling time. In this work we present a methodology, formulated as a stochastic filtering problem, to optimally predict the evolution of the aforementioned health factors based on noisy and irregularly sampled observations. In particular, a hidden Gamma process model is proposed to capture the nonnegativity and nonnegativity of the derivative of the health factor. As such filtering problem is not amenable to a closed form solution, a numerical Monte Carlo approach based on particle filtering is here employed. An adaptive parameter identification procedure is proposed to achieve the best trade-off between promptness and low noise sensitivity. Furthermore, a methodology to identify the risk function associated to the observed equipment based on previous maintenance data is proposed. The present study is motivated and tested on a real industrial Predictive Maintenance problem in semiconductor manufacturing, with reference to a dry etching equipment.

In semiconductor manufacturing, metrology is generally a high cost nonvalue-added operation that significantly impacts on cycle time. As such, reducing wafer metrology continues to be a major target in semiconductor manufacturing efficiency initiatives. A novel data-driven spatial dynamic sampling methodology is presented that minimizes the number of sites that need to be measured across a wafer surface while maintaining an acceptable level of wafer profile reconstruction accuracy. The methodology is based on analyzing historical metrology data using forward selection component analysis (FSCA) to determine, from a set of candidate wafer sites, the minimum set of sites that need to be monitored in order to reconstruct the full wafer profile using statistical regression techniques. Dynamic sampling is then implemented by clustering unmeasured sites in accordance with their similarity to the FSCA selected sites and temporally selecting a different sample from each cluster. In this way, the risk of not detecting previously unseen process behavior is mitigated. We demonstrate the efficacy of the proposed methodology using both simulation studies and metrology data from a semiconductor manufacturing process.

Fabric care home appliances are pervasive inhouses worldwide and manufactures are constantly working for improving product performance, efficiency, and usability. From a manufacturing perspective, increase of performancehas to be attained while minimizing the increase of production costs. In this context, a Soft Sensor for estimating the clothes load weight in a horizontal axis household washing machines ishere presented. The proposed Soft Sensor is based on Machine Learning approaches. Several methodologies, both time-seriesand feature-based, are employed and compared. The approach has been tested on real world data on commercial household washing machines.

Subjects affected by Type I Diabetes (T1D) are constantly confronted with the complicated problem of administering themselves an adequate amount of insulin, so as to keep their blood-glucose concentration in a nearly physiological range. Recently, powerful technological tools have been developed to better face this challenge, in particular the so-called Artificial Pancreas (AP). Unluckily, the AP actuator, an insulin pump, is subject to faults, with potential serious consequences for subjects' safety. This calls for the development of advanced fault detection (FD) methods, leveraging the unprecedented data availability in this application. In this paper we tackle the problem of detecting insulin pump malfunctioning using a model-free approach, so that the complex sub-task of identifying a model of patients physiology is avoided. Moreover, we employed unsupervised methods since labeled data are hardly available in practice. The adopted data-driven Anomaly Detection (AD) methods are Local Outlier Factor and Connectivity-based Outlier Factor. The methods are applied on a feature set able to account for the physiological dynamics of T1D patients. The proposed algorithms are tested on a synthetic dataset, generated using the “UVA/Padova Type 1 Diabetic Simulator”, an accurate nonlinear computer simulator of the T1D subject physiology. Both methods show precision ~75% and recall ~60%• The described approach is suitable both for embedding in medical devices, such as the AP, and implementation in cloud-based remote monitoring systems.

In laundry treatment appliances, the weight ofthe laundry loaded by the user inside the drum dramaticallyaffects the operating behavior. Therefore, it is important toobtain a good estimate of the said quantity in order tocorrectly configure the machine before the washing/dryingstarts. In Vertical Axis Washing Machinesthe laundry weightis computed by exploiting the quantity of water absorbed bythe clothes. However, such approach does not grant accurateresults because the water absorption depends on the clothesfabric. For this reason, we propose a Soft Sensing approachfor weight estimation that exploits the information obtainedfrom physical sensors available on board without added costs.Data-driven Soft Sensors are developed, where, using MachineLearning techniques, a statistical model of the phenomenon ofinterest is created from a set of sample data.

Wafer metrology is an expensive and time consuming activity in semiconductor manufacturing, but is essential to support advanced process control, predictive maintenance and other quality assurance functions. Keeping metrology to a minimum is therefore desirable. In the context of spatial sampling of wafers this has motivated the development of a number of data driven methodologies for optimizing wafer sampling plans. Two such methodologies are considered in this paper. The first combines Principal Component Analysis and Minimum Variance Estimation (PCA-MVE) to determine an optimum subset of sites from historical metrology data from a larger candidate set, while the second employs Forward Selection Component Analysis (FSCA), an unsupervised variable selection technique, to achieve the same result. We investigate the relationship between these two approaches and show that under specific conditions a regularized extension of FSCA, denoted FSCA-R, and PCA-MVE are equivalent. Numerical studies using simulated data verify the equivalence conditions. Results for simulated and industrial case studies show that the improvement in wafer profile reconstruction accuracy with regularization is not statistically significant for the case studies considered, and that when PCA-MVE is implemented with a denoising step as originally proposed, it is outperformed by FSCA. Therefore, FSCA is the preferred methodology.

The laundry weight of the loaded in the drum of a laundry treatment machine is an important piece of information; laundry weight can be used to set various washing/drying cycle parameters and to optimize performances and efficiency. Unfortunately, dedicated weight sensors cannot be included in consumer laundry equipment given the related costs. For this reason, we present in this work a soft sensor approach for estimating laundry weight based on sensors already in place in a laundry treatment equipment; in particular, we consider here a heat pump tumble dryer as case study. The proposed soft sensor is based on regularization, a popular approach in Machine Learning to provide models without overfitting the training data. Different studies are provided in this work, by considering different constrains on timing and complexity of the Soft Sensor solution. The developed Soft Sensors have been tested on laboratory data provided by an industrial partner.

The goal of Aspect-Based Sentiment Analysis is to identify opinions regarding specific targets and the corresponding sentiment polarity in a document. The proposed approach is designed for real-world scenarios, where the amount of available information and annotated data is often too limited to train supervised models. We focus on the two core tasks of Aspect-Based Sentiment Analysis: aspect and sentiment polarity classification. The first task – which consists in the identification of the opinion targets in a document – is tackled by means of a weakly-supervised technique based on Non-negative Matrix Factorization. This strategy allows users to easily embed some a priori domain knowledge by means of short seed terms lists. Experimental results on publicly available data sets related to online reviews suggest that the proposed approach is very flexible and can be easily adapted to different languages and domains.

Smart production monitoring is a crucial activity in advanced manufacturing for quality, control and maintenance purposes. Advanced Monitoring Systems aim to detect anomalies and trends; anomalies are data patterns that have different data characteristics from normal instances, while trends are tendencies of production to move in a particular direction over time. In this work, we compare state-of-the-art ML approaches (ABOD, LOF, onlinePCA and osPCA) to detect outliers and events in high-dimensional monitoring problems. The compared anomaly detection strategies have been tested on a real industrial dataset related to a Semiconductor Manufacturing Etching process

Advanced Monitoring Systems are fundamental in advanced manufacturing for control, quality and maintenance purposes. Nowadays, with the increasing availability of data in production and equipment, the need for high-dimensional Anomaly Detection techniques is thriving; anomalies are data patterns that have different data characteristics from normal production instances and that may be associated with faults or drifts in production. Tools for dealing with high-dimensional monitoring problems are provided by Machine Learning: in this paper, we test the performance of a state-of-the-art anomaly detection technique, called Isolation Forest, on a real industrial dataset related to Etching, one of the most important semiconductor manufacturing process. The monitoring has been performed exploiting Optical Spectroscopy Data.

Virtual Metrology is one of the most prominentAdvanced Process Control applications in SemiconductorManufacturing. The goal of Virtual Metrology is to provideestimations of quantities that are important for production andto assess process quality, but are costly or impossible to bemeasured. Virtual Metrology solutions are based on MachineLearning approaches. The bottleneck of developing VirtualMetrology solutions is generally the feature extraction phase thatcan be time-consuming, and can deeply affect the estimationperformance. In particular, in presence of data with additionaldimensions, such as time, feature extraction is typicallyperformed by means of heuristic approaches that may pickfeatures with poor predictive capabilities. In this work, wepropose the usage of modern Deep Learning approaches tobypass manual feature extraction and to provide highperformanceautomatic Virtual Metrology modules. Theproposed methodology is tested on a real industrial datasetrelated to Etching. The dataset at hand contains OpticalEmission Spectroscopy data and it is paradigmatic of the featureextraction problem under examination.

The invention relates to a method for the determination of a laundry weight in a laundry treatment appliance comprising: Selecting a laundry program in the laundry treatment appliance; Starting the selected laundry program; Sensing a plurality of parameters indicating operating conditions of the laundry treatment appliance during the laundry program; and Predicting a weight of the laundry present within the laundry treatment appliance based on said plurality of parameters by means of a data-driven soft sensor.

The diffusion in Power Systems of distributed renewable energy resources, electric vehicles and controllable loads has made advanced monitoring systems fundamental to cope with the consequent disturbances in power flows; advanced monitoring systems can be employed for Anomaly Detection, Root Cause Analysis and Control purposes.

Several Machine Learning-based approaches have been developed in the past recent years to detect if a power system is running under anomalous conditions and, eventually, to classify such situation with respect to known problems.

One of the aspects that makes Power Systems challenging to be tackled, is that the monitoring has to be performed on streams of data that have a time series evolution; this issue is generally tackled by performing a features extraction procedure before the classication phase. The features extraction phase consists of translating the informative content of time series data into scalar quantities: such procedure may be a time-consuming step that requires the involvement of process experts to avoid loss of information in the making; moreover, extracted features designed to capture certain behaviors of the system, may not be informative under unseen conditions leading to poor monitoring performances.

A different type of data-driven approaches, that will be reviewed in this chapter, allow to perform classication directly on the raw time series data, avoiding the features extraction phase: among these approaches, Dynamic Time Warping and Symbolic-based methodologies have been widely applied in many application areas.

In the following, pros and cons of each approach will be discussed and practical implementation guidelines will be provided.

In the context of activity recognition, wearable devices arenowadays the preferable hardware thanks to their usability, user expe-rience and performances; at the same time, these devices present limi-tations in terms of computational capability and memory, which forcethe algorithm design to be at the same time ecient and simple. Inthis work, we adopt Symbolic Aggregate Approximation (SAX), a sym-bolic approach for information retrieval in time series data that allowsdimensionality and numerosity reduction; SAX is employed here, in com-bination with 1-Nearest Neighbor classier, to identify activity phases incontinuous repetitive activities from inertial time-series data. The pro-posed approach is validated on a public activity recognition dataset.

In this paper an approach to deal with Predictive Maintenance (PdM) problems with time-series data is discussed. PdM is a important approach to tackle maintenance and it is gaining an increasing attention in advanced manufacturing to minimize scrap materials, downtime, and associated costs. PdM approaches are generally based on Machine Learning tools that require the availability of historical process and maintenance data. Given the exponential growth in data logging in modern equipment, time series dataset are increasingly available in PdM applications. To exploit time series data for PdM, a functional learning methodology, namely Supervised Aggregative Feature Extraction (SAFE), is here employed on a semiconductor manufacturing maintenance problem.

In many applications, and especially thosewhere batch processes are involved, a target scalar outputof interest is often dependent on one or more time seriesof data. With the exponential growth in data logging inmodern industries such time series are increasingly availablefor statistical modeling in soft sensing applications. In orderto exploit time series data for predictive modelling, it isnecessary to summarise the information they contain as aset of features to use as model regressors. Typically thisis done in an unsupervised fashion using simple techniquessuch as computing statistical moments, principalcomponents or wavelet decompositions, often leading tosignificant information loss and hence suboptimal predictivemodels. In this paper, a functional learning paradigm isexploited in a supervised fashion to derive continuous,smooth estimates of time series data (yielding aggregatedlocal information), while simultaneously estimating a continuousshape function yielding optimal predictions. Theproposed Supervised Aggregative Feature Extraction (SAFE)methodology can be extended to support nonlinear predictivemodels by embedding the functional learning framework ina Reproducing Kernel Hilbert Spaces setting. SAFE has anumber of attractive features including closed form solutionand the ability to explicitly incorporate first and secondorder derivative information. Using simulation studies and apractical semiconductor manufacturing case study we highlightthe strengths of the new methodology with respect tostandard unsupervised feature extraction approaches.

In this paper a multiple classifier machine learning methodology for Predictive Maintenance (PdM) is presented. PdM is a prominent strategy for dealing with maintenance issues given the increasing need to minimize downtime and associated costs. One of the challenges with PdM is generating so called ’health factors’ or quantitative indicators of the status of a system associated with a given maintenance issue, and determining their relationship to operating costs and failure risk. The proposed PdM methodology allows dynamical decision rules to be adopted for maintenance management and can be used with high-dimensionaland censored data problems. This is achieved by training multiple classification modules with different prediction horizons to provide different performance trade-offs in terms of frequency of unexpected breaks and unexploited lifetime and then employing this information in an operating cost based maintenance decision system to minimise expected costs. The effectiveness of the methodology is demonstrated using a simulated example and a benchmark semiconductor manufacturing maintenance problem.

In semiconductor manufacturing, wafer quality control strongly relies on product monitoring and physical metrology. However, the involved metrology operations, generally performed by means of scanning electron microscopes, are particularly cost-intensive and time-consuming. For this reason, in common practice a small subset of only a productive lot is measured at the metrology stations and it is devoted to represent the entire lot. Virtual Metrology (VM) methodologies are able to obtain reliable predictions of metrology results at process time, without actually performing physical measurements; this goal is usually achieved by means of statistical models and by linking process data and context information to target measurements. Since semiconductor manufacturing processes involve a high number of sequential operations, it is reasonable to assume that the quality features of a given wafer (such as layer thickness and critical dimensions) depend on the whole processing and not on the last step before measurement only. In this paper, we investigate the possibilities to enhance VM prediction accuracy by exploiting the knowledge collected from previous process steps. We present two different schemes of multi-step VM, along with dataset preparation indications; special consideration will be reserved to regression techniques capable of handling high-dimensional input spaces. The proposed multi-step approaches are tested on production data provided by a partner semiconductor manufacturing industry. 

Slow release drugs must be manufactured to meettarget speci?cations with respect to dissolution curve pro?les.In this paper we consider the problem of identifying thedrivers of dissolution curve variability of a drug from historicalmanufacturing data. Several data sources are considered: rawmaterial parameters, coating data, loss on drying and pellet sizestatistics. The methodology employed is to develop predictivemodels using LASSO, a powerful machine learning algorithmfor regression with high-dimensional datasets. LASSO providessparse solutions facilitating the identi?cation of the most importantcauses of variability in the drug fabrication process.The proposed methodology is illustrated using manufacturingdata for a slow release drug.

Faulty operations of Heating, Ventilation and Air Conditioning (HVAC) chiller systems can lead to discomfort for the occupants, energy wastage, unreliability and shorter equipment life. Such faults need to be detected early to prevent further escalation and energy losses. Commonly, data regarding unforeseen phenomena and abnormalities are rare or are not available at the moment of HVAC systems installation: for this reason in this paper an unsupervised One-Class SVM classifier employed as a novelty detection system to identify unknown status and possible faults is presented. The approach, that exploits Principal Component Analysis to accent novelties w.r.t. normal operations variability, has been tested on a HVAC literature dataset.

Process monitoring and Predictive Maintenance (PdM) are gaining increasing attention in most manufacturing environments as a means of reducing maintenance related costs and downtime. This is especially true in industries that are data intensive such as semiconductor manufacturing. In this paper an adaptive PdM based flexible maintenance scheduling decision support system, which pays particular attention to associated opportunity and risk costs, is presented. The proposed system, which employs Machine Learning and regularized regression methods, exploits new information as it becomes available from newly processed components to refine remaining useful life estimates and associated costs and risks. The system has been validated on a real industrial dataset related to an Ion Beam Etching process for semiconductor manufacturing.

In the semiconductor manufacturing environment it is very important to understand which factors have the most impact on process outcomes and to control them accordingly. This is usually achieved through design of experiments at process start-up and long term observation of production. As such it relies heavily on the expertise of the process engineer. In this work, we present an automatic approach to extracting useful insights about production processes and equipment based on state-of-the-art Machine Learning techniques. The main goal of this activity is to provide tools to process engineers to accelerate the learning-by-observation phase of process analysis. Using a Metal Deposition process as an example, we highlight various ways in which the extracted information can be employed.

Classification methods with embedded feature selection capability are very appealing for the analysis of complex processes since they allow the analysis of root causes even when the number of input variables is high. In this work, we investigate the performance of three techniques for classification within a Monte Carlo strategy with the aim of root cause analysis. We consider the naive Bayes classifier and the logistic regression model with two different implementations for controlling model complexity, namely, a LASSO-like implementation with a l1 norm regularization and a fully Bayesian implementation of the logistic model, the so called relevance vector machine. Several challenges can arise when estimating such models mainly linked to the characteristics of the data: a large number of input variables, high correlation among subsets of variables, the situation where the number of variables is higher than the number of available data points and the case of unbalanced datasets. Using an ecological and a semiconductor manufacturing dataset, we show advantages and drawbacks of each method, highlighting the superior performance in term of classification accuracy for the relevance vector machine with respect to the other classifiers. Moreover, we show how the combination of the proposed techniques and the Monte Carlo approach can be used to get more robust insights into the problem under analysis when faced with challenging modelling conditions.

In semiconductor fabrication processes, effectivemanagement of maintenance operations is fundamental todecrease costs associated with failures and downtime. PredictiveMaintenance (PdM) approaches, based on statistical methodsand historical data, are becoming popular for their predictivecapabilities and low (potentially zero) added costs. We presenthere a PdM module based on Support Vector Machines forprediction of integral type faults, that is, the kind of failuresthat happen due to machine usage and stress of equipmentparts. The proposed module may also be employed as a healthfactor indicator. The module has been applied to a frequentmaintenance problem in semiconductor manufacturing industry,namely the breaking of the filament in the ion-source ofion-implantation tools. The PdM has been tested on a realproduction dataset.

In semiconductor manufacturing plants, monitoring physical properties of all wafers is crucial to maintain good yield and high quality standards. However, such an approach is too costly, and in practice, only few wafers in a lot are actually monitored. Virtual metrology (VM) systems allow to partly overcome the lack of physical metrology. In a VM scheme, tool data are used to predict, for every wafer, metrology measurements. In this paper, we present a VM system for a chemical vapor deposition (CVD) process. On the basis of the available metrology results and of the knowledge, for every wafer, of equipment variables, it is possible to predict CVD thickness. In this work, we propose a VM module based on least angle regression to overcome the problem of high dimensionality and model interpretability. We also present a statistical distance-based clustering approach for the modeling of the whole tool production. The proposed VM models have been tested on industrial production data sets.

Smart management of maintenances has become fundamental in manufacturing environments in order to decrease downtime and costs associated with failures. Predictive Maintenance (PdM) systems based on Machine Learning (ML) techniques have the possibility with low added costs of drastically decrease failures-related expenses; given the increase of availability of data and capabilities of ML tools, PdM systems are becoming really popular, especially in semiconductor manufacturing. A PdM module based on Classification methods is presented here for the prediction of integral type faults that are related to machine usage and stress of equipment parts. The module has been applied to an important class of semiconductor processes, ion-implantation, for the predictionof ion-source tungsten filament breaks. The PdM has been tested on a real production dataset.

Semiconductor fabrication involves several sequentialprocessing steps with the result that critical productionvariables are often affected by a superposition of affects overmultiple steps. In this paper a Virtual Metrology (VM) systemfor early stage measurement of such variables is presented;the VM system seeks to express the contribution to theoutput variability that is due to a defined observable partof the production line. The outputs of the processed systemmay be used for process monitoring and control purposes. Asecond contribution of this work is the introduction of ElasticNets, a regularization and variable selection technique for themodelling of highly-correlated datasets, as a technique for thedevelopment of VM models. Elastic Nets and the proposed VMsystem are illustrated using real data from a multi-stage etchprocess used in the fabrication of disk drive read/write heads.

Ion Implantation is one of the most sensitiveprocesses in Semiconductor Manufacturing. It consists inimpacting accelerated ions with a material substrate and isperformed by an Implanter tool. The major maintenanceissue of such tool concerns the breaking of the tungstenfilament contained within the ion source of the tool. Thiskind of fault can happen on a weekly basis, and theassociated maintenance operations can last up to 3 hours.It is important to optimize the maintenance activities bysynchronizing the Filament change operations with otherminor maintenance interventions. In this paper, a PredictiveMaintenance (PdM) system is proposed to tackle such issue;the filament lifetime is estimated on a statistical basisexploiting the knowledge of physical variables acting onthe process. Given the high-dimensionality of the data,the statistical modeling has been based on RegularizationMethods: Lasso, Ridge Regression and Elastic Nets. Thepredictive performances of the aforementioned regularizationmethods and of the proposed PdM module have beentested on actual productive semiconductor data.

Silicon Epitaxial Deposition is a process strongly influenced by wafer temperature behaviour, that has to be constantly monitored to avoid the production of defective wafers. However, temperature measurements are not reliable and the sensors have to be appropriately calibrated with some dedicated procedure. A Predictive Maintenance (PdM) System is proposed here with the aim of predicting process behaviour and scheduling control actions on the sensors in advance. Two different prediction techniques have been employed and compared: the Kalman predictor and the Particle Filter with Gaussian Kernel Density Estimator. The accuracy of the PdM module has been tested on real industrial production datasets.

Soft Sensors (SSs) are on-line estimators of “hardly to be measured” quantities of a process. The difficultyin measuring can be related to economic or temporal costs that cannot be afforded in a high-intensivemanufacturing production. In semiconductor manufacturing this technology goes with the name of Virtual Metrology(VM) systems. While a lot of efforts in research have been produced in the past years to identify the bestregression algorithms for these statistical modules, small amount of work has been done to develop algorithms fordata clustering of the entire production. This paper contains a new Information Theory-based approach to dataclustering for Virtual Metrology and Soft Sensors; the proposed algorithm allows to automatically split the datasetinto groups to be equally modeled. The proposed approach has been tested on real industrial dataset.

Virtual Metrology (VM) module have become popular in the past years and they are now widely adopted in the semiconductor plants. However, nowadays, still few works have been presented to deal with the interaction between VM and Run-to-Run (R2R), the most common control approach in the fabs. We present in this paper a new strategy to integrate VM with R2R based on Information Theory measure. The proposed control method penalizes statistical measure based on their statistical distance from the physical measure. This new approach also cope with the virtual loop control, where the R2R runs for several process iterations without in-situ measures, but based only on VM predictions. The results are compared with the actual state-of-the-art. 

Semiconductor manufacturing is one of the most technologically advanced industrial sectors. Process quality and control are critical for decreasing costs and increasing yield. The contribution of automatic control and statistical modeling in this area can drastically impact production performance. For this reason in the past decade major collaborative research projects have been undertaken between fab industries and academia in the areas of Virtual Metrology, Predictive Maintenance, Fault Detection, Run-to-Run control and modeling. In this paper we review some this research, discuss its impact on production and highlight current challenges. 

Many modeling problems require to estimate ascalar output from one or more time series. Such problemsare usually tackled by extracting a fixed number of featuresfrom the time series (like their statistical moments), with aconsequent loss in information that leads to suboptimal predictivemodels. Moreover, feature extraction techniques usuallymake assumptions that are not met by real world settings (e.g.uniformly sampled time series of constant length), and failto deliver a thorough methodology to deal with noisy data.In this paper a methodology based on functional learningis proposed to overcome the aforementioned problems; theproposed Supervised Aggregative Feature Extraction (SAFE)approach allows to derive continuous, smooth estimates oftime series data (yielding aggregate local information), whilesimultaneously estimating a continuous shape function yieldingoptimal predictions. The SAFE paradigm enjoys severalproperties like closed form solution, incorporation of first andsecond order derivative information into the regressor matrix,interpretability of the generated functional predictor and thepossibility to exploit Reproducing Kernel Hilbert Spaces settingto yield nonlinear predictive models. Simulation studies areprovided to highlight the strengths of the new methodology withrespect to standard unsupervised feature selection approaches.

In semiconductor manufacturing plants, monitoringphysical properties of all wafers is fundamental in order tomaintain good yield and high quality standards. However, suchan approach is too costly and in practice only few wafers in a lotare actually monitored. Virtual Metrology (VM) systems allowto partly overcome the lack of physical metrology. In a VMscheme, tool data are used to predict, for every wafer, metrologymeasurements. In this paper, we present a VM system for aChemical Vapor Deposition (CVD) process. On the basis ofthe available metrology results and of the knowledge, for everywafer, of equipment variables, it is possible to predict CVDthickness. In this work we propose a VM module based onLARS to overcome the problem of high dimensionality andmodel interpretability. The proposed VM models have beentested on industrial production data sets.

In semiconductor manufacturing, state of the art for wafer quality control relies on product monitoring and feedback control loops; the involved metrology operations, performed by means of scanning electron microscopes, are particularly cost-intensive and time-consuming. For this reason, it is not possible to evaluate every wafer: in common practice, a small subset of a productive lot is measured at the metrology station and devoted to represent the whole lot. Virtual Metrology (VM) methodologies are able to obtain reliable predictions of metrology results at process time, without actually performing physical measurements; this goal is usually achieved by means of statistical models, linking process data and context information to target measurements. Since semiconductor manufacturing processes involve a high number of sequential operations, it is reasonable to assume that the quality features of a certain wafer (such as layer thickness, critical dimensions, electrical test results) depend on the whole processing and not only on the last step before measurement. In this paper, we investigate the possibilities to improve the Virtual Metrology quality relying on knowledge collected from previous process steps. We will present two different scheme of multistep VM, along with dataset preparation indications; special consideration will be reserved to regression techniques capable of handling high dimensional input spaces. The proposed multistep approaches will be tested against actual data from semiconductor manufacturing industry. 

Epitaxy is a process strongly dependent onwafer temperature. Unfortunately, the performance ofthe pyrometers in charge of sensing wafer temperaturedeteriorate with the usage. This represents the majormaintenance issue for epitaxy process engineers who haveto frequently calibrate pyrometers emissivity coefficient. Atthe present state the change of the emissivity coefficient isheuristically based on fab tradition and process engineersexperience. We present a statistical tool to map therelationship between change in the temperature readingsand emissivity adjustments. The module has been testedon real industrial dataset.

Silicon Epitaxial Deposition is a process strongly influenced by wafer temperature behavior, that has to be constantly monitored to avoid the production of defective wafers. A Predictive Maintenance (PdM) System is here proposed with the aim of predicting process behavior and scheduling control actions in advance. Two different prediction techniques have been employed and compared: the Kalman predictor and the Particle Filter with Gaussian Kernel Density Estimator. The accuracy of the PdM module has been tested on real fab data. The proposed approach is flexible and can handle the presence of different recipes on the same equipment.

In semiconductor manufacturing plants, monitoring of all wafers is fundamental in order to maintain good yield and high quality standards. However, this is a costly approach and in practice only few wafers in a lot are actually monitored. With a Virtual Metrology (VM) system it is possible to partly overcome the lack of physical metrology. In a VM scheme, tool data are used to predict, for every wafer, metrology measurements. In this paper, we present a VM system for a Chemical Vapor Deposition (CVD) process. Various data mining techniques are proposed. Due to the huge fragmentation of data derived from CVD's mixed production, several kind of data clustering have been adopted. The proposed models have been tested on real productive industrial data sets.

We extend several recent results on full-state feedback stabilization and state estimation of PDE–ODE cascades, where the PDEs are either of heat type or of wave type, from the previously considered cases where the interconnections are of Dirichlet type, to interconnections of Neumann type. The Neumann type interconnections constrain the PDE state to be subject to a Dirichlet boundary condition at the PDE–ODE interface, and employ the boundary value of the first spatial derivative of the PDE state to be the input to the ODE. In addition to considering heat-ODE and wave-ODE cascades, we also consider a cascade of a diffusion–convection PDE with an ODE, where the convection direction is “away” from the ODE. We refer to this case as a PDE–ODE cascade with “counter-convection.” This case is not only interesting because the PDE subsystem is unstable, but because the control signal is subject to competing effects of diffusion, which is in both directions in the one-dimensional domain, and counter-convection, which is in the direction that is opposite from the propagation direction of the standard delay (transport PDE) process. We rely on the diffusion process to propagate the control signal through the PDE towards the ODE, to stabilize the ODE.