Publications Lists     [ all BibTeX ]  

This paper aims to delineate a comprehensive method that integrates machine vision and deep learning for quality control within an industrial setting. The innovative approach proposed in this solution leverages a microservice architecture that ensures adaptability and flexibility to different scenarios while focusing on the employment of affordable, compact hardware, and achieves exceptionally high accuracy in performing the quality control task keeping a minimal computational time. Consequently, the developed system operates entirely on a portable smart camera, eliminating the need for additional sensors like photocells and external computation, which simplifies setup and commissioning phases and reduces the overall impact on the production line. By leveraging the integration of the embedded system with the machinery, this approach offers real-time monitoring and analysis capabilities, facilitating swift detection of defects and deviations from desired standards. Moreover, the low-cost nature of the solution makes it accessible to a wider range of manufacturing enterprises, democratizing quality processes in Industry 5.0. The system has been successfully implemented and is fully operational in a real industrial environment and the experimental results obtained from this implementation are also presented in the work.

Learning useful representations of complex data has been the subject of extensive research for many years. With the diffusion of Deep Neural Networks, Variational Autoencoders have gained lots of attention since they provide an explicit model of the data distribution based on an encoder/decoder architecture which is able to both generate images and encode them in a low-dimensional subspace. However, the latent space is not easily interpretable and the generation capabilities show some limitations since images typically look blurry and lack details. In this paper, we propose the Introspective Variational Classifier (IntroVAC), a model that learns interpretable latent subspaces by exploiting information from an additional label and provides improved image quality thanks to an adversarial training strategy.We show that IntroVAC is able to learn meaningful directions in the latent space enabling fine-grained manipulation of image attributes. We validate our approach on the CelebA dataset.

Recent results show that features of adversarially trained networks for classification, in addition to being robust, enable desirable properties such as invertibility. The latter property may seem counter-intuitive as it is widely accepted by the community that classification models should only capture the minimal information (features) required for the task. Motivated by this discrepancy, we investigate the dual relationship between Adversarial Training and Information Theory. We show that the Adversarial Training can improve linear transferability to new tasks, from which arises a new trade-off between transferability of representations and accuracy on the source task. We validate our results employing robust networks trained on CIFAR-10, CIFAR-100 and ImageNet on several datasets. Moreover, we show that Adversarial Training reduces Fisher information of representations about the input and of the weights about the task, and we provide a theoretical argument which explains the invertibility of deterministic networks without violating the principle of minimality. Finally, we leverage our theoretical insights to remarkably improve the quality of reconstructed images through inversion.

Visual inspection has recently gained increasing importance in the manufacturing industry and is often addressed by means of learning methodologies applied to data obtained from specific lighting and camera system setups. The industrial scenario becomes particularly challenging when the inspection regards reflective objects, which may affect both the data acquisition and the classification decision process, thus limiting the overall performance. In this context, we observe that the dynamics of the reflected light is the key aspect to characterize these surfaces and needs to be accurately exploited to improve the performances of the learning algorithms. To this aim, we propose a combined model-based and data-driven approach designed to detect defects on the reflective surfaces of industrial products, captured as video sequences under coaxial structured illumination. Specifically, a tunable spatial-temporal descriptor of the evolution of the reflected light (Dynamic Evolution of the Light, DEL) is designed and employed within a Hybrid Learning (HL) framework, where the learning process of a Convolutional Neural Network (CNN) is driven by the model-based descriptor. This approach is also extended by adopting the similar in nature descriptor Dynamic Image. The proposed HL solutions are validated against a whole spectrum of state-of-the-art learning procedures and different descriptors. Experiments run on a dataset coming from an actual industrial scenario confirm the ability of DEL to accurately characterize reflective surfaces and the validity of the HL method, which shows remarkably better performance in fault detection even with respect to modern 3D- CNNs with comparable computational effort.

The advent of the Industrial Internet of Things (IIoT) has led to the availability of huge amounts of data, that can be used to train advanced Machine Learning algorithms to perform tasks such as Anomaly Detection, Fault Classification and Predictive Maintenance. Even though not all pieces of equipment are equipped with sensors yet, usually most of them are already capable of logging warnings and alarms occurring during operation. Turning this data, which is easy to collect, into meaningful information about the health state of machinery can have a disruptive impact on the improvement of efficiency and up-time. The provided dataset consists of a sequence of alarms logged by packaging equipment in an industrial environment. The collection includes data logged by 20 machines, deployed in different plants around the world, from 2019-02-21 to 2020-06-17. There are 154 distinct alarm codes, whose distribution is highly unbalanced. This data can be used to address the following tasks: 

1. Next alarm forecasting: this problem can be framed as a supervised multi-class classification task, or a binary classification task when a specific alarm code is considered.
2. Predicting alarms occurring in a future time frame: here the goal is to forecast the occurrence of certain alarm types in a future time window. Since many alarms can occur, this is a supervised multi-label classification.
3. Future alarm sequence prediction: here the goal is predicting an ordered sequence of future alarms, in a sequence-to-sequence forecasting scenario.
4. Anomaly Detection: the task is to detect abnormal equipment conditions, based on the pattern of alarms sequence. This task can be either unsupervised, if only the input sequence is considered, or supervised if future alarms are taken into account to assess whether or not there is an anomaly.

 All of the above tasks can also be studied from a continual learning perspective. Indeed, information about the serial code of the specific piece of equipment can be used to train the model; however, a scalable model should also be easy to apply to new machines, without the need of a new training from scratch. The collection and release of this dataset has been supported by the Regione Veneto project PreMANI (MANIFATTURA PREDITTIVA: progettazione, sviluppo e implementazione di soluzioni di Digital Manufacturing per la previsione della Qualita e la Manutenzione Intelligente - PREDICTIVE MAINTENANCE: design, development and implementation of Digital Manufacturing solutions for the intelligent quality and maintenance systems).

In many real-world applications of Machine Learning it is of paramount importance not only to provide accurate predictions, but also to ensure certain levels of robustness. Adversarial Training is a training procedure aiming at providing models that are robust to worst-case perturbations around predefined points. Unfortunately, one of the main issues in adversarial training is that robustness w.r.t. gradient-based attackers is always achieved at the cost of prediction accuracy. In this paper, a new algorithm, called Wasserstein Projected Gradient Descent (WPGD), for adversarial training is proposed. WPGD provides a simple way to obtain cost-sensitive robustness, resulting in a finer control of the robustness-accuracy trade-off. Moreover, WPGD solves an optimal transport problem on the output space of the network and it can efficiently discover directions where robustness is required, allowing to control the directional trade-off between accuracy and robustness. The proposed WPGD is validated in this work on image recognition tasks with different benchmark datasets and architectures. Moreover, real world-like datasets are often unbalanced: this paper shows that when dealing with such type of datasets, the performance of adversarial training are mainly affected in term of standard accuracy.

One of the main challenges in developing Machine Learning-based solutions for Semiconductor Manu-facturing is the high number of machines in the production and their differences, even when consideringchambers of the same machine; this poses a challenge in the scalability of Machine Learning-based so-lutions in this context, since the development of chamber-specific models for all equipment in the fab isunsustainable. In this work, we present a domain adaptation approach for Virtual Metrology (VM), one ofthe most successful Machine Learning-based technology in this context. The approach provides a commonVM model for two identical-in-design chambers whose data follow different distributions. The approach isbased on Domain-Adversarial Neural Networks and it has the merit of exploiting raw trace data, avoidingthe loss of information that typically affects VM modules based on features. The effectiveness of theapproach is demonstrated on real-world Etching.

Solutions for the quality control of metallic surfaces are proposed. Specifically, we study a deflectometric apparatus based on coaxial structured light and the related algorithmic procedure, which is able to detect the faulty surface of a sample captured by a video sequence. First, by considering the metallic surface a dynamic scene illuminated under different light conditions, we develop the descriptor residuals of linear evolution of light (RLEL) that extracts the defectiveness information starting from the movement of the object without explicitly considering the physical characteristics of the light structure. Then, leveraging on RLEL, we present a hybrid learning (HL) technique capable of overcoming the data-driven approach used in classic deep learning (DL). By exploiting a multisteps training process, we combine the model-based descriptor RLEL and a classical data-driven convolutional neural network (CNN) to obtain an unconventional gray-box CNN, which exceeds the performance of popular DL solutions such as 3-D-inception and 3-D-residual DL networks. Remarkably, HL also shows its validity in comparing the performance of the same network structures trained not in a hybrid way, namely without the injection of the model-based information given by RLEL.

We propose a Deep Learning (DL)-based approach for production performance forecasting in fresh products packaging. On the one hand, this is a very demanding scenario where high throughput is mandatory; on the other, due to strict hygiene requirements, unexpected downtime caused by packaging machines can lead to huge product waste. Thus, our aim is predicting future values of key performance indexes such as Machine Mechanical Efficiency (MME) and Overall Equipment Effectiveness (OEE). We address this problem by leveraging DL-based approaches and historical production performance data related to measurements, warnings and alarms. Different architectures and prediction horizons are analyzed and compared to identify the most robust and effective solutions. We provide experimental results on a real industrial case, showing advantages with respect to current policies implemented by the industrial partner both in terms of forecasting accuracy and maintenance costs. The proposed architecture is shown to be effective on a real case study and it enables the development of predictive services in the area of Predictive Maintenance and Quality Monitoring for packaging equipment providers.

Virtual Metrology is one of the most prominentAdvanced Process Control applications in SemiconductorManufacturing. The goal of Virtual Metrology is to provideestimations of quantities that are important for production andto assess process quality, but are costly or impossible to bemeasured. Virtual Metrology solutions are based on MachineLearning approaches. The bottleneck of developing VirtualMetrology solutions is generally the feature extraction phase thatcan be time-consuming, and can deeply affect the estimationperformance. In particular, in presence of data with additionaldimensions, such as time, feature extraction is typicallyperformed by means of heuristic approaches that may pickfeatures with poor predictive capabilities. In this work, wepropose the usage of modern Deep Learning approaches tobypass manual feature extraction and to provide highperformanceautomatic Virtual Metrology modules. Theproposed methodology is tested on a real industrial datasetrelated to Etching. The dataset at hand contains OpticalEmission Spectroscopy data and it is paradigmatic of the featureextraction problem under examination.