Manufacturing Industry DX through Deep Learning Application Technologies
—From Non-Destructive Testing to Quality Control—
Release date: May 15, 2020
This article describes research and development of Fujitsu AI technologies toward digital transformation (DX) of the manufacturing industry. Manufacturing sites backed by high productivity and excellent quality control are now faced with key issues such as a decline in the working-age population and difficulties in worker training. However, to provide high added value and boost market competitiveness, there is a need for revolutionizing not only manufacturing sites but also the entire manufacturing process. In response to these challenges, Fujitsu Laboratories of Europe (FLE) is working on solutions that has the potential to revolutionize both the manufacturing industry as well as digitalizing manufacturing sites through AI technologies. At FLE, we have developed new anomaly detection technology for detecting problems on manufacturing lines and detecting manufacturing defects in a wide variety of materials and components with high accuracy. We are also in the process of developing a deep-learning application technology called an AI simulator to also expand the application range of AI technologies to the design process by means of massively accelerated simulations of physical phenomena.
- 1. Introduction
- 2. Issues in manufacturing industry DX
- 3. Anomaly detection technology requiring no anomaly data
- 4. High-speed physical simulator technology
- 5. Conclusion
1. Introduction
The age in which an effective workforce could establish a competitive advantage is over. Thus, in today’s manufacturing industry, technology is the determining factor in gaining competitive dominance while also being a source of innovation. Materials, components, and products that provide added value such as novel functions, durability, and high environmental performance have been achieved through engineering achievements and research and development of manufacturing technologies. At the same time, the importance of on-site quality control in actual manufacturing and supply processes is widely recognized. Nevertheless, despite the need for advanced quality-control operations, where product functions become increasingly sophisticated, quality control is still carried out with an emphasis on visual inspection by skilled workers, forcing them to take on an ever-growing burden. In addition, the shrinking of the working-age population in Japan is becoming an increasingly serious issue.
In its White Paper on Monodzukuri 2019 (FY2018 Measures to Promote Manufacturing Technology) [1], the Ministry of Economy, Trade and Industry (METI) proposed a digital transformation (DX) of the manufacturing industry centered around AI and IoT technologies.The aim of which is to change an acute labor shortage into a favorable tailwind via technology digitalization and extensive labor-saving processes. At Fujitsu Laboratories of Europe (FLE), this declaration was actually preceded by the launch in FY2017 of a research and development initiative toward a transformation of a number of applications in the manufacturing industry through AI technologies. In particular, we developed a deep-learning-based technology targeting non-destructive testing that digitalized the process of detecting manufacturing defects in wind turbine blades; a procedure that has traditionally been done manually [2, 3].This timely development dramatically shortened the task of interpreting ultrasonic echo images.
Continuing on from the digitalization of non-destructive-testing technology, we here report on a new deep-learning application technology for achieving extensive labor savings in manufacturing and inspection processes. We also report on the development of an AI simulator [4, 5, 6, 7, 8], a deep-learning-based technology for simulating physical phenomena with the aim of expanding the application range of AI technologies to the design and development processes. The remainder of this article is organized as follows. Section 2 discusses problems to be solved by the aforementioned new technologies. Thereafter, in relation to each of these technologies, Sections 3 and 4 describe issues and solutions as well as introduce application examples before touching upon future issues. Finally, Section 5 summarizes the content of this article and presents a future outlook.
2. Issues in manufacturing industry DX
A schematic representation of the research-target transition towards ‘manufacturing industry DX’ by means of applying FLE’s AI technologies is shown in Figure 1.As described in the previous section, the trend in exploiting recent advances in modern AI to address some of the challenges in manufacturing began in FLE with the topic of non-destructive testing. In such processes, the task of judging anomaly locations has traditionally been a very time-consuming process for humans. A skilled and experienced worker would use a high-precision measuring instrument to accurately determine the presence of faults. This is an excellent example of work in which a person’s know-how can be digitalized, a strong candidate for deep learning today. This technology has already been commercialized as a non-destructive-testing solution called Fujitsu Advanced Image Recognition (F|AIR) [3], which has the significant effect of shortening inspection time as well as making the accuracy of judgments more uniform.
Figure 1 Quality control targeted by FLE.
To build on these recent efforts, we decided to further expand the application of AI to inspections in production processes. This includes, for example, the task of detecting flaws or smudges on the surface of materials and the task of detecting errors in component processing or painting. The difficulty of such inspection tasks is wide-ranging. It extends from the difficulty of imagining all possible defect patterns upfront, to the fact that the rate of occurrence of defects is actually low. For these reasons, in addition to having to make a “good/bad” judgment on the quality of a product using only data, it must also be possible to identify fault locations. Deep learning has been advancing explosively in recent years, and while individual technologies have been studied for solving each of these issues, there was a clear need for developing a technique that could solve all of them simultaneously. The anomaly detection technology described below aims to fulfill this need. It also aims to achieve extensive labor savings even for tasks that have been considered in the past to only be possible to do visually by means of specialized inspection personnel.
In addition to the above, we also endeavored to expand the applications of AI from production processes to design and development processes. While computer simulation have been used for some time to evaluate performance at the design and development stage, the computational cost of creating a mathematical model of physical phenomena and performing detailed simulations can be enormous. The AI simulator is an AI-based technology that can replace such model-based numerical calculations. Therefore, it can simplify the iteration of design/development work and performance evaluation from learning only the relationship between physical conditions and phenomena using deep learning. The key to achieving this capability is also a technology that can further evaluate the reliability of an output [7, 8] when substituting model-based calculations with model-free deep learning.
In this way, we have come to expand the application range of AI technologies in quality control from the digitalization of inspection processes to the digital innovation in design and development processes. The following sections describe in more detail anomaly detection technology and an AI simulator based on deep learning.
3. Anomaly detection technology requiring no anomaly data
This section describes anomaly detection technology based on deep learning. The process flow of this technology during training and actual operation is shown in Figure 2.
Figure 2 Training/operation process flow of anomaly detection technology
At training time, the process first uses only normal (anomaly-free) image data that captures only good items and artificially adds anomalies to part of this normal image data through artificial anomaly insertion technology. It then trains a neural network to restore the normal image from that with the added anomalies. Next, at operation time, this trained neural network can be used to convert an input image with or without anomalies into an image that includes no anomalies. Thereafter, by comparing the input image with the post-conversion restored image, it becomes possible to not only ascertain the presence of an anomaly but also its location.
In short, this technology is designed to perform three key processes: (1) artificially add anomalies, (2) train with images that have no anomalies, and (3) evaluate anomaly location from the difference between input/output images. Additionally, since only image data of good items are used as training input, this technology has the advantage of completing training with a relatively small amount of data and it can also be introduced into the workplace in a relatively short period of time.
We evaluated this technology using benchmark data assuming an anomaly detection task in manufacturing. Regardless of whether the target was materials, components, or products, this evaluation revealed that our technology has significantly improved the average accuracy when compared with the results of prior research from other companies or universities. This provides excellent performance results in anomaly detection tasks from images. We are currently holding technology trials with a number of partner companies in various fields, such as, steel, beverages, foodstuffs, and automobiles. In one of these trials involving a task that detects problems on a manufacturing line, high-accuracy results were obtained about two weeks after receiving data. As a result, we are now in the process of customizing the technology toward the implementation of a full-scale system. We further expect this technology to be applied to image-based anomaly detection in other fields such as, for example, aging-infrastructure inspections for bridges, paved roads, and steel towers. We also plan to make improvements to this technology including reducing the scale of processing and boosting its speed during actual operation.
4. High-speed physical simulator technology
This section describes the AI simulator as a technology for approximating physical simulators with both high accuracy and extremely high speeds. As shown in Figure 3,the AI simulator first converts geometries with associated inputs of physical structures and corresponding outputs into equally spaced voxels that represent this information in regions of 3D spaces. By training with output data that is calculated by physical simulators on the basis of a mathematical model as ground-truth data with respect to the input data, the AI simulator trains as a generative neural network to simulate physical phenomena. For example, in the case of computational fluid dynamics (CFD) simulation, geometry information may consist of the placement and density of objects, coefficient of thermal conductivity, amongst others, that can be used to calculate a heat transfer distribution.
Figure 3 Configuration of an AI simulator.
An example of applying the AI simulator to fluid analysis and load analysis is shown in Figure 4.In the figure, (a) shows a heat transfer distribution from fluid calculations and (b) shows a displacement distribution from structural calculations. In both cases, it can be seen that the AI simulator achieves a level of accuracy on a par with that of the physical simulator.
Figure 4 Application example of an AI simulator.
A large-scale generative neural network is needed to simulate complex physical phenomena. A means of training to achieve a high-accuracy simulation of physical phenomena and a means of evaluating the reliability of output are also needed. Our technology satisfies these needs by solving the three issues of (1) converting geometry information into an appropriate voxel format, (2) training a large-scale generative neural network with good accuracy, and (3) evaluating the uncertainty in an output [7, 8]. Furthermore, if a voxel format, as is the case in today’s neural networks, is to be used as it is, the computational cost of simulation would be massive. We have overcome this problem by performing calculations only against a region that greatly affects the output. This cutting-edge technology was evaluated as having much potential in industrial applications and was the recipient of the Intel DevMesh AI Spotlight Award [9].
We are currently making improvements to this technology to enable it to handle more complex structures and on a larger scale than currently possible with the aim of holding technology trials in various sectors, including energy, automobiles, and other fields. Furthermore, considering that physical simulation includes mathematical models other than those for handling fluids and structures, we also plan to expand the range of applications and models this technology can support. Additionally, the generative network technology cultivated by the AI simulator can be effectively applied to other problems involving quality inspections. Therefore, we further plan to support the use of the AI simulator in other operations in the manufacturing industry.
5. Conclusion
In this article, we described our development of anomaly detection technology and an AI simulator based on deep learning as technologies focused on inspection work in the production process and on design and development tasks. The aim of these advanced technological efforts is achieving high quality control that extends as far as the manufacturing process. For both of these technologies, we first plan to use them in co-creation businesses within the manufacturing industry. We will then go on to provide them as solutions within FUJITSU Human Centric AI Zinrai and Fujitsu’s manufacturing business brand Fujitsu Manufacturing Industry Solution COLMINA [10] , whereby engaging in both technology testing and technology extensions and in collaboration with a variety of customers. Going forward, we will continue with the development of technologies that respond to customers’ needs in the manufacturing-industry fields with a focus on quality inspection as well as design and development. We also aim to contribute to digitalization and digital innovation in the manufacturing industry through the use and introduction of AI technologies that target the entire manufacturing processes.
All company and product names mentioned herein are trademarks or registered trademarks of their respective owners.
References and Notes
- A. Al-Jarro et al.: “DeepSim-HiPAC: Deep learning-based platform converts physics-based simulators into real-time AI Simulators.” In NVIDIA GPU Tech, USA, 2019.Back to Body
About the Authors
Mr. Inakoshi is currently engaged in research and development of applied artificial intelligence and mathematical technologies.
Dr. Hida is currently engaged in research and development of applied artificial intelligence and mathematical technologies.
Dr. Al-Jarro is currently engaged in research and development of applied artificial intelligence and mathematical technologies.
Dr. Georgescu is currently engaged in research and development of applied artificial intelligence and mathematical technologies.
