Working to accelerate the social implementation of AI (Operation Process Analysis & Product Defect Inspection chapter)

Atsunori (Operation Process Analysis): Our customers in the manufacturing industry need to analyze the work of each worker in order to improve the efficiency of their manufacturing sites and to pass on skills. However, in the past, it was necessary for a supervisor to observe and analyze the work as it was done, and then measure the time required with a stopwatch or other device, involving a time-consuming and costly exercise. The Operation Process Analysis component can automatically estimate the type of work being performed by workers based on camera images. As a result, it can optimize the manufacturing site by recognizing and eliminating wasteful work, noting any variations between workers, and identifying potentially dangerous postures that could lead to accidents and injuries, while reducing costs at the same time.

Shunichi (Product Defect Inspection): In the manufacturing industry, failing to identify a defective product can severely impact on user confidence. With our approach, it is possible to detect defective products from image data with high accuracy, thereby reducing the risk of defective products leaking in to the marketplace. When creating an AI model to detect defective products, it is common practice to use both good and defective product data. However, in actual manufacturing sites, defective product data is difficult to obtain, and it takes time to secure sufficient data. Therefore, we have developed a technology to construct a highly accurate AI model using only good data, constantly estimating and monitoring the accuracy of the AI model and automatically repairing the model when the accuracy tends to decline. This is important as the accuracy of the AI model may decline due to changes in the on-site environment. These technologies are aimed at realizing the improved operational stability of AI in Product Defect Inspection.

Atsunori: The Operation Process Analysis component uses Actlyzer (*3), a behavior analysis technology that recognizes human behavior from camera images. It is capable of recognizing approximately 100 basic behaviors that have been learned in advance, and, can also recognize complex behaviors by combining basic behaviors. For example, a "Smartphone Zombie" can be defined and recognized as a combination of the basic actions of "walking," "face down," and "arms forward”. In this way, detailed rules for work can be set according to the actual site situation. Real-time images from cameras installed at the work site can be used to analyze worker behavior, such as measuring work time and confirming work procedures. In addition, by recording the timing and location of falls and potentially dangerous postures, etc., and identifying on which tasks they occur, it is possible to identify risks based on an understanding of the current situation.

Shunichi: The Product Defect Inspection component uses two main technologies. First is the Defect Removing AutoEncoder (DeRA) (*4), an image AI for Product Defect Inspection, which creates AI models using only images of good products to detect various product defects and accurately locate them. The second is High Durability Learning (HDL) High Durability Technology (*5), which is responsible for the stable operation of AI. This technology can automatically control the degradation of AI accuracy when environmental or operational data changes that cause AI model degradation occur, thereby helping to solve the problem of AI accuracy degradation during operation. These technologies can continue to detect defective products generated in the manufacturing process with high accuracy, reducing the risk of outflow to the market due to missed products, as well as improving product productivity and quality.

Atsunori: Actlyzer won first place in Papers With Code, the well-known benchmark in the machine learning field, and was also accepted at the top conferences ICPR2020 (*6), ICPR2022 and ICIP2022 (*7). It also won second place in the open competition for multi-person tracking technology at ECCV2022 (*8). The reason for the explosive interest of generative AI services over the past few months is that the generative AI can easily be used by anyone, unlike conventional AI, that requires highly specialist skills. Actlyzer is also characterized by its no-code UI operation. By combining pre-prepared basic movements with drag-and-drop, complex movement rules can be set, allowing non-technical personnel to customize the work movements they wish to recognize.

Shunichi: DeRA excels at accurately identifying anomalies. In a benchmark for anomaly detection in 2021, DeRA achieved the world's highest level of 98% in AUROC (Area Under the ROC Curve), an index that measures the performance of AI models in detecting anomalies (*9). Not only is HDL faster than other methods, it also achieves a lower error rate than conventional methods in an online learning benchmark of corrupted data as of 2022 (*10), as well as a level of accuracy equivalent or better than the state-of-the-art technology of the time.

Atsunori: Fujitsu I-Network Systems Limited is actually using the Operation Process Analysis component. They are currently working on identifying priorities for improvement by using this component. For example, they can calculate unexpected work input or time spent not working by subtracting the actual work time from the allocated time. In addition, knowledge about the working time differences by operator and by product type represents important information for future improvement.

Atsunori: Of course, there were essential features that we discovered only as a result of our customers using the system. For example, the need for worker identification and more detailed recognition of hand movements. We hope to solve our customers' problems while deepening our technology to realize such functions.

Shunichi: We are currently inspecting vein sensor components at Fujitsu Frontech Limited as a demonstration test of this component. For vein sensor devices, we identify good ones and defective ones with scratches and stains. The objective is to achieve stable operation allowing for degradation of the AI model caused by environmental changes, especially during product inspection.

Atsunori: When we develop technology, we place great importance on ease of use and ease of implementation for our customers. In the early days of Actlyzer, we received comments from customers in the field that it was difficult for them to use the tool. The problem was that they could not operate it themselves, as the version being used was more or less the same as the researchers’ version used to perform the analysis. We have now made improvements and introduced a no-code interface, in which the customer specifies the combination of actions with a mouse operation, improving ease of use. We have also focused on overcoming another issue, involving differences in the environment of the customer's PCs at the site, making it difficult to install the system and to build the environment. We can now use the environment built in the cloud, which allows us to demonstrate the system quickly. For on-site demonstrations, we provide containers that can run on the customer's PC, making it easy to introduce the system to the site.

Atsunori: I believe it is important to combine the top-down approach of narrowing down the functions and technologies needed based on customer requests with the bottom-up approach of applying cutting-edge technologies to components. Although cutting-edge technologies are being announced rapidly in AI research, there is a long delay before they are used by customers. I believe that Fujitsu Kozuchi can support this process of implementation by directly focusing on the functions required by the customer. I am also in discussion and jointly developing elements with many development members globally, such as in our research center in China and India. Thanks to teleworking, we are able to communicate with them regardless of location and time.

Shunichi: I believe that having customers try out Fujitsu Kozuchi is one very important way to encourage researchers to realize their technological contributions. Also, although it is not limited to Fujitsu Kozuchi components, I believe that the most important thing is for the solution to fit the problem that the customer really wants to solve, so I cherish every opportunity to talk with each customer.

Atsunori: Changes in the environment surrounding the manufacturing industry are becoming more and more pronounced. AI is expected to provide solutions to issues such as the decrease, aging, and diversification of engineers, as well as responding to mass customization and to increasingly sophisticated after-sales support requests. Some may feel that the Operation Process Analysis component monitors workers, but I believe that this technology can protect workers and help create a more comfortable work environment in the rapidly changing manufacturing industry. Currently, some work contents are not yet fully covered, but I would like to expand its functionality as a component based on customer needs and the latest technologies. By accelerating the research and development of AI, I hope to contribute to making factory operation smarter throughout society.

Shunichi: With the recent rise of generative AI, the general public has become more familiar with the existence of AI. As evidenced by the surge in stock prices in the semiconductor sector, which is indispensable for AI development, this trend is beginning to boost the B-to-B AI business. As AI services become more widespread, I feel that the day is rapidly approaching when customers will face the next challenge of stable operation and quality improvement during operation. The HDL technology for Product Defect Inspection, will become indispensable technology for AI operation in the future. This technology is exactly what will be required when AI becomes a pillar of various social services, and I believe it will have a great impact.