Skip to main content

Fujitsu

Global

Archived content

NOTE: this is an archived page and the content is likely to be out of date.

Fujitsu Develops Technology to Automate Construction of Visual-Inspections Programs for Production-Lines

Uses Zinrai AI-based machine learning to quickly start production lines

Fujitsu Laboratories Ltd.

Kawasaki, Japan, November 18, 2015

Fujitsu Laboratories Ltd. today announced that it has developed a technology that automatically generates visual-inspection programs that use images to automatically detect part-mounting and surface defects on production lines for electronic devices.

Existing technologies for automatically generating image-processing programs work by recognizing certain geometric primitives, such as straight lines and circles, and recognizing the positions of assembled parts.

Fujitsu Laboratories has built on this technological base and developed two new technologies. The first technology is trained on the method for comparing against pre-registered images of parts with complex shapes and the whole board, and then applies the comparison method against images captured on the production line to recognize when parts are out of alignment. The second technology identifies part defects by recognizing a variety of image features, including brightness and line orientations, in the patterns of parts images and making accept/reject decisions.

Fujitsu Limited's Human Centric AI Zinrai technology now makes it possible to automatically generate visual-processing programs for the inspection process in one-fifth the time it would take to create them by hand, allowing for faster production-line start-ups and more responsiveness to changes in product specifications. Furthermore, achieving highly accurate automated inspections with little discrepancy ensures consistent high quality products.

This technology is being exhibited at Fujitsu Forum 2015, running November 18-19 in Munich, Germany. Details of this technology are being presented at Vision Engineering Workshop 2015 (ViEW2015), opening December 3 in Yokohama, Japan.

Background

While there has been progress in automating camera-based visual inspections of parts mounting and product assembly and surface appearance on production lines, adapting to different types of parts and materials quantities requires that the image-processing system be updated, which creates a need for a faster updating process. The technology that Fujitsu Laboratories had previously developed for automatically generating image-processing programs(1) was designed to check alignment of mounted parts and assembled products, and it worked by recognizing simple geometric primitives, such as straight lines and circles. This was found to have a dependable recognition rate that allowed production lines to run without stopping, but in order to improve overall line efficiency, this would need to be applied to the inspection process itself (Figure 1).

Figure 1: Use of cameras in production linesFigure 1: Use of cameras in production lines

Issues

The inspection process on a production line makes decisions by checking whether a part has been mounted, whether it is aligned correctly, and whether there are surface defects (such as scratches or attached foreign matter). These inspection criteria can become quite diverse depending on the product's specifications, and automated camera inspections need to be able to recognize a wide range of part shapes and make decisions based on image features (Figure 2).

Figure 2: Typical inspections of electronic-parts mountingFigure 2: Typical inspections of electronic-parts mounting

About the Technology

Fujitsu Laboratories has developed a technology that improves existing technologies for generating and evaluating programs, and that automatically generates visual-processing programs for the inspection process. Key advances of this technology are that it can detect misalignments in parts mounting and can identify part defects. This technology uses the machine-learning, image-processing, and optimizing capabilities of Fujitsu's Zinrai AI technology.

Features of the technology are as follows:

1. Detects misalignments in mounted parts

To be able to handle a wide range of part shapes, the conventional approach is to store images of all the parts to be recognized beforehand, and calculate the degree of similarity between each of those and photographs captured on the production line to identify the positions of parts. However, to avoid incorrect detections of similarly shaped parts, a number of parameters need to be tuned for each part, relating to image-correction processing and threshold values. With this technology, Fujitsu Laboratories has now automated this tuning process.

As a program that recognizes any given shape, Fujitsu Laboratories has developed a program structure whereby two images - one of the part and another of a whole board - are input, and by cross checking their respective image transformations the program outputs the degree of similarity distribution. This structure uses a machine-learning technique called genetic algorithms to optimize the parameters being compared (Figure 3). Simply by recording the correct position of parts as sample data with respect to the images of the parts to be recognized and of the whole boards being photographed in situ, the new technology can automatically generate programs that are little affected by differences in lighting, and can correctly recognize parts even when there are similarly shaped parts nearby, resulting in highly accurate alignment detection.

Figure 3: Structure and optimization of program for complex-shape recognitionFigure 3: Structure and optimization of program for complex-shape recognition

2. Identifies parts defects

This technology extracts feature values such as brightness, contrast, and edge orientation via a number of image transformations, and generates a model from that data that represents a good part. Taking the decisions of images of good and defective parts from previous training and selectivity based on a variety of feature values as evaluation criteria, the genetic algorithms optimize the model, and automatically generate a high-performance image-classification machine (Figure 4).

Figure 4: Structure and optimization of program for identifying part defectsFigure 4: Structure and optimization of program for identifying part defects

Results

Internal testing of image-processing programs for part-mounting inspections on actual production lines showed that a program, having the same performance as one developed by a specialist, could be automatically generated with roughly two hours of training. This allows inspection programs to be developed in roughly one-fifth the time, for faster production line starts and more agility in updating systems to accommodate changes in specifications, while maintaining dependable line operations. Furthermore, achieving highly accurate automated inspections with little discrepancy contributes to consistently high quality products.

Future Plans

Fujitsu Laboratories is working to improve the performance of inspection programs used in product assembly and inspection processes, with the goal of practical implementation of this technology as a "Monozukuri Solution" during fiscal 2015.


  • [1] Technology that Fujitsu Laboratories had previously developed for automatically generating image-processing programs

    Press release issued September 9, 2014: Fujitsu Laboratories Develops World's First Software to Automate Production-Line Image Recognition (http://www.fujitsu.com/global/about/resources/news/press-releases/2014/0909-01.html)

About Fujitsu

Fujitsu is the leading Japanese information and communication technology (ICT) company, offering a full range of technology products, solutions, and services. Approximately 159,000 Fujitsu people support customers in more than 100 countries. We use our experience and the power of ICT to shape the future of society with our customers. Fujitsu Limited (TSE: 6702) reported consolidated revenues of 4.8 trillion yen (US$40 billion) for the fiscal year ended March 31, 2015. For more information, please see http://www.fujitsu.com.

About Fujitsu Laboratories

Founded in 1968 as a wholly owned subsidiary of Fujitsu Limited, Fujitsu Laboratories Ltd. is one of the premier research centers in the world. With a global network of laboratories in Japan, China, the United States and Europe, the organization conducts a wide range of basic and applied research in the areas of Next-generation Services, Computer Servers, Networks, Electronic Devices and Advanced Materials. For more information, please see: http://www.fujitsu.com/jp/group/labs/en/.

Press Contacts

Public and Investor Relations Division
Inquiries

Company:Fujitsu Limited

Technical Contacts

Monozukuri Technologies Laboratory


E-mail: E-mail: fa-vision-2015@ml.labs.fujitsu.com
Company:Fujitsu Laboratories Ltd.


All company or product names mentioned herein are trademarks or registered trademarks of their respective owners. Information provided in this press release is accurate at time of publication and is subject to change without advance notice.

Date: 18 November, 2015
City: Kawasaki, Japan
Company: Fujitsu Laboratories Ltd.