Technology to automate construction of visual-inspections programs for production-lines
Fujitsu Laboratories today announced that it has developed a technology that automatically generates visual-inspection programs that use images to automatically detect part-mounting and surface defects on production lines for electronic devices. Existing technologies for automatically generating image-processing programs work by recognizing certain geometric primitives, such as straight lines and circles, and recognizing the positions of assembled parts. Fujitsu Laboratories has built on this technological base and developed two new technologies. The first technology is trained on the method for comparing against pre-registered images of parts with complex shapes and the whole board, and then applies the comparison method against images captured on the production line to recognize when parts are out of alignment. The second technology identifies part defects by recognizing a variety of image features, including brightness and line orientations, in the patterns of parts images and making accept/reject decisions.
Fujitsu Limited's Human Centric AI Zinrai technology now makes it possible to automatically generate visual-processing programs for the inspection process in one-fifth the time it would take to create them by hand, allowing for faster production-line start-ups and more responsiveness to changes in product specifications. Furthermore, achieving highly accurate automated inspections with little discrepancy ensures consistent high quality products. This technology is being exhibited at Fujitsu Forum 2015, running November 18-19 in Munich, Germany. Details of this technology are being presented at Vision Engineering Workshop 2015 (ViEW2015), opening December 3 in Yokohama, Japan.
While there has been progress in automating camera-based visual inspections of parts mounting and product assembly and surface appearance on production lines, adapting to different types of parts and materials quantities requires that the image-processing system be updated, which creates a need for a faster updating process. The technology that Fujitsu Laboratories had previously developed for automatically generating image-processing programs(1) was designed to check alignment of mounted parts and assembled products, and it worked by recognizing simple geometric primitives, such as straight lines and circles. This was found to have a dependable recognition rate that allowed production lines to run without stopping, but in order to improve overall line efficiency, this would need to be applied to the inspection process itself (Figure 1).
The inspection process on a production line makes decisions by checking whether a part has been mounted, whether it is aligned correctly, and whether there are surface defects (such as scratches or attached foreign matter). These inspection criteria can become quite diverse depending on the product's specifications, and automated camera inspections need to be able to recognize a wide range of part shapes and make decisions based on image features (Figure 2).
About the Technology
Fujitsu Laboratories has developed a technology that improves existing technologies for generating and evaluating programs, and that automatically generates visual-processing programs for the inspection process. Key advances of this technology are that it can detect misalignments in parts mounting and can identify part defects. This technology uses the machine-learning, image-processing, and optimizing capabilities of Fujitsu's Zinrai AI technology. Features of the technology are as follows:
1. Detects misalignments in mounted parts
To be able to handle a wide range of part shapes, the conventional approach is to store images of all the parts to be recognized beforehand, and calculate the degree of similarity between each of those and photographs captured on the production line to identify the positions of parts. However, to avoid incorrect detections of similarly shaped parts, a number of parameters need to be tuned for each part, relating to image-correction processing and threshold values. With this technology, Fujitsu Laboratories has now automated this tuning process. As a program that recognizes any given shape, Fujitsu Laboratories has developed a program structure whereby two images - one of the part and another of a whole board - are input, and by cross checking their respective image transformations the program outputs the degree of similarity distribution. This structure uses a machine-learning technique called genetic algorithms to optimize the parameters being compared (Figure 3). Simply by recording the correct position of parts as sample data with respect to the images of the parts to be recognized and of the whole boards being photographed in situ, the new technology can automatically generate programs that are little affected by differences in lighting, and can correctly recognize parts even when there are similarly shaped parts nearby, resulting in highly accurate alignment detection.
2. Identifies parts defects
This technology extracts feature values such as brightness, contrast, and edge orientation via a number of image transformations, and generates a model from that data that represents a good part. Taking the decisions of images of good and defective parts from previous training and selectivity based on a variety of feature values as evaluation criteria, the genetic algorithms optimize the model, and automatically generate a high-performance image-classification machine (Figure 4).
Internal testing of image-processing programs for part-mounting inspections on actual production lines showed that a program, having the same performance as one developed by a specialist, could be automatically generated with roughly two hours of training. This allows inspection programs to be developed in roughly one-fifth the time, for faster production line starts and more agility in updating systems to accommodate changes in specifications, while maintaining dependable line operations. Furthermore, achieving highly accurate automated inspections with little discrepancy contributes to consistently high quality products.
Fujitsu Laboratories is working to improve the performance of inspection programs used in product assembly and inspection processes, with the goal of practical implementation of this technology as a "Monozukuri Solution" during fiscal 2015.