Quality control at the touch of a finger
For production companies, quality assurance of the process chain is essential - it is the only way to detect problems at an early stage and reduce additional costs. Fraunhofer researchers have developed an efficient method of quality control: Using a pointing gesture, employees can enter and document defects detected in car body parts into the inspection system.
The quality inspector meticulously scrutinizes the bumper and examines it for paint damage - after all, only flawless body parts are allowed to enter final assembly. If he finds a defect in the paint, all he has to do is point a finger and the defect is forwarded to the inspection system, where it is stored and documented. The employee receives visual feedback via a monitor that displays a 3D reconstruction of the bumper.
Testing is massively facilitated
What seems futuristic at first glance could soon become part of everyday life in quality assurance: Researchers at the Fraunhofer Institute of Optronics, System Technologies and Image Exploitation IOSB in Karlsruhe have developed the intelligent gesture control system on behalf of the BMW Group. In the future, it will replace current, time-consuming testing procedures. Until now, the inspector has had to memorize all the detected defects, leave his workstation, go to the PC terminal, make several entries, and then check the results.
Interaction with the space
The user has to operate a mask and then record the position of the defects and the type of defect. This is cumbersome, time-consuming and error-prone. Gesture control, on the other hand, decisively improves the working conditions of the inspector and results in significant time savings - the employee can remain at the workstation and interact directly with the object under inspection. If the bumper is in order, he swipes it from left to right. In case of damage, he points to the position of the defect.
3D tracking captures people and objects in real time
The basis for contactless gesture recognition is 3D data. The complete workplace must therefore be reconstructed in 3D beforehand. This includes both the person and the object he or she is working with: What does the person look like, where is he located, how does he move, what does he do, where is the object - all this information is required to correctly link the pointing gestures with the bumper. To enable gesture control, 3D body tracking is used to capture the person's posture in real time. The body component is also "tracked". The hardware requirements are low: a standard PC and two Microsoft Kinect systems - consisting of camera and 3D sensors - are sufficient to realize the reconstruction.
The corresponding algorithms, which fuse several 2D and 3D images, were developed by the FhG team especially for this application and adapted to the requirements of the BMW Group.
The nucleus for this technology is the Smart Control Room, in which people interact naturally with the room. They can use pointing gestures to operate remote displays without additional devices. The room recognizes which actions are currently taking place and offers the appropriate information and tools. Since gesture recognition is independent of displays, you are able to implement applications that do not use monitors, such as gesture interaction with real objects in this case. It doesn't matter what kind of object it is. Instead of the bumper, another component could also be tracked.
The technology can be subsequently integrated into existing production systems with little effort. The scientists were able to integrate their effective process into the BMW Group's system via a special interface module. The plan is to install a demonstrator at the BMW plant in Landshut in 2013. In cooperation with the quality inspectors on site, the system will be refined before it is used in production in the future.
The gesture recognition system will be presented at the Hannover Messe 2013 from April 8 to 12 at the Fraunhofer joint booth in hall 2, booth D18.