Real-Time Scanner for Robot Interaction

While the development of human-machine interfaces is advancing fast (e.g. speech control, touch interfaces), accurate real-time measurement of human-machine interaction in 3D space is a field where much research is still outstanding.

To meet the requirements of this field, we have developed a real-time 3D measuring system which can provide accurate 3D data of complex human-machine interaction scenes at high frame rates with low latency. One of the key requirements of the system was that it is capable of working irritation free in the vicinity of humans.

To demonstrate the system, we have built an exhibit consisting of the 3D scanner, a robot arm, and an interaction surface (made of rubber). The surface is observed by the 3D scanner and continuously generates new target coordinates for the robot based on the human interaction with the rubber surface. In addition, two LCD displays show the 3D model reconstructed by the scanner to provide additional visual feedback of the scan.

The 3D measuring system consists of calibrated stereo cameras with an active high-speed projector using aperiodic sinusoidal patterns. The projector works with invisible infrared light (850 nm wavelength) and therefore does not irritate humans being scanned (glare-free). In real-time operation, the system provides a 3D data rate of 36 3D data sets per second with a latency of 80 ms. Each 3D dataset consists of one million 3D points.

The scanner‘s cameras operate synchronously and each of them captures 360 images per second. The 3D reconstruction calculation is carried out on a standard PC completely on the graphics hardware (GPU program with OpenCL) so that full CPU performance is available for subsequent evaluation of the 3D data. This enables a wide range of applications with different data processing requirements, such as continuous shape monitoring in assembly line systems, pick-and-place applications, face detection in security gates, or 3D gesture recognition.

 

Authors: Patrick Dietrich, Kevin Srokos, Ingo Gebhart, Peter Kühmstedt, Gunther Notni