Whereas the development of technical devices (e.g., smartphones, automotive technology, medical technology) rapidly progresses, the improvement of their interaction with the user plays a minor role. Usually, machines are tools whose operation has to be learned and mastered. In order to realize an adequate human-machine interaction instead, the machine needs to be able to detect and interpret human gestures and facial expressions.
At the Fraunhofer IOF, we have developed various systems that enable us to capture fast body movements three-dimensionally. A sequence of certain patterns is projected onto a person by a newly developed high-speed projector and observed by a stereo camera system. By correlating the camera images and subsequently performing triangulation, we can generate several hundred to thousand high-precision 3D point clouds per second.
Due to the high radiant flux of its projector, one of the developed 3D sensors is suited for large measurement fields and thus whole-body measurements. Using this sensor, we recorded a rope skipper in motion, a soccer player during a shot, and other highly dynamic processes at a 3D frame rate of more than 1,300 Hz and at a resolution of up to 1,000 x 1,000 points.
Furthermore, we developed a system which features the fast projection and detection of the patterns at a wavelength of 850 nm. By transitioning from the visible spectral range to the near infrared (NIR, being imperceptible to the human eye), disturbing glare effects are avoided. Besides, an additional RGB camera can be used to simultaneously obtain color information. Hence, the developed sensor can be used for detecting and analyzing, e.g., emotional face expression.
The spectrum of potential system configurations ranges from superfast (several kilohertz 3D rate), through very large measurement fields (several square meters) to irritation free. This opens up further possibilities for human-machine interaction, e.g., with regard to interactive training systems, car interior monitoring, or security technology in public space. Future investigations at the Fraunhofer IOF aim to optimize the hardware as well as the algorithms for 3D data computation.