Motion in depth

This dissertation is about sensing and utilizing various kinds of human body motion, ranging from large-scale limb movements down to subtle respiratory motion of the upper body. One key idea hereby is to simultaneously capture different types of limb and body movements from two inherently different input modalities and to combine that data in a complementary way. The used modalities are a depth camera on the one hand and body-worn inertial measurement units on the other hand. It will be shown how such a complementary sensing approach can be achieved and how it can lead to the emergence of completely new applications that cannot easily be accomplished by the respective modalities on their own. For this, a novel method is proposed that allows the matching of the motion data as obtained from a wearable inertial sensing device and from the pose estimation on a depth camera’s video stream. As a result, this method allows the identification of the person and limb the wearable device is worn on within the depth footage of an observing depth camera in the surroundings. Such an identification allows both modalities to establish a communication channel where person related data can be transmitted to the correct person and device within multi-person scenarios. An exemplary application would be indoor localization in places that use surveillance cameras anyway, for instance in an airport. Here, the position of multiple persons within the field of view of a camera in the surroundings could be tracked and, if a user decides to use this feature and enables it on a wearable device, its position can on demand be transmitted to the correct person and device.
To facilitate such applications, furthermore, a novel compression scheme for quaternion-based motion data will be elaborated. It has the purpose to reduce the amount of data to be transmitted in order to reduce energy consumption and to save precious bandwidth. The compression will be achieved by a novel piecewise linear approximation algorithm and relies on the fact that, similar to computer animations, only body postures at key positions, so-called keyframes, need to be stored or transmitted while the overall motion can be interpolated from these.
Finally, depth data will thoroughly be evaluated towards its usage for the remote sensing of respiration by measuring the subtle movements of the upper body caused by the elevation of the chest and abdomen during breathing. For this, a novel depthbased algorithm to robustly monitor human respiration from a distance is proposed. This method does not require any physical body contact, works reliably in distances up to 4 meters and, in contrast to available approaches, even works in the presence of occlusions and upper body movements as for instance are introduced while standing and keeping balance. This will be validated by comparing the proposed algorithm to a commercial respiration belt in a validation study. Furthermore, this method as well as the most common state-of-the-art depth-based respiration estimation methods will be compared on a thorough user study where a selection of the most relevant parameters that influence the respiration estimation are evaluated in depth.