Improved training approaches for embedded learning with heterogeneous sensor data

The papers published as part of this doctoral thesis address significant challenges in the areas of annotation and synchronization of datasets, data-driven studies in the field of human-computer interaction, and machine learning related to multimodal activity data. Specifically, annotation workflows are examined for their robustness, an algorithm for improved synchronization of different acceleration data is presented, and potential data distortions in supervised studies involving human subjects are investigated within the context of a data-driven study. Advanced deep learning methods are employed for the analysis of human activities based on sensor data.
Furthermore, an extensive dataset of motion data from basketball players has been released.
Finally, the applicability of deep learning techniques, specifically Transfer Learning and Data Augmentation, to sensor data is explored.

The first research field explores real-world user studies for HAR and introduces the Activate-System. This system enables ad-hoc data collection using a smartwatch and smartphone app.
Furthermore, a presented user study evaluates and improves data collection and annotation methodologies. A 2-week study compares self-report diaries and in-situ annotation techniques, finding visualizing sensor data as time series improves recall accuracy. Additionally, I present an algorithm that synchronizes signals from multiple on-body sensors by exploiting cross-correlations of acceleration signals.
Finally, the Hawthorne Effect is analyzed by collecting observed and unobserved data. This effect suggests that study participants alter their behavior once they are aware of being part of a study or under observation.

The second focus is recognizing activities in complex environments like sports, specifically basketball. A preliminary study shows the feasibility of detecting fine-grained basketball activities using a single wrist-worn inertial sensor. The Hang-Time HAR dataset, with data from 24 players, encompasses periodic, sporadic, and complex basketball movements, enabling comprehensive classification through deep learning.

The third contribution focuses on transfer learning and data augmentation for sensor-based activity recognition. Extensive experiments assess model transferability across sensor positions, modalities, and activity domains. Results reveal high variability depending on specific factors, such as body location, and deteriorating when source and target domains differ significantly.
Data augmentation with GAN is also explored, comparing user-wise and fold-wise synthetic data generation. Expanding the dataset size by a factor of five improves the F1-Score by 11.0% for user-wise augmentation and 5.1% for fold-wise augmentation.