Wearable-based affect recognition

Advances in wearable-based sensor technology like smartphones and watches allow to monitor users in a minimally intrusive way. At the point of writing, wearables are, for instance, used to count steps or estimate burned calories. Recently, a first generation of smartwatches entered the consumer market offering data driven insights into affective states.
Over the course of this thesis physiological and motion data recorded using wearables have been employed to detect the affective state (e.g., stress or amusement) of users. The contributions made are threefold: First, a comprehensive literature review of the state-of-the art in wearable-based affect recognition was conducted. Second, concluding from this review a lack of publicly available multimodal datasets was identified. This gap was closed by recording, benchmarking, and publishing a lab study dataset for WEarable Stress and Affect Detection (WESAD). Third, a field study was conducted recording physiological and motion data as well as affective labels from 11 healthy subjects. Prior to the field study guidelines for smartphone-based labelling apps were formulated and they were evaluated using the field study data. Furthermore, data and labels acquired during the field study were used to train both feature-based and latest end-to-end trainable machine learning classifiers, detecting affective states on different scales. Both types of classifiers performed on par (averaged F1 score across scales: ~ 45%). Hence, potential pitfalls for wearable-based affect recognition were discussed in detail and implications for further research were provided.