Bridging the Gap: Assessing the Realism of Simulators in Human-Computer Interaction

The "Automation Paradox" in safety-critical domains shifts human operators into supervisory roles, degrading their skills while still requiring them to act as the ultimate cognitive failsafe during critical events. This necessitates new methods to monitor the operator's cognitive state, but research is hampered by a "fidelity gap". High-fidelity simulators are often inaccessible, while accessible low-fidelity tools may not produce generalizable results, creating a need for a new approach to simulator-based research.
This dissertation addresses this gap by proposing and validating the concept of psychophysiological realism, a criterion for simulator validity based on whether an environment feels real to an operator's cognitive and autonomic nervous systems. The core methodological contribution is a framework for instrumenting accessible simulation platforms with multimodal sensor suites, turning them into valid scientific instruments for assessing operator state and behavior.
The research progressed through four empirical studies. First, the approach's feasibility was established by instrumenting a web-based emergency dispatch simulator (LstSim), showing it could capture psychophysiological responses to stress. This method was then validated by correlating objective respiration data, which increased by 13.08% under high load, with subjective NASA-TLX workload scores, confirming the simulator could reliably measure cognitive load. The framework's scalability was subsequently demonstrated by successfully generalizing the methodology to an Air Traffic Control (ATC) simulation with an expanded, integrated sensor suite. Finally, to assess behavioral fidelity, a study comparing cyclist movement in a virtual reality (VR) simulator to the real world empirically quantified a "fidelity gap". It revealed that while core biomechanics transferred well, significant divergences emerged in higher-order control strategies and emotional responses.
Collectively, this research delivers an integrated and validated methodology that connects the objective measurement of an operator's cognitive state within a simulator to the quantitative assessment of that environment's behavioral fidelity. The framework provides the sensory input for developing intelligent adaptive interfaces, enables more rigorous, state-aware training, and supports an evidence-based approach to designing cognitively ergonomic systems in safety-critical domains.