Conversations in action — designing conversational agents for co-performance

Conversational User Interfaces challenge traditional models of human-computer interaction as they shift from visual to auditory communication. While this shift initially seems to simplify human interaction with technology, recent user studies have shown that commercially available Voice Assistants for homes and mobile phones often fail to meet users’ expectations as interactive resources and assistants. The first attempts to improve interaction design have focused on emulating human-human conversations and behaviors. However, there is still a lack of design guidelines for transitioning from information design to communication and interaction design. This work aims to address this gap by conducting a formative study and four design case studies to understand users’ interactions, perceptions, and expectations to derive implications through grounded design. By viewing human language use as both cognitive and social, this research explores the design space of conversational agents using the theoretical lens of Social Practice Theory. Specifically, conceiving conversational agents as carriers of practices in co-performance with humans contributes to the design of supportive and interactive resources. Moreover, the sonification and multimodality of interaction complement voice-first interactions and thus enrich conversational user experiences and further enable engaging interactions. In summary, this thesis contributes to the following fields:

• For CUI, this work uses the theoretical concepts of informative, communicative, and expressive behaviors to analyze and inform the design of conversational agents while providing design implications for future voice-first interactions.
• In line with Practice-Based Computing, this thesis presents four design case studies that empirically investigate human practices as performances and entities to derive design implications for conversational agents that act as Carriers of Practices in Coperformance with humans.
• For Practice-Based Computing, this research contributes alternative visions for ubiquitous conversational agents that facilitate engaging and meaningful interactions by being proactive and multimodal while matching the home practices and needs of humans.