At this year’s Mensch und Computer conference in Switzerland, we will conduct a workshop on our DFG-funded HopE project. Specifically, we will concentrate in our discussions on methods and tools for long-term evaluations with respect to in-the-wild deployments of ambient displays. We invite participation and welcome any submissions to this workshop. The submission deadline is June 12th, 2023.
Title of the workshop
Methods and Tools for (Semi-)Automated Evaluation in Long-Term In-the-Wild Deployment Studies
Michael Koch, Julian Fietkau, Susanne Draheim, Jan Schwarzer, Kai von Luck
Human-Computer Interaction research increasingly focuses on long-term evaluation of real-world field deployments. This includes challenging tasks such as observing usage behavior for longer periods of time and in a 24/7 manner. Existing methods such as in-situ observations and manual video analysis hardly provide means to cope with these tasks adequately. Automated evaluation techniques, using different kinds of sensor hardware including body tracking cameras, have been used in recent studies to capture usage behavior in long-term evaluations more efficiently. In this workshop we aim to gather and discuss the state of the art regarding this development as well as outline future perspectives. Topics for discussion are, for example, existing methods and tools, noise and errors in sensor data, best practices for the correlation of automated observations with ground truth data, field work (e.g., interviews and observations) for the contextualization of findings, ethical concerns, and automated approaches for detecting patterns in data. We expect the outcome of this workshop to be twofold: firstly, we aim at providing insights on how the field is currently evolving and, secondly, we intend to envision guidance on how to advance from there practically and methodologically.