Title: Automated capture and delivery of assistive task guidance with an eyewear computer: The GlaciAR system,
Abstract: In this paper we describe and evaluate a mixed reality system that aims to
augment users in task guidance applications by combining automated and
unsupervised information collection with minimally invasive video guides. The
result is a self-contained system that we call GlaciAR (Glass-enabled
Contextual Interactions for Augmented Reality), that operates by extracting
contextual interactions from observing users performing actions. GlaciAR is
able to i) automatically determine moments of relevance based on a head motion
attention model, ii) automatically produce video guidance information, iii)
trigger these video guides based on an object detection method, iv) learn
without supervision from observing multiple users and v) operate fully on-board
a current eyewear computer (Google Glass). We describe the components of
GlaciAR together with evaluations on how users are able to use the system to
achieve three tasks. We see this work as a first step toward the development of
systems that aim to scale up the notoriously difficult authoring problem in
guidance systems and where people's natural abilities are enhanced via
minimally invasive visual guidance.