The movement of a user's face, easily detected by a smartphone's front camera, is an underexploited input modality for mobile interactions. We introduce three sets of face-engaged interaction techniques for augmenting the traditional mobile inputs, which leverages the combination of the head movements with touch gestures and device motions, all sensed via the phone's built-in sensors. We systematically present the space of design considerations for mobile interactions using one or more of the three input modalities (i.e., touch, motion, and head). The additional affordances of the proposed techniques expand the mobile interaction vocabulary, and can facilitate unique usage scenarios such as one-hand or touch-free interaction. An initial evaluation was conducted and users had positive reactions to the new techniques, indicating the promise of an intuitive and convenient user experience.
@cite_33: ScatterDice Mobile (SDM) is a novel visualization system that leverages embodied motion and orientation gestures for intuitive and effective exploration of multi-dimensional data on mobile devices. Inspired by 's recent work, SDM uses the gyroscope sensor available on mobile devices to establish an orientation aware "dice rolling" metaphor for browsing scatterplot matrix visualizations mapped to a cube on mainstream mobile devices without any hardware modification. SDM has the potential for applications that require prompt access and exploration of large scale, multi-dimensional data on mobile devices anytime, anywhere.
@cite_21: We explore techniques for hand-held devices that leverage the multimodal combination of touch and motion. Hybrid touch + motion gestures exhibit interaction properties that combine the strengths of multi-touch with those of motion-sensing. This affords touch-enhanced motion gestures, such as one-handed zooming by holding one's thumb on the screen while tilting a device. We also consider the reverse perspective, that of motion-enhanced touch, which uses motion sensors to probe what happens underneath the surface of touch. Touching the screen induces secondary accelerations and angular velocities in the sensors. For example, our prototype uses motion sensors to distinguish gently swiping a finger on the screen from 'Sdrags with a hard onset' - to enable more expressive touch interactions.
@cite_32: Most studies on tilt based interaction can be classified as point-designs that demonstrate the utility of wrist-tilt as an input medium; tilt parameters are tailored to suit the specific interaction at hand. In this paper, we systematically analyze the design space of wrist-based interactions and focus on the level of control possible with the wrist. In a first study, we investigate the various factors that can influence tilt control, separately along the three axes of wrist movement: flexion extension, pronation supination, and ulnar radial deviation. Results show that users can control comfortably at least 16 levels on the pronation supination axis and that using a quadratic mapping function for discretization of tilt space significantly improves user performance across all tilt axes. We discuss the findings of our results in the context of several interaction techniques and identify several general design recommendations.
@cite_0: TiltText, a new technique for entering text into a mobile phone is described. The standard 12-button text entry keypad of a mobile phone forces ambiguity when the 26- letter Roman alphabet is mapped in the traditional manner onto keys 2-9. The TiltText technique uses the orientation of the phone to resolve this ambiguity, by tilting the phone in one of four directions to choose which character on a particular key to enter. We first discuss implementation strategies, and then present the results of a controlled experiment comparing TiltText to MultiTap, the most common text entry technique. The experiment included 10 participants who each entered a total of 640 phrases of text chosen from a standard corpus, over a period of about five hours. The results show that text entry speed including correction for errors using TiltText was 23 faster than MultiTap by the end of the experiment, despite a higher error rate for TiltText. TiltText is thus amongst the fastest known language-independent techniques for entering text into mobile phones.
@cite_31: This paper reports on the design and use of tactile user interfaces embedded within or wrapped around the devices that they control. We discuss three different interaction prototypes that we built These interfaces were embedded onto two handheld devices of dramatically different form factors. We describe the design and implementation challenges, and user feedback and reactions to these prototypes. Implications for future design in the area of manipulative or haptic user interfaces are highlighted.
Combining touch and motion further extends our interaction vocabulary with mobile phones. Hinckley al presents a summary of such techniques and how the creation of novel gestures is possible with touch and motion sensors @cite_21 . Device orientation inferred from motion sensors is one important input to be used simultaneously with touch. TiltText employs tilting angles to resolve text input ambiguities of the traditional phone keypad @cite_0 . Other usages of tilting and touch include measuring wrist deflection angles @cite_32 and scrolling @cite_31 . ScatterDice Mobile supports the exploration of multi-dimensional data on mobile phones by mapping its orientation to different chart viewing perspectives @cite_33 .