The movement of a user's face, easily detected by a smartphone's front camera, is an underexploited input modality for mobile interactions. We introduce three sets of face-engaged interaction techniques for augmenting the traditional mobile inputs, which leverages the combination of the head movements with touch gestures and device motions, all sensed via the phone's built-in sensors. We systematically present the space of design considerations for mobile interactions using one or more of the three input modalities (i.e., touch, motion, and head). The additional affordances of the proposed techniques expand the mobile interaction vocabulary, and can facilitate unique usage scenarios such as one-hand or touch-free interaction. An initial evaluation was conducted and users had positive reactions to the new techniques, indicating the promise of an intuitive and convenient user experience.
@cite_26: In this paper, we propose a new way for measuring key typing pressure when using off-the-shelf laptop computers. Accelerometers embedded in laptop computers to protect hard discs from sudden motion are becoming very common. This paper explores the concept of utilizing this accelerometer for sensing non-verbal aspects of key typing, such as key typing pressure. This possibility enables a wide variety of pressure-sensitive user interfaces through the use of software without requiring any additional hardware sensors. Such software can be distributed easily to a substantial number of potential users. To confirm the feasibility of this idea, we compared typing finger velocities (obtained by high-speed camera images) with sensor data from an accelerometer embedded in a laptop computer. We then confirmed that there is a clear correlation between these two sets of data. We also investigated differences in typing pressure patterns among different users. By combining keystroke speeds and typing pressure, we found it is possible to distinguish among users. This feature can be used for security purposes such as preventing a laptop computer from being used by non-owners. We also present possible application ideas such as rich text expression, new types of user interface elements, and authentication.
@cite_22: In 1992, proposed remotely operating machines in a factory by manipulating a live video image on a computer screen. In this paper we revisit this metaphor and investigate its suitability for mobile use. We present Touch Projector, a system that enables users to interact with remote screens through a live video image on their mobile device. The handheld device tracks itself with respect to the surrounding displays. Touch on the video image is "projected" onto the target display in view, as if it had occurred there. This literal adaptation of Tani's idea, however, fails because handheld video does not offer enough stability and control to enable precise manipulation. We address this with a series of improvements, including zooming and freezing the video image. In a user study, participants selected targets and dragged targets between displays using the literal and three improved versions. We found that participants achieved highest performance with automatic zooming and temporary image freezing.
@cite_7: We envision a nomadic model of interaction where the personal computer fits in your pocket. Such a computer is extremely limited in screen space. A technique is described for "spilling" the display of a hand held computer onto a much larger table top display surface. Because our model of nomadic computing frequently involves the use of untrusted display services we restrict interactive input to the hand held. Navigation techniques such as scrolling or turning the display can be expressed through the table top. The orientation and position of the hand held on the table top is detected using three conductive feet that appear to the touch table like three finger touches. An algorithm is given for detecting the three touch positions from the table's sensing mechanism.
@cite_21: We explore techniques for hand-held devices that leverage the multimodal combination of touch and motion. Hybrid touch + motion gestures exhibit interaction properties that combine the strengths of multi-touch with those of motion-sensing. This affords touch-enhanced motion gestures, such as one-handed zooming by holding one's thumb on the screen while tilting a device. We also consider the reverse perspective, that of motion-enhanced touch, which uses motion sensors to probe what happens underneath the surface of touch. Touching the screen induces secondary accelerations and angular velocities in the sensors. For example, our prototype uses motion sensors to distinguish gently swiping a finger on the screen from 'Sdrags with a hard onset' - to enable more expressive touch interactions.
@cite_1: 
@cite_19: We review the Boom Chameleon, a novel input output device consisting of a flat-panel display mounted on a tracked mechanical armature. The display acts as a physical window into 3D virtual environments, through which a one-to-one mapping between real and virtual space is preserved. The Boom Chameleon is further augmented with a touch-screen and a microphone speaker combination. We created a 3D annotation application that exploits this unique configuration in order to simultaneously capture viewpoint, voice and gesture information. Results of an informal user study show that the Boom Chameleon annotation facilities have the potential to be an effective, and intuitive system for reviewing 3D designs.
In addition to orientation, device movement can provide more input modalities. For example, Boom Chameleon demonstrates a display tracked with 6 degrees-of-freedom in space as a window in a virtual 3D world @cite_19 . TouchProjector @cite_22 and Spilling @cite_7 allow the user to manipulate an object by direct touch on the screen augmented by movement of the phone. Another type of motion involves hard force gestures such as shaking, tapping, and striking the device. Examples include using accelerometer to measure typing pressure @cite_26 and the “chucking” technique @cite_1 . Hinckley al also present a number of techniques in this category, such as “hold and shake” and “hard drag” @cite_21 .