Touch and Gesture Interaction
Touch and gesture interaction lets people control devices by touching a surface or moving their hands and bodies, underpinning smartphones, tablets, and many natural user interfaces.
Definition
Touch interaction is direct input made by contacting a sensing surface, often with multiple simultaneous contacts, while gesture interaction interprets the movement and configuration of fingers, hands, or the body as commands; together they support direct, often natural, control of interactive systems.
Scope
This topic covers direct-touch and gesture-based input: single- and multi-touch interaction on screens and surfaces, touch-target sizing and accuracy strategies, the design and elicitation of gesture vocabularies, and mid-air and free-form gestures. It addresses the human-performance and design issues specific to these modalities. It does not cover pointing with indirect devices, treated under pointing and target acquisition, nor speech and combined modalities, treated under multimodal and voice interaction.
Core questions
- How do touch input and the 'fat finger' problem affect target size and accuracy?
- How are gesture vocabularies designed so that gestures are guessable and memorable?
- What are the strengths and limits of multi-touch compared with pointing?
- How does direct touch change the design of feedback and controls?
Key concepts
- multi-touch
- touch target size
- fat finger problem
- gesture vocabulary
- gesture elicitation
- direct manipulation by touch
- mid-air gesture
- natural user interface
Key theories
- Touch accuracy and target sizing
- Direct touch is imprecise because the finger occludes the target and contacts a broad area; strategies such as offset cursors, take-off selection, and adequate target sizes measurably improve touch accuracy.
- User-defined gestures
- Eliciting gestures from users rather than imposing designer-invented ones produces vocabularies that are more guessable and agreed-upon, improving learnability of gesture interfaces.
- Natural user interface design
- Designing for touch and gesture requires rethinking conventions from the mouse era; effective natural user interfaces exploit directness and immediacy while managing the lack of precision and the absence of hover and right-click.
Clinical relevance
Touch and gesture interaction is the dominant way people use smartphones, tablets, kiosks, and many appliances; good touch-target sizing and intuitive gestures are central to usability on these devices and to making them workable for users with varying dexterity.
History
Touchscreen research dates to the 1980s, including work on improving touch accuracy. Multi-touch interaction moved into the mainstream with smartphones and tablets around 2007, prompting research into gesture design, including user-elicitation studies, and into design principles for natural user interfaces that exploit direct touch and gesture.
Key figures
- Jacob O. Wobbrock
- Ben Shneiderman
- Daniel Wigdor
- Dennis Wixon
- Bill Buxton
Related topics
Seminal works
- wobbrock2009
- potter1988
- wigdor2011
Frequently asked questions
- Why are touch targets often larger than mouse targets?
- A fingertip contacts a broad, soft area and hides the target beneath it, so precise selection is harder than with a pixel-accurate cursor. To keep touch reliable, designers make interactive targets larger and space them apart, and may use techniques like selection on lift-off to improve accuracy.
- Why elicit gestures from users instead of designing them?
- Gestures invented by designers are often hard for users to guess or remember. Eliciting gestures by asking many users how they would perform actions reveals common, intuitive choices, leading to gesture sets that more people can discover and recall without instruction.