Usability Metrics and Measurement
Usability metrics quantify how well people use a system, capturing performance such as task success and time as well as subjective satisfaction through standardized questionnaires.
Definition
Usability metrics are quantitative measures of interaction quality, including performance measures such as effectiveness and efficiency and self-reported measures of satisfaction, used to benchmark, compare, and track the usability of a system.
Scope
This topic covers the quantitative side of evaluation: performance metrics such as task success rate, time on task, and error counts; self-report metrics from standardized instruments such as the System Usability Scale; and the analysis and reporting of these measures, including confidence intervals and benchmarking. It addresses how to collect reliable numbers and interpret them. It does not cover the qualitative observation of behaviour, treated under usability testing, nor predictive cognitive models, treated under cognitive models of interaction.
Core questions
- What performance metrics capture effectiveness and efficiency?
- How do standardized questionnaires measure perceived usability and satisfaction?
- How should usability data be summarized with appropriate uncertainty?
- How can metrics be used to benchmark and compare designs?
Key concepts
- task success rate
- time on task
- error rate
- System Usability Scale (SUS)
- satisfaction rating
- confidence interval
- benchmarking
- efficiency and effectiveness metrics
Key theories
- Performance and satisfaction metrics
- Usability is measured along complementary dimensions: objective performance such as task completion, time, and errors, and subjective satisfaction captured through ratings, which together operationalize the effectiveness, efficiency, and satisfaction components of usability.
- Standardized usability questionnaires
- Validated instruments such as the System Usability Scale and the IBM questionnaires give reliable, comparable scores of perceived usability, allowing benchmarking across systems and over time.
- Statistics for small-sample user research
- Because usability studies often have small samples, appropriate methods, confidence intervals, adjusted-Wald intervals for proportions, and care with significance testing, are needed to draw defensible conclusions.
Clinical relevance
Quantitative usability metrics let teams set targets, track progress, and justify design decisions to stakeholders; standardized scores such as SUS provide a common language for comparing products and are used in industry reporting and in some regulatory usability documentation.
History
As usability work matured, the field developed standardized instruments: Brooke's System Usability Scale appeared in 1996 and the IBM satisfaction questionnaires shortly before. Texts by Tullis and Albert and by Sauro and Lewis consolidated metrics and small-sample statistics in the 2000s and 2010s, making quantitative UX measurement a routine part of practice.
Key figures
- Thomas Tullis
- Bill Albert
- Jeff Sauro
- James R. Lewis
- John Brooke
Related topics
Seminal works
- brooke1996
- tullis2013
- sauro2016
Frequently asked questions
- What is the System Usability Scale?
- The System Usability Scale (SUS) is a short ten-item questionnaire that yields a single score from 0 to 100 reflecting users' perceived usability of a system. Because it is quick, reliable, and widely used, SUS scores can be compared across products and against established benchmarks.
- Why report confidence intervals for usability metrics?
- Usability studies often have small samples, so a single average can be misleading. Confidence intervals show the range of plausible values for the true metric, communicating how much uncertainty surrounds an estimate and preventing overconfident conclusions from limited data.