Critical Appraisal and Individual Evidence Evaluation
Critical appraisal is the structured process of systematically examining a single piece of research to judge its validity, the size and precision of its results, and its relevance to a given question. It is the practical skill that turns reading a study into evaluating one, and it sits at the centre of evidence-based practice: before evidence can be applied or synthesised, each study must first be appraised on its own terms.
Definition
Critical appraisal is the explicit, criteria-based evaluation of an individual research study to determine the trustworthiness of its methods (internal validity), the meaning and precision of its findings, and the extent to which those findings can be applied to other settings or populations (external validity).
Scope
This area orients the reader to how individual studies are evaluated rather than how bodies of evidence are pooled. It covers the generic appraisal questions (is it valid? what are the results? are they applicable?), the structured tools and checklists used to apply those questions, and the reporting standards (such as CONSORT and STROBE) that make a study transparent enough to appraise in the first place. It is reference-educational and does not provide clinical recommendations.
Sub-topics
Core questions
- Are the results of this individual study valid, given how it was designed and conducted?
- What are the results, and how precise and clinically or practically important are they?
- Can the results be applied to the population, setting, or question at hand?
- How do structured tools and reporting standards make a study appraisable and reduce subjective judgement?
Key concepts
- Internal validity
- External validity (generalisability)
- Risk of bias
- Validity-results-applicability appraisal framework
- Reporting standards versus methodological quality
- Critical appraisal tools and checklists
- Reproducibility and transparency of reporting
Mechanisms
Appraisal proceeds by mapping the design of a study to the biases that design is prone to, then asking whether the authors took adequate steps against them. The widely taught Users' Guides framework organises this into three sequential questions — are the results valid, what are the results, and will they help locally — so that judgement of validity precedes interpretation of effect size, which in turn precedes any judgement about applicability (Guyatt 1993; Greenhalgh 1997). Distinct from appraisal of methods, reporting standards specify what a paper must disclose so that a reader has the information needed to appraise it at all; the two are complementary, because a well-conducted study that is poorly reported cannot be appraised, and a transparently reported study can still be judged at high risk of bias.
Clinical relevance
Critical appraisal is a core competency of evidence-based practice across the health professions, because applying research responsibly requires first judging whether a study is trustworthy and relevant. This area describes how individual studies are evaluated and reported; it characterises evidence rather than prescribing diagnostic or treatment decisions, and it is not a substitute for clinical guidelines.
Evidence & guidelines
Appraisal is supported by validated, design-specific instruments rather than a single universal checklist; a systematic review found wide variability across more than a hundred published tools with no agreed gold standard for any design (Katrak 2004). Reporting standards endorsed by journals and the EQUATOR Network — CONSORT for randomised trials and STROBE for observational studies — complement appraisal by standardising disclosure, while frameworks such as GRADE extend appraisal from the single study to the certainty of a body of evidence (Guyatt 2008).
History
Structured critical appraisal grew out of the clinical epidemiology developed at McMaster University and the evidence-based medicine movement of the 1990s. Sackett and colleagues' definition of evidence-based medicine framed appraisal of individual studies as a routine clinical skill (Sackett 1996), and the JAMA Users' Guides series codified explicit, sequential appraisal questions for each study type (Guyatt 1993). Greenhalgh's widely read primers brought the same approach to a broad clinical audience (Greenhalgh 1997). In parallel, reporting standards and dedicated appraisal tools proliferated from the late 1990s onward, eventually coordinated through the EQUATOR Network.
Debates
- Does reporting quality measure study quality?
- Reporting standards document what was disclosed, not whether the study was well conducted; treating adherence to a reporting checklist as a measure of methodological quality conflates transparency with validity, and most standards' authors caution against using them as quality scores.
- Is there a gold-standard appraisal tool?
- The large number of competing checklists, with little overlap in content and no agreed reference instrument for any design, leaves appraisal partly dependent on the tool chosen and on reviewer judgement.
Key figures
- David Sackett
- Gordon Guyatt
- Trisha Greenhalgh
- Andrew Oxman
Related topics
Seminal works
- sackett-1996
- guyatt-1993
- greenhalgh-1997
Frequently asked questions
- How is critical appraisal different from evidence synthesis?
- Critical appraisal evaluates one study at a time, judging its validity and relevance. Evidence synthesis combines the findings of many appraised studies; appraisal of each individual study is a prerequisite step within synthesis.
- Is following a reporting checklist the same as appraising a study?
- No. Reporting standards tell you whether a study disclosed enough information; appraisal uses that information to judge whether the study's methods and results can be trusted and applied. A fully reported study can still be at high risk of bias.