Mixed Methods Appraisal Tool: Assessing Quality Across Study Designs
Also known as: MMAT, MMAT 2018
MMAT (Mixed Methods Appraisal Tool) is a practical, design-agnostic quality assessment tool developed by Pluye et al. (2014, updated 2018) to evaluate the methodological quality of quantitative (RCTs, non-randomized studies), qualitative, and mixed-methods studies. Unlike tools designed for single paradigms (e.g., Cochrane RoB 2 for RCTs), MMAT provides unified criteria applicable across diverse research methodologies, making it particularly useful for systematic reviews incorporating multiple study designs.
Key highlights
- Design-agnostic framework: single tool applicable to RCTs, observational quantitative studies, qualitative research, and mixed-methods designs—eliminates need for multiple appraisal tools
- Practical and concise (5 questions per design): fast to administer (5–10 minutes per study) without sacrificing rigor
- Percentage scoring (0–100%) is intuitive and comparable across studies; enables ranking and sensitivity analyses in systematic reviews
- Explicitly addresses design-specific quality criteria (e.g., reflexivity for qualitative studies, allocation concealment for RCTs); avoids inappropriate appraisal of qualitative work by quantitative standards
- Accommodates mixed-methods designs, which often combine quantitative and qualitative components; MMAT includes specific criteria for integration quality
Intuition
This section is available to Pro members. Upgrade to Pro
How it works
This section is available to Pro members. Upgrade to Pro
When to use it
MMAT is valuable for systematic reviews or qualitative syntheses incorporating diverse study designs. Use when appraising mixed-methods interventions (quantitative effectiveness + qualitative barriers/facilitators), implementation research, realist reviews, or evidence syntheses spanning paradigms. Particularly useful when a single-design tool (RCT-focused or qualitative-focused) would be inappropriate.
Strengths & limitations
- Design-agnostic framework: single tool applicable to RCTs, observational quantitative studies, qualitative research, and mixed-methods designs—eliminates need for multiple appraisal tools
- Practical and concise (5 questions per design): fast to administer (5–10 minutes per study) without sacrificing rigor
- Percentage scoring (0–100%) is intuitive and comparable across studies; enables ranking and sensitivity analyses in systematic reviews
- Explicitly addresses design-specific quality criteria (e.g., reflexivity for qualitative studies, allocation concealment for RCTs); avoids inappropriate appraisal of qualitative work by quantitative standards
- Accommodates mixed-methods designs, which often combine quantitative and qualitative components; MMAT includes specific criteria for integration quality
- Yes/no binary scoring lacks nuance; a study meeting all 5 criteria at a basic level scores 100%, equivalent to a study meeting all criteria at an exemplary level
- No evidence of reliability (inter-rater agreement, kappa) or validation against external quality standards; MMAT developers have not conducted formal psychometric evaluation
- Percentage scoring does not distinguish between domain importance; all 5 criteria are weighted equally despite some domains (e.g., allocation concealment in RCTs) having greater impact on bias
- Minimal guidance for complex study designs (e.g., cluster-randomized, crossover, network meta-analysis); users must adapt criteria or use design-specific tools
- Limited validation in practice; some reviewers report difficulty interpreting criteria (e.g., what constitutes 'appropriate' data collection?) without additional training
Common pitfalls
This section is available to Pro members. Upgrade to Pro
Applications
This section is available to Pro members. Upgrade to Pro
Frequently asked
Can I use MMAT for systematic reviews of only RCTs or only qualitative studies?
MMAT can be used, but design-specific tools are often preferable and more detailed. For RCTs, Cochrane RoB 2 is gold-standard; for qualitative studies, CASP Qualitative Checklist or JBI Critical Appraisal Tools are more comprehensive. Use MMAT when your review truly includes multiple design types and you prefer a unified framework.
What does 'appropriate data analysis' mean for qualitative studies (MMAT question 3)?
The analysis should be systematic and transparent. For example, thematic analysis should describe the coding process and data reduction; grounded theory should follow standard procedures. The criterion is met if the analysis method matches the stated research objective and is executed rigorously (not merely cherry-picking quotes).
Does a study scoring 100% on MMAT have low risk of bias?
Not necessarily. MMAT evaluates methodological rigor in design execution (e.g., randomization, data collection, analysis clarity). However, a well-executed study can still have other sources of error (e.g., publication bias, poor outcome selection, inadequate reporting). Use MMAT as one component of quality assessment, alongside domain-specific bias tools.
How do I assess MMAT quality for a study with mixed quantitative and qualitative components?
Many studies have a primary component (e.g., RCT + qualitative interviews). Score the dominant component using its corresponding MMAT criteria. Document whether the qualitative or quantitative aspect is primary. Some reviewers score both components separately and average or report both scores.
Is MMAT suitable for rapid reviews or evidence gap analyses?
Yes. MMAT is one of the fastest quality assessment tools (5–10 minutes per study). Its brevity makes it practical for rapid evidence reviews when detailed appraisal is infeasible, though at some cost to detail compared to longer tools.
Sources
- 1.Pluye, P., & Hong, Q. N. (2014). Combining the power of stories and the power of numbers: mixed methods research and mixed studies reviews. Annual Review of Public Health, 35, 29–45.
You have read it. What now?
Cite this page
ScholarGate. (2026, June 3). MMAT. ScholarGate. https://scholargate.app/research-methodology/mixed-methods-appraisal-tool