Pilot-tested Mobile Experience Sampling — Validated In-the-Moment Data Collection via Smartphone
Pilot-tested Mobile Experience Sampling Method · Also known as: pilot-tested mobile ESM, pretested mESM, validated mobile experience sampling, mobile ESM with pilot phase
Pilot-tested mobile experience sampling (mESM) is a data collection approach that combines smartphone-delivered, real-time self-report prompts — the Experience Sampling Method — with a structured pilot phase to validate the instrument, signal timing, burden level, and response quality before full deployment. The pilot phase is not optional decoration; it is the core quality gate that separates a rigorously validated mESM study from an ad hoc one.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
When to use it
Use pilot-tested mobile ESM when you need ecologically valid, real-time self-report data on psychological states, behaviors, or contexts as they occur in everyday life, and when measurement quality is a priority. It is especially appropriate for studies of mood, stress, social interaction, physical activity, or cognitive experience over multiple days. The pilot phase is indispensable when deploying a new or adapted instrument, using an unfamiliar platform, studying a population with varying technical literacy, or when compliance rates are expected to be a concern. Do not use this approach when the research question does not require repeated within-day measurement, when the study population cannot commit to a multi-day protocol, or when resources (time, budget, technical support) are insufficient for both a pilot and a main study.
Strengths & limitations
- Captures experience in context and in real time, substantially reducing retrospective recall bias compared to end-of-day or weekly surveys.
- The pilot phase detects instrument, platform, and compliance problems before they contaminate the main dataset — protecting data quality and study validity.
- Produces rich multilevel data enabling within-person and between-person analyses that cross-sectional surveys cannot support.
- Ecologically valid — participants respond in their natural environments, preserving the authenticity of self-reports.
- Pilot findings can be reported as a methodological contribution, improving transparency and replicability.
- More resource-intensive than a single-administration survey — requires platform setup, participant onboarding, compliance monitoring, and two recruitment phases (pilot and main).
- Participant burden from repeated prompts can cause fatigue and dropout, especially in longer studies or with high signaling frequency.
- Requires multilevel statistical expertise; naive averaging of ESM responses across time points ignores the nested data structure and produces biased estimates.
- Findings from one context or population may not replicate in others even with full pilot validation, since compliance and comprehension vary across groups.
Frequently asked
How large should the pilot sample be for a mobile ESM study?
Most methodologists recommend 5–20 participants for a pilot phase, drawn from a population similar to the main study sample. The goal is not statistical power but identification of practical problems — item confusion, technical glitches, and burden-related dropout. A pilot of fewer than five participants risks missing population-specific issues; more than 20 is generally unnecessary unless the target population is highly heterogeneous.
How many prompts per day is appropriate?
Four to ten random prompts per day is the most common range in the ESM literature. Pilot compliance data should guide the final frequency. If pilot participants complete fewer than 70% of prompts, consider reducing frequency. Compliance tends to decline over multi-week studies, so a shorter, more intensive sampling window often produces better data than a longer low-compliance one.
What platform should I use to deliver mobile ESM?
Widely used platforms include m-Path, ExperienceSampler, PACO, and Qualtrics with SMS delivery. Choice depends on your budget, required features (GPS tagging, sensor integration), and your participants' devices. The pilot phase should test the platform on both Android and iOS devices to catch OS-specific notification failures.
Can pilot data be included in the final analysis?
Generally, no. Pilot data are collected under a different protocol (shorter duration, possibly revised items, and often with participants who are aware they are testing the system). Pooling them with main-study data introduces protocol heterogeneity. If the protocol is unchanged between pilot and main study, inclusion may be considered but must be transparently reported and sensitivity-tested.
How is mESM different from ecological momentary assessment (EMA)?
The two terms are largely synonymous in the behavioral and health sciences. EMA is the dominant label in clinical and health psychology; ESM is more common in social and organizational psychology. Both involve repeated real-time self-reports prompted in daily life. Mobile ESM specifically emphasizes smartphone delivery; EMA may also use paper forms or dedicated devices in some older or low-resource study designs.
Sources
- Larson, R., & Csikszentmihalyi, M. (1983). The experience sampling method. New Directions for Methodology of Social and Behavioral Science, 15, 41–56. link ↗
- van Berkel, N., Ferreira, D., & Kostakos, V. (2017). The experience sampling method on mobile devices. ACM Computing Surveys, 50(6), 1–40. DOI: 10.1145/3123988 ↗
How to cite this page
ScholarGate. (2026, June 3). Pilot-tested Mobile Experience Sampling Method. ScholarGate. https://scholargate.app/en/survey-methodology/pilot-tested-mobile-experience-sampling
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- Diary MethodSurvey Methodology↔ compare
- Mobile Experience SamplingSurvey Methodology↔ compare
- Pilot-tested SurveySurvey Methodology↔ compare
- Sensor Data CollectionSurvey Methodology↔ compare