Facial Coding in Advertising Research
Also known as: Facial Expression Analysis, Automated Facial Coding, Emotion AI for Ads, Facial Action Coding for Marketing
Facial coding measures consumers' emotional responses to advertising by analyzing the movements of their faces while they watch. It rests on Paul Ekman and Wallace Friesen's Facial Action Coding System (FACS), which decomposes any expression into elemental action units, the contractions of individual facial muscles such as the lip-corner pull of a smile or the brow lowering of a frown. Manual FACS coding is precise but slow, so the field has shifted to automated facial coding, in which computer-vision models detect landmarks and action units frame by frame and map them to emotions and to continuous valence and arousal. Daniel McDuff, Rana el Kaliouby, and colleagues showed at scale that these automatically measured facial responses to ads predict ad liking and even changes in purchase intent. Aggregated across viewers, the result is a second-by-second emotional response curve over the ad, revealing where it amuses, surprises, bores, or repels. Facial coding thus turns spontaneous, fleeting expressions into a quantitative, time-resolved index of how an ad makes people feel.
Key highlights
- Grounded in the anatomically defined Facial Action Coding System, giving an objective vocabulary for expression rather than subjective impressions.
- Provides a second-by-second emotional response curve that localizes exactly where an ad delights or loses viewers.
- Automated webcam-based coding scales to large remote samples quickly and inexpensively relative to lab biometrics.
- Validated at scale to predict ad liking and, with brand-timed positive expression, changes in purchase intent.
Intuition
This section is available to Pro members. Upgrade to Pro
How it works
This section is available to Pro members. Upgrade to Pro
When to use it
Use facial coding when you want a moment-to-moment, behavioral measure of emotional response to time-based marketing content, especially video advertising, where it can pinpoint which scenes amuse, surprise, bore, or repel viewers and whether positive emotion lands on the brand. Automated webcam-based coding is well suited to large, fast, remote samples for ad pre-testing and creative comparison. It is appropriate when emotion is the construct of interest and self-report is too coarse or too retrospective. It is less appropriate for static stimuli with little expressive response, for subtle attitudes that do not register on the face, or for cultures and contexts where overt facial expression is suppressed. Because expressions can be polite, suppressed, or ambiguous, facial coding is best combined with eye-tracking, biometrics, or self-report rather than treated as a complete account of how an ad works.
Strengths & limitations
- Grounded in the anatomically defined Facial Action Coding System, giving an objective vocabulary for expression rather than subjective impressions.
- Provides a second-by-second emotional response curve that localizes exactly where an ad delights or loses viewers.
- Automated webcam-based coding scales to large remote samples quickly and inexpensively relative to lab biometrics.
- Validated at scale to predict ad liking and, with brand-timed positive expression, changes in purchase intent.
- Captures expressed emotion only; suppressed, polite, or felt-but-unexpressed reactions go unmeasured.
- Webcam coding is sensitive to lighting, head pose, occlusion, and camera quality, which degrade action-unit detection.
- Mapping action units to discrete emotions is probabilistic and can misread ambiguous or blended expressions.
- Cultural and individual differences in display rules limit cross-population comparability of expression intensity.
Common pitfalls
This section is available to Pro members. Upgrade to Pro
Applications
This section is available to Pro members. Upgrade to Pro
Frequently asked
What is an action unit and why build emotion measurement on it?
An action unit is the movement produced by an individual facial muscle or a small group of muscles, such as the lip-corner puller that pulls the mouth into a smile or the brow lowerer that knits the eyebrows. Ekman and Friesen's Facial Action Coding System catalogs these units so that any expression can be described as a combination of them. Building emotion measurement on action units makes it objective and reproducible: instead of an analyst guessing that someone looks happy, the method records which specific muscles moved and how intensely. Emotion labels are then derived from action-unit configurations, which keeps the higher-level interpretation anchored to verifiable physical movements and allows ambiguous readings to be checked against the underlying units.
Can automated facial coding really predict whether an ad works?
Within limits, yes. McDuff and colleagues analyzed thousands of facial responses to ads and showed that ad liking can be predicted from facial expressions, particularly positive ones, with substantial accuracy, and that changes in purchase intent can also be predicted, though less strongly. Importantly, they found that simply making people smile is not enough for purchase intent: positive expressions that occur right after the brand appears are far more effective. So facial coding predicts outcomes best when it captures not just whether people react but when, relative to brand and message moments. It is a meaningful predictor that complements other measures, not a guarantee, and its accuracy depends on video quality and adequate sample sizes.
Does a smile during an ad always mean the viewer liked it?
Not necessarily. Smiles can be polite, social, ironic, or even nervous, and they can occur during an awkward or embarrassing moment rather than a genuinely enjoyable one. Ekman's work distinguishes felt enjoyment, which involves both the lip-corner puller and the cheek raiser around the eyes, from social smiles that may involve only the mouth, so the action-unit detail matters for interpretation. In practice, analysts examine the full configuration of action units, the timing relative to the content, and the context rather than treating any upturned mouth as endorsement. This is why facial coding reports the time course and the underlying units, and why it is often paired with other measures to confirm what an expression really signifies.
Sources
- 1.Ekman, P., & Friesen, W. V. (1978). Facial Action Coding System: A Technique for the Measurement of Facial Movement. Palo Alto, CA: Consulting Psychologists Press.ISBN 9780931835018
- 2.McDuff, D., El Kaliouby, R., Cohn, J. F., & Picard, R. W. (2015). Predicting Ad Liking and Purchase Intent: Large-Scale Analysis of Facial Responses to Ads. IEEE Transactions on Affective Computing, 6(3), 223-235.
You have read it. What now?
Cite this page
ScholarGate. (2026, June 23). Facial Coding in Advertising Research. ScholarGate. https://scholargate.app/marketing/facial-coding-advertising