Why getting emotion right matters
In all forms of advertising, it’s important to drive brand engagement, convey meaningfully different associations and drive long and short-term predisposition. People are more likely to buy from brands they have a meaningful connection with, so understanding how advertising works means understanding the emotional engagement an ad generates.
At Kantar, we test advertising with a number of different measures, designed to capture the ‘what’, as well as the ‘why’, of consumer responses. Testing advertising results in better ROI. And better-quality creative can even compensate for slightly reduced media spend. Moving an ad’s creative quality from average to best means a 32% increase in ROI. And we know that creativity is the second most important driver of ad profitability – so it’s worth getting it right.
In our survey solutions we ask a range of validated diagnostic questions about branding, enjoyment and the persuasiveness of ads, all of which can be compared to normative data. We also use a facial coding add-on, for a second-by-second time series read of emotional responses during the ad. Integrated with survey data, this is a powerful combination from which we derive actionable insights and recommendations on how to optimise the creative.
What is facial coding?
Facial coding is an AI-based analysis of facial muscle movements that allows us to measure facial expressions such as smile, brow furrow or raised eyebrows. We capture facial expressions from our panelists via webcams with explicit consent, the data is anonymised and aggregated.
In other words, the algorithms recognise expressions, not people. It measures and quantifies emotional reactions that people would find difficult to articulate otherwise. While people know that they smile or frown, they cannot precisely report the timing or strength of these responses. So, this information is extremely helpful to understand viewers’ emotional reactions to advertising. It means we can test ads with people in their natural environment, to capture the wide range of reactions they show while watching ads.
The ease of data collection makes our technique scalable and allows for testing big, reliable samples, the value of which cannot be overestimated in market research. Full integration with survey responses means we can compare the emotional reactions of various groups, for example, those who liked vs disliked the ad. This in turn enables us to explain an ad’s scores and understand how each scene affected the ad’s overall evaluation, along with survey and open-ended responses. Based on these conclusions, we can confidently give our clients recommendations for ad edits and improvements.
How does facial coding work?
Facial coding has an AI outcome: while algorithms provide information about the type, strength and timing of emotional reactions, the interpretation is done by analysts. This is crucial for reliable analysis, as the same facial expression can mean different things depending on the context. Kantar uses Affectiva’s emotion AI technology which is trained on this diversity, and the human analyst assures the context is taken into account and can be explained. Our facial coding outputs are easier to interpret and understand than EEG, for example.
Contemporary, high-quality AI-based facial coding is grounded in scientific knowledge collected over more than 100 years and validated through numerous academic studies. The Affectiva technology is proven to be highly accurate as well as validated and published in peer-reviewed articles. It is also being constantly improved as more data is accumulated. Moreover, Kantar has captured well over two million face videos and researched over 49,000 ads using facial coding and we have validated the link between emotion and sales success. We have proven that our measure of overall Expressiveness captured via facial coding drives shifts in sales volume share, especially for more established brands.
So facial coding provides a lot of useful information about people’s emotions that cannot be gathered otherwise, while keeping the testing situation as natural as possible. It tells us not just whether people are paying attention, but also how they feel. It enhances survey measures and exceeds all other implicit measures of emotions in terms of both ease of administration and utility of data.
Understanding different cultures
Not all facial coding technology is equally reliable. Some people question whether facial coding technology is potentially biased against different groups. Affectiva’s technology has been upgraded many times over the last 10 years and each time the algorithms are trained on highly diverse datasets. The technology uses more real-world face data than any other organisation – over 13 billion frames of data from over 90 countries, and as a result, it is trained on predominantly non-Caucasian datasets with a balance in gender and age. Great care is taken to ensure equal accuracy across demographic groups so we can be confident that the system exhibits minimal bias. Affectiva’s metrics are also interpreted in the context of the country of data collection – the system includes normative comparisons for every country, so analytics can allow for the fact that some cultures are more expressive on different dimensions than others.
The reliability and sensitivity thresholds used by facial coding are important. The Affectiva facial coding technology is trained rigorously and delivers high published accuracy and test-rest reliability. The technology uses a high threshold to minimise false positives (i.e. insignificant facial movements), but we still see measurable responses to advertising across almost all scenes – and the variation we see is meaningfully linked to engagement with the content. There are very few ads that don’t produce a detectable facial response across an audience, but there is a wide variety of degree of response – reflecting the variation in emotional power in real-world advertising. It is naïve to assume that all ads will produce a significant emotional response all the time. Affectiva’s facial coding successfully discriminates between engaging (and sales effective) ads and non-engaging ads, and also between engaging and less engaging scenes within an ad, which is exactly what is required in sensitive and diagnostic ad testing.
Facial coding in action
Sustainability is an area that evokes strong emotions and opinions, and sustainability campaigns are no different. We see that ads with social and/or environmental messaging evoke stronger emotions – both positive and negative. Campaigns that speak to these issues should do the same. And that means getting the emotional tone right.
In ads with a sustainability message, for example, we know it is important to leave people hopeful, confident and empowered to change.
Seventh Generation - Trees and B’s
Humour is an effective way to use emotion in advertising. Seventh Generation’s ad for recycled toilet paper uses humour to communicate its sustainable message.
Facial coding helps to understand how humour is resonating, and at which points in the ad. In this example, we looked at different groups of people based on how committed they are to sustainability.
With facial coding, we can see that the humour makes the message more accessible for groups that are less committed to sustainability (Considerers). The ad evokes more smiles for considerers throughout, including during the ‘save the trees’ call to action at the end, so a useful tactic to get the sustainability message out to a broader audience.
Another ad we tested for a client had a very different response to what was intended. With facial coding and open-ended responses we found out that the scenes intended to be funny were not humorous. People weren’t understanding the jokes, and one scene was even found to be offensive. A re-edit was required to ensure that the setup and the payoffs were paced correctly.
What about EEG? (Electroencephalography)
When we piloted EEG (recording brain activity) to understand advertising, we found it intrusive, vague in meaning, and unscalable. It provides information across far fewer dimensions compared to facial coding, and the accuracy is often lower. Academic papers have cited low comparability across different studies, and the intrusive measurement in unnatural conditions has potential to alter respondents' emotions, introducing an additional factor we don't want to measure.
Context matters for both EEG and facial coding, but with facial coding we measure several expressions, versus just one dimension in EEG. Even if a smile covers different emotions, it is easier for people to decode them (each of us does this every day, in every social interaction) than to understand the complex scientific terms and dimensions associated with EEG encoding. This means that many people can become skilled facial coding analysts while EEG requires trained neuroscientists to read the data.
Summary
We think it’s important to measure emotion in advertising – and we will soon be rolling out facial coding into our automated digital ad testing around the world via Kantar Marketplace. With increased investment in digital advertising, understanding how the ads work and why is even more vital. In the Attention Economy, every second counts. Watch our on-demand webinar with our partner Affectiva to see how facial coding can enhance your creative and get in touch to find out more.
