Marketers’ Top Measurement Concerns
CIMM: the Coalition for Innovative Media Measurement, has identified critical issues that should be addressed this year. Read more »
audience measurement
- Research & Data Quality
- Article
The ARF Attention Measurement Validation Initiative: Phase 1 Report Updated
Attention metrics have drawn a high degree of energy in the last few years, for many reasons, including the loss of behavioral signals due to privacy restrictions, growing frustration with ad viewability and its perceived limitations, attention metrics’ impact on the cross-platform measurement debate and that biometric technologies can now be applied “in the wild,” rather than just in labs. The ARF’s Attention Measurement Validation Initiative aims to describe the attention measurement space in detail, illuminating this nascent sector. The Phase One findings include a comprehensive literature review and a report that maps out the vendor landscape in this increasingly diverse specialty. The report includes two sections. The first section describes what methods are being used, what these companies report and how and what they measure, be it ad creative or the media environment. The second section includes in-depth overviews of the 29 participating attention measurement companies. The Phase One Report is a must-read for anyone interested in attention metrics or what companies are operating in the space.
Member Only Access
- Past Event Highlights
- Article
OOH Measurement’s Game Has Changed
Christina Radigan – SVP, Research & Insights, Outfront
Christina Radigan of Outfront explored the advantages of out-of-home advertising (OOH) and discussed advancements in its measurement techniques. Christina noted that with the loss of cookies and third-party data, contextual ad placement will see a renewed sense of importance, and in OOH, location is a proxy for context, driving content. She further indicated the benefits of OOH citing a recent study by Omnicom, using marketing mix modeling (MMM), which found that increased OOH spend drives revenue return on ad spend (RROAS). This research also highlighted that OOH is underfunded, representing only 4% to 5% of the total media marketplace. Following up on this, Christina pointed to attribution metrics, measuring the impact of OOH ad exposure on brand metrics and consumer behaviors, to demonstrate OOH's effectiveness at the campaign level. Expanding on their work in attribution, she noted changes stemming from the pandemic: Format proliferation and greater digitization, privacy-compliant mobile measurement ramping up (opt-in survey panel and SDK) and performance marketing and measurement becoming table stakes for budget allocations. New measurement opportunities from OOH intercepts included brand lift studies, footfall, website visitation, app download and app activity and tune in. Finally, she examined brand studies conducted for Nissan and Professional Bull Riders (PBR), showcasing the effectiveness of OOH advertising in driving recall, ticket sales and revenue. Key takeaways:- MMMs return to the forefront, as models become more campaign sensitive and are privacy compliant (powered by ML and AI).
- A study from Omnicom, using MMM, found that optimizing OOH spend in automotive increased brand consideration (11%) and brand awareness (19%). In CPG food, optimizing OOH spend increased purchase intent (24%) and optimizing OOH spend in retail grocery increased awareness (9%).
- OOH now represents a plethora of formats (e.g., roadside ads, rail and bus ads, digital and print) and has the ability to surround the consumer across their journey, providing the ability to measure up and down the funnel, in addition to fueling behavioral research.
- Key factors for successful measurement in OOH: feasibility (e.g., scale and scope of the campaign, reach and frequency), the right KPIs (e.g., campaign goal) and creative best practices (Is the creative made for OOH?).
- OOH advertising is yielding tangible outcomes by boosting consumer attention (+49%). Additionally, there has been a notable surge in advertiser engagement (+200%).
- Ad recall rates in OOH continue to increase (e.g., 30% in 2020 vs. 44% in 2023).
- Past Event Highlights
- Article
AUDIENCEXSCIENCE 2024
The ARF’s annual AUDIENCExSCIENCE conference highlighted the most critical audience measurement issues. Through keynotes, panels, debates and rigorously peer-reviewed research presentations, attendees learned about a wide array of new and evergreen industry topics, endemic to our industry changes. World-class thinkers joined us in NYC to share their perspectives on the future of advertising research and measurement, and how tomorrow’s technologies and data trends will impact advertising and media.
Member Only Access
- Past Event Highlights
- Article
Human Experience: Why Attention AI Needs Human Input
Dr. Matthias Rothensee – CSO & Partner, eye square
Stefan Schoenherr – VP Brand and Media & Partner, eye square
Speakers Matthias Rothensee and Stefan Schoenherr of eye square discussed the need for a human element and oversight of AI. Beginning their discussion on the state of attention and AI, Matthias acknowledge that race for attention is one of the defining challenges of our time for modern marketers. He quoted author Rex Briggs, who noted the "conundrum at the heart of AI: its greatest strength can also be its greatest weakness." Matthias indicated that AI is incredibly powerful in recognizing pattern from big data sets but at the same time there are some risks attached to it (e.g., finding spurious patterns, hallucinations, etc.). Stefan examined a case study using an advertisement for the candy M&Ms, which considered real humans using eye tracking technology and compared it to results using AI. The goal was to better understand where AI is good at predicting attention and where does it still have to optimize or get better. Results from a case study indicated areas for AI improvements in terms of gaze cueing, movement, contrast, complexity and nonhuman entities (e.g., a dog). The static nature of AI (e.g., AI prediction models are often built based on static attention databases) can become a challenge when comparing dynamic attention trends. Key takeaways:- Predictive AI is good at replicating human attention for basic face and eye images, high-contrast scenes (e.g., probability of looking at things that stand out) and slow-paced scene cuts where AI can detect details.
- AI seems unaware of a common phenomenon called the "cueing effect" (e.g., humans not only pay attention to people's faces but also to where they're looking), which leads to an incorrect prediction.
- AI has difficulties deciphering scenes with fast movements (e.g., AI shows inertia) in contrast to slow-paced scenes where AI excels in replicating human feedback. In this case human feedback is more accurate.
- AI is more consumed with attention towards contrast (e.g., in an ad featuring a runner, AI gave attention to trees surrounding the runner), whereas humans can decipher the main aspect of an image.
- AI decomposes human faces (e.g., AI is obsessed with human ears), whereas humans can detect the focal point of a human face. In addition, AI hallucinates, underestimating facial effects.
- AI has difficulties interpreting more complex visual layouts (e.g., complex product pack shots are misinterpreted).
- AI is human centric and does not focus well on nonhuman entities such as a dog (e.g., in scenes where a dog was present, AI disregarded the dog altogether).
- AI tends to be more static in nature (e.g., AI prediction models are often built based on static attention databases), which can be a problem when comparing this to dynamic attention trends.
- Past Event Highlights
- Article
Optimizing Big Data + Panel Measurement Through Calibration
David Kurzynski – SVP, Data Science, Nielsen
Kyle Poppie – VP, Data Science, Nielsen
It is challenging to measure the smaller audiences of local TV and measurement challenges include false zero audience metrics and instability. Kyle Poppie (Nielsen) reviewed the evolution of local TV measurement, and this presentation demonstrated how Nielsen’s approach enables accurate measurement. Calibrating big data to a probabilistic panel controls for biases in the big data population that cannot be accounted for by weighting alone. The panel provides accurate and unbiased measurement at aggregate levels while big data provides greater coverage of granular behavior. An example demonstrated how the calibration of panel data and big data resulted in a more accurate weighted audience size. David Kurzynski (Nielsen) presented a case study that applied calibration to live data from a secondary station in New York. The improved result included fewer zero ratings and smoother trends. Key takeaways:- The goal of calibration is to achieve local TV measurement that provides accuracy and stability for audience levels and audience flows.
- Both big data and panel data are critical as inputs to calibration to achieve these goals. Audience levels are informed by both big data and panel homes, and audience flows are influenced by big data.
- Relative and total errors decrease as a result of calibration compared to panel-only currency.
- Past Event Highlights
- Article
The Power of AI for Effective Advertising in an ID-free World
Rachel Gantz – Managing Director, Proximic by Comscore
Amidst heightened regulations in the advertising ecosystem, Rachel Gantz of Proximic by Comscore delved into a discussion of diverse AI applications and implementation tactics, in an increasingly ID-free environment, to effectively reach audiences. Rachel highlighted signal loss as a "massive industry challenge," to provide a framework for the research she examined. She remarked that the digital advertising environment was built on ID-based audience targeting, but with the loss of this data and the increase in privacy regulations, advertisers have placed their focus on first-party and contextual targeting (which includes predictive modeling). In her discussion, she focused on the many impacts predictive AI is having on contextual targeting, in a world increasingly void of third-party data, providing results from a supporting experiment. The research aimed to understand how the performance of AI-powered ID-free audience targeting tactics compared to their ID-based counterparts. The experiment considered audience reach, cost efficiency (eCPM), in-target accuracy and inventory placement quality. Key takeaways:- Fifty to sixty percent of programmatic inventory has no IDs associated with it and that includes alternative IDs.
- Specific to mobile advertising, many advertisers saw 80% of their IOS scale disappear overnight.
- In an experiment, two groups were exposed to two simultaneous campaigns, focused on holiday shoppers. The first group (campaign A) was an ID-based audience, while the second group was an ID-free predictive audience.
- Analyzing reach: ID-free targeting nearly doubled the advertisers’ reach, vs. the same audience, with ID-based tactics.
- Results from cost efficiency (eCPM): ID-free AI-powered contextual audiences saw 32% lower eCPMs than ID-based counterparts.
- In-target rate results: Significant accuracy was confirmed (84%) when validating if users reached with the ID-free audience matched the targeting criteria.
- Inventory placement quality: ID-free audience ads appeared on higher quality inventory, compared to the same ID-based audience (ID-free 27% vs. ID-based 21%).
- Past Event Highlights
- Article
CTV Ads: Viewer Attention & Brand Metrics
Rohan Castelino – CMO, IRIS.TV
Mike Treon – Programmatic Lead, PMG
Representing the Alliance for Video Level Contextual Advertising (AVCA), Rohan Castelino (IRIS.TV) and Mike Treon (PMG) examined research conducted with eye tracking and attention computing company, Tobii. The research endeavor focused on the impact of AI-enabled contextual targeting on viewer attention and brand perception in CTV. Beginning the discussion, Rohan examined challenges with CTV advertising. He noted that advances in machine learning (ML) have empowered advertisers to explore AI enabled contextual targeting, which analyzes video frame by frame, uses computer vision, natural language, understanding, sentiment analysis, etc., to create standardized contextual and brand suitability segments. Highlighting a study of participants in U.S. households, the research specifically aimed to understand if AI-enabled contextual targeting outperformed standard demo and pub-declared metadata in CTV. Additionally, they wanted to understand if brand suitability had an impact on CTV viewers’ attention and brand perception. Results from the research found that AI-enabled contextual targeting outperformed standard demo and pub-declared metadata in CTV and increased viewer engagement. In closing, Mike provided the marketers’ perspective on the use of AI-enabled contextual targeted ads and its practical applications. Key takeaways:- Challenges with CTV advertising: Ads can be repetitive, offensive and sometimes irrelevant, in addition to ads being placed in problematic context.
- In addition, buyers are unsure who saw the ad or what type of content the ad appeared within. A recent study by GumGum showed that 20% of CTV ad breaks in children’s content were illegal (e.g., ads shown for alcohol and casino gambling).
- Advertisers have begun experimentation with contextual targeting in CTV, as a path to relevance.
- A study conducted with U.S. participants that examined the effects of watching 90 minutes of control and test advertisements, using a combination of eye tracking, microphones, interviews and surveys to gather data found that:
- AI-enabled contextual targeting attracts and holds attention (e.g., 4x fewer ads missed, 22% more ads seen from the beginning and 15% more total ad attention).
- AI-enabled contextual targeting drives brand metrics (e.g., 2x higher unaided recall and 4x higher aided recall).
- AI-enabled contextual targeting increases brand interest (e.g., 42% more interested in the product, 38% gained a deeper understanding).
- Research to understand if brand suitability had an impact on CTV viewers’ attention and brand perception found that:
- Poor brand suitability makes CTV viewers tune out ads and reduces brand favorability (e.g., 54% were less interested in the product, 31% liked the brand less).
- AI-enabled contextual targeted ads are as engaging as the show.
Top Topics at AxS 2024
Our analysis of the presentations during this year’s AUDIENCExSCIENCE conference shows which issues are marketers’ current priorities.
Insights from ARF Councils
ARF Councils have issued reports that provide unique insights on important issues, such as optimal terminology for how to reach diverse audiences and cross-platform measurement. Read more »