White paper Optimizing Audience Buying on Facebook and ...€¦ · Optimizing Audience Buying on...

White paper

Optimizing Audience Buying on Facebook and Instagram

July 2016

Contents

1 Introduction2 Methodology5 Results – Campaign delivery

– Brand lift

– Cost per lift

11 What it means for marketers

Optimizing Audience Buying on Facebook and Instagram 1

Running campaigns simultaneously across different digital platforms should provide advertisers with extended reach into new audiences as well as the ability to reach pre-existing audiences in a more cost-effective manner. However, in practice, it isn’t immediately obvious what the most optimal way is of achieving these benefits and how significant they will be for any given campaign.

Over the past few years, Facebook’s evolution into a family of apps and services has given advertisers new ways of extending their campaigns to platforms such as Instagram and the Audience Network. While some advertisers and agencies have been experimenting with manual allocation of budgets, Facebook fully launched placement optimization in late 2015 as a way of providing advertisers with an easier way of optimizing campaign delivery across Facebook, Instagram and the Audience Network. This paper aims to measure the effectiveness of optimizing audience buying to deliver value to brand advertisers.

Introduction

How placement optimization worksPrevious Marketing Science research has shown that over the course of a campaign, cost per outcome — from mobile app installs to brand awareness or online conversions — tends to vary between and within platforms due to audience behavior and characteristics. While advertisers and agencies can manually allocate budgets across platforms to account for these changes, placement optimization leverages Facebook’s ad delivery system to dynamically seek out the lowest cost per outcome at any given point in time wherever it’s available, whether it be on Facebook or, as in the illustrative example in Figure 1, on Instagram. As a result, placement optimization should provide lower cost per outcome than advertising on a single platform or even trying to manually allocate budgets across various platforms.1

Facebook IQ White paper2

Introduction

In this context, placement optimization is both an important and interesting topic for the Facebook Marketing Science team to study. Marketing Science’s mission is to help marketers understand how to leverage the maximum possible value from their advertising dollars. Determining whether placement optimization provides additional value and quantifying those gains is thus an important and straightforward goal for the team.

We are also interested in placement optimization, as it offers a novel opportunity to robustly measure multichannel, cross-platform campaigns. Cross-platform and cross-device campaigns have traditionally been difficult to measure effectively; cookie-based measurement does a poor job of tracking individuals across devices and platforms and can tend to undervalue the true contribution of mobile.2 Not surprisingly, a recent eMarketer survey showed that

when it comes to cross-platform measurement, 35% of advertisers are “not using a robust measurement technique,” while 34% say they “evaluate each channel individually and optimize based on channel-specific performance.”3 Rather than cookies, Facebook measurement solutions instead rely on people-based identities and conversion pixels, meaning we are in the unique position of being able to accurately measure the impact of a cross-platform campaign and better understand what factors might be contributing to any efficiency gains we observe.

For this study, we’ve used placement optimization across Facebook and Instagram as a starting point to measure placement optimization’s effectiveness for brand advertisers. In the future, we plan to extend our methodology to look at direct response objectives as well as expand it across our family of apps.

Cost

per

out

com

e

Time elapsed

Instagram-only avgcost per outcome

Facebook-only avgcost per outcome

Placement optimization avg cost per outcome

Facebook measurement solutions rely on people- based identity and conversion pixels rather than cookies

Figure 1. Graphic illustration of placement optimization mechanism


Methodology

Test design We focused our placement optimization research on brand campaigns that ran across Facebook and Instagram and used randomized controlled trials (RCT) as the primary methodology. We worked with a total of 10 brand advertisers across a range of verticals, countries, budgets and target audiences. While we deliberately chose a variety of advertisers and campaigns to test, we used the same underlying methodological approach throughout. Working closely with each advertiser, we split their budget and audience size equally between a placement optimization test cell in which an ad impression could be either on Facebook or on Instagram and a Facebook-only test cell where all impressions were delivered exclusively on Facebook.

Our choice of test cells was deliberate. We could have potentially added other test cells or combinations of test cells, such as including an Instagram-only cell or cells with manually allocated budget across Facebook and Instagram. While these are both valid test designs that would have provided potentially meaningful insights, our consideration was driven primarily by a desire to focus on the most common question we get from advertisers that tend to be comfortable with their Facebook campaigns and that are looking for answers on how best to incorporate Instagram into their campaign strategy. With that question as our priority, we felt the Facebook-only versus placement optimization test design was the most logical question to start understanding this topic.

With the two test cells decided up-front, every Facebook and Instagram user in a given test was randomly assigned to either the Facebook-only or the placement optimization cell prior to the campaign launch. Each test cell also had its own control group that did not receive any ads. This setup meant that the only difference between the two test cells was that consumers in the placement optimization cell had the opportunity to be reached across both Facebook and Instagram whereas consumers in the Facebook-only cell could only be reached on Facebook. Apart from that important difference, every other campaign feature — such as campaign objectives, optimization method, maximum bid (if using the auction), audience targeting and creative — were exactly the same across both test cells. Figure 2 provides an overview of our test design.

This randomization of consumers across rigidly defined cells is the hallmark of the RCT methodology and ultimately provides for greater measurement accuracy compared to observational studies.4


Methodology

Measurement of brand lift

Placement optimization

Randomize groups Potential placement Brand polling

Facebook-only

Facebook & Instagram(+ other media)

Facebook(+ other media)

Note: Campaign budget, target audience, creative and campaign durations are hold constant for both cells.

Figure 2. Test design


Methodology

Data collection procedureDespite some limitations, surveying has traditionally been the data collection method of choice for brand advertisers that want to understand the impact of a given campaign on their brand equity. With that in mind, we measured the impact of each of the two test cells (placement optimization and Facebook-only) by surveying a randomly selected subset of Facebook and Instagram users from both the exposed and control groups. We collected on average about 10,000 survey completions per test. The surveys consisted of three questions asked sequentially to the same person: while all the campaigns we measured asked ad recall as the first question, advertisers were allowed to customize the other two survey questions they wanted to ask. These questions generally fell into either upper-mid funnel metrics around awareness and affinity to lower-funnel metrics around purchase intent or willingness to recommend.5

One important decision regarding our test design was about choosing the platform on which to administer the surveys. We ultimately elected to survey on Facebook mobile feed regardless of whether respondent had seen ad impressions on Facebook or Instagram. The main reason for this was to control for the fact that survey respondents on Instagram are not necessarily the same kind of survey respondents we see on Facebook; they likely have different baseline

attitudes and perceptions of the brands we were testing, which could complicate comparing survey results between platforms and between test cells. We therefore chose to limit this potential bias by surveying exclusively on Facebook. The tradeoff of this approach is that respondents who saw ad impressions on Instagram may not have been active on Facebook and thus not eligible to respond to the survey on Facebook. This in turn could have led to an outcome where we underestimated Instagram’s true impact since we might have collected relatively fewer respondents who were exposed on Instagram.

To address this potential bias, we compared the distribution of Instagram survey respondents in the placement optimization cell to the actual Instagram reach numbers for the campaigns in question. Overall, we did find that Instagram was slightly underrepresented in terms of total survey respondents who were exposed to the Instagram campaigns when compared to actual Instagram reach. However, the responses from both groups — those exposed on Facebook and those exposed on Instagram — showed no statistically significant differences. Given that there were no meaningful differences in survey responses between those exposed on Facebook and those exposed on Instagram, we concluded that our survey approach for this campaign did not appear to introduce any noticeable biases.


Methodology

Hypotheses

In thinking about our test design, we considered three different ways in which placement optimization could impact a given campaign and formulated a hypothesis for each.

Hypothesis 1 (Campaign delivery): Placement optimization is more efficient at campaign delivery than Facebook-only. By allowing ad spend to flow freely across Facebook and Instagram, placement optimization ultimately has access to a wider audience, which should lead to higher reach with lower cost per impression (CPM) and/or cost per reach (CPR).

Hypothesis 2 (Brand lift): Placement optimization generates higher lift than Facebook-only. Lift here is defined as the difference between the percentage of favorable answers between the test and the control groups. One reason why placement optimization might perform better is that it is more effective at reaching audiences that are more likely to be influenced by a campaign. Another reason might be that all things held equal, seeing an ad on two different platforms might create a more lasting impact than only seeing it on one platform. For this study, we compared the brand lift between placement optimization versus the Facebook-only cell by using a difference in difference framework (i.e., is the difference between the control and exposed for a given brand metric higher for placement optimization than for Facebook-only?).

Hypothesis 3 (Cost efficiency): Placement optimization is more cost-effective in driving lift. If both Hypothesis 1 and 2 hold — e.g., you’ve reached more people and created more lift for the same budget — it follows that placement optimization would offer a more cost-effective way of achieving a campaign’s objectives. However, it’s also possible for Hypothesis 3 to be true even if either Hypothesis 1 or Hypothesis 2 is not supported. For instance, it is theoretically possible to observe the same level of lift between the two test cells but emerge with more people being impacted by the campaign via increased reach. The metric we use for testing this hypothesis is cost per lift (CPL). It is a concept the Marketing Science team has previously encouraged advertisers to consider6 and is essentially a function of three interdependent factors: campaign spend, ad effectiveness and reach and frequency. Cost per lift captures the relationship of these factors:

Cost per lift = f (Reach and Frequency, Ad Effectiveness | Spend) = Spend/(Reach × Lift)

Simply put, cost per lift estimates how much it costs an advertiser to change the attitude/perception of a given person influenced by a campaign.


Results

Between December 2015 and May 2016, we worked with 10 brand advertisers to run placement optimization tests. These advertisers came from a range of countries (Brazil, Canada, UK and US) and verticals including CPG (3), Retail (2), Automotive (2), Entertainment (1), Politics (1) and Charity (1). On average, campaigns had an audience size of 13 million and an impression frequency of 1.8 per week over three weeks (which we consider to be in an optimal range). Across the campaigns, we also observed that on average, 83% of impressions were delivered on Facebook and 17% of impressions were delivered on Instagram.

For each campaign, we measured the following metrics:

• Cost per reach/impression (Hypothesis 1)

• Brand lift (Hypothesis 2)

• Cost per lift between the placement optimization and the Facebook-only cell (Hypothesis 3)

Campaign delivery

Result 1 (Campaign delivery): Hypothesis 1 is supported. On average, placement optimization campaigns had 4.1% more reach and a 5.2% lower cost per impression (CPM), which in turn led to a lower cost per reach (CPR) of 5.8%. For the average campaign, this translated into an additional half million people reached for the same budget when compared to the Facebook-only cells. The increased reach and lower CPMs were even more pronounced for campaigns that were booked using Reach and Frequency (+6.4% for reach and -7% for CPM).

As illustrated in Figure 3, we also found evidence that campaign spend might be positively correlated to placement optimization efficiency. For example, the larger the campaign, the more value placement optimization delivers.

0.0%

5.0%

-5.0%

10.0%

-10.0%

15.0%

-15.0%

20.0%

-20.0%Cost

of r

each

ing

1000

peo

ple

impr

ovem

ent

Total spend

Relationship between total spend and cost of 1000 reach improvement

Figure 3: The relationship between total spend and cost of reach improvement

Across the campaigns, we observed that on average, 83% of impressions were delivered on Facebook and 17% of impressions delivered on Instagram


Results

Brand lift

Result 2 (Brand lift): Hypothesis 2 is rejected. There is not a statistically significant difference in lift between the placement optimization and the Facebook-only test cells. The hypothesis is rejected for both upper- and lower-funnel brand metrics.

At the outset, Hypothesis 2 was going to be the most difficult one to prove. Even though we collected a relatively large number of survey responses (~10,000 respondents per campaign), the margin of error around the surveys means that in some instances placement optimization would have had to perform several percentage points better than Facebook-only for us to register a statistically significant lift. In essence, even with larger sample sizes, survey is still prone to Type II error when it comes to hypothesis testing. Given the above reasons, we are not willing to unambiguously reject Hypothesis 2 that placement optimization may indeed be providing higher lift but have to conclude that any gains it does provide may be marginal and difficult to fully measure with conventional survey.

While placement optimization did not provide additional lift per se, it is important to note that it did provide equivalent lift, as it reached more people overall. In other words, placement optimization produced the same level of lift even though its budget was ultimately spread across a wider exposed audience. This is because despite the increased reach in placement optimization, there was only a marginal difference in average frequency between the two cells (2.94 for placement optimization and 2.99 for Facebook-only). Recent Facebook research7 has shown a positive and causal relationship between frequency and brand lift and in the case of our tests. Hence, we believe brand lift was constant across the two cells because placement optimization’s greater reach didn’t come at the expense of less frequency.

Placement optimization produced the same level of lift even though its budget was ultimately spread across a wider exposed audience


Results

Cost per lift

Result 3 (Cost per lift): Hypothesis 3 is supported. Across the various types of brand questions we asked, from top-funnel questions like ad recall to lower-funnel questions like purchase intent, cost per lift on placement optimization was on average 10% to 27% lower. As mentioned in the Hypothesis section of this paper, cost per lift essentially tells us how much an advertiser paid, on average, to change the attitudes or raise the awareness level of a given person influenced by the campaign. In the case of our placement optimization tests, when we combine the fact that 1) lift is comparable between placement optimization and Facebook-only, 2) campaign spend is the same and 3) placement optimization had a larger reach, the cost of impacting a person is lower for placement optimization than the Facebook-only cell.

Illustration of placement optimization cost efficiency calculation

Cost per lift =x Reach

Spend

Brand lift

Note: - Spend is the same for the 2 test cells- Brand lift is the same for the 2 test cells- Reach is higher for placement optimization

A look at how sales outcomes were impacted by placement optimization While all of our campaigns used brand survey as the primary measurement tool, one of the 10 campaigns also concurrently ran a DLX study, giving us an opportunity to look at how sales outcomes were impacted (if at all) by placement optimization. The overall results for the campaign (across both test cells) were statistically significant and return on ad spend (ROAS) positive. Within the test cells, placement optimization showed a 5% improvement over the Facebook-only cell. While inferences cannot be made based on a single data point, the results point to the possibility that the incremental reach offered by placement optimization might also be able to drive more real-world outcomes.

Figure 4: Illustration of placement optimization cost efficiency calculation


Conclusion

What it means for marketers

While our initial research is somewhat narrow in scope, the results of our research nevertheless offer a clear answer to a specific question many brand advertisers have about whether placement optimization performs better than Facebook-only campaigns. We plan to build on this initial set of findings in future research, looking at things like direct response objectives, creative differentiation and other apps and services, such as the Audience Network. Our findings show that advertisers should use placement optimization for brand campaigns. Across the 10 brand studies we looked at, placement optimization consistently reached more people at lower cost, generated comparable lift to Facebook-only campaigns and provided better cost efficiency in moving brand metrics. It is worth noting that all of our tests used the same creative across both Facebook and Instagram. Future tests will look at the impact of customizing creative across both platforms. While there may be specific instances where it makes sense for advertisers to think about running Facebook and Instagram campaigns separately, we believe that, at a minimum, brand advertisers should think of placement optimization with identical creative as the default option over a Facebook-only campaign.

As a starting point, we think this research is useful both in terms of its insights and the validation it provided for the multi-cell lift test. We have three key recommendations for marketers interested in running brand campaigns across Facebook and Instagram:

1. Focus attention on the overall outcome instead of guaranteed delivery on a specific platform. It is normal for campaign delivery to take place mostly on one platform rather than be evenly distributed across both platforms. When seeing skewed delivery, do not interpret that as a sign of placement optimization underperforming — rather see it as placement optimization doing its job and focusing your spend on the platform(s) where it is finding the most value for a given campaign and audience.

2. Be mindful of ensuring sufficient campaign spend and keep a large audience. Placement optimization appears to deliver better results with larger audiences and budgets. Smaller placement optimization campaigns will likely never do worse than a Facebook-only campaign but the benefits might not be as evident.

3. Set up your own randomized control tests to scientifically measure placement optimization versus Facebook-only. Some advertisers tend to evaluate a new ad product or strategy by comparing results to previous campaigns. Although this might sound intuitive and can potentially provide some level of insight, this approach could generate heavy biases based on the differences, however small, that take place between campaigns. We therefore encourage marketers to set up their own randomized control tests (either using the lift tool or our split tool) to validate how effective placement optimization works based on their business objectives.

Placement optimization consistently reached more people at lower cost, generated comparable lift to Facebook-only campaigns and provided better cost-efficiency in moving brand metrics


Appendix

Notes & Sources

1 Placement optimization delivers impressions only where they are most cost-efficient instead of evenly across platforms. There may be instances when ad impressions have zero delivery on a platform due to different audience activities across different times of day.

2 “People-Based Measurement: In a World of Increasing Consumer Choice,” Facebook, Feb 2016.

³ “Marketers Still Struggle with Cross-Channel Measurement” by eMarketer, Nov 11, 2015.

⁴ The specific benefits of the RCT methodology, compared to more traditional observations studies, are discussed in depth in a recent joint white paper between Facebook and the Kellogg School of Management of Northwestern University, “A Comparison of Approaches to Advertising Measurement: Evidence from Big Field Experiments at Facebook.” See also, “Demystifying Measurement: Why Methodology Matters” by Facebook IQ, Mar 2016.

5 Ad recall: Do you recall seeing an ad on a mobile device in the past 7 days?

Awareness: Have you heard of ____ campaign? / Which brand comes to your mind when thinking of ___?

Familiarity: How familiar are you with _____? Favorability: What is your opinion of _____ ? Intent: How likely are you to consider _____?

6 “Measuring for Success: Facebook on Facebook and Instagram” by Facebook IQ, Sep 2015.

7 “Efficient Frequency” by Facebook IQ, Jul 2016.

White paper Optimizing Audience Buying on Facebook and ...€¦ · Optimizing Audience Buying on...

Documents

Transcript of White paper Optimizing Audience Buying on Facebook and ...€¦ · Optimizing Audience Buying on...