Impact Evaluation of a Landing Page Used in Mercy For Animals’ Facebook Advertising Campaign

Results from an Experimental Design

January 20, 2015

2

Table of Contents

Background & Objectives 3

Survey Methodology 4

Sample Consideration & Analysis 5

Summary of Findings 6

Reported Changes in Dietary Habits 7

Reported Consumption of Animal Products 10

Attitudes 11

3

Background & Objectives

One tactic employed by Mercy For Animals (MFA) towards its overarching goal of preventing animal cruelty is funding Facebook ads that link to landing pages with videos designed to convince viewers to reduce or eliminate eating animal products. The idea is that reducing demand for animal products will reduce the number of animals raised in the meat, dairy, and egg industries. MFA focuses its Facebook ad buys on females ages 13-24 because they are viewed as a demographic sympathetic to preventing animal cruelty and are more likely to change their behavior than other groups.

The Facebook ads are viewed by MFA, and by many in the animal cruelty prevention community, as a cost-effective method for increasing awareness and educating target demographics about the conditions endured by farmed animals.

The purpose of this study is to understand the extent to which the landing page used in the Facebook ad intervention has an impact on changing meat, dairy, and egg consumption behavior, as well as attitudes and diet intentions, among the target audience.

4

Survey MethodologyPro-vegetarian ads were run on Facebook targeting women ages 13 to 24. Of those who clicked on the ad, half were assigned to the experimental group and directed to a landing page where they were shown a video on cruelty to farm animals called Farm To Fridge. The other half were assigned to the control group and taken to a landing page with a video focused on combatting tropical diseases. Both landing pages had links to additional resources on their respective topics. 

Facebook re-targeting pixels were used so that experimental and control groups could be targeted separately with follow-up surveys in the future.

Two to four months after the program intervention, the retargeting pixels were used to promote a survey to the two groups. A Facebook page called “Nonprofit Research” advertised the survey, so viewers would not know the survey was connected to the intervention. The ads offered the ability to win an Amazon gift card for participating in an academic survey. Control and experimental group participants who clicked on the ad were taken to separate (but identical) survey pages, where they completed the follow-up survey. Timeframe Experimental ControlFebruary 2015 Watch MFA video Directed to neutral video

May-July 2015 Take survey (post-exposure)

Take survey (no exposure to video)

5

Sample Considerations and Analysis

Results are reported among 1,433 females age 13-25. MFA’s Facebook ads target females in this age group, and therefore, this demographic is the focus of this survey and analysis. A small number of males and individuals of other ages who completed the survey were excluded from analysis. Survey data was also weighted to control for age and nationality.

As with any sample frame, the sample size allows a certain degree of precision. To detect small effect sizes with confidence (i.e., power), very large sample sizes are needed. Many of the effect sizes found in this study are small, and may fall outside the study’s margin of error due to the study’s sample size. Specifically, although no statistically significant difference was found in animal product consumption between Experimental and Control, due to the low power it is unclear whether the landing page with factory farming video had a positive, neutral or negative impact on animal product consumption.

Statistically significant differences between Experimental and Control samples are noted in this report at the 95% confidence interval where applicable.

Age 13-16 44%Age 17-20 31%Age 21-25 25%

Female 99%Other 1%Male --

United States 59%United Kingdom 17%Canada 15%Australia 8%Other 1%No response 0%

6

Summary of Findings

The experimental design did not uncover a difference in the number of servings of animal products that participants reported as having eaten in the previous two days. Viewing the landing page with factory farming video did not appear to alter the amount of animal products being eaten two to four months later.

The Experimental group was more likely than the Control group to report that they intended to eat less meat or to eat no meat four months in the future. Self-identification as vegetarian was also higher among the Experimental group than the Control group, even though the actual incidence of vegetarianism/veganism - as measured by reported consumption of animal products over the past two days - was the same.

Did not eat meat then, do not eat meat

now

Decreased in last 4 months

No changeIncreased in last 4 months

7%

26%

58%

17%10%

24%

55%

16%

Reported Changes in Dietary Habits (1 of 3)

7

One fourth report a decrease in meat consumption – however, both experimental and control samples show the same decreased consumption, suggesting that the reported decline is not a result of viewing the landing page with factory farm video.

Self-reported vegetarianism is higher among the Experimental group than Control group.

Changes During Ad DeploymentAt any time in the past four months, have you increased or decreased the total amount of meat, chicken and fish you ate? This may be a choice you made for a

few days or for more than a few days.

Experimental Control higher at the 95% CI for that age category

32%

55%

13%

32%

53%

15%

Reported Changes in Dietary Habits (2 of 3)

8

A small proportion report they are eating more meat now than four months ago, with the Experimental group slightly more likely to report eating more today (statistically significant at the 80% level).

Are Changes in Meat Consumption Sustained?

Compared to four months ago, are you eating more, less, or about the same amount of meat, chicken and fish (in total)?

Experimental Control

I will probably not be eating

any

I will probably be eating less (in

total) than I was four month ago

I will probably be eating about the same amount (in

total) as I was four months ago

I will probably be eating more (in total) than I was four months ago

9%

17%

61%

14% 11%

20%

55%

14%

Reported Changes in Dietary Habits (3 of 3)

9

Self-reported intent to reduce or exclude meat, chicken, and fish from the diet four months from now is higher among the Experimental group than the Control group.

Intended Future Meat Consumption

Four months from now, how much meat, chicken and fish (in total) do you think you will be eating?

Experimental Control higher at the 95% CI for that age category

Exp: Cont:31% 26%

10

Reported Consumption of Animal Products

Total

 Control Experimental

Base (n=684) (n=749)

Pork 1.115 1.225

Beef 1.595 1.580

Dairy 3.297 3.416

Eggs 1.389 1.447

Chicken 1.811 1.941

Fish 0.835 0.829

Reported Servings in Past Two Days

There is no observed and statistically significant difference in reported consumption of animal products between the Control and Experimental groups overall or by age groups.

37%

33%

26%

26%

62%

59%

24%

20%

23%

27%

47%

47%

16%

14%

20%

21%

36%

35%

Attitudes

11

Cows, pigs and chickens are intelligent, emotional

individuals with unique personalities

I know how to replace meat, chicken and fish dishes with appealing non-meat options

Eating less meat, chicken and fish (in total) is the right

thing to do

Experimental

Control

% “Strongly + Somewhat Agree”

Strongly agree Somewhat agree

For additional information, please contact:Gayle Vogel | 703-842-0204 | vogel@edgeresearch.com

Matthew Maher | 703-842-0210 | maher@edgeresearch.com

Edge Research1560 Wilson Blvd, Suite 475

Arlington, VA 22209www.edgeresearch.com