Experimental evaluation of anti- tobacco PSAs · Broad Objective To conduct experimental research...
Transcript of Experimental evaluation of anti- tobacco PSAs · Broad Objective To conduct experimental research...
Experimental evaluation of anti-tobacco PSAs
Supported by: NCI Center of Excellence in Cancer Communication Research Grant at the ASC/APPC University of Pennsylvania
Andrew Strasser, Joseph Cappella, Christopher Jepson, Robert Hornik, Martin Fishbein, Caryn Lerman
©2011 Caryn Lerman, DO NOT COPY OR USE
Improving Anti-Tobacco Public Service Announcements
•Mass media anti-smoking campaigns have had some promising results, but outcomes have been inconsistent
•Self-report data on message effectiveness have limitations
•Experimental research studies can be used for PSA design and evaluation
©2011 Caryn Lerman, DO NOT COPY OR USE
Broad ObjectiveTo conduct experimental research on features of anti-tobacco PSAs that influence persuasiveness, and translate this research to the development of more effective media campaigns
To test the main and interacting effects of PSA message sensation value (MSV) and argument strength (AS) on physiological responses, persuasion, and behavior change.
Specific Objective
©2011 Caryn Lerman, DO NOT COPY OR USE
Phase I: PSA Rating and Selection
1. Categorizing Topics
2. Evaluating Message Sensation Value
3. Evaluating Arguments
©2011 Caryn Lerman, DO NOT COPY OR USE
•569 smoking-related PSAs were categorized.
•Exclusion Criteria included:Target – ChildrenTarget – AdolescentsPSAs other than 30-second durationnon-English speakingsecond hand smoke-focus
1. PSA Categorization
©2011 Caryn Lerman, DO NOT COPY OR USE
Reliability of Categorization
2 Annenberg and 2 TTURC raters categorized the PSAs:
- TTURC raters were >90% in agreementwith Annenberg ratings
- TTURC raters were >95% in agreement with each other
- Overall Kappa = 0.89, p<.001
©2011 Caryn Lerman, DO NOT COPY OR USE
Target Audience (adults)
155942
010203040506070
Promote Tx-Seeking Neg. Conseq. (any) Knowledge about smoking
# of PSAs (n=99)
©2011 Caryn Lerman, DO NOT COPY OR USE
2. Evaluating Message Sensation Value PSAs differ in format, content and audiovisual features. Message Sensation Value (MSV) is a measure of these attributes.
Perceived Message Sensation Value (PMSV) is the subjective report of the MSV features.
MSV is coded for visual, audio and content dimensions.
Morgan, Palmgreen, Stephenson, Hoyle, Lorch: J of Comm, 2003.©2011 Caryn Lerman, DO NOT COPY OR USE
MSV Scoring (Kentucky/rPMSV Morgan et al)Category Item Scoring
Visual
Length Time (seconds)
Animation 0/1
Cuts 0=1-6; 1=7-14; 2=15+
Edits 0=1-6; 1=7-14; 2=15+
Faces count
Special Visual Effect 0/1
Slow Motion 0/1
Fast Motion 0/1
Unusual Colors 0/1
Intense Moments 0/1
©2011 Caryn Lerman, DO NOT COPY OR USE
MSV Scoring (Kentucky/rPMSV Morgan et al)
Category Item Scoring
Audio
Sound Saturation 0/1
Music 0/1
Sound Effects 0/1
Slow Voice 0/1
Fast Voice 0/1
Content
Acted out/ Talking Head 1 vs 0
Narrative 0/1
Unexpected Format 0/1
Surprise/Twist Ending 0/1
Celebrity 0/1
©2011 Caryn Lerman, DO NOT COPY OR USE
Reliability of MSV scoresInter-rater reliability of MSV scoring (2 raters):
MSV scores from subset of 45 of 99 PSAs:Kendall’s Tau = 0.906, p<0.001.
Correlations
1 .961** .523** .139.000 .000 .171
99 99 99 99.961** 1 .353** -.031.000 .000 .762
99 99 99 99.523** .353** 1 -.118.000 .000 .244
99 99 99 99.139 -.031 -.118 1.171 .762 .244
99 99 99 99
Pearson CorrelationSig. (2-tailed)NPearson CorrelationSig. (2-tailed)NPearson CorrelationSig. (2-tailed)NPearson CorrelationSig. (2-tailed)N
msv
visual
audio
content
msv visual audio content
Correlation is significant at the 0.01 level (2-tailed).**.
Internal Consistency (MSV, visual, audio, content dimensions)
©2011 Caryn Lerman, DO NOT COPY OR USE
3. Evaluating ArgumentsStep 1: Argument Extraction
Arguments were extracted in a 2-step process.1) 2 raters viewed the PSAs and typed explicit and
implicit messages from each PSA.2) 2 new raters viewed PSAs and read previous arguments to form one statement reflecting argument of PSA.
Step 2: Argument EvaluationShopping mall-intercept study of 300 participants (149 female) who were each presented 12 PSA arguments such that each argument was presented a minimum of 36 times (36-38 times) during the study.
All participants were current smokers (mean 17.5, sd = 11.7)who had smoked at least 100 cigarettes during their lifetime.
©2011 Caryn Lerman, DO NOT COPY OR USE
After each statement, participant was asked if the statement (rated 1 strongly disagree to 5 strongly agree):
Is a reason for quitting smoking that is BELIEVABLE
Is a reason for quitting smoking that is CONVINCING
Is a reason for quitting smoking that is NEW to me
Is a reason for quitting smoking that APPLIES to me
Gives a reason for quitting smoking that is IMPORTANT TO ME
Put THOUGHTS in my mind about quitting smoking
Put THOUGHTS in my mind about wanting to continue smoking
Helped me feel CONFIDENT about quitting smoking
Would help my friends quit smoking
Made me want to quit smoking
Overall, how much do you agree or disagree with the statement
Is the reason given for quitting smoking a strong or weak reason* Argument Evaluation Survey adapted from Zhao, Cappella et al (NIDA study)©2011 Caryn Lerman, DO NOT COPY OR USE
Descriptive Data on Argument Evaluation
Count 99
Mean 29.68
SD 2.23
Minimum 20.85
Maximum 33.03
©2011 Caryn Lerman, DO NOT COPY OR USE
Pearson CorrelationP-value
Arg. Eval. NFC Perc. Vul. Try to quit Will QuitInterest in Quit
Argument Evaluation Total
.061
.294.510 *.000
.424 *
.000.350 *.000
.524 *
.000
Need ForCognition
.061
.294.193 *.001
.135*
.019.056.337
.172*
.003
Perceived Vulnerability
.510 *
.000.193 *.001
.229 *
.000.182*.002
.476*
.000
Try to quit .424 *.000
.135*
.019.229 *.000
.767*
.000.662*.000
Will quit .350 *.000
.056
.337.182*.002
.767*
.000.604*.000
Interest in quit .524 *.000
.172*
.003.476*.000
.662*
.000.604*.000
Association of Argument Evaluation with Intention Items (N=300)
©2011 Caryn Lerman, DO NOT COPY OR USE
PSA Selection
0.00 1.00 2.00 3.00 4.00 5.00
rPMSV
-2.00
-1.00
0.00
1.00
2.00z_
scor
e
1.00
2.00
3.00
4.00
5.00
6.00 7.00
8.00
9.00
10.00
11.00
12.00
13.00
14.00
15.00
16.00
17.00
18.00
19.00
20.00
21.00
22.00
23.00
24.00
25.00
26.00
27.00
28.0029.00
30.00
31.00
32.00
33.00
34.00
35.00
36.00
37.00
38.00
39.00
40.00
41.00
42.00
43.00
44.0045.00 46.00
47.00
48.00
49.00
50.00
51.00
52.00
53.00
54.00
55.00
56.00
57.00
58.00
59.00
60.00
61.00
62.00
63.0064.00
65.00
66.00
67.00
68.00
69.00
70.00
71.00
72.00
73.00
74.0075.00
76.00
77.00
79.00
80.00
81.00
82.00 83.0084.00
85.00
86.00
87.00
88.00
89.00
90.00
91.00
92.00
93.00
94.00
95.00
97.00
98.00
99.00
-1SD Mean +1SD
+1SD -1SD
©2011 Caryn Lerman, DO NOT COPY OR USE
Study Design and Procedures
Baseline Questionnaire
View PSAs (as a set of 4)•Psychophysiology assessment
Outcome Measures•Beliefs, attitudes, intentions, etc
View PSAs (individually)•Processing,effectiveness
Recall Measures
N=50
N=50
N=50
N=50
Argument Strength
Weak Strong
MSV
Low
High
Calling TTURC Quit Line4 PSAs/condition
©2011 Caryn Lerman, DO NOT COPY OR USE
MSV
ARG
PSA•Arousal•Process•Recall
Beliefs Norms Intention
Attitude
Efficacy
Activation Theory Integrated Model
Conceptual Model
©2011 Caryn Lerman, DO NOT COPY OR USE
N=200
High MSV Low MSV
High Arg(N=49)
Low Arg(N=48)
High Arg(N=50)
Low Arg(N=53)
Age 46.4 (12.2) 42.4 (11.7) 44.2 (13.9) 41.3 (11.5)
Sex (% male) 47.5 44.6 40.8 50.1
Race (% White) 65.0 60.0 57.0 60.3
Education (%HS) 62.3 58.3 57.1 58.0
Cigs per day 21.7 (8.8) 20.4 (9.4) 23.3 (14.1) 18.6 (8.2)
FTND 5.2 (2.2) 5.4 (2.3) 5.5 (2.2) 5.3 (2.5)
Sensation Seeking 20.8 (4.3) 20.4 (4.4) 21.2 (3.9) 20.4 (4.8)
Descriptive Statistics by PSA Condition
©2011 Caryn Lerman, DO NOT COPY OR USE
MSV
ARG
PSA•Physio.•Process•Recall
Beliefs Norms Intention
Attitude
Efficacy
Activation Theory Integrated Model
Conceptual Model
©2011 Caryn Lerman, DO NOT COPY OR USE
Zygomatic and Corrugator
©2011 Caryn Lerman, DO NOT COPY OR USE
0
0.02
0.04
0.060.08
0.1
0.12
0.14
0.16
High Low
highlow
Physiological Measures -- Corrugator (mV)
Arg
MSVAS: ns MSV: p=.012
©2011 Caryn Lerman, DO NOT COPY OR USE
0
0.02
0.04
0.060.08
0.1
0.12
0.14
0.16
High Low
highlow
Physiological Measures -- Zygomatic (mV)
Arg
MSVAS: nsMSV: ns
©2011 Caryn Lerman, DO NOT COPY OR USE
MSV
ARG
PSA•Physio.•Process•Recall
Beliefs Norms Intention
Attitude
Efficacy
Activation Theory Integrated Model
Conceptual Model
©2011 Caryn Lerman, DO NOT COPY OR USE
4
4.2
4.4
4.6
4.8
5
5.2
5.4
High Low
highlow
Cognitive Processing
Arg
MSVAS: nsMSV: p=.013“…think about the consequences of quitting smoking as
shown in the PSAs” ©2011 Caryn Lerman, DO NOT COPY OR USE
MSV
ARG
PSA•Physio.•Process•Recall
Beliefs Norms Intention
Attitude
Efficacy
Activation Theory Integrated Model
Conceptual Model
©2011 Caryn Lerman, DO NOT COPY OR USE
0
20
40
60
80
100
High Low
highlow
Recall -- Percent Correct
Arg
MSV
AS x MSV: p=.001
©2011 Caryn Lerman, DO NOT COPY OR USE
MSV
ARG
PSA•Physio.•Process•Recall
Beliefs Norms Intention
Attitude
Efficacy
Activation Theory Integrated Model
Conceptual Model
©2011 Caryn Lerman, DO NOT COPY OR USE
3.9
4
4.1
4.2
4.3
4.4
4.5
High Low
highlow
Positive Beliefs (Mean: 5 items, 5 point scale)
Arg
MSVAS: nsMSV: ns
©2011 Caryn Lerman, DO NOT COPY OR USE
2.7
2.8
2.9
3
3.1
3.2
3.3
3.4
High Low
highlow
Negative Beliefs (Mean: 7 items, 5 point scale)
Arg
MSVMSV: p=.03
©2011 Caryn Lerman, DO NOT COPY OR USE
MSV
ARG
PSA•Physio.•Process•Recall
Beliefs Norms Intention
Attitude
Efficacy
Activation Theory Integrated Model
Conceptual Model
©2011 Caryn Lerman, DO NOT COPY OR USE
24
25
26
27
28
29
30
31
High Low
highlow
Efficacy (10 item; 1-4 scale)
Arg
MSVMSV: p=.035
©2011 Caryn Lerman, DO NOT COPY OR USE
MSV
ARG
PSA•Physio.•Process•Recall
Beliefs Norms Intention
Attitude
Efficacy
Activation Theory Integrated Model
Conceptual Model
NS
But… Thanks to Michael Hennessy, we find support for the Integrated Model when using perceived PSA parameters, rather than conditions…. Stay tuned..©2011 Caryn Lerman, DO NOT COPY OR USE
This is Your Brain on PSAs….
©2011 Caryn Lerman, DO NOT COPY OR USE
PSA Video:“Family Reunion”
30 sec
Frame #100:01:32
Frame #200:11:56
Frame #300:21:02
Frame #400:25:77
©2011 Caryn Lerman, DO NOT COPY OR USE
Recognition by MSV Level
1200
1250
1300
1350
1400
Hi MSV Lo MSV
RT
RT
©2011 Caryn Lerman, DO NOT COPY OR USE
Brain Effects of Anti-Tobacco PSAs
Recognition of PSA frames is positively correlated with prefrontal and temporal, and negatively correlated with occipital activation
Low MSV PSAs are associated with greater prefrontal and temporal activation; High MSV produced occipital activation
Langleben et al. Neuroimage, 2008©2011 Caryn Lerman, DO NOT COPY OR USE
Effects of Smoking Cues in Anti-Tobacco PSAs
Argument Strength
(between-subject)
Smoking Cue (within-subject)
No Yes
Strong3 PSAs 3 PSAs
Weak3 PSAs 3 PSAs
Yahui KangJ. Cappella
advisor
N=82©2011 Caryn Lerman, DO NOT COPY OR USE
3.103.153.203.253.303.353.403.453.503.55
pre no-cueads
post no-cue ads
pre smk-cue ads
post smk-cue ads
Smoking Cue
Sm
okin
g U
rge
Weak ArgumentStrong Argument
Smoking Cues in PSAs with Weak Arguments Increase Urges
Kang et al., 2009©2011 Caryn Lerman, DO NOT COPY OR USE
Eye Tracking Smoking Cues
• Ashley Sanders Jackson, Cappella et al• Visual attention to smoking cues in ad segment• 84 adult smokers• 16 PSAs
EOS = eyes on screenP < .01
©2011 Caryn Lerman, DO NOT COPY OR USE
Smoking Cues in Anti-Tobacco PSAs
Model of Cue Effects on Persuasion
We will also explore moderating influences on cues effects, such as nicotine dependence, motivation to quit, and gender.
Argument strength
-
-
+SmokingCues
Message Processing•Recall•Ad effectiveness
Cue-ResponseManipulation
urges
physio.Persuasion•Attitudes•Self-efficacy
Intention to quit
Smokingbehavior
Manipulation
+
+
3 (no cue, central cue, peripheral cue) x 2 (weak, strong argument) factorial design (n=300)…ads balanced for MSV, themes
©2011 Caryn Lerman, DO NOT COPY OR USE
Implications
• “Attention-grabbing” high MSV format could impede the learning and retention of PSA message
• Smoking cues in PSAs may increase urges and distract attention from PSA message, thereby reducing effectiveness (effect may depend on argument strength and whether cue is central to PSA argument)
• Further experimental support could influence PSA design and effectiveness of future anti-tobacco media campaigns
©2011 Caryn Lerman, DO NOT COPY OR USE
CollaboratorsAndrew Strasser, Robert Hornik, Joseph Cappella, Martin Fishbein
Paul Wileyto, Christopher Jepson, Paul Sanborn
Daniel Langleben, James Loughead, Kosha Ruparel
FUNDING: NCI P50©2011 Caryn Lerman, DO NOT COPY OR USE