In Design Group 3 White Paper

8/2/2019 In Design Group 3 White Paper

1/19

To: Christa TestonCc: Melissa ClarkFrom: Brian Bowles,Dylan Brown, Scott Linja, Brian Lockner, Maxine Major

Infographics Effectiveness

White Paper

Introduction

Wildfire danger is a prevalent concern in the Northwestern United States. Currently land managers

still do not have enough resources to manage the increasing severity of wildfires and costs

associated with wildfire damage and prevention. Melissa Clark from the University of Idahos

Department of Natural Resources Department will be conducting a study on how to engage

residents of the Northwest in supporting legislation to help prevent and treat wildfires, and to take

personal action to minimize the danger of wildfires.

Our study was a pilot study to assist Melissa Clark by testing the effectiveness of a set of

infographics, one gains-framed and one loss-framed infographic, to spread knowledge about the

dangers of wildfires and influence the views of the participants viewing either infographic toward

supporting wildfire control methods. The results from our study will allow us to provide Ms. Clark

with data on which infographic tends to influence a greater change in participants responses, and

to provide feedback on how to maximize the effectiveness of either infographic.

This white paper outlines our study to test the effectiveness of these infographics, and to determineany areas in which the effectiveness of these infographics might be improved. The infographics weused were developed by a technical writing class at the University of Idaho during the Springsemester 2012.

Gains-framed and loss framed infographic versions of the infographic were presented to differentgroups of people. We conducted knowledge and opinion tests before and after participants viewedthe infographics, and the answers to the questions were interpreted to evaluate the participantsknowledge of wildfires and prevention. We also attempted to evaluate how either viewing a gains-framed or loss-framed infographic may have influenced participants to change their opinion aboutthe dangers of wildfires, and helped influence participants toward taking action to preventwildfires.

We aimed to discover the following through this study:

Is either infographic successful at increasing the participants wildfire preventionknowledge?

Does either infographic successfully influence the participants views on wildfires? If either or both of the above is true, is one of the infographics more successful than the

other at this?


2/19

In order to determine whether or not viewing an infographic was effective, we evaluated howmuch the individual participants gained knowledge and if their responses indicated a changedperception of fire danger and willingness to participate in wildfire prevention measures.

This study was not designed to tell us how either infographic could be improved to affect

the participants views, or how to convey more knowledge to the participant. This studywas designed to only tell us which infographic, gains-based or loss-based, was moreeffective to convey information, and if participants viewing either infographic were morelikely to take action.

Methods

Our group used three separate tests in order to determine how effective both the gains-framed and

loss-framed infographics were on participants from different demographics. The participants were

selected based upon proximity to the various group members administering the separate tests. The

three groups included a random sample of 15 FarmHouse fraternity members (Tests administeredby S. Linja), 10 Argonaut staff members (Tests adminstered by D. Brown) and 6 Moscow residents

(Tests administered by B. Lockner). A total of 31 people participated in this test.

The tests were administered in this order:

Pre-treatment:

1. Interview

2. Knowledge Quiz

3. Survey

Treatment:

4. Infographic

Post-treatment

5. Knowledge Quiz

6. Survey

Interview (see Appendix B)

A fill-in-the-blank interview was used to examine independent variables within the various test

groups. The variables included gender, major, hometown, home state, rural/urban community,

political affiliation, and previous wildfire experience. The data collected would be analyzed to

determine significant trends between certain demographics compared to the results of the survey.

Knowledge Quiz (see Appendix C)After completing the survey, the participants were given a six question knowledge quiz prior to

viewing the infographic. The knowledge quiz questions were relevant to understanding wildfire

dangers and related information that could be answered by a cursory viewing of either infographic.

Each participant completed the knowledge test, and after treatment, the participant took the

knowledge test again. The responses were analyzed to determine the participants level of wildfire


3/19

knowledge prior to viewing the infographic, and how much their amount of wildfire knowledge

changed after treatment.

Survey

After completing the interview, each test group was given a survey that examined each

participant's perceptions and beliefs pertaining to wildfires and prevention. The survey, formulated

by Melissa Clark, investigated the participants existing beliefs/perceptions prior to an examination

of either infographic. After the infographics had been examined during the treatment, each group

was given the survey once more, and results were analyzed to determine if the participants views

had changed.

Treatment (see Appendix A)

After taking all three pre-treatment tests, participants were divided randomly into two groups forthe treatment session. For one minute, each group was allowed to view one of the twoinfographics, either the gains-based infographic, or the loss-based infographic. After one minute,

the infographics were removed from the participants sight, and they proceeded to take the post-treatment tests.

The responses from all three methods were compiled and analyzed to determine if there was a

marked difference between pre-treatment and post-treatment responses.

Results

Note: After testing had completed, it was discovered that it had not been noted which participants

from the Argonaut Staff group had viewed either the gains-based or loss-based infographic. Sinceeach participant did complete the treatment according to the test plan, their overall scores were

included in overall totals, but for the purpose of segregating results based on type of infographic

viewed, their results were excluded entirely.

Knowledge Test Results (See Appendix C)

Prior to viewing the infographics, participants overall scored 48.9% on the knowledge quiz, and

after treatment, overall scores increased by 31.7%. The range of pre-treatment knowledge test

scores differed among the three demographics by 15%, but post-treatment, the range narrowed to a

4.4% difference in scores. (Figure 1)

The group with the lowest pre-treatment scores was the FarmHouse Fraternity, and this group hadthe highest post-treatment increase in test scores. In contrast, the Moscow Residents group startedwith the highest pre-treatment scores, and their post-treatment scores indicated the lowest increasein knowledge gained of all three groups. The Argonaut staff had the highest overall scores at theend of testing. (Figure 1)


4/19

Figure 1: Knowledge Test Scores by Demographic

Although recipients of either infographic were selected at random, participants who would receivethe loss-based infographic did worse on the pre-treatment knowledge test than participants whowould receive the gains-based infographic. This is why the difference in scores was higher afterviewing the infographic, despite receiving overall lower post-treatment scores. (Figure 2)

Figure 2: Knowledge Test Scores by Infographic Viewed

Knowledge Test AnalysisThe Moscow Residents demographic is expected to be the most closely related to the actual

intended audience for these infographics, and the results that the infographics had on the Moscow

Residents represent the expected effectiveness of these infographics. Because Moscow residents

had the highest pre-treatment knowledge scores, it could be hypothesized that the average reader of

the infographic may already have a high level of wildfire knowledge, and that the majority of


5/19

readers may not have a knowledge increase comparable to the FarmHouse and Argonaut

demographics.

However, the average audience is still expected to gain approximately 30% more knowledge of the

subject matter by viewing either infographic. This may indicate that these infographics could be

useful to educate residents of the Northern Rockies in Idaho and Montana.

The gains-framed infographic readers saw a knowledge gain of 30% while the loss-framed

infographic readers a 33% knowledge gain. Due to the initial lower score of the loss-gains pre-

treatment test, this 3% discrepancy may be considered insignificant when compared to the average

knowledge gain of either infographic. (Figure 2)

All three demographics had significant increases in knowledge. The Farmhouse fraternity scored

low on the pre-treatment knowledge test, and the Moscow Residents scored highest out of all three

groups. After treatment, all three demographics scored significantly higher than their pre-treatmentscores, and the Farmhouse fraternity scores were nearly equal to those of the Moscow Residents.

This indicates that these infographics educated different demographics equally well. (Figure 1)

Knowledge Test Discussion

This knowledge test was simple, brief, and tested participants short term memory to recall wildfire

data successfully. This did not test the longevity of participants memory, and their ability to recall

important information at a later date, such as would be useful in a real-world setting. In addition,

while the infographics gave the participants information, it could have been better developed tohelp participants to recall information reflecting the importance of wildfires and on actions that

individuals can take to prevent wildfires.

Survey Results (See Appendix D)

The survey given to participants was divided into three sections. For the purpose of this study, in

order to evaluate the infographics ability to affect the views of the participants, we focused on the

results of Question #3 only. Question #3 asked participants to rate the perceived effectiveness of

several action items, and how likely the participant was to personally take action on that item. The

scale ranged from -3 (least effective / least likely to take action) to +3 (most effective / most likely

to take action).

Six participants neglected to complete this section, and to maintain the integrity of data collected,

their incomplete answers were not included in the averages for this section.


6/19

Figure 3. Total Effectiveness vs. Willingness to Take Action

Participants tended to assign a higher numerical value to their perceived effectiveness of asuggestion than to the likelihood of personally taking action on that suggestion. Both pre-treatment and post-treatment data supported this trend, as noted in Figure 3.

The rated effectiveness of suggestions generally increased post-treatment for participants whoviewed either infographic, with the exception of the fourth listed suggestion: Donating to localfire management organizations. Rated effectiveness scores decreased by 0.2 and willingness totake action decreased by 0.1 post-treatment. (Figures 4 and 5) Willingness to take action inregards to the third suggestion, Taking extra precautions to ensure that campsites are free of trashand debris. also declined 0.3, post-treatment. (Figure 5)

Figure 4: Effectiveness of Suggestions


7/19

Figure 5: Likelihood of Taking Action

Participants who viewed either infographic generally rated the effectiveness of suggestions higher

post-treatment. Participants who had viewed the gains-framed infographic reported 0.2 points

higher effectiveness, and those who had viewed the loss-framed infographic responded with 0.1

points higher, post-treatment. (Figure 6)

Figure 6: Effectiveness of Suggestions by Infographic Viewed

Participants who viewed the gains-framed infographic marked their willingness to take action 0.2

points higher than pre-treatment, and participants who viewed the loss-framed infographic ranked

their willingness to take action an average of 0.1 points lower, post-treatment. (Figure 7)


8/19

Figure 7: Willingness to Take Action by Infographic Viewed

Survey Analysis

Both infographics were effective to increase the overall effectiveness ratings of suggestions, butpost-treatment, willingness to take action was still significantly lower than rated effectiveness. Thisimplies that these infographics may have increased concern on a public level, but not enough thatparticipants felt a need to take personal responsibility to help prevent wildfires.

Although the rated effectiveness of ensuring that campsites are free of debris increased, the

participants willingness to take action decreased, indicating that the infographics had a negative

effect on participants desire to take action. Because the rated effectiveness and willingness to

donate to local fire management organizations both decreased post-treatment, this indicates that theinfographics were defective in influencing participants to donate money. (Figure 4)

Participants who viewed the gains-framed infographic increased their rated effectiveness ofsuggestions by twice as much as the loss-framed infographic, implying that the gains-framedinfographic was more effective to influence participants to the importance of the suggestions.

The gains-framed infographic increased participants willingness to take action while the loss-framed infographic lowered their willingness to take action. This would indicate that the loss-framed infographic deterred participants from taking action, and could be considered defective inthis aspect. (Figure 3)

Most of the suggestions were rated higher post-treatment in both effectiveness and participant

willingness, indicating that the infographics were minimally effective to increase participants

concern. (Figure 3)

Survey Discussion


9/19

Survey results revealed a strong discrepancy between the rated effectiveness of a suggestion andthe willingness of the participant to take action on that suggestion. It might be worth the efforts ofinfographic development to attempt to close this gap, and influence the readers of eitherinfographic to take action. This might be possible if the infographic were developed with morepersonal or region-specific information, such as land management contact information, information

on how to support legislation, or easy methods individuals can use to take preventative measuresagainst wildfires.

Since post-treatment scores declined for participants rated effectiveness and willingness to donatemoney to fire management organizations, it appears that these infographics were defective to givethe participants any sort of fiscal responsibility. It might be concluded that prior to viewing theinfographics, participants may not have known what measures were already in place to managewildfires. After viewing the infographics, participants may have learned that land managementpractices are already in place, and felt less pressure to become personally responsible to offerfinancial assistance. To better influence the target audience, it might be best to modify theinfographics to include information that shows a need for increased financial support and the

benefits of increased funding for wildfire prevention.

Participant Interview Results (See Appendix B)

We collected information about each of the participants in an attempt to draw correlations betweennot only the specific demographics we selected, but others we may not have intended. The figuresbelow are the results of demographic data compared with the survey statement,Im concernedabout wildfires near my community.

All participants pre-treatment rated their wildfire concern between -0.7 and -0.6 points. Post-treatment, participants who viewed the gains-framed infographic increased wildfire concern to apositive score of 0.4. Although participants who viewed the loss-framed infographic increased

their scores by 0.5, their overall wildfire concern still ranked negative. (Figure 8)

Figure 8: Wildfire Concern by Infographic Viewed


10/19

Pre-treatment, wildfire concern was an average of 0.0 for Moscow Residents, and negative for allother demographics. Every demographic increased their post-treatment concern levels, but onlyMoscow Residents reported a positive level of concern. (Figure 9)

Figure 9: Wildfire Concern by Demographic

Residents of Idaho and Washington scored negative levels of concern pre-treatment, and positiveconcern post-treatment, while residents of Alaska and Oregon scored neutral and positive concernpre-treatment, but changed to negative concern post-treatment. Residents of California seemedunaffected by the treatment, and rated perfectly negative scores before and after treatment. (Figure10)

Figure 10: Wildfire Concern by State of Residency

Some political responses received may be considered to be errors (i.e. Buddhism is generally notconsidered to be a political view), but all views expressed were considered to best represent theentire group of people tested. Overall, there was a mix between which political affiliations changed


11/19

their level of concern. Buddhism, Communist, Democrat and Moderate all reported negative levelsof concern, although Conservative increased concern slightly post-treatment, and Moderatedecreased concern.Democrats and Libertarian decreased concern after treatment, but Independent, Liberal,Republican, and Undecided all increased concern post-treatment. (Figure 11)

Figure 11: Wildfire Concern by Political Views

Participants who ranked their hometown as urban increased their wildfire concern from -1.2 to1.0, but participants who stated that their hometown was rural decreased wildfire concern from-0.3 to -0.4 post-treatment. (Figure 12)

Figure 12: Wildfire Concern by Community Type

Interview Analysis

Those who viewed the gains-framed infographic yielded a positive level of concern, post-

treatment, but participants who viewed the loss-framed infographic, despite an increased level of

concern, still remained negative regarding their concerns about wildfires. This would indicate that


12/19

the loss-framed infographic had failed to adequately increase concern about wildfires to a desirable

level. (Figure 8)

Although Moscow Residents rated their wildfire concern higher post-treatment, the other

demographics still failed to express a positive amount of concern for wildfire danger. This may

indicate that either infographic had failed to inspire the majority of our participants to be concerned

about wildfires. (Figure 9)

Data indicating that residents of Idaho and Washington were less concerned before treatment, and

more concerned post-treatment may indicate that they were unaware of wildfire dangers before the

test, but were willing to concede that healthy wildfire concern is necessary post-treatment.

However, residents of Alaska and Oregon, who indicated positive concern pre-treatment, lost

concern, which indicates that the infographics may have been ineffective to generate concern. This

is supported by California residents having completely negative and unaffected concern before and

after treatment. (Figure 10)

Political views and levels of concern seemed to be as randomly increased or decreased, and

although strong correlations could be made between individual groups, groups we would assume

would test at similar levels of concern (i.e. Conservative and Republican) failed to do so. This

would indicate that the size of our test groups was not large enough to be a true sample of the

population. (Figure 11)

As expected, participants from rural communities seemed to have higher levels of wildfire concern

than those of participants from urban communities. However, the infographics seemed to be much

more effective on urban dwellers, since their levels of concern appeared to increase. The fact thatparticipants from rural communities decreased their levels of concern after treatment indicates a

failure for the infographic to raise concern for all demographics. (Figure 12)

Interview Discussion

The infographics could be improved to encourage participants to rank wildfire concern higher

overall. Even though Moscow Residents rated post-treatment wildfire concern much higher than

the other groups, the other demographics failed to yield a positive amount of concern. Despite

increased post-treatment concern, low concern will yield low support of wildfire prevention

measures, indicating that these infographics fail to generate concern.

The survey generating inconsistent scores between demographics from different states would be

worth pursuing as a greater, separate research project. The scores received from our smaller test

indicate either a complete failure of the infographics to generate concern, or a very strong

difference in both knowledge and concern between different states. Further testing would be

necessary, and might be recommended since the intended research will encompass several different

states in the Northwest.


13/19

Whether or not there is a true correlation between political affiliation and levels of concern would

also be better determined through a more extensive test. The data we collected on this did not seem

to indicate any true strong correlations since there were very few members tested from any one

political party. However, performing a large-scale test to discover the effectiveness of the

infographics on different political parties might be quite useful if the purpose of the demographics

is to encourage residents of the Northwest to support land management legislation.

Since the infographics failed to increase levels of concern for participants from rural communities,

this could also be an indicator that the infographics information may have educated an already

concerned demographic about fire prevention measures already in place, and helped to ease

concerns about wildfire dangers. This indicates that perhaps the infographics were merely

informative, and did not convey the real and present dangers that individuals living in the

Northwest region could face. The infographics may be improved by ensuring that these dangers are

expressed.

Overall, there is a lack of real evidence to indicate that either of these infographics truly raise

levels of wildfire concern. Most of these tests indicate that at least half of the participants are not

actively concerned about wildfires. The infographics should be modified to concern levels in the

participants. Our study, however, did not perform tests to determine how this could be done.

Other Findings

Participants did not indicate any difficulty or discomfort in answering questions on the interview.

Participants did not indicate any problems with the knowledge quiz.

The test administrators for all groups stated that many participants asked the administrator to

explain what a prescribed burn is. The administrators did not answer the question to ensure

fairness in evaluating knowledge scores, but the question was prevalent enough that it became a

concern.

Participants had several problems with Question #2 on the survey, indicating difficulty in

answering the statements with the given ranking. FarmHouse and Moscow Residents did not write

their concerns on the survey itself, but both the Moscow Residents and FarmHouse test

administrators reported general confusion, and told participants to answer as best they could

regardless.

Argonaut participants written comments included:

This question does not work.

On statement Prescribed fire is too uncontrollable to be a forest management tool,


14/19

participant commented, Important in my life doesnt work.

Relabeled categories to disagree, neutral, and agree

On Question #2 instructions, This doesnt make sense.

On statement Prescribed fire is too uncontrollable to be a forest management tool,

participant commented, I dont get it.

Relabeled categories to disagree on left and agree on right.

Scribbled out the categories altogether.



Commented entire section: I think these questions are better suited to agree or disagree,

and thats how I answered.

Discussion

Effectiveness of Infographics

After the tests were tabulated and analyzed, it was easy to see that the infographic had done its job

effectively; there was a substantial amount of knowledge gained on average by nearly all test

participants. Both infographics resulted in around a 30% gain in knowledge post-treatment

compared to the pre-treatment knowledge test. Since there was an almost equal percentage of

knowledge gain in both the gains and loss frames of the infographic, it cannot be concluded that

one was more effective than the other by a large margin. However, the loss-framed infographic did

not seem to effectively influence participants to increase willingness to take action on any of the

survey suggestions. Survey results showed that the willingness to take action dropped by a small

margin after treatment. Overall, results reveal that the participants seemed to gain more knowledgeand motivation to take action from the gains-framed infographic as opposed to the loss-framed

infographic.

Survey Critiques

As the tests were administered to the participants, it was clear that there was confusion among

them in regards to Melissa Clarks survey: primarily Question #2. Many of the participants voiced

their confusion while taking the survey, and many others wrote notes about how they decided to

answer the questions in a different manner.

After administering the tests, our group reviewed the survey again to analyze how the questions

wording was indeed misleading and confusing.

Question #2 of the survey included statements such as prescribed fire is too uncontrollable to be a

forest management tool, and instead of the options being ranked on an agree to strongly

disagree scale, participants were asked to rate the options on a scale of extremely important to

not important at all. If the scale for this question had been more closely related to the individual


15/19

statements, it would have reduced confusion and made the survey a more useful tool to gain

opinions.

In addition to the confusion about this question, some participants overlooked that there wasactually a back side to the survey, and simply did not complete it. For future studies, it might be

beneficial for the test administrator to inform participants prior to taking these tests that the surveyconsists of a double-sided copy.

Limitations of Study

While this test collected a large amount of information, not everything that would be informative

to the development of a large-scale study could be extracted from the data we collected. Even with

our group performing the test on a larger group of participants than was required, the information

we collected only gave results for the information we sought after: to determine if the infographics

were an effective way of conveying knowledge to participants, if the infographics were effective to

change participants views regarding wildfires, and whether a loss-frame or gains-framed

infographic was overall more effective to perform either of these tasks.These tests did not inform us of how the infographics could be improved to increase concern about

wildfires, how to encourage participants to donate money to fire management organizations, how

to help participants feel stronger emotions of loss or gain through the framing of the infographics.

We also did not test a large enough sample of the local population to have only a small margin of

error in our tests.

Recommendations for Melissa Clark

Some aspects of the test require some modification, primarily the survey. Question #2 needs the

evaluation criteria to be modified into reasonable answers to the statements. Test administrators

will need to inform participants about the back page of the survey, or ensure that all surveys given

are printed on one-sided paper.


16/19

Appendix A

Gains-based Infographic Loss-based Infographic


17/19

Appendix B

Pretest Interview Participant Number .

1. Gender .2. Major .

3. Hometown . State .4. Do you currently live in a rural area, or an urban area? Circle one: Rural/Urban5. Political affiliation/preference .6. Have you had to deal with a wildfire before? Explain:

.

.

.


18/19

Appendix C

Knowledge Quiz Participant Number .

1. Approximately, how many wildfires were there in the Northern Rockies (Idaho/Montana)in 2011?

a. 100b. 1000c. 2000d. 10,000

2. What is the projected worth of Idahos forests?a. $13 millionb. $152 millionc. $1 billiond. $3 billion

3. Since the creation of the Forest Service in the early 1900s has wildfire frequency...a. decreased

b. increasedc. remained the samed. is obsolete because wildfire is no longer a problem

4. Since the creation of the Forest Service in the early 1900s has wildfire severity...a. decreasedb. increasedc. remained the samed. is obsolete because wildfire is no longer a problem

5. Prescribed burns, initiated by the Forest Service...a. greatly reduced fire hazardb. reduced wildfire severity

c. consume possible fuel sources (debris, vegetation, etc.)d. all of the above6. What is a measure that homeowners can take to combat wildfire?

a. install fire retardant sidingb. clear roof surface of fire fuelc. keep the yard clear of leavesd. all of the above


19/19

Appendix D

(survey added as PDF in Microsoft Word edited version of this white paper)

In Design Group 3 White Paper

Documents

Transcript of In Design Group 3 White Paper