DOCUMENT RESUME ED 327 898 AUTHOR Arkin, Elaine Bratic … · 2014-03-24 · DOCUMENT RESUME. CS...

ED 327 898

AUTHORTITLE

INSTITUTIONREPORT NO-PUB DATENOTEAVAILABLE FROM

PUB TYPE

EDRS PRICEDESCRIPTORS

IDENTIFIERS

ABSTRACT

DOCUMENT RESUME

CS 507 377

Arkin, Elaine BraticMaking health Communication Programs Work. APlanner's Guide.National Cancer Inst. (NIH), Bethesda, Md.NIB-89-1493Apr 89120p.

Rose Mary Romano, Office of Cancer Communications,National Cancer Institute, Building 31, Room 4B43,Bethesda, MD 20892.Books (010)

MF01/PC05 Plus Postage.*Cancer; *Health Education; *Health Programs; HealthPromotion; Health Services; Instructional Materials;Professional Services; Program Design; *ProgramDevelopment; Pesource Materials; SpeechCommunication*Health Communication; National Cancer Institute

This manual, designed to assist professionals inhealth and health-related agencies, offers guidance for planning ahealth communication program about cancer based on social marketingand other principles as well as the experi9nces of National CancerInstitv.te staff and other practitioners. The six chapters arearranged by sequentially crdered stages of program development: (1)Planning and Strategy Selection; (2) Selecting Channels andMaterials; (3) Developing Materials and Pretesting; (4) ImplementingYour Program; (5) Assessing Effectiveness; and (6) Feedback to RefineProgram. Each chapter contains selected readings for more completeinformation about specific Lubjects as well as materials (checklistsand planning questions) for duplication and use. The appendixesinclude a glossary and other sources of planning and health-relatedinformation including sample forms and questionnaires. (KEH)

Reproductions supplied by EDRS are the best that can be madefrom the original document.

AYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAY AYAYAYAY AYAYAY AYAYAYAY AYAYAYAY AYAYAY AYAYAYAY AYAYAYAYAYAYAY AYAYAYAYAYAYAY AYAYAYAY AYAYAYAYAYAYAY AYAYAYAY AYAYAY AYAYAYAY AYAYAY AYAYAVAYAYAYAYAYAYAYAY AYAYAYAY AYAYAY AYAYAYAY AYAYAYAY AnYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAY AYAYAYAY AYAYAY AYAYAYAY AYAYAYAY AYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYIAY AYAYAYAY AYAYAYAYAYAYAY AYAYAYAY AYAY'AYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYA,YAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAYAvAYAYAYAYA,YAYAYAYALAGNelAYAYAYAYAYAYAYAYAYAYAYA"lAYAYAY-AYAYAYAYAYAYAli

Preface IN

kknowied.vmsatv

How to Use This look all

Introduction 1

le Role of Communication in Disease Prevention and Control 1

ilelevant Health Education, Mass Communication and Social Marketing Theories, Models, and Practices 1

What Evaluation IsAnd How It Fits Into Communication Programs 3All Health Communication Programs Are Not Alike 3

The Health Communications Process: Overview 5

$tage 1: Planning and Strategy Selection 9Where Do You Start?Review Available DataIdentify Existing Activities and Gaps 9Write Goals and Objectives 10Gather New Data 10Determine Your Target Audiences 11

Establiai Audience Tracking System 12Assess Resources 12Draft Communication Strategies 13Write Program Plan and Timetable 14Selected Readings 14

Program Plan: Outline 15Cancer Communication Program Planning Guide 16

Stage 2: Selecting Channels and Materials 21

Identify Messages and Materials 21Choose Channels 22

How to Reprint Camera-Ready Materials 23Characteristics of Mass Media Channels 24The Role of Public Service Media Campaigns 25

Selected Readings 2$Selecting Channels that Work 26How to Select an Advertising Agency 27

Stage 3: Dovoloping listorials and Probating 31Develop and Test Message Concepts 31

How the Public Perceives Health Messages 32Considerations for Message Construction 33

Develop Draft Materials 34Using Celebrity Spokespersons 35

PretestingWhat It Can and Cannot Do 36Tips for Developing TV PSAs 36Make Print Materials Easier to Read 37Producing Materials for Special Audiences 38Examples of What Pretesting Can Do 39

Pretesting Methods 39Self-Administered Questionnaires 39Central Location Intercept Interviews 40Theater Testing 42Focus Group Interviews 42Readability Testing 44

Readability Scores of Selected Magazine Articles 44Gatekeeper Review 45

Determine What and How Much to Test 45Plan and Conduct Pretests 46

Designing the Questionnaire 46Recruiting Respondents 46

Pretest Methods: Summary 47ApplicaWity of Pretesting Methods 48

Identifying Interviewers 49Facilities 49Getting Help 49Summary 50

Estimated Costs of Pretesting, 1988 50Selected Readings 51

Excuses for Avoiding Pretesting 51

Stage 4: Implementing Your Program 55Prepare to Introduce Program 55The Importance of Tracking Progress 55Establish Process Evaluation Measures 56Work With Intermediaries 56

Steps for Involving Intermediaries in Your Program 57Types of Organizations and ActMties to Consider in Health Communication Programs 58

Consider Working with Business 59Review and Revise Program Components 59Selected Readings 60

Stage 5: Assessing Effectiveness 63Outcome Evaluation 63Impact Studies 63Determine What Evaluation to Do 63

Types of Evaluation 63Elements of an Evaluation Design 65

Evaluation Options Based on Available Resources 65Examples of Program Assessment Questions 66

Selected Readings 66

Stag* 6: Feedback to Refin Program 69Apply What You Have Learned 69Revise Program 69

Write an Evaluation Report 69Share What You Have Learned 70Selected Readings 71

AppendicesA. Information Sources 75B. How to Test fcr Readability 77C. Sample Forms

Educational Materials Review Forms 83PrePost Booklet Testing Form (Self-Administered) 87Central Location Intercept Questionnaire 89Focus Group Moderator's Guide 95Gatekeeper/Professional Review Questionnaire 97Standard PSA Pretest Questions 99

D. How to Design a "Theater" Style lest for PSAs 101

E Glossary 125F. Bibliography 127 -

G. Comment Form 131 ;.)

communication plays anessential role in diseaseprevention and health

promotion. Programs designedto promote changes in healthbehaviors and to encourageearly detection and prompttreatment of illness havedemonstrated that mass mediaand other communicationstrategies can be effective inreducing the risk of seriousillness.

Communicating effectivelyabout health is a difficult task.Health information is oftencomplex and technical. Inaddition, the information may beinconclusive, controversial,contradictory and subject tochange as new researchfindings are released. Manydiseases such as cancer arefear-arousing; individualresponses may be emotional.New health information mayconflict with long-held personalbeliefs. As a result, the potentialexists for misdirecting oralienating the public.

Careful planning and de-velopment of health communi-cation programs are important toavoid these undesirable effects,and to assure that communica-tion activifies have the greatestpotential for success.

For more than a decade theOffice of Cancer Communica-tions (OCC), National CancerInstitute, has been developingcommunication messages andi.,..ograms for health profession-als, patients, and the public. Wehave learned a great deal abouthow to develop effective com-munications. Over the years wehave received many requests forassistance in planning programsthis guide is largely directedto the questions we have beenasked.

The purpose of this manual isto learn from and share ourexperiences, and those of othersthose who are faced withplanning health communicationprograms. This book discussessome key principles relative tospecific steps in program devel-opment, and includes examplesof their use. Sources of addition-al information on each subjectare included at the end of thechapters. A glossary, a bibH-ography, and other resourcescan be found in the appendices.

6

This guide expands upon andreplaces "Pretesting in HealthCommunicr-tions." Informationabout pretesting is incorporatedinto this book; in addition, wehope that this new text will helpexplain the factors beyond ma-terials pretesting that contributeto a successful health communi-cation program.

To request additional copiesof the guide please contact:

Rose Mary RomanoOffice of Cancer

CommunicationsNational Cancer Institute,Building 31, Room 4B43Bethesda, MD 20892

III

4, ,

%

4

Two groups of advisorswere convened by theOffice of Cancer Commu-

nications to help design thisguide. These advisors and manyothers reviewed the book indraft. For their invaluableassistance, we would like tothank:

z

--,..

Charles DarbyProspect Associates

Rene DurazzoStanford Center for Disease Prevention

Vicki Freimuth, Ph.D.University of Maryland

Kathleen Jamieson, Ph.D.University of Texas

William D. Novel liPorter/Novelli

James W Swinenart, Ph.D.Public Communication ResourceE

Robert W. DennistonOffice for Substance Abuse Prevention

Brian R. Flay, D. PhiLUniversity of Illinois

Rachel GreenbergProspect Associates

William MorriscrtNational Institutes of Health

Douglas ShiffletD.K. Shifflet and Associates, Ltd.

Lawrence Wallack, Dr. RH.University of California, Berkeley

Derry AllenEnvironmental Protection Agency

Ann Fisher, Ph.D.Environmental Protection Agency

Susan MaloneyOffice of Disease Prevention andHealth Promotion

William Smith, Ed.D.Academy for Educational Development

Terry BellichaNational Heart, Lung, and BloodInstitute

Paul KnottAmerican Red Cross

Richard Windsor, Ph.D., M.RH.University of Alabama at Birmingham

We would especially like tothank Elaine Bratic Arkin, whowas responsible for writing thisbook.

v

This book is designed tohelp professionals inhealth and health-related

agencies communicate with thepublic. It offers guidance forplanning a health communica-tion program based on socialmarketing and other principlesand the experiences of NationalCancer Institute staff and otherpractitioners. It Li designed tohelp you identify what to do andwhyand where to go forassistance.

An overview of each step inprogram development is in-cluded in the section entitled"The Health CommunicationsProcess" (pages 5-6). Thissection has been designed sothat you can duplicate and shareit, use it to substantiate yourprogram plans, or to increasethe awareness of your own staff.

Each chapter subheading listswhat is covered withinto helpyou select those sections thatrelate to a specific task you face.Checklists and planning ques-tions are included to help youshape your tasks, and theselected readings at the end ofeach chapter direct you to morecomplete information aboutspecific subjects. The Appen-dices include a glossary andother sources of planning andhealth-related information.

Samp!e forms, questionnaires,and a theater test methodologyare also included to help you indeveloping your own pretests.

We request suggestions youmay have for future editions ofthis guide; a comment form ison the last page of this guide.We welcome your comments.

vil

in 1980 theDepartment of Healthand Human Servicespublished theObjectives for theNation: 1990. Thesehealth objectives,compiled by

groups of experts addressing 15priority areas, projected animprovement in our national healthstatus that could be reached by 1990 ifcurrent knowledge were to be applied.Better use of existing healthknowledge requires communicationamong health care and social serviceprofessionals, related organizations,government agencies, the privatesector, and indMdual citizens. In fact,communication between leaders inhealth, education, industry, labor,community organizations and otherswas cited as essential in reaching the1990 objectives. Healthcommunication programs can bedesigned to inform, influence andmotivate institutional or publicaudiences.

Communication can:

0 increase awareness of a healthissue, problem or solution

0 affect attitudes to create support forindividual or collective action

o demonstrate or illustrat skills

0 increase demand for health services

o remind about or reinforceknowledge, attitudes or behavior.

Health communication programscannot:

0 compensate for a lack of health careservices

0 produce behavior change withoutsupportive program components

0 be equally effective in addressing a//issues or relaying all messages.

Therefore, communication should beincluded as one component in pro-grams designed to address a healthproblem.

The National Cancer Institute hasincorporated aspects of various com-munication models, theories andpractices into its planning process.Each discipline offers a differentperspective on the consumer and thesteps leading to behavior change. Forexample, social marketing practiceconsiders the perceptions and per-ceived needs of the target audiencesas an essential element of planning.Health education models involve anexploration of the components ofbehavioral intention that will influencean individual's willingness to act. Masscommunication theories help explainfactors that influence message trans-mission between the source and thetarget audience and the expectedeffects. Summarized below are someof the theories, models and practicesthat have been applied in developingthis guide.

Social MarkMingSocial marketing models, first articu-lated by Philip 1<otler and based oncommercial marketing practices, showthat the consumer (target.audience)

10

should be the central focus forplanning and conducting a program.The program's components focus onthe:

o pricewhat the consumer mustgive up in order to receive theprogram's benefits. These "costs"may be intangible (eg., changes inbeliefs or habits) or tangible (e.g.,money, time or travel)

o productwhat the program is tryingto change within the target audience

0 promotionhow the exchange iscommunicated (eg., appeals used)

0 placewhat channels the programuses to reach the target audience(e.g., mass media, community,interpersonal)

The formulation of price, product,promotion and place evolves from re-search with the consumer to deter-mine what benefits and "costs" theywould consider acceptable, and howthey might be reached. Lessonslearned from social marketing stressthe importance of understanding thetarget a .dience and designing strate-gies based on their wants and needsrather than what good health practicedirects that they "should" do.

Behavioral IntentionsStudies of behavioral intentions sug-gest that the likelihood of the targetaudience adopting a desired behaviorcan be predicted by assessing (andsubsequently trying to change orinfluence) their attitudes toward andperceptions of benefits of the behavior,along with how they think that theirpeers will view their behavior. Re-search by Fishbein and Ajzen supportthe idea that an individual's, andsociety's (perceived) attitudes are animportant predecessor to action.Therefore, an important step towardinfluencing behavior is a preliminaryassessment of target audience atti-tudes, and subsequent tracking toidentify any attitudinal changes.

Communications for PersuasionWilliam McGuire has described thesteps an individual must be persuadedto pass through in order to assimilate adesired behavior. These steps are:

0 exposure to the message

0 attention to the message

0 interest in or personal relevance ofthe message

0 understanding of the message

0 personalizing the behavior to fitone's life

0 accepting the change

0 remembering the message andcontinuing to agree with it

0 being able to think of it

0 making decisions based on bringingthe message to mind

0 behaving as decided

0 receiving (positive) reinforcement forbehavior

0 accepting the behavior into one's life

To communicate the message suc-cessfully, five communication compo-nents all must work:

0 the credibility of the messagesource

0 the message design

0 the delivery channel

0 the target audience

0 the targeted behavior

Attention to McGuire's considera-tions helps assure that the communi-cation program plan addresses allfactors that determine whether amessage is ;eceived and absorbed,aod that the program is staged overtime to address audience needs asthey differ over time while progressingtoward behavior change.

2

Diffusion of InnovationsThe health policy makers call it "tech-nology transfer;" Everett Rogersdescribes the process whereby newproducts or ideas are introduced or"diffused" to an audience. Whetherthe message is accepted (or thebehavior adopted) depends uponwhether the audience:

0 perceives it as beneficial

0 sees it as in accordance with theirneeds and values

0 finds it easy or difficult to under-stand or adopt

0 can try the behavior

0 feels that the results of the trial oracceptance are viewed positively bytheir peers.

Rogers suggests that the massmedia are a quick and effective routefor introducing new information ortrying to influence attitudes, especiallyin the early stages of reaching audi-ences predisposed toward acceptingnew ideas. However, at the point oftrial, or "adoption," interpersonalchannels are more influential. Thismeans that a communications strategymight consist of using the mass mediato introduce the message, provideknowledge, influence attitudes, andreinforce behavior, and using commu-nity or interpersonal intervention toteach and encourage the adoption ofthe behavior. This is especiallyimportant for groups known not toadopt new behaviors quickly

PRECEDE ModelLawrence Green developed thePRECEDE model, an approach toplanning that examines the factorswhich contribute to behavior change.These include:

0 predisposing factorsthe indi-vidual's knowledge, attitudes,behavior, beliefs, and values prior tointervention that affect their willing-ness to change

0 enabling factorsthe structure ofthe environment or community andan individual's situation that facilitateor present obstacles to change

0 reinforcing factorsthe positive ornegative effects of adopting thebehavior (including social support)that influence continuing thebehavior.

These factors require that theindividual be considered in the contextof their community and social struc-tures, and not in isolation, whenplanning communication or healtheducation strategies.

Understanding these conceptsrelated to health communicationplanning can help assure a successfulprogram. A more thorough review maybe valuable for planning a particularprogram. Sources of more informationinclude:

r ,hbein, Martin and Ajzen, I , Belief,Attitude, Intention and Behavior. Reading.MA: Addison-Wesley, 1975.

Green, Lawrence W., Kreuter, Marshall W.Deeds, Sigrid, G., and Patridge, Kay B.,Health Education Planning. A DiagnosticApproach, Palo Alto. Mayfield PublishingCompany, 1980.

Kotler, Philip and Andreasen, Alan R..Strategic Marketing for NonprofitOrganizations, 3rd Edition, EnglewoodCliffs, NJ: Prentice-Ha 1987.

Kotler, Philip, Ferrell, OC., and Lamb,Charles. Strategic Marketing i'or NonprofitOrganizations; Cases and Readings.Englewood Cliffs, NJ: Prentice-Hall. 1987

McGuire, William. "TheoreticalFoundations of Campaigns,- in Rice. R.E ,and Paisley, W.J. (Eds.), PublicCommunication Campaigns, Beverly HillsSage Publications, 1981, pp. 41-70.

McOuail, Denis and Wendahl, Sven,Communications Models for the Study ofMass Communications, New York:Longman, Inc , 1981.

Rogers, Everett. M., Diffusion ofInnovations, New York: Free Press. 1983

Additional sources are included inAppendix F (Bibliography)

While it Is true that there are spe-clailsts who use sophisticated tech-niques to evaluate programs, it is alsotrue that evaluation is a naturalprocess. We all take actions that areassessed by consciously or uncon-sciously reviewing the available facts,considering them in the light of theoriginal intent, and drawing aconclusiJn.

For example, you might find that thenews media rarely report your news asyou think they should. A close look atthe situationthe content of yournews releases, how and when they arereleased, and reactions of reportersreceiving themmight identify andlead to solving the problem. Thepurpose of any evaluation is to learnfrom actions so that improvementscan be made.

*easy

MomAGossiglimil

V.* tiftwor boaWar *oo Pao*risf. Olisilm. b 0 Ow*now 0 000.111111.MY b. V.I01.0WWI. pi 0.11101.00.10 eV 0011

0.1 OM OWN*Sap PP 10401impb. in 4.0. Too'501. 01,05or y401.110CPNIpomp 0110 IMAMilis/ MINIM SY. Me 0/1MI MOM MOMPt 0 40.100 OWNpyre 1. p.m...,,,,, ..., ..., A pew # wow* MIN fampilill a 0OP.0100 S. NM 001411001011. 00 Mt MI 0. Mem,. pp% Yypi IM.Iv V? 0011011 01000P11.04Amp asp ,....,r 0 Ow SW 01. 010106411.10.00WW1 .0 rt.* .100.0..... CPO 101.4 aley..a. du 01 Not 4. a 0.01100 tb WPM/ PO ~IW V ON parys .....100 ppm 010011.100.1.1000.11.00111.1 00.111,40.0.00a Prow Ir.0Ore 0 Pp INN VP 180.0 POO4501 TO 5. Pe pa 00.111101.11100104.114110 000 00 MOWsyq my. am PM V WWI. '. WI"......., pc @PPM .5151011: 14.001. 414.10.10..6 ....550100541. re 1.00. Vim..01400.y. I. 'Arm avourftre frown oareweird..* 0. I The ....P WOO. ay. 110.0 Ponprpor Merb Map aluill V NI re ray/ 400 p

0.00.00 pa mem "be

ligAllsd

Everyone reaches conclusionsabout the relative success or failure ofprograms and actMties. Formal eval-uation helps assure that those con-clusions are based on objective data.

Formal evaluation takes the naturalprocess and makes it a conscious,orderly effort, using objective tech-niques for gathering and analyzingdata and reaching conclusions in orderto:

o improve current and future efforts

o certify the degree of change thathas occurred

0 identify programs, or elements otprograms, that are not working.

Evaluation is one of many toolsavailable to help communicationprogram planners and other decision .makers do their jobs well. However, itis important to recognize that there aremany forms and types of evaluation,from the very informal and simple tothe very formal and complex.

Evaluation is not a task that istacPed onto the end of a program.Assessment and careful planning areinterdependent, integral functions ofprogram development and implemen-tation. Just as each step of a programcontributes to its effect, each step canbe subjected to evaluation. Evenbefore program development begins,evaluative discipline demands that thedesired program outcome be described as specifically as possibleOnce set, these goals and objectivesdirect how each aspect of the programwill be develooed. In this guide, evalua-tion strategies are incorporated intoeach stage of program development.In addition, a more complete discus-sion of program evaluation is includedin Stage 5: Assessing Effectiveness.

1 2

Some examples cited in this workbookere large-ocale nationwide programs;others were developed for a state orcommunity. Most of the planning stepsand considerations apply regardless ofgeographic span. Similarly, somehealth problems are more complexand controversial than others. Theymay irwolve a select or many diversetaiyet audiences; simple information orcomplex behavior change; one ormany channels; generous budgetsover many years, or almost no budget.

This guide describes what we havefound to be a practical scheme forplanning and implementing healthcommunication programs. However.you may encounter a situation thatdoes not permit or require each stepoutlined here. We hope you willconsider each issue and step includedand make a conscious decisionregarding whether it applies to yoursituation. Clearly, there are no hard-and-fast rules; consider the followingchapters as they are intendedassuggested guidelines.

3

Afundamental premise of thisworkbook is that to be viable,health communication programs

must be based on an understanding ofthe needs and perceptions of theirtarget audiences. The diagram belowillustrates an approach to healthcommunication incorporating assess-meras of target audience needs andperceptions at critical points inprogram development and implemen-tation. The six stages constitute acircular process, in which the laststags feeds back to the first in acontinuous process of planning andimprovement.

Each of the six stages is sum-marized in this chapter to provide youwith an overview 0 this approach,then in more detail in the followingchapters. The steps outlined belowconstitute an ideal process, one thatmay require more time and moneythan many agencies can afford. All ofthe steps may not be feasible, or insome cases even essential. However,carefully following the steps in eachstage of the process can make the

next program phase more productiveIn general, however, you must applyyour profewional judgment to decidewhich steps &re appropriate for yourparticular program.

%

The planning stage of a programprovides the foundation for the entirehealth communication process. Facaltydecision making at this point can leadto the development of a program thatis "off the mark!' Careful assessmentof a problem in the beginning canreduce the need for costly midcoursecorrections.

Key Issues

What is already known about thehealth problem? (Analyze existingdata.)

Figure 1

Nam in Health Communication

- Stages when pretesting is used.,II.s....

I 3

What new kinds of information willbe needed before planning theprogram? (Generate new data ifneeded.)

Who is the target audience? What isknown about them?

Owerall, what change is planned tosave or lessen the problem? (goals)

What measurable objectives can beestablished to define success?

How can progress be measured?(Plan evaluation strategies.)

What should the target audience betold? (Draft communicationstrategies.)

The decisions you make in stage 1 willguide you in selecting the appropriatecommunication channeks) and pro-ducing effective materials. Withoutclear objectives and a knowledge ofyour target audience, you risk pro-ducing .naterials which are inappro-priate for the target audience or theissue being addressed.

Key lawsAre there any existing materialswhich could be adapted for theprogram?

Which channels are most appro-priate for reaching the targetaudience? (e.g., worksite, massmedia, face-to-face)

What materials formats will best suitthe channels and the messages?(e.g., booklets, videotapes, curricula)

5

In stages 1 and 2 most programplanning is completed; this planningprovides the basis fer developingmessages and materials. Often severaldifferent concepts will be developedand tested with target audiences.Feedback from the intended audienceis critical in stage 3.

Key team

What are the different ways that themessage can be presented?

How does the target audience reactto the message concept(s)?

Does the audience:Understand the messa.,?Recall it?Accept its importance?Agree with the value of the

solution?

I How does the audience respond tothe message format?

Based on responses from the targetaudience, do changes need to bemade in the message or its format?

How could the message be pro-moted, the materials distributed, andprogress tracked?

I

The fully developed program isintroduced to the target audience;promotion and distribution beginthrough all ctannels. Program com-ponents are periodically reviewed andrevised if necessary. Audienceexposure and reaction are tracked topermit alterations if needed.

Key Imes

Is the message making it throughthe intended channels of communi-cation?

Is the target audience paying atten-tion, and reacting?

Do any existing channels need to bereplaced, or new channels added?

Which aspects of the program arehaving the strongest effect?

Do changes need to be made toimpiove program effect?

The program should be assessed byanalyzing the results of measurementsplanned in stage 1 and used through-out the program's lifespan.

Key Issues

Were the program objectives met?

Were the changes which took placethe result of the program, otherfactors or a combination of both?

How well was each stage of pro-gram planning, implementation andassessment handled?

JrIP1

At each stage useful information wasgathered about the auolence, the mes-sage, the channels of communicationand the program's intended effect. ANof this information helps prepare for anew cycle of program development.The more information that can bereviewed at the end of the firstprogram phase, the more likely it isthat these questions can be answered:

Why did the program work, or notwork?

Are there program changes or im-provements that should be made toincrease the likelihood .-...f success orto address change in the audience,or problem or other situations?

Are there lessons learr,ed that couldhelp make future programs moresuccessful?

ik,r5c

.cct"Y

t

V'fic

, cs0

0a)Q.)

,11,k,/,4-:',,,,z(-11`..1.,,,,,'',1v,*.-:"+

;-;$=

7",1,7;q,?<

,'

-355.44,55`,4t.?-":°.,itr-

.5,:

c

,,,,,,,, ..:c4 ,

,,*

.,,,

.,..,,,

,'5',1,,

'',

cccf ,

4 -.c.",

c*Cc*cr

715

(7)

01,1

C15

qr cy).s

a)

c=

400C

C5

;,,

,^k

epea

14.ccn

1..

The first purpose of planning is todetermine whether the problemcan be addressed through com-

munication. If it can be, the planningstage provides the foundation for theentire health communication process.Flawed decision making at this pointcan lead to the development of acommunication program that is "offthe mark." Careful assessment of aproblem in the beginning can reducethe need for costly midcoursecorrections.

Where Do You Start?When faced with developing a newcommunication program, it may seemas though you must do everything atonce. To organize your thoughts aboutplanning, ask yourself questions likethese:

a What is the health problem to beaddressed?

a Who is affected by it, and hole

a Are they aware that the problemcould affect them?

a Who is interested in the problem?

0 What activities have beenaddressing the problem?

a Are the media or other organizationsdoing anything?

a What can we say or do to helpameliorate the problem?

a To whom?

0 What do we want to accomplish?

a What resources are available'?

This chapter is designed to help youfind the answers to these questionsand design a program plan.

Review Available Data(Secondary Research)The more you understand about anissue or health problem the better youcan target corrective measures. Checkall potential sources of information inyour agency; identify gaps in what you

have, and seek outside sources of in-formation. The types of information youshould (ideally) have to plan yourcommunications program include:

a a description of the problema incidence0 effects on indMdual and

communitya causes and preventive measures0 solutions, treatments or remedies

a a description of who is affected(potential target audiences)0 age, sex, ethnicity, places of work

and residence0 causative/preventive behaviors0 related knowledge, attitudes and

behaviors0 patterns of use of health-related

services0 media preferences am! habits0 information sources that are

considered credible by potentialtarget audiences

The purpose of this data collectionis to describe the health problem orissue, who is affected, and what theyknow, believe, and do. This investiga-tion should also help identify relatedactivities.

Sources of information will, ofcourse, vary by issue and by wheth^ryour program will relate to a specificcommunity or broader geographicarea. Examples of data sourcesinclude:

a library searches

a sources of health statistics (a localhospital, a state health department,the National Center for HealthStatistics)

....,

411.-

I G

0 government agencies, universities,voluntary and health professionalorganizations

0 clearinghouses

0 advertising agencies, newspapers,radio and television stations (formedia use data, buying and con-sumption patterns)

a community service agencies (forrelated service use data)

0 corporations (e.g., General Mills),trade associations and foundations

a polling companies (for audienceknowledge and attitudes)

a depositories of polling information(e.g., the Roper Center, University ofConnecticut)

0 local Chambers of Commerce

Both published and unpublishedreports may be available from thesesources.

A number of Federal health infor-mation clearinghouses provide infor-mation, products, materials andsources of further assistance forspecific health subjects. A helpful firststep in planning may be to contact theappropriate clearinghouse and yourhealth department to obtain informa-tion on the health issue. A clearing-house referral source and otherpotential sources of information areincluded in appendix A.

Identify Existing Activitiesand GapsIf another organization is already ad-dressing the problem, you may want tocontact them to discuss

m what they have learned

a what information or advice they mayhave to help you plan

la what else is needed (what gapsexist)

a opportunities for cooperativeventures.

9

Write Goals and ObjectivesThe gcals and objectives establish theshape of the programwhat it is de-signed to accomplish. The programgoal or goals describe the overallchange (e.g., a specific improvement inone aspect of the health of a certainpopulation). Reaching the goal mayencompass service delivery, financialand societal support and othersituational changes tha, can only bepartially addressed by communicationstrategies. The Office of DiseasePrevention and Health Promotion'sPromoting Health/Prevoting Disease(the 1990 Health Objectives for theNation) can help you set goals forsome health issues.

Objectives describe the inter-mediate steps that must be reached toaccomplish the broader goals; theydescribe the desired outcome, but notthe steps involved in obtaining it (you'lldesign strategies for getting therelater). Some of these objectives maybe beyond the scope of a communica-tion program; this guide only ad-dresses translating the communicationobjectives into action.

Objectives are written to articulatewhat your program is intended to do.Therefore, communication objectivesshould be:

I specific

s attainable

prioritized to direct the allocation ofresources

O measurable to assess progresstowards the goal

O time specific

These objectives are the foundationfor program development and evalua-tion. If they are not clear and "action-able," your program may be unfocusedand ineffective. Once written, goalsand objectives serve as a kind ofprogram "contract," or agreementregarding program purpose.

10

For example:

Gosh

:To Inc** 010 Percent lhenumbeiofp0001tWithOntrolled

high WOO** bY 1991

To detectarktrefat, to.treatment

ihdivkJuèJ eleiatedbloOd pressure.

For this objective, a communicationsactivity might include a campaign totell middle-aged blanks (or anotherdesignated high risk group) where togo for screening, and motivate them togo. If screening services are notconveniently available, a critical firststep would be to identify whether andwhere screening, referral and patientfollowup seMces could be madeavailable.

Another example:

OakTo reduce by 5 percent the

nuniber of parents of young

children who smoke by 1991.

OWstlw'To Increase by x percent the

number of pediatricians who

counsel their patients parents

about how to quit smoking

by 1991.

For this objective, it would benecessary to measure how manypediatricians currently counsel theseparents, and establish a numericalobjective based on your resources andrealistic expectations for change.Strategies might include providingpediatricians with the latest infor-mation about the effects of parentalsmoking on their children, designingcounseling guidelines for pediatricians,and encouraging parents to ask theirchildren's pediatrician for help inquitting.

17

Goals and objectives should berealistic. You don't want your programto be considered a "failure" becauseyou set unrealistic expectations. Forexample, it is generally impossible toachieve a goal of 100 percent. Anepidemiologist or statistician may helpyou determine recent rates of changerelated to your issue so that you havesome guidance for determining whatadditional changes could be affected.

Gather New Data(Primary Research)You may find that there is not enoughknown about the health problem, itsresolution, or those who are affectedto develop a communication strategy.If the health problem cannot be de-fined, or there is no way to ameliorateit, you may decide wisely that a com-munication program is an inappro-priate response until more informationor appropriate actions becomeavailable.

If it appears that there is no actionan individual can take because theissue is beyond individual control (e.g.,the regulation of the use of toxicsubstances), not subject to collectiveaction, services are not available (e.g.,for a high risk, low income population),or there is no treatment, you mustdecide whether a communication pro-gram should be developed. In somecases, you may need to make surethat other program elements are inplace prior to developing a commu-nication program. (However, in thiscase, you could decide to developcommunication strategies directed topolicymakers to seek their help indefining and addressing the problem.)

More often, you can define theproblem and who is affected, butinformation about that population maybe unavailable or outdated. At thispoint, you may decide to gather newdata before planning a communicationprogram. This investigation can involvea probability sample of respondentswith results which can be projected tothe whole target audienceas well as

its subparts(quantitative research),or fewer representatives typical of thegroup to be reached but with a lessstringently designed sample (qualita-tive research). Or it may fall some-where in betweenusing a combina-tion of research methods (eg., focusgroups along with a small-scale tele-phone survey). Agencies frequentlyrely on qualitative research because itis faster and less expensive

Quantitative research such assurveys with large, statisticallyrepresen'ative samples usually usestructured questions administeredthrough:

O personal interviews (at home orintercepted at another location)

telephone

mail

Quantitative research may also takeother forms such as collecting vitalstatistics data. This process canestimate what percentage of apopulation is aware of the issue, orthinks or behaves in a certain way.These data provide:

o a valuable baseline for trackingchanges as a result of yourcommunication intervention

12 information for setting priorities (e.g.,according to which groups of peopleseem to lack awareness, or aremost likely to behave in an unhealthymanner)

o information about segments within atarget population who are more inneed of attention.

Usually, quantitative data will notreveal why individuals think or act asthey do; also, because information isavailable only in response to specificquestions asked, unexpected factorsor influences may not be identified.

Qualitative investigation can providethe exploration into the "why."Examples of qualitative studiesinclude:

focused group discussions ("focusgroups")

4,

0 a small number of "open-ended" orin-depth interviews.

These and other methods arediscussed in Stage 3. These methodscan provide:

information about target audienceperceptions

identification by the target audienceof issues they perceive as relatedand important

0 in-depth discussion of what theparticular health issue means to thetarget audience.

Although qualitative research canhelp identify and explore issues,because of the small numbers ofrespondents and the lack of stringentsampling, the findings cannot beprojected to a population as a whole.

Determine Your TargetAudiencesSpecifically describing the audience(or audiences) for your programwhoyou want to reach and influence withyour messageswill help you developrelevant messages and materials andidentify the channels most likely toreach them. Few messages are appro-

18

7"4

priate for everyone included in the"general public:' given the diverseinterests, needs, concerns andpriorities among different segments ofthe public. Trying to reach everyonewith one message or strategy maydilute your message so that it appealsto few rather than many people.

Doscribing Your Target AudloncoTry to think about a// of the physical,demographic (and, perhaps, psycho-graphic) characteristics of the peopleyou are trying to reach to help divide"the public" into more manageablegroups or target audiences.

These characteristics include:o physicalsex, age, type and degree

of exposure to health risks, medicalcondition, disorders and illnesses,health history of family

o behavioralmedia exposure,membership in organizations, health-related activities or actions andother lifestyle characteristics

ei demographicoccupation, income,educational attainment, familysituation, places of residence andwork, cultural characteristics

11

psychographicattitudes, opinions,beliefs, values, selfappraisal andother personality traits

The more complete a "profile" ordescription you can develop of youraudience, the better you will beprepared to develop a program suitedto them.

"kgmenting" Thrget AudimcOnce you describe what you knowabout who you want to reach, youshould be able to "segment" orseparate your target audience from therest of the general population. You mayfind that you want to reach severaldistinct population groups.

Primary target audiences are thoseyou want to affect in some way; youmay have several primary targetaudiences. If so, you should setpriorities among them to help orderyour planning and allocate yourresources. Secondary targetaudiences are those with influence onthe primary audience or those whomust do something in order to helpcause the change in the primary targetaudience.

The process of identifying anddefining audiences should lead tosetting audience priorities, that is:

0 deciding which one (or several)audience segments is mostimportant

12

,

O deciding which audiences areimportant but less critical becauseof their health risk/health status,influence or link to the primary targetaudience(s) or limited programresources

O deciding who will not be a targetaudience for the program. This deci-sion provides valuable programdirection for decisions regardingmessage development and dissemi-nation, helping assure that all pro-gram resources are spentproductively.

Establish Audience 'RackingSystemIf you are designing a major communi-cation program that will stretch over along period of time, building audiencetracking into your program plans willassure that you can:

0 find out what your target audienceknows, thinks and does before youbegin activities (baseline data)

periodically survey to assess pro-gress and the need for modificationor new activities

identify the change in status amongthe target audience when your pro-gram is completed.

All too frequently, audience surveysonly are undertaken during and afterthe program, or are inappropriately

timed to occur too far after the pro-gram completion, or are sporadic, orincompatible and results cannot becompared. To avoid these problems,consider whether audience tracking isappropriate for your program at thisearly planning stage.

Assess ResourcesIt is important that you assess yourresources to determine what and howmuch you realistically will be able toaccomplish. Setting realistic expecta-tions can help you avoid the frustrationof not accomplishing as much asanticipated. To set realistic objectives,think about these questions:

1. What are the greatest areas ofneed?

2. Which activities will contribute themost to answering these needs?

3. What resources are available?Include:ci staff and other "people" re-

sourcessuch as committeemembers, associates from otherprograms, and volunteers

o budgetfunds and "in kind"resources such as computertime, mailing costs, printingservices available from anothersource, educational materialsfree or at cost

o informationabout the issue, thetarget audience, the communityand media structures, or aboutavailable educational materialstimethat is, how many weeks,months or years are available tocomplete the program.

4 What community activities, organi-zations or other contributing factors

<1st?

. What barriers (such as approvalobstacles, absence of funding,hard-to-reach target audience) arethere?

6. Which activities would best utilizethe resources you have identifiedand best fit within the identifiedconstraints?

Those activities identified in question 6should become your priorities.

Coping with LIMN! Romulus"Resources" includes a lot more thanfunding, as demonstrated in question 3above. Sometimes you may feel soconstrained by a lank of funds thatdeveloping a program appears impos-sible. An honest assessment may leadto the conclusion that a productiveprogram is not possible; on the otherhand, sufficient intangible resources(that is, other than "hard funding")may be available to proceed.

Remember: adequate funding alonewon't guarantee program success. Inaddition to careful program devel-opment, you'll need the cooperationand help of your own associates and,perhaps, other organizations. That'swhy questions 5 and 6 above areimportant to consider as you decidewhether and how to develop a newhealth communication program.

Draft CommunicationStrategiesNow that you have defined what needsto be done (goals and objectives) withwhom (target audiences), it la time todesign communication strategies mostlikely to get you there. The strategystatement begins with:

the program objectives

the primary and secondaryaudiences,

and adds:

the target information to becommunicated

benefit, as perceived by audience.

The strategy statement provides allprogram staffincluding writers andcreative staffwith the same directionfor developing all messages andmaterials. It also may contain thetactics that will be used to reach targetaudiences with the appropriatemessages.

The audience benefitwhat theywill gain that they perceive as im-portant or valuablemay be differentfrom your percepons of what thebenefit is (e.g., ir 'proved health)andmay be different )or each audienceidentified. Stage 3 discusses focusgroups and other means to helpidentify what the target audienceperceives as important.

The strategy statement also shoulddescribe why the audience would wantthe benefit, to help direct creativedevelopment.

Developing the strategy statementprovides a good test of whether youhave enough information to begindeveloping messages. You may betempted to skip this step, but thestrategy statement forms a foundationand the boundaries for all creativedevelopment. Seeking agency (andperhaps, community) approval of thestrategies at this early stage can makethem feel informed and ease theapprovals and cooperation you mayneed later.

Once you have decided on the com-munication strategies, all programelements should be compatible withthese strategiesthat means everyprogram task should contribute to theestablished objectives, and be tar-geted to the identified audiences; allmessages and materials shouldincorporate the benefits and otherinformation in the strategy statement.

As you develop the program andlearn more about the audience andtheir perceptions, you may need toalter or refine the strategy statement.However, it only should be changed toreflect improved information that willstrengthen your capability to reach theprogram's goals. It should not bealtered to accommodate a great ideathat is "off strategy."

13

Write Program Plan andTimetableAll of the elements of your planningshould be recorded in a programplanyour first program product. Theprogram plan is your "blueprint." lishould be used to:

design all program tasks

explain your plans within youragency and with cthers

provide a record of where youbegan.

A sample outline for a program plan isincluded on page 15.

You will note flat the program planoutline includes your evaluation plans.Because evaluation occurs at manypointsbefore, during and after theprogramdescriptions of evaluativestrategies are dispersed throughoutthis workbook, within the stages wherethe evaluation would occur. Formativeevaluation is discussed in stages 1and 3, process evaluation in stage 4and summative evaluation in stage 5.A summary of the various kinds ofevaluation can be found on page 64(stage 5).

Program TimetableA final planning step is producing atime schedule for program devel-opment and implementation. Theschedule should include every taskyou can think of from the time youwrite the plan until the time you intendto complete the program. The moretasks you build into the timetable now,the more likely you will remember toassign the work, and keep onschedule If you forget importantintermediate steps, your costs andtiming might change. The timetablecould look li's this:

14

Goal

Olgoctive 1:

Planning Tasks Person Responsible Due Date Resources Required

Implementing Tasks Person Responsible Due Date Resources Required

Evaluation Tasks Person Responsible Due Date Resources Requited

Objectiva 2: etc.

Of course, nothing ever goes quiteaccording to plans! The timetableshould be considered a flexiblemanagement tool. You may want toupdate it regularly (e.g., once a month)so that it can function dually tomanage and track progress.

Selected Readings

Alcohol, Drug Abuse, and Mental HealthAdministration, Communicating with YouthAbout Alcohol: Methods, Messages andMaterials, DHHS Publication No. (ADM)86-1429, 1986.

Chisnall, Peter M., Marketing Research,New York: McGraw-Hill, 1986.

Funkhouser, Judith E., "Before theCameras Turn: The Research Base of theYouth Alcohol Prevention Campaign,"Alcohol Health and Research World,Summer 1987, Vol. II, No. 4.

General Mills, American Family Report19784979: Family Health in an Era ofStress, Minneapolis, MN: General Mills,1979.

General Mills, A Summary Report on U.S.Consumer Knowledge, Attitudes andPractices About Nutrition, Minneapolis,MN: General Mills, 1979.

Getting the Word Out, WisconsinClearinghouse, 1954 East WashingtonAvenue, Madison, WI, 53704-5291, 1979.

National Cancer Institute, CancerPrevention Awareness Survey (TechnicalReport, WAVE II), NIH Publication No.87-2907, July 1987.

National Digestive Disease EducationProgram, A National Survey of PublicPerceptions of Digestive Health andDisease, 1984, contact Dr. Brent D. Ruben,Dept. of Communication, Rutgers, NewBrunswick, NJ 08903.

Office of Disease Prevention and HealthPromotion, Promoting Health/PreventingDisease, Department of Health andHuman Services, Washington, DC: 1980.(to be revised)

Desorintion of Need: (why the prograir is fring

itiiieurees Required: ( ff, art shop ordevefoped)

computer tiMe)

Primary Target Audienoei:. (In priority order)(Include age, gander, ethnic group and other

pertinent characteristics)

Potential Pniakomm (scheduling conflicts,clearances, policies and approvals that you and other

Staff Must address)

Key Strategies: (list for each target audience,

Methods of Rvaluation: (include formative,process and summative (outcome) evaluation strategies)

Secondary Target Audienoes: (in priority order) _

22

15

Mseesimit1. State program goal

2. a) Whatare the modifiablerisk (adore

b) What risk faCtors/issues arethe highest priority forprogram development?

3. a) What are the behaviorsassociated with the identifiedrisk factors/issues?

b) Whiah behaviors are highpriority for intervention?

4. What audiences should betargeted?

S. What are the key factorsinfluencing the behaviors in thetarget audience? (KAB, skills,availability of resources,professional attitudes)

6. Develop and state programobjectives.

Agency mission statement

Role of communications

literature searches

Epidemiologic studies

Consultation with experts

Agency policies

Literature searches

Epidemiologic studies


Agency policies

Review of morbidity/mortalitystatistics.

Review of knowledge/attitudes/behavior (KAB) surveys

Agency policies and politics

Review of other Federal programs

Consultation with experts (advisorygroups, working groups)

Secondary data tracking

Literature reviews


Focus groups (audience members,professionals)

Psychographic surveys

NCI surveys


Literature reviews


Focus groups (audience members,professionals)

Psychographic surveys

NCI surveys


Prevention:To reduce the incidence of cancerthrough information and educationprograms that promote publicadoption of risk-reducing behaviors.

Smoking

Diet

Prevention:Smoking; fat and fiber intake

adults

family caretakers

low-SES blacks

Hispanics

Cancer Prevention In low-SES blacksfear and avoidance of cancer

(attitude)medIcal care is not effective (belief)smoking cessation, food selection

skills

Smokingincrease the number of health

professionals who counsel theirpatients to quit smoking

increase the number of national/regional organizations that sponsortobacco education programs

le

23

Pregram00$1eparivi:._7. Whitarethe beet channels to

osetaiettch the targetaudience(media,:worksites,nationatand Community'voluntary organizations,churches, the medical sector)

I. a) What is the best program/products to use in the selec-ted channel with the targetaudience? (Define messages,and straterm develop prod-ucts to support them.)

b) Are the selected productsclear and effective to theintermediary? To the targetaudience?

c) Are program/products likelyto produce the desiredoutcome with the inter-mediary? Target audience

C. How will the program/product bedistributed/promoted?

10. Develop an action plan withstaff and resource allocations,timetables, expected outcomes.

lbooklegPreeees liveleadsa11. a) Is the product reaching the

intermediary?

b) Is the intermediary using theproduct with the targetaudience?

c) What percentage of thetarget audience is beingreached?

d) Is the product producing thedesired outcome with tha tar-get audience (e.g., calls, at-tendance at presentations)?

12. Assess results of tracking andevaluations. What changes areindicated?

lieectirc*Oin:dfrrosi.eis

Analysis of the reach and influence ofthe channel on the target audiencethrough::

literature reviewsrevieW of other programstargeting the audience

'focus groupsconsultation with experts

Assessment of resources/roie (staff,budget)Agency policies and politics

a) Resources

Past successes

Concept testing

Focus groups


b) PretestsReadability tests

c) Review of evaluations of similarprograms/productsPilot tests

Analysis of various intermediarynetworksReview oi past NCI programsReview of other comparableprograms

See page 14.

Standard tracking procedures

Consultation with evaluationstaff/experts


Secondary data tacking


Evaluation and tracking studiesAnecdotal and observationalinformation

Benicitis

Blacksblack newspapers and magazinesblack national organizations with

regional and local affIggtesblack colleges and churches

The Good News media campaign

The "Quit for Good" program forphysicians

Partners in Prevention Network

Placing staff in policy-settingpositions

Mall-intercept PSA tests

Focus group poster tests

Pilot test of pharmacy-based smokingcessation program

Pilot test of modular TV nutritionProgram

Mailed announcements of "Quitfor Good" kit using professionalsocietirss' mailing listsNewsletter announcementsTV PSA campaign

See pdge 15.

TV PSAs

a) bounceback cards analysissurveys of stations

b) BAR public service monitoring data

c) BAR data/Inquiry data

d) Analysis of inquiry data

Review might indicate the need forimproved promotion, revised products,selection of different intermediaries

17

24

,4

4,

,^"; ", ,

ith investigation and analysiscompleted and your plan inhand, you are ready to devel-

op the communication tools. It is vitalto have the right tools for the job. Instage 2 you'll decide what kinds ofmaterials fit your budget and yourtarget audience, and which channelswill work the best. Choosing mes-sages, formats and channels areinterdependent stepseach must suitthe other, and each decision must fitthe strategy statement.

Idedify Messages andModalsMaterials production can be a timeconsuming and costly process.Because it is creative and has tangibleresults, it is frequently seen as the keydevelopmr. ntal step for a communica-tion program. However, you shouldconsider whether creating newmaterials is really necessary.

11111411/telle.

-"713

;VWI

lbu may have discovered existingcommunication materials (booklets,leaflets, posters, public serviceannouncements, videotapes) whilegathering data to plan your program. Ifnot, look now. Sources include:

health Jepartments (in your state orother states)

university or public libraries

voluntary organizations

health professional associations

community-based health promotioncoalitions

the ODPHP National HealthInformation Center (see page 75).

If you find materials related to yourhealth issue, decide whether theYmight be appropriate for your program"as is" or witn modification. Considerthese issues in conjunction with yourstrategy statemero-

Do they offer accurate, completeand relevant messages?

Are they appropriate for your targetaudience in format, style and read-ability level?

Are they available and affordable?

Could they be modified to becomeaPPropriate?

If you are considering the use ofexisting materials, you will want todiscuss with the producer:

how the messages were developed

whether the materials were tested

how they have been used

were they effective

b if you could receive permission tomodify or use parts of the materials(especially if materials are r.:opy-righted).

Sample forms developed by OSAPare included in appendix C to help youconduct your own materials review.

NW may want to test promisingmaterials with the target audience atthis point (see stage 3 for a descriptionof testing methods). If the materialsprove to be inappropriate, you will havevaluable information for developingnew materials.

Discussions about what materialsformat (e.g., print or audiovisual) willbest suit your program will bedetermined by:

26

the message (e.g., its complexitysensitNity style, purpose)

the audience (e.g., will they want toread about the subject, or wJuldthey rather watch a videotape)

the channels (e.g., whether you willbe most likely to reach the audiencethrough a school, library, physician.the media, or a combination c:these). and the formats thf ..thannelsprefer and

your budget and other availableresourcel,.

Communication materials willprobably represent a majct expendi-ture for your program. Make sure thatyou choose a format that you canafford in sufficient quantities to makeyour program work, and that you don't

",

allot so much of your budget to materi-als that you can't afford distribution(e.g., mailing lists, mailing services,postage costs), promotion and processevaluation.

Chan ChannelsThe decision regarding the channelsyour program will use is interdepend-ent with the decision about materialsformat. Message delivery channelsinclude:

face-to-face (e.g., health care profes-sional to patient, peers, familymembers)

group delivery (e.g., worksite orclassroom)

organizational (e.g., constituents of aprofessional or voluntary association)

mass media (e.g., radio, television,magazines, direct mail, billboards,transit cards, newspapers)

community (e.g., libraries, employ-ers, schools, malls, health fairs, localgovernment agencies1

a comNnarion of any or all of these(this choice is likely to work best).

Each channel offers different benefitsand may require different messagedesign (althovgh not necessarilydifferent messages) to fit the channelin length and format.

The mass media can transmit newsquickly to a broad audience, but can-not alone be expected to motivatepeople to change their behavior. Themass media are generally the public'sprimary source of information out maybe less trusted than more intimatesources of information, and arecons*.rained try time, space, andnewsworthiness, among other factors,to the extent they can explain complexinformation properly and fully. Media(news) may focus too much attentionon new information or informationaffecting limited segments of thepopulation; may increase the chancesfor miscommunication of complex orcontroversial news; may communicateincomplet: information (most crucially,leaving out information explaining what

22

should be done about a healthproblem). Wss of control over how theinformation is communicated may be atrade-off for broad and rapidtransmission.

Whether mass media is intentionallyselected as one channel, or whether ahealth issue appears as news, youshould remember that the purpose ofthe mass media is to inform and enter-tain, not educate. Therefore, if thecomplete message is too complicated,or simply not considered interestingenough for use by the media, you willbe obligated to redesign the messageso that it is more appealing to mediaprofessionals (media "gatekeepers")and their perceptions about what theiraudience wants. Working with mediaprofessionals will help assure thatmessages are interesting as well asaccurate, and may help you obtaingreater exposure for your program.

Mass media offers many opportu-nities for messages beyond publicservice announcements, includingmentions in news programs, enter-tainment programming, public affairs,"magazine" and interview shows(including radio audience callins), liveremote broadcasts, editorials (tele-vision, radio, newspapers, magazines)and health and political columns innewspapers and magazines. Eachformat offers a particular advantage for

2 e"

communicating messages, and eachformat may reach a different audiencemix. You may decide to include avariety of formats and media channels,always choosing hom among thosemost likely to reach your targetaudiences.

Interpersonal channels put healthmessages in a more familiar context.These channels are more likely to betrusted and influential (physicians,friends and family members of thetarget audiences are usually verycredible sources). Developingmessaoes, materials and links intointerpersonal channe's may requiretime-consuming and costly develop-ment. Influence through interpersonalcontacts requires familiarity with themessage and, therefore, may moreappropriately follow long-term expo-sure to and acceptance of mediamessages.

Community channels (includingschools, employers and communityorganizations) can reinforce andexpand upon media messages, andoffer instruction. Establishing links withcommunity institutions and organiza-tions can shortcut the development ofinterpersonal routes of influence withthe target audience. Interpersonal andcommunity channels can offer support

28

for action and are two-way, allowingdiscussion and clarification, encour-aging motivation, and reinforcingaction.

Using several different channels willincrease the repetition of the informa-tion, increasing the chance thi.t theaudience will be exposed to the infor-mation a sufficient number of times toabsorb and remember it. Channelselection should be determined prior tcmaterials production since messageformat will be different for variouschannels.

Consider these questions as youmake decisions about channels:

Which channels are most appro-priate for the health problem/issueand message?

Which channels are most likely to becredible to and accessible by thetarget audience(s)?

Which channels fit the programpurpose (e.g., inform, influenceattitudes, change behavior)?

Which and how many channels arefeasible, considering your timeschedule and budget?

You may choose a different mix ofchannels for each audience to matchwhat you have learned about whatinformation sources the audienceconsiders credible and how they aremost likely to be reacheo.

23

24

29

,

Selected ReadingsBurrelle's Media Directories, Burrelle'sSpecial Groups Media Directory, Update(updated regularly), Livingston, NJ:Eurrelle's Media Directories.

Cable TV Publicity Outlets, Cable TVPublicity Outlets Nationwide,Washington, Depot, CT Cable TV PublicityOutlets, updated regularly.

Gannett Newspapers, Media MarketGuide, New York: Gannett Newspapers,updated regularly.

Papazian, E., The Media Book, New York:The Media Book, 1980.

Roman, Kenneth, and Maas, Jane, How toAdvertise, St. Martin's Press, 175 FifthAvenue, New York, NY 10010, 1976.

Simmons Market Research Bureau,Simmons 1985 Study of Media andMarkets, Media Series, Vols. 1-13, NawYork: Simmons Market Research Bureau,Inc., 1985.

Storer Communications, BroadcastingYearbook, Hollywood and New York:Broadcasting Publications Inc., updatedyearly. -Traub, James, "The World According toNielson: Who Watches Television andWhr Channels, January/February 1985,p. 26.

4 -, 30

25

.,1'",,,WSktt.:A6'Z'V'l 'ki" `"1:?:

.14,,A."" ::41 , ,",,i ,,, ,,,

-,,,,,,A,-.., ,,, :-..; , . fI ..,..

.7,

.',,,...-, , ' ft

-,- '.7,-- --- ,0. 6 -,- .

0,0- - -..- . , - --- i '4' ' -' i ' 1 `4, \,","' ..' ,'1.,-1, 0 , A, - 1 ' ..,, .

i1 , ' ^,:, `'''TAAAA;VA.,-; -''''' , 'A ,':::',/,;;A`A'A'`%-"' ' ,4;04A',i:':`-'-i ''Z' ',,,&';',"'i-A.I'R ;' :' ,:; A.,A;7")A-4' r -,:;"'*::'1.''':

-. '''

,'

40.4.'t-' '4-, -A4

34,

OW41.?

A

te - A

; A

en, ? ,

r',"ete ";"'0, "

1100;-. 4, -,-er 4-

,

`. *;;,; %, *

/4.',""; `"

'63e

'A*,,a

A`;', 71, , 'A, '2,-","

''7"").N1.4t;t.`4- 1,-

tn31.431,f.'i':g,4L.

o

Develop end Test Message ConceptsMdiovisOat MaterialsPrinfyitérialSUsing Celebrity SpokespersonsHoW the POPO, Perceives Health MessagesConsiderationSfor Message Construction

a DeveIà Draft' MaterialsTipelor.DevelOping TV PSAsMake Print Materials Easier to Readprockicing Materialt for Special Audiences

Pretestincfr.MhatitiCan and Cannot Do`04friPITSA.What:Pretesting Can Do

Preteeting:MetliodeSett "OniniStered QuestionnairesCentrailocation Intercept InterviewsTheater*ting'''FocUS PeOtipinterviewsReadabil,itffesting

-.ReadabilityScores of Selected Magazine ArticlesGatekeeper:ReviewPretesting:MethOds: Summary

Estimated pireOt Costs of Pretesting160tiiiity, of-Pretesting Methods

Determining Whatand'How Much to Test

Plan ind, COneiliict- PretestsDesign the,QueStionnaire

,'RegruitReSpOndentsldentkIntervieWers

Gifting:HelpStOhlarY:

USing PretesfReSultsExCUSeilck AVOICling Pretesting

a SeleCted Readings

3 3

y"ith program planning corn-pleted, you are ready to pro-duce messages and materials

for your program. This chapter ad-dresses:

developing message concepts

testing message concepts and draftmaterials

production tips

content issues, including messageappeal, spokespersons and consid-erations for "hard-to-reach" or otherspecial audiences.

If you have located suitable materialsfor your program, you will be able toskip or simplify this process. If you areunsure whether the materials you haveIccated are appropriateor think thatthey may need alterationsyou canmove directly to pretesting.

Develop and Test MessageConceptsYour communication strategy state-ment and the inlormation you gatheredabout the target audiences (stage 1)form the basis for developing messageconcepts. These message conceptsare your messages in "rough draft,"and represent different ways ofpresenting the information to the targetaudiences. You may want to preparetwo or more message concepts usingdifferent:

spokespersons (e.g., a physician, apeer)

appeals (e.g., humor, fear, factual)

styles (e.g., photographs, graphs)

formats (e.g., audiovisual with music,instructional poster) for testing withgatekeepers and the target audi-ence.

Audiovisual MaterialsIf you have decided to produce PSAsor other audiovisual materials, here aresome presentation options to consider:

demonstrationthe et - is u al

format is ideal for demonstrating thedesired health behavior, especially ifskills must be taught.

testimonialsa credible presenter(e.g., the Surgeon General) can lendcredibility to your message. A rec-ognizable spokesperson may beattention-getting. The most credibleand relevant presenter may vary fordifferent audiencesand may be atarget audience representative, anauthority (e.g., a physician) or acelebrity connected with the healthissue.

slice of lifea dramatization withinan "everyday"or familiarsettingmay help the audience associate withyour message. A simple story may beeasy to remember; you might chooseto present the health problem andshow the solution. This style may beboth credible and memorable, but itmay also be "corny."

animationbeyond use for youngchildren, animation can be eye-catching for adults (older children mayconsider some animation "silly" or"babyish"). You can use animation todemonstrate desired actions; it mayalso be a good choice for addressingabstract subjects (e.g., explaining"respite care") or sensitive subjects(such as AIDS), or several disparatetarget audiences (e.g., different ethnicgroups) at once.

humorcan be memorable, heart-warming and effective. However,humor is difficult to do well. Thelighthearted can also be silly, and apunchline can become stale quickly.For some audiences or subjects,humor can be offensive. ("There isnothing funny about cancer," agreedone men's focus group.) It can also beexpensive.

emotioncan make a messagereal, and personal; it can also "turnoff" the ,?,wer. Emotional approachesrange from warm and caring to feararousing and disturbing. As withhumor, emaional appeals may be"high risk" production choicesthesechoices should be pretested andproduced with care.

use of musiccan lend to a moodyou are trying to creaie it can alsocompete with the message.

31

q -0 LI

r`; e 1, li

Print MaterialsYou may decide to produce a singlebooklet, or your program and budgetmay call for a range of materials, eachfor a different purpose (e.g., a posterfor attention, a booklet for explanationor teaching) or use (in a supermarketdisplay, a phl Acian's office or a class-room).

As you decide what print materialsto produce, consider that:

all messages in all the media youchouse should reinforce each otherand follow the communicationsstrategy

a no matter how creative, compellingor wonderful a message is, if it doesnot fit the strategy statement, objec-tives and identified audiences, throwit out. Don't compete for attentionwith your own campaign

whatever approach or style you havechosen should be echoed in alicampaign elements. In print ma-terials, use the same or compatiblecolors, types of illustrations andtypefaces throughout the campaign.If there is a logo or theme, use it inall print and audiovisual materials

33

use illustrations to gain attention, aidunderstanding and recall; make surethey reinforce, and don't competewith your message. Use captions,headlines and summary statementsfor additional reinforcement.

As you consider a particular tone orstyle for either audiovisual or printmaterials, make sure that you haveaccess to the expertise and budget tosuit your choice. Think about whetherthe style will enhance or compete withyour message. Simplicity may offer thegreatest chance of success.

Production WoosNo matter which approach youchoose, high quality production isnecessary to make the message work.If you feel you have to skimp on pro-duction, choose a simpler way ofpresenting the message. Producingpoor quality materials wastes fundsand can damage your program'sandyour owncredibility.

flood for Audionco TestingEven if you think you've chosen thepresentation style most suited to themessage and audience. you shouldpretest it to be sure. Check eachconcept to make sure that it complieswith your communications strategyand objectives. Testing alternativeconcepts with the target audiencemay:

help identify which has the strongestappeal and potential for effect

identify new concepts

identify confusing terms or concepts

identify language used by the targetaudience

help eliminate weaker concepts andsave production costs.

34

Focus groups are most commonlyused for testing at this stage becausethey permit open and extended discus-sion about concepts and ideas. Youmay show rough illustrations or a tele-vision "storyboard" (frame-by-frameillustrated description) to the group, orjust discuss the messages and presen-tation style with them and ask them tovisualin the product. A description offocus groups and alternative testmethods begins on page 39.

Develop Draft MaterialsBased on findings from testing mes-sage concepts, you will want to refinethe most promising approach andproduce materials in draft or "rough"form. A graphic designer or audio-visual producer will frequently providea facsimile version of a poster orpamphlet, or a storyboard of atelevision PSA for your review andapproval. Materials inor close tothis format should be tested at thisstage to:

assess comprehension

identify strong and weak points

a determine personal relevance

gauge confusing, sensitive orcontroversial elements.

37

Testing at the concept stage helps tochoose the strongest from amongmore than one potential approach topresenting your message. Now, youhave refined the strongest conceptinto the language, style and format youwant to use. Additional testing is de-signed to strengthen the approach youhave chosenbefore your productionfunds are spent.

Because materials production isone of the most costly steps in pro-gram development, it makes sense totest before you invest in final produc-tion. Draft materials should resemblethe final product as closely aspossible, but without expensive pro-duction. For example, radio announce-ents may be produced in a nonst0Iosetting and with nonprofessionalvoices for testing. A teievision an-ouncement may be tested wi:h a story-board, or with an animatic (videotapedsketches with voice over). A bookletmay be p "epared using good qualityword processing copy, rough illus-trations, and a copy machine. Similarly,posters, print ads and flyers can beproduced in rough form for pretesting.

Pretest graphics with the targetaudience. People interpret graphics indifferent ways, just as they do thewritten word. If your graphics style orillustrations depart from what"gatekeepers" (e.g., PSA directors,physicians, teachers) expect, test withthem, or ask them to review, as well.Use favorable responses from thetarget audience as a "selling point"with gatekeepers.

38

35

ProtestingWhat it Can andCannot DoPretesting draft materials is a type offormative evaluation used to helpensure that communications materialswill work. Pretesting is used to answerquestions about whether materials are:

understandable

relevant

attention-getting and memorable

a attractive

credible

acceptable to the target audience.

These are factors that can make thedifference in whether materials work ordon't work with a particular group: theyalso involve value judgments on thepart of thd respondents and yourinterpretation of what they mean. Mostpretesting involves a few personschosen as representative of intendedtarget audiences, and not a statistical-ly valid sample (in number or selectionmethod). That is, pretesting is general-ly considered "qualitative research"research which can be interpretedsomewhat loosely to provide cluesabout audience acceptance anddirection regarding materials pro-duction and use

I

...

.,

c-)

30 3 9

[

4 0

A variety of procedures may beused to test messages and materialsThe best methods for a particularprogram depend upon the nature ofthe materials, the target audience andthe amount of time and resourcesavailable for pretesting. There is noformula for selecting a pretestmethodology, nor is there a "perfect"method for pretesting. Methods shouldbe selected and shaped to fit eachpretesting requirement, consideringthe objectives of and resourcesavailable for each project.

Included here are descriptions ofsome frequently used methods ofpretesting health concepts, messagesand materials. In addition, samplequestionnaires or other forms thathave been developed for some ofthese methods are included inappendix C, for you to adapt. Eachmethod carnes with it benefits andlimitations. Sometimes using severe:methods in combination will helpovercome the limitations of inds,dualprocedures For example, focus groupinterviews may be used to identifyissues and concerns relative to aparticular audience, followed byindividual interviews to discussidentified concerns in greater deptnReadability testing should be used asa first step in pretesting draft manu-scripts, followed by individual questionnaires or interviews regarding materi-als with target audience respondentsCentral location interviews or theatertesting of messages for television orradio permit contact with largernumbers of target audience respond-entsespecially useful prior to finalproduction of materials Following thedescriptions of pretesting methods is adiscussion of how to choose the mostsuitable method for a particularsituation

37

Given the qualitative nature of mostpretesting research, it is important torecognize its fimitations:

Pretesting cannot absolutely predictor guarantee learning, persuasion,behavior change or other measuresof communication effectiveness.

Pretesting in health communicationis seldom designed to quantitativelymeasure small differences amonglarge samples; it '3 not statisticallypreci 3e. It will not reveal that bookletA is 2.5 percent better than bookletB. (Presumably, pretests of suchprecision could be applied, but thecost of obtaining such data would behigh, and the findings may be nomore useful than the diagnosticinformation frm more affordableapproaches.)

Pretesting is not a substitute forexperienced judgment. Rather, it canprovide additional information fromwhich you can make sounddecisions.

It is important to avoid misuse ofpretest results. Perhaps the mostcommon error is to overgeneralize.Qualitative, diagnostic pretest methodsshould not be used to estimate broad-scale results. If 5 of the 10 respond-ents in a focus group interview do notunderstand portions of a pamphlet, itdoes lot necessarily mean that 50percent of the total target populationwill be confused. The lack of under-standing among those pretest re-spondents suggests, however, that thepamphlet may need to be revised toimprove comprehension. In sum,pretesting is indicative, not predictive.

Another problem that arises inhealth communication pretestingconcerns interpretation of respondentreactions to a sensitive or emotionalsubject such as breast cancer orAIDS. Respondents may becomeunusually rational when reacting tosuch pretest materials, and cover uptheir true concerns, feelings andbehavior. As a result, the pretestermust examine and interpret responsescarefully.

384 1

IN

42

Pretesting offers both the opportu-nity and the temptation to structure thetest and interpret the results to supportor justify a preconceived point of view.R is natural to want your favorite con-cepts or messages to test well, butthere is no need to test unless you arewilling to consider the results ob-jectively.

One final point: pretesting does notguarantee success. Good planningand sound pretesting can be negatedby mistakes in final production. Themessage in a television ?SA on cancertreatment, for instance, may pretestwell, but then be flawed by an execu-tion that uses an actress who seemstoo happy to be awaiting the results ofa biopsy report. Similarly, leaflet copythat pretests well may be renderedineffective by a poor layout, hard-to-read type, and inappropriateillustrations.

Pretesting MethodsThe most frequently used pretestingmethods are described below. Theseinclude:

self-administered questionnaires

central location intercept interviews

theater testing

focus group interviews

readability testing

gatekeeper review.

Following these descriptions there isa summary chart on page 47 to helpyou compare the advantages and dis-advantages of each method.

1. kitadministered QuestionnairesSelf-administered questionnaires:

enable program planners to elicitdetailed information from respond-ents who may not be accessible forpersonal interviews (e.g., doctors,teachers or residents of rural areas)

allow respondents to maintain theiranonymity and reconsider theirresponses

do not require interviewer time andcan be done relatively inexpensively

n

can be answered by many respond-ents at once

require time to locate respondentsand secure their cooperation

require follow up to increase re-sponse rates if mailed.

Selfadministered questionnairesalso can be:

mailed to respondents along with thepretest materials

distributed to respondents gatheredat a central location

used where personal interviews arenot feasible

an inexpensive pretesting techniquefor agencies with minimal resources.

A self-administered questionnaireshould be designed (see sample inappendix C), then pilot tested with 5 to10 respondents. Usually, question-naires and pretest materials are dis-tributed to respondents whose partici-pation is sought in advance, but theyalso may be mailed to potentialrespondents without advance noti-ficatior Respondents are asked toreview the materials on their own, tocomplete the questionnaire and thento return it within a specified time.

The questionnaire should be rela-tively short and clear or respondentsmay not complete it. Clear, conciseinstructions to the respondent areimportant because there is nointerviewer to offer clarification. Open-ended questions may be used toassess comprehension and overallreactions to materials and close-endedquestions to assess such factors aspersonal relevance and believability ofthe material. Measures of attention orrecall may not be reliable when usedwith this technique since respondentsmay refer back to the material.

Resources are invested primarily inquestionnaire development and anal-ysis of results. The latter expense canbe kept to a minimum by using manycloseended questions.

40

Ilk

Self-administered questionnaireshave certain disadvantages:

The primary problem with thistechnique is the possibility of a lowresponse rate.

I It is important to over-recruit re-spondents and recontact respond-ents to encourage them to returntheir questionnaires to ensure asufficient number of returns.

The data collection may take longerthan with other methods (e.g.,central location intercept interviews)because of delays in responses,especially if the questionnaires aremailed.

The type of respondents who returnthe questionnaires may be differentfrom those who do not respond, andthis approach cannot be used withrespondents who have reading andwriting limitations. Hence, a certaindegree of bias may be introducedand results should be interpretedwith t-iis in mind. (Contacting thosewho did not respond by telephonewill permit a comparison ofrespondent/nonrespondentanswers.)

1 ,;

2. Central Location Intarcopt InterviawsCentral location intercept interviews in-volve stationing interviewers at a pointfrequented by individuals from thetarget audience and asking them toparticipate in the pretest. There aretwo advr:otages to this:

a high traffic area (e.g., a shoppingmall, hospital waiting area or schoolyard) can yield a number of inter-views in a reasonably short ti,re

a central location for hard-to-reac1target audiences can be a cost-effective means of gathering data.

A typical central location interviewbegins with the intercept. Potentialrespondents are stopped and askedwhether they will participate. Then,specific screening questions are askedto see whether they fit the criteria ofthe target audience. If so, they aretaken to the interviewing stationaquiet spot at a shopping mall or other

siteare shown the pretest materials,and asked questions. The questionsmay help assess:

comprehension

individual reaction

personal relevance

credibility

recall (if test situation includesexposure to the materials prior to theinterview).

Although the respondents inter-cepted through central locationinterviews may not be statisticallyrepresentative of the target population,the sample is usually larger than thoseused in focus groups or individual in-depth interviews.

Unlike focus groups or in-depthinterviews, the questionnaire used incentral location intercept pretesting ishighly structured and contains pri-marily multiple choice or close-endedquestions to permit quick response.Open-ended questions, which allow"free flowing" answers, should be keptto a minimum because they take toomuch time for the respondent toanswer and for the interviewer torecord responses. The questionnaire,as in any type of research, should bepilot tested before it is used in thefield. A sampla questionnairP isincluded in appendix C.

A number of market researchcompanies throughout the countryconduct central location interceptinterviews in shopping malls. Clinicwaiting rooms, churches, SocialSecurity offices, schools, worksites orother locations frequented by individu-als representative of the target audi-ence also can be used for thispurpose. It is advisable to obtainclearances or permission to set upinterviewing stations in these locationswell in advance.

Posters can be tested in the kindof setting (e.g., a clinic waiting room orschoolroom) where they will be used.Posters should be mounted on a wallalong with other materialsjust asthey are expected to be usedwhere

the target audience passes, gathers orwaits. Selecting respondents fromamong those who have been "ex-posed" to the poster in its "naturalsetting" prior to the interview, and thenmoving to a nearby but separatelocation to ask questions will permit anassessment of factors such as com-prehension and personal relevance,and also whether:

the material attracts attention

the iespondent can recall thematerial when exposed to it in a"natural" setting

The major aaVantage of the centrallocation intercept approach is its cost-effectiveness for interviewing largenumbers of respondents in a shortamount of time. Because theseinterviews are intended to provideguidance ("qualitative" information),the size of the sample should only belarge enough to give you answers toyour Pretest questions. If you haveinterviewed 50 respondents and mostof them feel similarly about yourmaterials, you are probably ready tostop. If, however, there are substantialdisagreements or differences betweenrespondents, or their responses haveraised new questions, additional

4,1

interviews should be conducted untilyou are satisfied that you have cleardirection from the respondents. Youmay decide to revise (and perhaps testagain) after fewer interviews if it isclear that changes are needed.

Designing a central location inter-cept pretest can be relatively easy. Afew simple questions ("Do yousmoke?" "How old are you?" "Do youhave teenaged children?") can identifyrespondents typical of the target audi-ence quickly at the point of interceptQuestions to assess comprehensionand target audience perceptions of thepretest materials form the core of thequestionnaire (see appendix C). A fewadditional questions, tauored to thespecific item or items being tested("Do you prefer this pictureor thisone?"), also may be constructed tomeet program planners' particularneeds. The interview should be nolonger than 15-20 minutes. If it mustbe longer, you may need to designspecial incentives to convince therespondent to continue the interview(e.g., a small fee or gift, or a plearegarding the importance of thesubject and their opinions).

41

Central locatir'n intercept interviewsshould not be used if respondentsmust be interviewed in depth or onemotional or very sensitive subjects.The intercept approach also may notbe suitable if respondents are likely tobe skeptical or resistant to beinginterviewed on the spot (e.g.,commuters anxious to return home).Although it is time-consuming to set upprearranged appointments, they mayactually save time if respondents maynot be willing to cooperate in a centrallocation.

3. Theater Tnting"Theater" tests are so-called becausethey gather a large group of respond-ents to react, usuahy to audio oraudiovisual materials, into a room (or"theater"-style setting) at once.Commercial services conduct theater-style tests for advertising agencies;this technique can be adopted forhealth messages. In commercialtheater testing, approximately 300respondents are recruited by tele-phone to a central location, such as ahotel. Respondents are asked to watcha "pilot" television program to judgewhether it should be aired. Commer-cials are included in the program;some are control (constant) spots,others are being tested. At the conclu-sion of the program, respondents areasked whether they recalled anycommercials (or PSAs), and thenasked questions regarding content andpersonal relevance. A similar se-quence can be used to test radiocommercials.

Theater testing quickly gathers alarge number of responses. Unlikesome other pretest methods, thematerials being tested are imbeddedwithin a program, with commercials, tosimulate a natural viewing situation.This permits the assessment of howhkely the audience is to pay attentionto and remember the message.

Because commercial testingservices are costly, a guide to con-ducting your own theater-style tests isincluded in appendix D. You can

42

choose a settilig where the targetaudience gathers and where they canassemble in a large group (e.g., asenior citizen center, a schoolauditorium) to conduct your owntheater-style test.

4. Focus Group InterviewsFocus group interviews are a form ofqualitative research adapted by marketresearchers from group therapy. Alsocalled exploratory group sessions, theyare used to obtain insights into targetaudience perceptions, beliefs andlanguage. Focus group interviews areconducted with a group of about 8 to10 people. Using a discussion outline,a moderator keeps the session ontrack while allowing respondents totalk freely and spontaneously. As newtopics related to the outline emerge,the moderator probes further to gainuseful insights.

Focus group interviews are es-pecially useful in the concept develop-ment stage of the communicationprocess. They provide insights intotarget audience beliefs on a healthissue, alloy, program planners toexplore perceptions of messageconcepts, and help trigger the creative

45

thinking of communication profession-als. The group discussion stimulatesrespondents to talk freely, providingvaluable clues for developing materialsin the consumers' own language, andsuggestions for changes or newdirections.

In the planning stages of programdevelopment focus groups can beused to develop the hypotheses (orbroad study issues) for larger quantita-tive studies. Focus groups also canhelp determine public perceptions,misconceptions and attitudes before aquestionnaire is developed and thefield research is conducted.

Focus groups also can be used tosupplement quantitative research.Market researchers originally devel-oped this technique to explore ingreater depth the data from large scaleconsumer surveys. Obtaining in-depthinformation from individuals typical ofthe target audience can provideinsights into what the statistical datamean, or why individuals respond incertain ways.

As with all respondents, thoseselected for focus groups should betypical of the intended target audienceVarious subgroups within the targetaudience may be represented in sep-arate group discussions, especiallywhen discussing sensitive or emotion-al subjects, to segregate respondentsby age, sex, race or whatever othervariable is likely to hinder freedom ofexpression. Teenage girls are lesslikely to be inhibited in discussingsexual activity, for instance, if theirparents, or teenage boys, are not inthe group. Respondents are recruited

1 to 3 weeks in advance of the inter-view sessions, usually by telephone.They may be recruited using thetelephone directory, and interviewed byphone to determine if they qualify forthe group. Or, they may be recruitedfrom among members of a relevantorganization, place of employment orother source. Recruiting respondents"at random" is not required becausethe results from focus group researchare not intended to be statisticallyrepresentative.

There are several important criteriafor conducting effective group inter-views. Ideally, respondents should notknow the specific subject of thesessions in advance, and they shouldnot know each other. Knowing thesubject may result in respondentsformulating ideas in advance and nottalking spontaneously about the topicduring the session. Knowing otherrespondents may inhibit individualsfrom talking freely. Finally, all respond-ents should be relative "newcomers"to focus group interviews. This permitsmore spontaneity in reactions andeliminates the problem of "profession-al" respondents who may lead ormonopolize the discussion. For thesame reasons you may want toexclude health professionals andmarket researchers from focus groups

There is no firm rule about thenumber of focus groups that should beconducted. The number of groupsdepends upon program needs andresources. If target audience percep-tions appear to be comparable after afew focus groups (you'll need at leasttwo groups to make this decision), youmay not find out any more by con-vening additional sessions. If per-ceptions vary, and the direction formessage development is unclear,additional groups may be beneficial. Inthis case, reviions in the discussionouthne after several groups can helpclarify unresolved issues in theadditional groups.

4 6

An experienced, capable moderator,who can skillfully handle the groupprocess, should be used. The moder-ator does not need to be an expert inthe subject matter being discussed;rather, a good moderator builds rap-port and trust and should proberespondents without reacting to, orinfluencing, their opinions. Themoderator must be able to lead thediscussion, and not be led by thegroup. The moderator must emphasizethat there are no right or wronganswers to questions posed. A goodmoderator understands the process ofeliciting comments, keeps the dis-cussion on track, and makes it clearthat he or she is not an expert on thesubject. You will need tc -ehearse withthe moderator to point out any topicsor concerns you want emphasized, ordiscussed in more depth.

As noted earlier, the results of focusgroup interviews should be interpreiedcarefully. It is useful for an unseenobserver (e.g., behind a one-waymirror) to take notes as well as to taperecord or videotape the session forlater review. In interpreting the findingsfrom group interviews, you should lookfor trends and patterns in target audi-ehce perceptions rather than just a "hesaid she said" kind of analysis.

43

Group discussion should not beused when individual responses orquantitative information are needed.For example, when assessing the finalcopy for a booklet, it is more importantto gather individual rather than groupreactions to indicate the indMdual'sactual comprehension, perceptionsand potential use. However, self-administered questionnaires can becompleted by each participant prior tobeginning a group discussion tocombine individual and groupreactions.

5. Roadability Testing"Readability testing" simply predictsthe approximate educational level aperson must have in order to under-stand written materials. Health infor-mation materials such as pamphlets,flyers, posters and magazine articlesare designed for distinct target groups;a readability test will indicate if aprinted piece is written at a level mostof the audience can understand. As-sessing the readability of a pamphletor another printed message will notguarantee its effectiveness, and is byno means an absolute indicator ofsuccess.

Readability formulae use counts oflanguage variables such as word andsentence length. The formulae havebeen devised statistically to predictreadability scores. Generally speaking,the reading level required to under-stand a given pamphlet will be higherwhen its sentences are long and/orwhen a large number of polysyllabicwords is found within the text.

It is important to note that readabil-ity formulae measure only the structur-al difficulty (i.e., vocabulary, sentencestructure and word density) of writtentext. They do not measure otherfactors related to how "readable" acertain text is, such as sentence"flow," conceptual difficulty, organiza-tion of material, the influence of formator design of materials on comprehen-sion, accuracy or credibility. Readabili-ty tests are conducted by program

44

staff and do not include participationby the audience for whom the materi-als are being produced. Consequently,readability testing supplements butdoes not supplant the need to pretestwith the target audience.

Despite its limitations, readabilitytesting is useful because it:

is quick

is virtually without cost

provides a tangible measure

reminds the writer to choose wordsand terms carefully.

Based on a review of the advan-tages, disadvantages and predictivevalidity of 12 selected readabilityformulae, the NCI Office of Cancer

-.-, .....

4 7

Communications chose the SMOGgrading formula for testing thereadability levels of its public andpatient education materials. SMOGwas chosen because it is both simpleto use and accurate. Completeinstructions for using the SMOGreadability test to print materials areincluded in appendix B.

Health and medical subjects ofteninclude many polysyllabic words andcomplex terms; readability formulaehave not been designed to take intoaccount the special terminology usedin describing health subjects. In somecases, extensive use of multisyllabicwords known to be understandable toa particular audience (e.g., "cigarette")may lead to an unwarrantedly high

readability score. Therefore, as with allpretesting, readability test resultsshould be used as indicative and notpredictive of problems or success.

O. Gatekeeper ReviewOften, public and patient educationmaterials are routed to their intendedtarget audiences through health pro-fessionals or other intermediaries suchas organizations that can communi-cate for you to their members. Theseintermediaries act as "gatekeepers,"controlling the distribution channels forreaching your target audiences. Theirapproval or disapproval of materialsmay be a critical factor in a program'ssuccess. If they do not like a poster ora booklet, or do not believe it to becredible or scientifically accurate, itmay never reach the intended audi-ence. Also, because they may be incloser touch with the target audiencethan you are, they may provide goodadvice about whether the audiencewill accept the materials.

Although not a pretesting techniquein the strictest c;ense of the term,gatekeeper :eview of rough materialsis important and should be consideredpart of the formative evaluation pro-cess. It is not a substitute for pretest-ing materials with target audiencerepresentatives. Neither is it asubstitute for obtaining clearances orexpert review for technical accuracy;these should be completed before

pretesting is undertaken. Sometimestelling the gatekeeper that technicalexperts have reviewed the material foraccuracy may provide reassuranceand hasten gatekeeper approval.

Gatekeeper reviews may beconducted simultaneously with targetaudience pretesting so that data fromboth groups can be gathered, analyzedand synthesized to provide directionfor revising materials. A short, self-administered questionnaire may bedirected to individuals representativeof the gatekeeper population. Asample is included in appendix C.Questions may include overall reac-tions to the materials and assess-ments of whether the information isappropriate and useful.

In other cases, there may be noformal questionnaire, but rather atelephone or pth sonal conversation ormeeting held to review and commenton (or approve) materials. If there is noquestionnaire, you should consider inadvance what kind of questions youwant to ask in the meeting or interviewand whether or not you need formalapproval of the materials. A discussionwith gatekeepers (e.g , a television PSAdirector, the executive director of amedical society) at this point can alsobe used to introduce your program sndsolicit their involvement in a variety utways beyond materials development.(See Stage 4: Implementing YourProgram.)

- mr....J1111

48

Determine What and HowMuch to TestQualitative research should beconducted in the early stages ofprogram development before full fundshave been committed to materialsproduction and messages can bechanged if necessary. As noted earlier,testing can be useful at the conceptdevelopment stage, once audiencesand communication strategies havebeen determined, and prior to mes-sage development. Exploration withthe target audience at this stage, mostfrequently through focus groupdiscussions, can help determineappropriate message appeals (e.g.,fear arousing vs. factual), spokes-porson (e.g., a scientist, public officialor member of the target audience) andappropriate language (determined bylistening to the group discussion).

Testing of drafted materials prior tofinal production permits identificationof flaws prior to the expenditure offunds for final production, andespecially prior to the use of materialswith target audiences.

Completed information materials aresometimes tested prior to beginning anew phase of a preexisting program

A combination of methods can beused to assess an audience's compre-hension, the message's believability,personal relevance, acceptability andother strong and weak points. Methodsshould be selected to suit the purposeof the testing, the sensitivity of thesubject and the resources available fortesting. Adequate investigation isespecially important when developingsensitive or potentially frighteningmessages, presenting complex, newinformation or designing a newprogram. In these cases, pretestingcan reveal potential problems, butmust be carefully structured, con-ducted and analyzed.

45

Qualitative research responsescannot be considered representative ofthe public or projectable to the popu-lation as a whole. If projectable dataare required, more formal methodolo-gies should be used. However, formost retesting purposes, qualitativemethods may be more valuablebecause they provide insights intothinking and reasons for attitudes ormisunderstandings that are vital tohelp -ohne messages and materials.

When deciding when, whether andhow much you should use pretestmethods in developing your program,consider:

46

How much do you know about thetarget audience?

How much do you know about themin relation to your health problem orissue?

Is your issue or program new,controversial, sensitive or complex?

Have you conducted relatedresearch that can be applied to thistopic?

Can you afford to make a mistakewith a particular message oraudience?

Plan and Conduct PretestsThe level of effort and staff resourcesrequired will vary considerably fromone pretest to the next. Most pretest-ing is conducted with small samplesconsisting of respondents who aretypica! of the target audience and whoare easily accessible. These results,combined with your professionaljudgment, provide important directionfor improving messages and materials.

This section provides practicalsuggestions for how to plan andimplement pretests. These sug-gestions should help you reduce thetime and costs involved, whether ornot commercial research firms arehired to supply field work and tabula-tion. The cost estimates in the charton page 50 are for direct costs only;not included are staff time to providedirection or other support you wouldprovide to the firm conducting the test.In some cases, you may reduce thesecosts by conducting pretests on yourown, with the help of an expert. Somemarket researchers will tell you thatbad research is worse than no re-search, and you must use profession-als; others say that with proper in-struction, you can do some testing onyour own. Both points of view arevalid; venture on your own with care.

Designing the QuestionnaireAs in the planning stage of progr rndevelopment, a first step in planning apretest is to formulate the researchobjectives. These objectives should be

43

stated specifically to provide a clearunderstanding of what you want tolearn. Measures of attention, compre-hension, believability and personalrelevance are key. Other specificquestions to identify strengths andweaknesses in rough messages andmaterials also should be developedbased on the pretest objectives.Questions should not be asked just tosatisfy someone's curiosity.

There are several ways to keeppretesting costs down:

keep the questionnaire short and tothe point

try to use as many closeended ormultiple choice questions aspossible for easy tabulation andanalysis

try to develop codes for quantifyingresponses in advance when open-ended questions are necessary

whenever possible, borrow questionsfrom other pretesting research.

Sample questionnaires are included inappendix C as one resource.

Recruiting RespondentsIf your budget does not allow you tohire a market research firm to recruit,you can recruit respondents yourself.A small donation may encouragemembers of local church, school, civicsocial organizations to participate in apretest.

An incentive is often used to helpensure that respondents participate ina pretest. Small amounts of money($5-$30), gifts, movie passes or a freedinner may be offered as an incentiveto participants. Another way to ensuresufficient participation is to recruitmore people than are actually needed.Often respondents who agree to parti-cipate do not show up. If all partici-pants do show up, they should beincluded in the pretest, or the "extra"respondents should be informed thattoo many respondents are present,given the agreed-upon incentive,thanked, and asked to leave.

Other ways to increase participationinclude:

a scheduling the pretest at a time thatis most convenient for respondents(e.g., at lunch or after work)

choosing a safc? and convenient site

providing transpol'ation

arranging for child care during thetime of the pretest, if necessary.

Recruiting patients or their familiesmust be given special consideration.Clinics, hospitals or local HMOs canbe contacted for help, and adequateplans should be made to ensure thatthe respondents are not inconveni-enced. Human subjects' clearancemay be needed before proceedingCooperation with the medical staff anda concern for the physical andemotional status of the patient and

family (especially if the patient needs afamily member's assistance to attend)must be considered in planning thepretest.

Identifying InterviewersTrained interviewers should be usedwhenever possible. For focus groupand in depth interviews, this is essen-tial. If your agency has no experiencein focus group studies, you mightconsider hiring a good, experiencedmoderator, observing and taping thesessions and using them as training todevelop in-house skills. Local adver-tising agencies may be of assistancein identifying a good moderator.Continuing education courses ininterpersonal communication or groupinteraction also may be useful for stafftraining or identifying potentialinterviewers.

5

For conducting central locationinterviews, university and collegedepartments of marketing, communi-cations or health education might beable to provide interviewer training andstudent interviewers. Pretesting aposter or an advertisement is anexcellent "real world" project for afaculty member to adopt as a classproject. Students in these departmentsare being trained in research methods,and pretesting can give them achance to develop their skills.

FacifidesPretesting facilities should be quietand comfortable. Meeting rooms atchurches, office buildings or otherinstitutions can be used for conductingfocus group or individual in-depthinterviews. If an observation room witha one-way mirror is not available, youmay allow staff to listen by hooking upspeakers in a room nearby, or byaudiotaping or videotaping the session.

Getting HelpMany resources exist for obtainingprofessional assistance in pretesting.As mentioned in the previous section,the faculty at university departments ofmarketing, communications, healtheducation, psychology or sociologycan be helpful in designing and con-ducting pretests. Marketing researchfirms that specialize in respondentrecruitment, interviewing, tabulationand other services may have facilitiesfor conducting group sessions andother techniques. The AmericanMarketing Association's MarketingServices Guide lists suppliers andservices geographically throughout theUnited States. Also, advertising clubs(many affiliated with the AmericanAdvertising Federation), and chaptersof the Public Relations Society ofAmerica may undertake public serviceprojects at no charge to nonprofitorganizations. Other sources includethe Marketing Research Association,and the Association of Public OpinionResearchers.

49

505 3

One caution: indMduals trained incommercial testing may not becompletely aware of all the nuancesand subtleties involved in healthcommunication. They will be able todraw on their commercial experiencefor selecting the appropriate pretestmethodology. However, there are otherfactors such as the wording andinterpretation of questions and resultsthat are influbnced by the complexitiesof health informztIon. Th:i old adagethat managers shc!ild !snow enoughabout each facet of their business tomanage their experts holds true forpretesting. You should be prepared tosupervise and guide your consultants.

SammyTo yield useful results, a pretest shouldbe planned carefully. Ample timeshould be allowed for:

contracting with research firms (ifnecessary)

arranging for the required facilities(1-2 weeks)

developing and testing the question-naire (2-3 weeks)

recruiting interviewers and respond-ents (2-4 weeks)

gathering the data (1-2 weeks)

analyzing the results (1 week)

making the appropriate alterations inmessages or materials

pretesting again, if needed.

And adequate pretesting shouldinclude:

carefully defining the targetaudience

recruiting from that audience

considering tests with "gatekeepers" or intermediaries

defining the purpose of materialsprior to designing questionnaire

locating a trained interviewer andinterpreter for some tests

carefully assessing results

considering using a "mix" ofmethods to tailor your pretesting toyour needs.

Without adequate planning. pre-testing may not serve its intendedpurposeto improve your messagesand materials. Instead. it couldbecome expensive research that is oflittle or no use.

Selected ReadingsAllman, William F., "Staying Alive in the20th Century:* Science 85, pp. 31-41.October 1985.

American Marketing Association.Marketing Services Guide, Chicago.published yearly.

Basch, Charles E., "Focus GroupInterview: An Underutilized ResearchTechnique for Improving Tneory andPractice in Health Education." HealthEducation Quarterly, Winter 1987, vol.14(4), pp. 411-448.

Bertrand. Jane E., CommunicationsPretesting, Community and Family StudyCenter, Media Monograph 6. University ofChicago, 1978.

Sudman, Seymour and Bradburn, NormanM., Asking Questions A Practical Guide toQuestionnaire Design, San Francisco. CA:Jossey-Bass Publishers. 1986.

t_ '4446P

_

,

r

51

"I-"Z

'`r,""' ' '4'

'1,./0",), I,'Y

.:,H

4,3

'

,,

,''',

':.

,,

(,

I

,,

,,.,

',,,,

'.

'''',,

' '''''

." '",''.' (', L

',,,,,"

''''"'''..,' .

'',''' '',%

'+,,

:..,,,','

.",'.:' ''

',

, .,,,M, 4 '',.'i,'-, ,,,,,.

'''

','':

''..,'

..'.1',

.,.'''',,'

'

,,:s..',41

'.., ,''''':.*.,'..,.:::',1

':';.!'',':

''',,'?eC

''' V.,

''';.,^.,'. ',,

'','''''.,',

; '"0_,''

''''''

r.,s,-,

',t,,

'',,`'d'", ^,,,',,,,.,::,:`

-,,s'i

.11'

i''

' "''

'."

"'4'.

:('''',''

.,,,

.,

,

,,,.,;

''''''''''''':,"

., , ',,,, f '''',1 0.';.

',:'..''','.

'''

,

'IL

".m

r-

'

Prepare to Introduce ProgramAt this point a number of actions occursimultaneously in order to get ready tointroduce the program. Before it be-gins, materials must be available insufficient quantities, program "kickoff"and promotion plans in place, "gate-keepers" representing channels ofimplementation briefed and progresstracking measures developed. Forexample, if you plan to:

use the mass media to introduce theissue and your new program (e.g., ata press conference)

work with voluntary organizations toprovide followup (e.g., presertationsat their meetings) and

ask the interested public to seekadditional information from theirphysicians.

Before you "kickoff" your program,consider:

Do you have a list of all the relevantmedia outlets to be contacted?

Does every organization that shouldbe involved know about your pro-gram?

Have you prepared your staff andothers to respond to inquiries?

Do you have sufficient materials tostart the program (e.g., PSAs andpress kits) and respond (e.g., leafletsfor the public)?

Are the materials in place (e.g., intelevision stations, physicians'offices, schools or supermarkets)?

Are professionals in the communityaware of the new program andprepared to respond if asked (e.g.,counsel their patients)?

Do you have mechanisms in placeto track progress (e.g., number andnature of inquiries) and identify po-tential problems (e.g i^cientsupplies of materials)?

Your program implementationshould indicate how and when re-sources WI! be needed, when specificevents will occur and at what pointsyou will assess your efforts.

The Importance of TrackingProgressOnce your program is under way, youmay not be able to anticipate allcontingencies that may arise, but youcan plan ways to identify potentialproblems. You should build a moni-toring system into your program tohelp you identify any problems, flawsor oversights regarding materials,implementation strategies or channelselection before they become majorimpediments to success.

Often, problems are quickly cor-rectable if you can identify them, butcan cause harm if you don't. Forexample, if you ask the public to callfor more information, you shouldprovide a mechanism (e.g., a simpleresponse form) for telephone operatorsto record questions asked andanswers given. A frequent review ofresponses will identify whether incor-rect or inadequate information is beinggiven, any new information required torespond, and inquiry patterns.

Frequently, program implementationtakes longer than you might expectmaterials may be delayed at theprinter, a major news story may pre-empt your publicity or a new prioritymay delay community participation. Aperiodic review of planned tasks andtime schedule will help you alter anyplans that might be affected by unex-pected events or delays. There isnothing wrong with altering your plans

to fit the situatior -keeping in mindwhat you are trying to achieve. In fact,you may risk damaging your programif you aren't willing to be flexible andalter specific activities when needed.

Process evaluationtracking howand how well your program is workingcan provide tangible evidence ofprogram progress, often useful toprovide encouragement and reward toparticipants and evidence of successto your own agency. It can also assurethat the program is working the way inwhich you planneda vital assurancepnor to undertaking any more formaloutcome evaluation. (A more completediscussion of evaluation is included instage 5.)

\

55

Establish Process EvaluationMeasuresTo help avoid overall programdysfunction because specific tasksaren't working, you should make surethat program checks are in place.Mechanisms in place should track:

work performed, time schedules andexpenditures (internal measures)

publicity, promotion and otheroutreach

participation, inquiries or otherresponses

functioning and quality of responsesystems (distribution, inquiriesresponse)

interim changes of audienceawareness, knowledge or actions.

Some ways of tracking include:

I weekly materials inventory review

clipping services of print mediacoverage

"bounceback" cards or followupphone calls with television and radiostations

monitoring logs of television/radiostations for frequency and time ofPSA airings

monitoring volume of inquiries andlength of time to reply

't :

56

reviewing telephone responses foraccuracy and appropriateness

checking distribufion points toassess materials use (and makesure that materials are still available)

phone calls or meetings with parti-cipating organizations to reviewprogress and problems

focus groups or telephone interviewswith program participants/targetaudience members

followup with teachers, physicians orother community professionals tocheck their preparedness, interestand to identify problems.

Work With intermediariesYou may have planned to use just oneor a combination of media, interper-sonal and community channels foryour program. However, once yourprogram is visible, others may bewilling to help. You may haveopportunities to expand the numberand kinds of channels involved.Although how your program expandsdepends upon your resources andcommunication strategies (stage 1),these opportunities can make programimplementation exciting ar.d chal-lenging. Some program managers maybelieve tha he most creative (and

JI

rewarding) step is materials develop-ment. Actually, creative use ofmessages and materials is essential tosuccess, creates tangible evidence ofprogress and can be very rewarding.

Woridng with organizations or indi-viduals outside of your own agency isalmost always necessary to reach atarget audience. These organizations(e.g., a television station, hospital, PTA)or individuals (e.g., a family physician,pharmacist, visiting nurse) are "inter-mediary" channels to reach the targetaudience.

"Intermediaries" can help you byproviding:

access to a target audience

r ^re credibility for your message orprogram because the target audi-ence considers them to be a trustedsource

additional resourceseither tan-gible or intangible (eg., volunteers)

added expertise (e.g., trainingcapabilities)

co-sponsorship of communityevents.

Although the Office of Cancer Commu-nications has found using intermedi-aries essential to fulfilling theirprogram expectations, there aredrawbacks that you should recognizeand prepare for. Working with otherorganizations can:

be time consumingto locate,convince them to work with you,gain internal approvals, undergoplanning and/or training

require altering your programevery organization has differentpriorities and perspectives, andintermediaries may want to makeminoror major programchanges to accommodate theirstructure or needs

result in loss of "ownership" andcontrolbecause other organiza-tions may change the time schedule,functions, or even the messages,and take credit for their part (or all)of the program.

You should be aware of thesepossibilities and be prepared to decidehow much "give and take" you will beable to allow without violating theintegrity of your program, its direction,and your own agency procedures.Balancing these decisions will requirestrategic thinking. Youor the staffperson responsible for working withintermedianesshould also be:

a good manager, to balance allprogram components

a team player, to work with theseorganizations

diplomatic and willing to negotiate

willing to share credit for success.

Choose IntermediariesWorking with other orwnizations istime consuming as well as profitable.In order to make the best use of yourstaff time available, follow these steps:

make a list of potential intermediaryorganizations

find out which agencies would reachthe most of your audience

consider which might have thegreatest influence with youraudience

think about which organizations youare most likely to be able to con-vince to work with you (e.g., whereyou know a person to contact)

think about which would require lesssupport from you (e.g., fewer of yourresources)

prioritize list of organizations accord-ing to the factors listed above

make contacts in priority order

5 fu 57

Consider Working withBusinessCooperative ventures with the busi-ness sector (for-profit companies) canalso help your program work, withbenefits accruing to your program andthe company.

Private sector companies with aninterest in health information include'

supermarkets

pharmacies

insurance companies

hospitals and HMOs

producers of pharmaceuticals, foodsand other health-related products.

Other companies may be interested inworking with you even if their productor service is not related to yourprogram to:

provide a useful public service

vi improve their corporate image andcredibility

attract the attention of a particularsector of the public.

In addition to providing funds, youmight ask for these kinds of help:

vi distribution of matenals to theirconsumers (e.g., in a supermarket)or to their employees

access to free or paid media spaceand time

corporate endorsement and help insecuring support from othercorporations

access to their market resew( h orother data

use of their staff time or facilities

sponsorship of related activities fortheir employees or for the public

printing, mailing, production facilitiesor other in-kind contributions.

Review and Revise ProgramComponentsWhether or not you continue to expandand involve more media outlets ororganizations in your program, youshould periodically assess whether.

activities are "on track" and on time

the target audience is being reached

vi some strategies appear to be moresuccessful than others

some aspects of the program needmore attention, alteration orelimination

time schedules are being met

resource expenditures are accept-able

The process evaluation and othertracking measures you establishedshould permit this assessment. Youshould establish specific intervals toreview progress. Preparing progressreportswith successes, modifiedplans and schedulescan help youkeep all your agency and program"players" informed and synchronized.

59

Selected ReadingsGross, G M., Active Communities. HealthyFamilies, Madison, WI Center for PublicRepresentation, September 1987

Morns, Lynn Lyons and Fitz-Gibbon, CardTaylor, How to Measure ProgramImplementation, Beverly Htl Is, CA SagePublications, 1978.

National Heart, Lung, and Blood Institute,Churches as an Avenue to High BloodPressure Control, NIH Publication No87-2725, October 1987

National Heart, Lung, and Blood Institute,Community Guide to High Blood PressureControl, NIH Publication No 82-2333, May1982

National Heart, Lung, and Blood Institute,Strategies for Diffusing Health Informationto Minority Populations, September 1987

National Heart, Lung, and Blood Institute,With Every Beat of Your Heart, Anldeabook for Community Hear' HealthPrograms, NIH Publication No 87-2641,September 1987

National Library Service for the Blind andPhysically Handicapped, Reaching People,Washington DC , Library of Congress,1980

ODPHP Health Information Center,Working with the Print Media (for theHealthy Older People Program),Publication No Y0182, $2 00

Peter Glenn Publications, National RadioPublicity Directory, New York Peter GlennPublications, 1982

Television Digest, Television Factbook,Washington, DC Television Digest, 1982

TV Publicity Outlets, TV PublicityOutletsNationwide, Washington Depot,CT TV Publicity Outlets, 1982

Working Press of the Nation, TV and RadioDirectory The Media Encyclopedia,Chicago The National Research Bureau,updated regularly

60

- -

:

essf001COrne*Bla

2

.'2

Smarts: an.

4 4-

t Eyalua oRrogram Assesament Questiont-

an Ayai ftesources

ISefAct

-,

,

-

;

The process measures discussedin stage 4 are designed to mon-itor the program in progress.

Tracking the number of materials dis-tributed, meetings attended or articlesprinted will tell you how the program isoperating, and may tell you whetherthe target audience is responding;these measures will not tell you aboutthe program effects: whether thetarget audience learned, acted ormade a change as a result. Therefore,it is important to evaluate the results ofyour programits effect or outcome.

Most outcome measures are de-signed to tell you what effect wasachieved, but not how or whytheseare the subjects of formative researchand process measures. The effect oroutcome is paramount, but you alsoneed to know what happened, howand why which elements worked, andto analyze what should be changed infuture programs. Therefore, plans foroutcome measures are combined withother evaluation strategies duringstage 1 planning. A review of all of thekinds of evaluation discussed in thisguide can be found on page 64.

Outcome EvaluationOutcome evaluation methodologiesusually consist of a comparisonbetween the target audience aware-ness, attitudes and/or behavior beforeand after the program. Unlike thepretesting methods ("formativeevaluation") described in stage 3,these are quantitative measures,necessary to draw conclusions aboutthe program effect. Going a stepbeyond process measures, outcomeevaluation should provide moreinformation about value than quantityof activity. The measures may be self-reported (e.g., inteMews with thetarget audience) or observational (e.g..changes in clinic visits or diseasemorbidity). Comparisons between acontrol group (one that did not receivethe program, but is similar in otherrespects to the target audience) andthe target audience receiving theprogram are desirable.

1

Impact StudiesAs discussed in the introduction to thisguide, communication programs areone contributor to the improvement ofthe public's health. In a "real world"environment, there are many factorswhich influence an individual's healthbehavior, including peer support andapproval, self-esteem and otherindividual characteristics, advertisingand mass media coverage of health,community and institutional factors(such as the availability of services). Itis usually extremely difficult to separate the impact of your communica-tion program from the effects of otherfactors ("confounding variables") onan individual's behavior. For thisreason, impact studies are rarelyinitiated as a part of programs usingonly communication strategies.

Determine What Evaluation to DoLimited resources may force you tochoose between process evaluation oroutcome evaluation. Neither, independ-ently, will provide you with a completepicture of what happened. Someexperts will tell you that if you mustchoose, you should choose outcomeevaluationthe only way to certifythat you accomplished your objectives.However, process evaluation can helpyou understand why you did or did notaccomplish your objectives. Therefore,others will advise that process meas-ures are more importantto allow youto manage your program well.

Every prcgram planner facesconstraints to undertaking evaluationtasks, just as there are constraints to

designing other aspects of a communi-cation program. These constraints mayinclude:

a limited funds

limited staff time and capabilities

length of time allotted to theprogram

limited access to computer facilities

agency restrictions to hiringconsultants or contractors

policies limiting the ability to gatherinformation from the public

management perceptions regardingthe va:. of evaluation

levels of management support forwell-designed evaluation activities

difficulties in defining the objectivesof the program, or in establishingagency consensus

difficulties in designing appropriatemeasures for communicationprograms

difficulties in separating the effectsof program influences from otherinfluences on the target audience in"real world" situations.

These constraints make it neces-sary to accommodate to existing limi-tations as well as the requirements ofa specific program. However, it is nottrue that "something is better thannothing." If an evaluation design, datacollection or analysis must be com-promised to fit limitations, the programmust make a decision regardingwhether:

the required compromises will makethe evaluation results invalid

an evaluation strategy is essentialfor the particular situation, com-pared with other compelling uses forexisting resources.

63

64

These are some of the questionsyou should consider before decidingwhat kind of evaluation will be best foryour program:

How long will your program iast? Willthe implementation phase be longenough to permit measurement ofsignificant effects and periodicadjustment?

Do you want to repeat or continueyour program?

Are your objectives measurable inthe foreseeable future?

Which program components aremost important to you?

Is there management support orpublic demand for programaccountability?

What aspects of the program fit bestwith your agency's priorities?

Will an evaliation report helpcommunication efforts compete withother agency priorities for futurefunding?

There are a number of sources foryou to find help when you design anevaluation. Several pertinent texts areincluded in the suggested readings atthe end of this chapter. If there is not aplanning and evaluation staff in youragency, you may find help at a nearbyuniversity. Also, you may contact anappropriate clearinghouse or Federalagency and ask for evaluation reportsthat may have been prepared (butgenerally are not published) on similarprograms.

A frequent response to "What kindof evaluation measures are you plan-ning?" is "I don't have enough moneyfor evaluation." Rarely does anyonehave access to resources for an idealhealth communication program, muchless an ideal evaluation component.Nevertheless, there are practicalbenefits to including evaluation as apart of your work: to tell you whetheryour program is on track and how wellit worked. The previous discussion ofprocess, outcome and impact evalua-tion included some examples of thekinds of questions you might ask. Witha little creative thinking, you will findthat you can include some form ofevaluation for almost any size ofbudget. The chart below gives ex-amples of evaluation tasks you mightconsider if you don't really have anevaluation budget ("minimal re-sources"), and if you have a moderatebudget for evaluation. It also gives youexamples of the kinds of evaluationsyou might ideally consider ("substan-tial resources"). The matrix is additivefrom left to right. That is, each ascend-

ing program level could be expected toinclude the evaluation techniquedescribed at lower levels in addition tothose described at the higher level.

Elements of an EvaluationDesignEvery formal design, whether forma-tive, process, outcome, impact or acombination, must contain certainbasic elements. Tfr 3e include:

1. A Statement of CommunicationOb;cctivesUnless there is an adequate definitionof desired achievements, evaluationcannot measure them. Evaluatorsneed clear and definite objectives inorder to measure program effects.

2. Definition of Data to be CollectedThis is the determination of what is tobe mefisured in relation to the objec-tives.

3. MethodologyA study design is formulated to permitmeasurement in a valid and reliablemanner.

4. instrumentationData collection instruments are de-signed and pretested. These instru-ments range from simple tally sheetsfor counting public inquiries tocomplex survey and interview forms.

5. Data CollectionThe actual process of gathering data.

6. Data ProcessingPutting the data into usable form foranalysis.

7. Data AnalysisThe application of statistical tech-niques to the data to discover signifi-cant relationships.

8. ReportingCompiling and recording evaluationresults. These results rarely pronouncea program a complete success orfailure. To some extent all programshave good elements and bad. It isimportant to appreciate that lessonscan be leamed from both if results areproperly analyzed. These lessonsshould be applied to altering theexisting program or as a guide toplanning new efforts.

6 r" 15

"".....

,

""-

,

%a

-,

666 G

Selected ReadingsFink, Arlene and Kosecoff, Jacqueline, AnEvaluation Primer and Workbook: PracticalExercises for Health Professionals,Beverly Hills, CA: Sage Publications, 1978.

Fitz-Gibbon, Carol Taylor and Morris, LynnLyons, How to Design a ProgramEvaluation, Beverly Hills, CA: SagePublications, 1978.

French, John F., Fisher, Court C., Costa,Jr., Samuel J. (ed.) Working withEvaluators: A Guide for Drug AbusePrevention Program Managers, U.S.Department of Health and HumanServices. Alcohol, Drug Abuse and MentalHealth Administration, Publication No(ADM) 831233, 1983.

Hawkins, J. David and Britt Nederhood,Handbook for Evaluating Drug and AlcoholPrevention Programs, ll S. Department ofHaalth and Human Services, Alcohol,Drug and Mental Health Administratior.,DHHS Pubhcation No (ADM) 87-1512,1987,

National Heart, Lung. ard Blood Institute,Measuring Progress in High BloodPressure Control. An EvaluationHandbook, N1H Publication No. 86.2647,April 1986

Rossi, Peter H., Freeman, Howard E .Evaluation, Beverly Hills SagePublications, 1985

Survey Research Center, Interviewer'sManual (revised edition). University ofMictrgan, Ann Arbor Institute for SocialResearch, 1978

t,

,

4

5

A N

a

1

N

r

,

I

i ,

A I41

. f

rrrl

f

1111111111111111111111111111

Apply What You Have LeamatiThe ideal way to apply evaluationfindings is to impro .our ongoingprogram. You also r. 'JSE what yo..learn from process or outcome evalua-tion measures to:

justify your program with manage-meltprovide evidence of need for addi-tional funds or other resources

increase institutional understandinpof and support for health coi hmuni-cation actMties

encourage ongoing cooperativeventures with other organizations.

Revise ProgramIf your program is continuing or youhave an opportunity to advise otherswho may plan similar programs, takethe time to apply what yciu havelearned. For example,

Reassess goals and objectivesHas anything changed (e.g., withthe target audience, tho commu-nity, or your agency's mission) torequire revisions in the originalgoals and objectives?Is there new information about thehealth issue that should be in-corporated into the program mes-ages or design?

Determine areas where additionaleffort is needed

Are there objectives that are notbeing met? Why)Are there strategies or activitiesthat did not succeed? Are moreresources required? Do you needto review why they didn't work alidwhat can be done to correct anyproblems?

Identify effective activities or strate-gies

Have some objectives been metas a result of successful activi-ties?Should these be continued andstrengthened because they appearto work well?

69

Or should they be considered suc-cessful and completed?Can they be expanded to apply toother audiences or situations?

Compare costs and results of dif-erent activitiesw What were the relative costs

(including staff time) and results ofdifferent aspects of your program?

a Are there some activities thatappear to work as well but costless than others?

I Reaffirm support for the programHave you shared the results ofyour activities with the leadershipof your agency?Did you remember to share thisinformation with the individualsand organizations outside youragency who contributed?Do you have evidence of programeffectiveness and continued needto convince your agency tocontinue your program?Do you have new or continuingactivities that suggest the involve-ment of additional organizafions?

fts*,44

a Determine to end a program that didnot work.

4 -

Share What You Have LearnedIt is frequently difficult to find the timeto analyze and report on what youhave teamed and share it with others.You may find that other responsibilitiesleave you littie time to prepare formaldocumentation of your program or tosubmit findings for publication. Never-theless, what you learn from imple-menting a communication programmight be invaluable to someone who isfaced with a similar responsibility. Youmay not have to prepare a formalreport or article to let others knowwhat you have learned. Consider.

ce,

letters about your findings to appro-priate medical, public health orhealth education journals

a poster presentation at a relevantprofessional meeting

70 C.)

I a program description and samplematerials sent to a related cleanng.house, Federal or state agency

local professional newsletters

letters, phone calls, brief reports ormeetings with your peers in similarorganizations.

Letting others know about yourprogram may prompt them to tell youabout similar experiences, lessons.new ideas or potential resources.

Selected ReadingsGreen, Lawrence W. Lewis. FrancesMarcus. Measurement and Evaluation inHealth Education and Health PromotionPalo Alto- Mayfield Publishing Co , 1986

Hawkins, J David and Britt Nederhood.Handbook tor Evaluating Drug and AlconolPreventron Programs, U S Department ofHealth and Human Services. Alcohol.Drug and Mental Health AdministrationDHHS Publication No (ADM) 87-1512.1987

Moms. Lynn Lyons and FitzGibbon. CarolTaylor, How to Present an EvaluationReport, Beverly Hills. CA Sagr;Publications, 1978

( ' .

71

National Sources ofHealth.Related Data

The National Center for HealthStatistics (NCHS)U.S. Public Health Service, NCHS3700 East-West HighwayHyattsville, MD 20782

NCHS tracks and analyzes changes inhealth status in the U.S. A summary ofstatus and trends is published eachyear as Health/United States.

Morbidity and Mortality Weekly Report(MMWR)Centers for Disease ControlAtlanta, GA 30333

The MMWR provides immediate alertsregarding disease incidence, morbidityand mortality as reported to CDC byhealth departments nationwide.

The Roper CenterP.O. Box 440Storrs, CT 06268

The Roper Center collects and storespublic opinion data collected by surveyorganizations including NORC, Gallup,Roper, Harris, Opinion ResearchCorporation, Yankelovich, GordonBlack Company, various news pollsand special studies.

The ODPHP National HealthInformation CenterP. 0. Box 1133Washington, DC 20013-1133(800) 336-4797 (from outside of theWashington Metropolitan area)

The center helps the public locatehealth information through theidentification of health informationresources and a referral service.

Journals

These journals cover health communi-cations program development andrelated topics discussed in this book:

A/coho/ Health and Research World, aquarterly publication of the (DHHS)National Institute on Alcohol Abuseand Alcoholism, ADAMHA, 5600Fishers Lane, Rockville, MD 20857.

American Demographics, Box 68, 127West State St., Ithaca, NY 14851.

American Journal of Health Promotion,a quarterly publication, Box 1287,Royal Oak, MI 48068.

American Journal of Public Health, amonthly publication of the AmericanPublic Health Association, 1015 15thSt. NW, Washington, DC 20005.

Evaluation and the Health Professions,quarterly, Sage Publications, Inc., 2111W. Hillcrest Dr., Newbury Park, CA91320.

Health Education, bimonthly, apublication of the American Alliancefor Health, Physical Education,Recreation and Dance, 1900Association Drive, Reston, VA 22091.

Health Education Quarterly, apublication of the Society for PublicHealth Education, John Wiley & Sons,Inc., Periodicals Division, 605 ThirdAvenue, New York, NY 10158.

Health Education Research, quarterly,IRL Press Inc., P.0 Box 0, McLean,VA 22101-0850.

HealthLink, a quarterly publication ofthe National Center for HealthEducation, 30 East 29th St., r lw York,NY 10016.

Journal of Communication, quarterlypublication by Oxford University Press,University of Pennsylvania, 3620Walnut St., Philadelphia, PA19104-3858.

72

Public Health Reports, a bimonthlypublication of the U.S. Public HealthService, Dept. of Health and HumanServices, Hubert Humphrey Bldg., Rm.721-H, 20'J Independence Ave. SW,Washington, DC 20201.

Public Opinion Quarterly, a quarterlypublication of the AmericanAssociation for Public OpinionResearch, University of Chicago Press,5801 S. Ellis Ave., Chicago, IL 60637.

75

The SMOG Readability FormulaTo calculate the SMOG reading grade level,begin with the entire written work that is beingassessed, and follow these four steps:

1. Count off 10 consecutive sentences near thebeginning, in the middle, and near the end ofthe text.

2. From this sample of 30 sentences, circle allof the words containing three or moresyllables (polysyllabic), including repetitions ofthe same word, and total the number of wordscircled.

3. Estimate the square root of the total numberof polysyllabic words counted. This is done byfinding the nearest perfect square, and takingits square root.

4. Finally, add a constant of three to the squareroot. This number gives the SMOG grade, orthe reading grade level that a person musthave reached if he or she is to fully under-stand the text being assessed.

A few additional guidelines will help to clarifythese directions:

A sentence is defined as a string of wordspunctuated with a period (.), an exclamationpoint (!) or a question mark (?).

Hyphenated words are considered as oneword.

Numbers which are written out should also beconsidered, and if in numeric form in the text,they should be pronounced to determine ifthey are polysyllabic.

Proper nouns, if polysyllabic, should becounted, too.

Abbreviations should be read asunabbreviated to determine if they arepolysyllabic.

Not all pamphlets, fact sheets, or otherprinted materials contain 30 sentences. To test atext that has fewer than 30 sentences:

1.Count all of the polysyllabic words in the text.

2.Count the number of sentences.

3. Find the average number of polysyliabicwords per sentence as follows:

Total # of_polysyllabic wordsaverage lotal # of sentences

4. Multiply that average by the number ofsentences short of 30.

5.Add that figure on to the total number ofpolysyllabic words.

6. Find the square root and add the constant of3.

Perhaps the quickest way to administer theSMOG grading test is by using the SMOGconversion table. Simply count the number ofpolysyllabic words in your chain of 30 sentencesand look up the approximate grade level on thechart.

An example of how to use the SMOGReadability Formula and the SMOG ConversionTable is provided on the following page.

73 77

Example Using theSMOG Readability Formula:I.

InYou MIL2

2. 3.(The key is action)(1bu can help protect yourself against

cancer) Act promptly to:

(*Prevent some cancers through simple changes inlifestyle.)

(cFmd out about early ette_ctio)tests in your home.)

(4.Gain peace of mind through (iii3,/leckulps.)

Cancers You Should Know About(7. Lung Cancer is the number one cancer men, both

in the n of new cases each 9,000 and deaths.1-#):.....11r.:4rates are due mainly to

1;:Whia- sin° : not smoking, you can largelyprevent lung cancer risk is reduced by anoking less,and by using lower tar and 11013) brands) But quittingaltogether is by far the most effective safeguard. TheAmerican Cancer Society offers Quit Smoking Clinicsand self-help materials.

Colorectal Cancer is second in cancer deaths (25,100)an0 third in new cases (49,000). When it is found early,chances of cure are good. A regular general physicalusually includes a digital examination of the rectum and aguaiac slide test of a stool specimen to check for invisibleblood. Now there are also Do-It-Yourself Guaiac Slidesfor home use. Ask your doctor about them. After youreach the age of 40, your regular check-up may include a"Procto," in which the rectum and part of the colon areinspected through a hollow, lighted tube.

01- Prostate Cancer is second in the number of new caseseach year a an, d third in deaths CAVV))(ff Occursolmainly in men over 60XKigigf, rectal exam of-theprostate by your doctor is the best

A Check-Up Pays OffO'Be sure to hive a gligt, ifghfli Fsiail including an

oral exantikis your best taIMI, of-good health)*This pamphlet is from the American Cancer Society.

78

Sample only. Information may not be current.

,ffow Cancer WorksCif we know something about how cancer works, we can

act more dreiark to protect ourselves against thedisease) Here are the basics.

(111:. Cancer spreads; time co --Cancer isgrowth of 116nornui small and ifunchecked, spreads) m an early, local stage,the chances for cure are best.)

(02. Risk iteei with ageThis is not a reason to worry,but a signal to have moret thorough415W4check-ups)Your doctor or clinic can advise you onwhat tests to get and how often they should beperformed.

3. What you can doDon't smoke and you will sharplyreduce your chances of getting lung cancer. Avoid toomuch sun, a major cause of skin cancer. Learncancer's Seven Warning Signals, listed on the back ofthis leaflet, and see your doctor promptly if theypersist. Pain usually is a late symptom of cancer, don'twait for it.

Unproven RemediesBeware of unproven cancer remedies. They may soundappealing, but they are usually worthless. Relying onthem can delay good treatment until it is too late(Checkwith your doctor or the Cancer 11,t)

More Information02For more i of any kind about cancerfree of

costcontact your local unit of th Mar Cancere=isKnow Cancer's Seven Warning Signals

(73- 1. Change in bowel or bladder habits)(24.2. A sore that does not heal.)(25. 3. Unusual bleeding or discharge.)(2.4. 4. Thickening or kiwi!' breast or elsewhere)(27. 5. I ....estiat(20.6. change in wart or mole.)

er.7. agging cough or hoarseness.)

(36.If you have a warning signal, see your doctor)

7

We have calculated the reading grade levelfor this example. Compare your results to ours,then check both with the SMOG conversiontable:

Readability Test CalculationsTotal Number of Polysyllabic Words =38Nearest Perfect Square =36Square Root = 6Constant = 3SMOG Reading Grade Level = 9

SMOG Conwrsion TWOTotal Polysyllabic

Word CountsApproximate GradeLevel (t1.5 Grades)

0-2 4

3-6 57-12 61320 7

21-30 831-42 943-56 10

57-72 11

73-90 12

91-110 13

111-132 14

133156 15

157-182 16

183210 17

211-240 18

*Developed by Harold C McGraw, Office of Educational Research,Baltimore County Schools, Towson, Maryland

'?579

Educational Materials Review Forms

Pre-Post Booklet Testing Form (Self-Administered)

Central Location Intercept Questionnaire

Focus Group Moderators GuideGatekeeper/Professional Review Questionnaire

PSA Pretest Questions

G

Educational Materials Review FormsSource: Office for Substance Abuse Prevention, ADAMHA

I. PRODUCT DESCRIPTION

Title of Product

Producer/Author

Organization

Address

Accession No

Date

Screener

Publication Date

Contact

Other Sponsors/Endorsers

Format:Fact SheetBrochureBookletBookPosterPrint Ad

_ 3/4" Video_ VHS Video

16mm FilmSlidesAudiotapeScriptInformation PackageSoftwareOther (describe)

PSA

Magazine

ArticlePerformanceTaped MessageNewsletterContestWorkbookCurriculumClassroom MaterialComic Book

Length:

Context(s): (check all that apply)Part of a Program (describe)

With Other MaterialsStands AloneHas Training Component (please enclose)

Topic(s): (check all that apply)AlcoholDrugsDrug (specify)

__ Both Alcohol and DrugsAwarenessPreventionInterventionTreatmentGeneral Health/SafetyOther (describe)

Phone

Mode(s) of Delivery: (check all that apply)_ Individual

GroupInstructor-ledMass MediaSelf.Instructional

Setting(s): (check all that apply)_ Home

SchoolCommunityHealth/Mental Health CareWorksiteJustice SystemSocial Services SystemMass MediaOther (describe)

Target Audience(s): (check all that apply)_ General Public,

Parents (specify age of child)BlacksHispanicsNative AmericansAsian and Pacific IslandersHealth Care Providers (specify specialty)

_ High Risk Families/YouthPolicymakers/AdministratorsYouth (specify ages)Young Adults (18-25 years)Educators (sriecify gradefspA/D Treatment ProfessionalsA/D Prevention ProfessionalsEmployers

_ Scientists and ReseachersOther (descnbe)

7 ,---1

i

Language(s): (check all that apply)English

SpanishBilingual (specify)Other (indicate)

Readability:Low Literacy (grade level 3-5)Very Easy (grade level 6-7)Easy (8)Average (9-10)Fairly Difficult (11-13)Difficult (14.16)N/A

Pretested/Evaluated:Yes _ UnknownNo

(If yes, describe and include copy of report ifpossible)_

Current Scope:NationalRegionalStateLocal

Availability:Unknown

_ Restrictions on UsePermission Required to ReproduceCitation Required

Available FreeNegatives Available on Loan

_ Payment RequiredPrice $

_ Available Through Free AV Loan ProgramSource (if different from above)

83

Description: (Please describe the product in two.three sentences )

Comments:

84 7 ',

II. FIELD REVIEW FORM

Target Audience (if different from Screening Form)Topic (if different from Screening Form)Major Messages (list)

Persuasive Technique (describe)

Distinguishing Qualities (describe)'

Acc No

Date

Reviewer

Production Quality

Excellent

5 4 3 2

Poor

1

(Comments)

Content 5 4 3 2 1

(Comments)

Credibility 5 4 3 2 1

(Comments)

Ability to Attract Attention 5 4 3 2 1

(Comments)

Ability to Convey Information* 5 4 3 2 1

(Comments)

Ability to Change Attitudes 5 4 3 2 1

(Comments)

Ability to Elicit Appropriate Action 5 4 3 2 1

(Comments)

Appropriate for National Distril--,ion Yes No Limited Use (describe)

(Comments)

Overall Rating 5 4 3 2 1

(specify any particular strengths/weaknesses)

Recommend for further consideration (eg , promotion, replication, purchase, adaptation, testing or evaluation)"

Yes No

Please explain recommendation

Return to'

85

7,)

86

Considerations tor Field Review

1. Target audienceWhat audience is the material best suited for? For whom should it not be use& Consider thelanguage style, use of terminology, length, appropriateness of examples and format in determining the targetaudience.

2. Persuasive techniqueAre the messages positive and upbeat? Are positive role models used? Fear appeals?Authority figures (who)? Peer pressure?

3. Distinguishing qualitiesInnovative or unique presentation, format or style? Fills a need for specific audience ormessage?

4. Production qualitiesls the material professional in appearance, attractive, well-written? Is the production formatappropriate for the intended use (e.g., setting, equipment required)? Should production changes be considered(e.g., use of less or more color)?

5. ContentClear and accurate? Up to date? Appropriate message, tone and appeal? Stimulating? New knowiedge?Perpetuate myths or stereotypes? Balanced and credible? Biased or judgmental?

6. Elicit actionDescribes desired behavior? Illustrates skills required? Demonstrates appropriate behavior?

7. Credibilityls product,on or distribution source credible for target audience? For intermediaries (e.g., teachers orparents)? Is message, theme, presentation credible?

8. Appropriate for national distributionWill materials starr alone, or require ming for use? Inappropriate for someaudiences (e.g., culturally inappropriate) or geographic areas?

9. Recommendation for evaluationAre there questions or uncertainties that need to be resolved prior to determin-ing disposition? Should materials be tested?

8u

PrePost Booklet Testing FormSource* U S Environmental Protection Agency

I. Pretest QuestionsAs you probably are aware, Toms River is the site of a pilot project designed to inform residents aboutpotential environmental hazards associated with the Superfund site, and to encourage their irvolvement in EPA's decision-making process for cleanup of the site.

We would appreciate your wilhngness to share your reactions to the attached fact sheet by readingit and answering a few questions. We do not ask your name and all inform ition you provide wil; re-

main confidential.Because only a few Toms River citizens are being asked to help judge ihis material, your response

is particularly valuable.

Before you begin, please check the appropnate answers to these four questions

1. How much would you say you know about the Toms River Superfund study"A little Some A lot

2. Is there anyt!mng in particular you want to know about the study?Yes NoIf yes, please specify.(Note: more knowledge questions can be added here.)

3 Are you or any member of your family an employee/former employee of (Superfund site company)"

Yes No

4. Are you a member of any group particularly concerned about the environment?

Yes No

Now, please turn the page and read the fact sheet.

87

II. Posttest QuestionsNow that you have finished reading the fact sheet, please answer the questions below Toi_, may rt?tt,back to the fact sheet as you consider your response if you wish

1 In your own words, what would you say is the purpose of the Superfund study?(Note: additional 'row ledge questions can be added here)

2. How much of the information in the fact sheet was new to you?Most ot it Some of it None

3 Do you have questions about the Superfund study which were not answered in 1f-it fact sheet'Yes NoIf yes, please list.

4 Was there anything you particularly liked about the fact sheet?Yes NoIf yes, what?

5 Was there anything you particularly disliked about the fact sheet, or found confusingYes NoIf yes, what?

6 This fact sheet is most appropuate for (ch?ck all that apply)General public College graduates Professionals

7 Would you recommend the fact sheet to a friend or family member?Yes No

8 The following are a senes of phrases describing the fact sheeteach line that most closely reflects your opiniona very interesting somewhat interestingb very informative somewhat informativec accurate partially accurated very clear somewhat cleara very useful somewhat usefulf unbiased biased towards governmentg easy to read understandableh complete scmewhat complete

Please circle the one Lnoice Lir

not at all interestincnot informativeinaccurateconfusingnot usefulbiased towards Indutrvhard to understandincomplete

9 Would you like to say anyihing else about the fact sheet? Please comment

Thank you very much for your help in reviewing this fact sheet

88

Central Location Intercept (Respondent's name and ID No)

Questionnaire (Location Houston =1. Chicago=2)Source National Heart, Lung and Blood Institute

(Phone Number)

IntroductionGood afternoon (evening).

My name is (interviewer's initials) , and I'm with Quick Test, a marketing research firm.We are conducting a research study in this mall with people who are between 25 and 55 years of ageWould you be wilhng to spend a few minutes answering some questions?

ScreenerS1 What is your age? (RECORD)

1 Under 25 (THANK AND TERMINATE)2. Between 25 and 40 (CONTINUE)3 Between 41 and 55 (CONTINUE)4 Over 55 (THANK AND TERMINATE)

S 2 Have you ever been told you have any of the following?

YES NO1 Diabetes CONTINUE CONTINUE2 High blood pressure CONTINUE TERMINATE3 High cholesterol CONTINUE CONTINUE

S 3 Which of the foHowing best describes the highest level ci education you've completed (CIRCLEONE)

1 Elementary or grade school (CONTINUE)2 Some high school (CONTINUE)3 High school graduate (CONTINUE)4 Some college (CONTINUE)c Sul lege graduate (CONTINUEMAXIMUM OF 2 PER SUBGROUP)6 Some g'aduate work (THANK AND TERMINATE)7 Graduate degree ( CHANK AND TERMINATE)

S 4 Respondent's sex

1 Male2 Female

S 5 Respondent's race

1 White (CONTINUE)2 Black (CONTINUE)3 Other (THANK AND TERMINATE)

89

5.6 I wcyild hke to show you three television messages which have been produced as a publicservice and get your reactions to them. This will not take more than 15 minutes and we wouldlike to offer you an appreciation of $2.00 Els a means of saying "thank you" for your time. If youwould like to see these public service messages, would you please come with me into ourviewing room?

1. No (THANK AND TERMINATE)2. Yes (CONTINUE)

NOTE: Participants are to be recruited in the following subgroups.

15 black males, ages 25-4015 white males, ages 25-4015 black males, ages 41-5515 white males, ages 41-5515 black females, ages 41-5515 white females, ages 41-55

In each of the subgroups listed above, one-third of the respondents should follow Order 1. one-thirdshould follow Order 2, and one-third should follow Order 3, as specified below.

ORDER 1. Show "Randy" first and administer questionnaire. Then show "Red Balloon" andadminister questionnaire, followed by "Wild Horse" and questionnaire Thank and pdy $2appreciation.

ORDER 2. Show "Red Balloon- and administer questionnaire Then show "Randy" and administerquestionnaire. Show "Wild Horse," administer questionnaire, and thank and pay $2appreciation.

ORDER 3. Show "Wild Horse- and administer questionnaire. Then show "Red Balloon" and adminis-ter questionnaire, followed by "Randy'. and questionnaire Thank and pay $2 appreciation

CHECK ORDER BEING USED:

ORDER 1 ("Randy," "Red Balloon," "Wild Horse")ORDER 2 ("Red Balloon," "Randy," "Wild Horse")ORDER 3 ("Wild Horse," "Red Balloon," "Randy")

FOR EACH RESPONDENT FASTEN TOGETHER THE SCREENER, "RANDY" QUESTIONNAIRE,"RED BALLOON" QUESTIONNAIRE AND "WILD HORSE" QUESTIONNAIRE IN THAT ORDER INPREPARATION FOR COMPUTER ENTRY REMEMBER TO RECORD TIME OF FINISH AT BOTTOM OFSCREENER.

Time start:Time finish.

90

Questionnaire fnr "Randy"IF ORDER 1, READ THE FOLLOWING.

I'd like to show you tne first public service message and then ask you a few questions about it Themessage is still in rough form, so you will need to use your imagination a little to picture what it willlook like after final production. In this version you will see drawings used to get the idea across, but inthe final version there will be live action and it will have a more finished look Please try to concen-trate more on the concept than on the drawings as you watch them

IF ORDER 2, READ THE FOLLOWING.

Now I'd like to show you the second message and aet your reactions Again, please remember thatthis is in rough form and is not yet in finished form

IF ORDER 3, READ THE FOLLOWING

I'd like to show you the third message now. Again. I'll be asking for your reactions Please rememberthat it is still in rough form,

PLAY "RANDY"NOTE: Throughout the questionnaire, "HBP. should be read as "high blood pressure'S

1. What did you think was the main idea of this announcement? (RECORD FIRST MENTION) Anythingelse? (REC9RD ALL OTHER MENTIONS.)

First Mention(Circle only one)

1 High blood/HBP 1

2 It's up to you your responsibility to take care of HBP 2

3 If you don't take care of HBP. won't bc around for other things 3

4 Take care of yourself (general) 4

5 Take care of HBP control HBP 5

6 You can enjoy life (have a normal/active/iong life) by taking care ofHBP

7 Stay on HBP treatment, keep up fight. work at it don t ease up 7

8 Take care of HBP for your loved ones 8

9 Treat your HBP, do what your doctor says 9

10 Control HBP or could become ill (stroke/die) 10

11 Take care of your HBP every day, treat it for life 11

12 Other XX

RECORD

Anything Else(Circle all other

mentions that apply)

91

2. What, if anything, about the announcement did you particularly like? (RECORD FIRST MENTION)Anything else'? (RECORD ALL OTHER MENTIONS)

First Mention(Circle only one)


mentions that apply)1 Everything 1 1

2. Nothing 2 1

3 Liked message in general 3 1

4. Easy to follow/understand 4

5. Attentiongetting/interesting 5 1

6 Message is important 6 1

7. Message contains useful information 7 1

8 Good reminder 8 1

9. Liked idea of having control/being able to do something about HBP tohelp yourself 8 1

10 Message is direct and to the point 9 1

11 Focuses on blacks 10 1

12 Message is dramatic 11 1

13 Message speaks directly to me 12 1

14 Liked encouraging tone/can have a full Me if treat HBP 11 1

15 Message and pictures fit well together 12 1

16 Don't know 13 1

17 Other XX 1

RECORD

3. What, if anything, about the announcement did you particularly dislike'? (RECORD FIRST MENTION)Anything else'? (RECORD ALL OTHER MENTIONS)

First Mention(Circle only --ie)


mentions that apply)1 Everything 1 1

2. Nothing 2 1

3. Dishked message in general 3 1

4 Message was difficult to follow/understand 4 1

5 Voices hard to understand 5 1

6 Too light/not serious enough 6 1

7 Too serious/scary/overly dramatic 7 1

8 Disliked black voice/image 8 1

9 Not especially attention-getting or interesting 9 1

10 Not enough information, information not new or useful 10 1

11 Too peczimistic, grim 11 1

12. Message and pictures didn't fit well together 12 1

13 Message didn't relate to me 13 1

14 Don't know 14 1

15 Other XX 1

RECORD

92t i,

4. Was there anything in the announcement that you found confusing or hard to understancr(RECORD FIRST MENTION) Anything else? (RECORD ALL OTHER MENTIONS)

Anything ElseFirst Mention (Circle all other

(Circle only one) mentions that apply)1 Nothing (everything was clear) 1 1

2 Confusing in general 2 1

3. Message not clear 3 1

4. Words were hard to understand 4 1

5. Didn't understand that the father had died 5 1

6 Didn't understand why the father didn't live past fifty 6 1

7 Too much information presented 7 1

8 Not enough information presented 8 1

9. Message didn't relate to me 9 1

10. Didn't underotand who was speaking 10 1

11 Other XX 1

RECORD

5. Was there anything in the announcement you found hard to believe'? (RECORD FIRST MENTION)Anything else? (RECORD ALL OTHER MENTIONS)


(Circie only one) mentions that apply)1 No, nothing 1 1

2 Mesr;age not behevable in general 2 1

3 Not believable that someone would die at fifty because didn I takecare of HBP 3 1

4 Not believable that HBP is that dangerous to health 4 1

5 Not behevable that HBP is that hard to control 5 1

6 Other XX 1

RECORD

6. (ASK ONLY FOR THOSE FOLLOWING ORDER 1) Can you tell me who the sponsor of this messagewas?


(Circle only one) mentions that apply)1 National Cholesterol Education Program 1 1

2 The government 2 1

3. National Heart Association 3 1

4 Don't remember 4 1

5 Other XX 1

RECORD

93

7. I'm going to read to you a set of statements describing the message you just saw For eachstatement p'ease tell me whether you strongly agree, agree, neither agree nor disagree, disagree,or strongly disagree with the statement. (READ STATEMENTS AND SHOW SCALE)

NeitherStrongly

Agree AgreeAgree NorDisagree Disagree

StronglyDisagree

1 The message was interesting 1 2 3 4 5

9 The message was convincing 1 2 3 4 5

3. The message was Irritating 1 2 3 4 5

4 The message was confusing 1 2 3 4 5

5 The message made its point 1 2 3 4 5

6. The message was not serious enough 1 2 3 4 5

7. The message was offensive 1 2 3 4 5

8 The message was scary 1 2 3 4 5

9. The message was believable 1 2 3 4 5

10 The message gave me useful information 1 2 3 4 5

11 The message gave useful information for otherpeople 1 2 3 4 5

12 The message captured my attention 1 2 3 4 5

13 The message will capture the attention of thosewith HBP 1 2 3 4 5

14 The message was a good reminder to take careof HBP 2 3 4 5

15. The message had an overall encouraging tone 1 2 3 4 5

16 The message was too mild. it should be stronger 1 2 3 4 5

17 I will be more conscientious about my HBPtreatment 1 2 3 4 5

18 Staying on my HBP treatment program is astruggle for me 1 2 3 4 5

19 The message convinced me that it's importantto control HBP 2 3 4 5

94 6

Focus Group Moderator's GuideSource. National Institute on Aging

Wannup and Explanation1. Introductiona Thanks for comingb Your presence is important.c Describe what a focus group is

like an opinion scvey, but verygeneral, broad questions

2. Purposea We will be discussing some issues

related to healthb I'm interested in all of your ideas,

comments, and suggestionsc There are no right or wrong

answers.d All commentsboth positive and

negativeare welcomee Please feel free to disagree with

one another. We would like to havemany points of view

3. Procedurea Explain use of tape recorder (one-

way mirror). All comments areconfidentialused for researchpurposes only

b I want this to be a group discussion,so you needn't wait for me to call onyou Please speak one at a time, sothat the tape recorder can pick upeverything

c We have a iot of ground to cover, soI may change the subject or moveahead Please stop me if you wantto add something

4. Self Introductionsa Ask each participant to introduce

himself/herselftell the group yourname, and where yOU live

General Attitudes Health and Aging1. Is your health something that you

actively think about? (If not, whynot?)

2. When you think of health, what doyou think about?

3. What about the health of yourfamily'? What issues or topics doyou think about?

4. (45-54 and over 55 groups) Do youthink about growing older, and thechanges that might bring? What doyou think about? (Probe re healthchanges independence, changes inlifestyle )

5. What do you think other people/yourparents feel about growing older?(Probe re health changes, inde-pendence, changes in hfestyle )

Relationship to Older Family Members1. Do you have parents, other older

relatives or friends who are close toyou?

2. What is your relationship with theseolder relatives/friends? (How muchcontact, do you feel close to them?Responsible?)

3. Do you think about their health andwell-being? What, specifically, doyou think about? (Probe re healthconcerns, independence, livingsituations )

Knowledge of Alzheimer's Disease(If Alzheimer's Disease hasn't comeup in the conversation)1. Have you ever heard of Alzheimer's

Disease? What have you heard?/.. What do you think it is? Is it different

from just getting old? (Probe for ageof onset. how many people it af-fects, cause(s), symptoms, preven-tion, how it is diagnosed, how it istrcated, how quickly it progresses,is it contagious, what happens, newresearch they have heard about,what kind of care is requiredathome, nursing homes, relatedcosts )

3. Is it (senility or Alzheimer's) aninevitable consequence of aging?

4. Where have you heard aboutAlzheimer's Disease (probe forsource)?

5. Do you have questions? Whatwould you like to know about it?What else?

6. Do you ever have any thoughts orconcerns about Alzheimer'sDisease? For yourself or forsomeone else? (Who?)

7. What kind of concerns?

Sources of Information (General)1. If you wanted to find out more

about Alzheimer's Disea e (hadquestions or concerns), how wouldyou go about it? (Probe foradditional sources, e.g., familyphysician, friends, workpla esource, where else in owncommunity, library or nationalagencies.)

2. Do you know of any agencies ororganizations that deal especiallywith older people or Alzheimer'sDisease?

Facing Alzheimer's Disease1. Now I want you to think about

particular situation where you zieconcerned about a friend orrelative You think they might haveAlzheimer's Disease What wouldyou do? (Probe how would youinteract with that individual [eg , aparent], where would you turn forhelp [call, wnte, gor?)

2. In particular, what would you needto know to deal with the situation?Issues to probe for-' more information about what it IS:i diagnosis- consequenceswhat happens

treatment-, physician referral/or how to find

oneresearch they've heard about

-, serviceswhere to go, whatkinds of services existfinancial burden

95

Sources of Information and Assistance(When Facing Alzheimer's Disease)1. Would you be more likely to seek

help in your own community or tocontact a national organization'?

2. What if your relative lived in anothertown'? What would you do then'?

3. Have you ever heard of the NationalInstitute on Aging'?

4. How have you heard of them/Whathave you heard'? (If not it is a partof the National Institutes of Healthin Bethesda, Maryland The NIH isthe Federal agency responsIble formedical research )

5. Would you be likely to call or writeto the National Institute on Aging'?(Why or why nor))

a. Have you ever heard of theAlzheimer's Disease and RelatedDisorders Association (ADRDA)'? (Ifnot mentioned previously) (If not.explain ADRDA and its chapters )

7. How have you heard of it/What haveyou heard'?

8. If you had questions would you bemore likely to contact a nationalorganization or a local organization'?

9. Would you prefer to be able to talkwith someone (eg . a "hotline") orwould you want written information?Or would you prefer to speak withsomeone in person'? Why'?

96

Attitudes Toward Proposed Center1. The National Institute on Aging is

planning to establish a NationalAlzheimer's Disease Center TheCenter will be a national source ofinformation about Alzheimer'sDisease, including information aboutdiagnosis, treatment, conse-quences, new research and serv-ices available for its victims andtheir families What do you thinkabout this idea (the new Center)?Do you think that it is a good idea ornot'? Why?

2. What kind of information would bemost valuable for you. if you had tocope with someone who hadAlzheimer's Disease (diagnostic.treatment. medical referral. socialservices referral. support groups,respite careprobe for meaning ofrespite care)?

3. What advice would you have for theInstitute as they clan for this newCenter'?

I wiH make sure that they know aboutyour opinions

Thank you very much for yourcontributions

GatekeeperlProfessional Review QuestionnaireSource: Office of Disease Prevention and Health Promotion, USPHS

INSTRUCTIONS: Please review the attached skillsheet and then answer the questions below Returnthe completed questionnaires and skill sheets in theenclosed stamped envelope within the next threedays

Your Name

Skill Sheet Reviewed'

1. What would you say are the main messages being communicated in this skill sheet?

2. How important do you think th_oe messages are for older people'?Very importantSomewhat important

______ Not particularly important

3. In your professional opinion, are the recommendations made here appropriate for older people'?Yes No

Why?

4. !s there anything in the skill sheet that you found vague, confusing or unclear'?Yes Nn

If yes, what? (Please note in space below or directly on skill sheet.)

5. In your opinion, is there anything in the skill sheet that is inaccurate or misleading?Yes No

If yes, what? (Please note bei.ow or directly on skill sheet )

6. Do you feel there are any important points relating to this topic that have been omitted'?Yes No

If yes, what are they?

7. Is there anything in the skill sheet you think older people might find offensive'?Yes No

If yes, what?

8. Which of the following phrases best descnbes how you feel about the visuals (illustrations, photos, etc )

used in the skill sheet? (Circle one from each pair)a. Visuals effectively support a Visuals are relevant to the target

the message audienceb. Visuals do not effectively b Visuals are not relevant to the

support the message target audience

97

9. What, if anything, do you particularly like about the skill sheet?

10. What, if anything, do you particularly dislike about the skill sheet?

11. Which of the following groups (audiences) do you feel this skill sheet is addressing9 (Check one)

All adultsAll adults but especially older peopleOnly older people

12. How responsive do you feel older people will be to the message contained in the skill sheet9Very responsiveSomewhat responsive

______ Not responsiveWhy do you feel this way?

13. Which of the following phrases best describesare likely to view it?(Circle one in each group)a, Very informativeb. Somewhat informativec. Not very informative

a Very easy to understandb Fairly easy to understandc Not so easy to understand

a. Very encouraging in toneb. Somewhat encouraging in tonec. Not particularly encouraging in tone

your opinion of this skill sheet in terms of how older people

14. How could the message in this skill sheet be improved?

a Very realisticb Somewhat realisticc. Not particularly realistic

a Very appealingb Somewhat appealingc Not particularly appealing

a Very motivatingb Somewhat motivatingc Not particularly motivating

15. Based on what you know about the older target audience, are there any other comments you want to make

about this skill sheet?

THANK YOU FOR YOUR TIME PLEASE RETURN THE QUESTIONNAIRE AND SKILL SHEET TOIN THE ENCLOSED, SELF-ADDRESSED, STAMPED ENVELOPE.

98

Standard PSA Pretest Questions

1. Main Idea Communication/ComprehensionWhat was the main idea this message was trying to get across to you?

What does this message ask you to do?

What action, if any, is the message recommending that people take?

In your opinion, was there anything in the message that was confusing'?

Which of these phrases best describes the message_ Easy to understandHard to understand

2. Likes/DislikesIn your opinion, was there anything in particular that was worth remembering about the message

What, if anything, did you particularly like about the message?

Was there anything in the message that you particularly disliked or that bothered you? If yes, what?

3. BelievabilityIn your opinion, was there anything in the message that was hard to believe If yes, what'?

Which of these words or phrases best describes how you feel about the message?BelievableNot believable

29

4. Personal Relevance /InterestIn your opinion, what type of person was this message talking to

Was it talking to . .

_ Someone like meSomeone else, not me

Was it talking to ..All peopleAll people but especially (the target audience)Only (the taraet audienc

Which of these words or phrases best describes how you feel about the message'?InterestingNot interesting

InformativeNot informative

Did you learn anytning new about (health subject).from the message'? If yes, what'?

5. Other Target Audience ReactionsTarget audience reactions to messages can be assessed using pairs of words or phrases or using a 5-pointscale. The following is an example of how this is done.

Listed below are several pairs of words or phrases with the numbers 1 to 5 between them I'd like you toindicate which number best describes how you feel about the message The higher the ni. iber, the moreyou think the phrase on the right describes it. The lower the number, the more you think the phrase on theleft describes it. You could also pick any number in between Now let's go through each set of words Pleasetell me which number best describes your reaction to the message.

Too Short 1 2 3 4 5 Too LongDiscouraging 1 2 3 4 5 Encouraging

Comforting 1 2 3 4 5 AlarmingWell Done 1 2 3 4 5 Fborly Done

Not Informative 1 2 3 4 5 InformativeIs there anything in the message ttlat would bother or offend people you know9

6. Impressions of AnnouncerPlease select the one answer from each pair of phrases which describes your teeltrig abryit the announcer_ Believable

Not believable

Appropriate to the messageNot appropriate to the message

Gets the message acrossDoesn't get the message across

100

SummaryThis television PSA pretesting metho-dology is a modified version of thetheater-testing techniques used bycommercial advertisers. Here is how itworks:

Individuals typical of your PSA'stary6t audience are invited to a con-veniently located meeting room. Theroom should be set up for screening atelevision program. Participants shouldnot be told the real purpose of thisgathering; instead, that their reactionsto a television program are beingsought.

At the session, participants watch atelevision program. The program canbe any non-health, entertaining 15-30minute (approximately) videotape. Thevideotape is interrupted about halfwaythrough by a sequence of four com-mercials. Your test PSA should beinserted between the second and thirdcommercials (as indicated in thediagram below).

At the end of the program, themeeting host (you or one of your col-leagues) asks participants to write intheir answers to questions designed togauge their reactions to the program.Then the host asks about the adver-tisements participants saw. Then, yourtest PSA is played again, by itse!f, andthe host asks participants specificquestions about their reactions.

Videotape Sequence

Program Program(1st half) (2nd half)

Preparing ior the PretestPreparing involves:1. producing the PSA in rough form2. planning the pretest3. preparing the questionnaire4. recruiting respondents.

Step 1: Producing the PU In RoughFormThe three most commonly producedforms of rough messages are "ani-matics," "photomatics," and "roughlive action:' If line drawings are used,the rough PSA is called an "animatic":if photographs are used, the roughPSA is called a "photomatic." A"rough live action" message isproduced by filming or videotaping anactual run-through of the scriptusingsimplified sets, live actors, easily ac-Lessible locations, or simulated back-grounds (e.g., rear screen projection ofthe set). In each case, the videotapeincludes antudio delivery of thescript.

The storyboard serves as the blue-print for producing animatics, photo-matics or rough live action messages.When producing animatics or photo-matics, the illustrations or photographsmust be large enough and clearenough for videotaping. Each illus-tration or photograph should be atleast 9" x 12" so that the camera cancapture sufficient detail. Larger sizesalso permit camera movement (e.g.,moving left to right) within the frame tocreate a sense of motion or action.

If you produce a photomatic, usingslides projected on a screen allowsyou to create whatever size scene youwant.

YourTest PSAs

Control 7 IIYour Control 7

Commercials Test PSA Commercials1 & 2 3 & 4

The more the rough messageapproximates final production quality,the more likely the pretest results willpredict audience response accuratelythe illustrations should be realistic,the characters should look like thecharacters you plan to use in the finalspot, facial expressions should reflectthe mood and tone of the script andthe settings should be sketched indetail. If photographs are used, thepictures of people and places shouldbe clear and should resemble those tobe used in the final. Pay attention tothe setting, wardrobe, props, thecamera angle and perspective. Reviewthe processed photographs and selectthe clearest ones for producing therough message.

The video portion of an animatic ora photomatic is produced by video-taping each scene, frame by frame.Simulate motion by moving the cern-era in or out ("zooming"), left to right("panning"), or up and down. Recordthe aw portion of the rough mes-sage and then edit to exactly the rightlength.

Finally, edit the videotape "to time"(30 or 60 seconds) using the sound-track as a guide. The video and audiotracks are then "mixed" together toproduce the rough message.

101

SummaryProducing the Rough PSA

A. Animatics and Photomatics1. Develop a storyboard.2. For photomatics, arrange talent, Ir tion setting and props.3. Record video portion frame by fr,4. Record audio portion and edit it to time."5. Edit videotaped footage "to time:'6. Mix audio and video tracks.

B. Rough Live Action1. Develop a storybart.i.2. Arrange talent, location, setti.lgs and props.3. Rehearse the spa4. Videotape (with sound) the rough live action.

If you decide to videotape liveaction instead of pictures, use nonpro-fessional actors (e.g., friends or co-workers) to enact the script in a settingthat closely approximates that to beused in final production. The visualsand the sound should be recorded atthe same time. To minimize the number of times the live action must betaped, the actors should rehearse andthe production crew should be briefed.A detailed production plan (disci.IssedbeloW) will guide the crew.

How can I keep rough messageproduction costs down?Animatics, photomatics and rough liveaction PSAs z..an be produced with thehelp of a professional productioncompany, your agency's audiovisualstaff, a local television station or alocal college or university. The pro-fessional production company willprobably be the most expern..iveoption. The latter three options can beless expensive. For example, a localtelevision stal;or. may donate Rs ser-vices for producing your spot Or, atelevision production instructor mayassist you '41 no charge (except for thecost of the videotape) by makingproduction of your rough rnessage astudent assignment.

102

Tnere are several wa,control the costs and tht p. Juuctionquality of your rough PSA.

1. Create a detailed production planincluding:ID What scenes will appear on the

screen and in what order')o How long will each scene be on

the screen"O What camera movements will be

needed in each scene?o How will scenes be edited

together (e.g., fades, dissolves ordirect cuts)?

o ,AThat portions of ihe soundtrackgo with each scene,

9 Send your production plan inadvance to the person who will behelping you, and have extra cop:eson hand at the production sess:on

3. Make sure ycur scnpt and produc-tion plan for the rough message arecomplete and timed in adva;.ce.

4. Rerecord the audio track before thevisuals are shot

5. Make sure your illustrations, photo-graphs and/or slides are in the rightorder when you arrive for the pro-ductbn session.

6 Finally, remember 11-al you areproducing a -nugh message. Save"perfection' ;or final productionl

Rough message production costsrnay be reduced further by usingamateur talent, friends or co-workerswho can do a respectable job '..e.lord-ing the PSA script. These same peoplemay be used for photographs or in arough live echo^ ST:1 Shooting thevisuals and recording the soui.c.t.trlckon the same day alfio may save timeand money. Finatiy, consider contact-ing a local radio station for recordingthe soundtrackthe station mayorovide their facilities free of charge orat a lower rate than a recording studio

Animatics may be less expensiveand quicker to produce than photo-matics, since they are composed ofartists' renditions. While you rn:y he:qeto buy the artwork, you eliminate costsfor tal,...nt (actors), location sett:ng (t eobtaining clearance to rent or useproperty), props and travel HoweN.er,drawiogs may not communicate therealities or subtleties of the visualportion of the message as well asphotographs or rough live actionThese drawbacks can be minimized byiising a g 3d ffiustrak and a good,clear soundhack.

Photomatics may be expen.sive than animatics, depend.ng .iponthe cost of the I-holography. [heavailabil.ty of people who can nsent the characters, the acces,libilityof an appropriate location, and th2 rateyour photographer charges are 1,1emajor factors that affect costs. Thesecost:. :an be kept down by workingwith people who are readily avallNeand by arranging all the details iradvance The major adwarlage otproducing a photornatic over Enenimatic is that you can produi a amore realistic and believable renditionof the final message. Compare thecosts of an illustrator versus aphotographer and factor in thelogistics involved when decidingwhether you should produce ananimatic or a pholomatic

A rough live action message can bethe least expensive way to produceyour message in rough form, if:

0 you are using amateur talent whocan do a respectable run-through ofthe script;

o your script can be produced in areadily accessible location that doesnot require a lot of set-up time; and

a your production crew can videotapethe run-through with a minimumnumber of "takes."

We recommend that you videotaperather than film the rough message forpretesting becaus- editing and pro-cessing are wally less dxpensive.Home or studio video equipment canbe used to produce any of the threerough message forms.

What do 1 do once I've producedthe rough message?You need to locate a dramatic or non-health documentary videotape to serveas the program to be "reviewed" by

respondents. You may find a suitableprogram through your library, a free-loan service (ask your librarian) or youmay tape a television program (try PBSfor a program your respondents maynot have seen). You'll also need to tapefour non-health commercials fromtelevision.

The PSA pretesting program isinterrupted about halfway through by aseries of commercials. Insert twocommercials, your test PSA, then twomore commercials. Also edit your PSAonto the videotape about five secondsafter the end of the test program (seethe diagram on page 101).

Step 2: Planning the PretestThe purpose of the pretest is to deter-mine what improvements, if any,should be made before final produc-tion. To guide you in preparing thepretest questionnaire and in analyzingthe results, write a clear statement ofyour message's objectives, the targetaudience and the main point or pointsyour PSA is trying to get acro3s. In

Fraz:77''

J%

addition, identify any special concernsyou have about the PSA.

PSA producers often have certainconcerns about target audience reac-

StmumnyAdvadages and Disadvadoge of Rough PSA

Rough PSA Type Advantages Disadvantages

Animatic o Less expensive than others

o Fewer logistical arrangements

O No props required

o No location settings

o Easiest to produce

o May not approximate script

o Needs artist who can render clearline drawings

Photomatic o More realistic than animatic

o Can be inexpensive and easy toproduce if location, props, andactors are available

o More costly than anirnatic

o More logistical arrangements thananimatic

Rough Live Action o More realistic than animatic orphotomatic

o Can be inexpensive if bcation,props, and actors are readilyavailable

o Many logistical arrangements,rehearsals required

o May be more difficult to makechanges

103

tions to their messages. For example,technical words or phrases could beconfusing. Or, the characters in thespot could be perceived as unbeliev-able or inappropriate. Or the musiccould be a problem.

Now is the time to consider eachelement in the message that mayaffect audience recall, comprehensionor other reactions, and to determirewhich of these concerns should beaddressed in pretesting.

What are your specific objectives intesting this television PSA?Fill out the pretest planning form onpage 113 for the PSA you will be`..-bting. Consult this form as youdevelop your questionnaire

3loP 3: Developing the QuestionnaireTo gather useful information from thepretest, the questionnaire must becarefully constructed. It will containfour parts:

1. Program Questionsregardingaudience reactions to the televisionprogram to be presented at the testsession. All you need to do is copy thePart I questions from the SampleQuestionnaire on page 115. Use it aspage 1 of your questionnaire.

2. Recall and Main IdeaCommunicationthree standardquestions to assess your PSA's abilityto attract attention and to convey itsmain point will appear on page 2 ofyour questionnaire. Again, simply copythe Part II questions from the SampleQuestionnaire.

3. Audience Reactionssewralstandard audience reaction questions,and questions you will developspecifically for your test message.

4. Demographicsquestions torecord the characteristics of theparticipants (e.g., their sex, age, levelof education, nealth status, etc.).

Each part of the questionnaireshould be separated by a piece ofpaper telling respondents not to gofurther until they are instructed to doso.

104

SummaryPlanning the Pretest

1. Write out message objectives, intended target audience, and mainpoints.

2. Identify special concems, eg., technical words, believability music.3. Consider each message element with audience recall, comprehension

or other reactions in mind. Decide which concerns take priority

How does the questionnaire work?After participants have viewed thetelevision program and the commercialsequence, the meeting host asks thequestions about the television programon page 1 of the questionnaire. Thesequestons are asked because partici-pants were told that the purpose of thetest session is to get their reactions toa new television program. (In fact,page 1 can be discarded after thepretest session.)

After giving participants time towrite their answers on page 1, the hostreads aloud standard pretesting ques-tions (Part II) and gives participantstime to fill in their responses.

The first two questions measure"audience recall" or the ability of yourmessage to attract attention. The thirdquestion is designed to measure theextent to which your message commu-nicated its main idea or point. It is notenough for an announcement to beremembered; it must also communi-cate the key information to itsaudience.

The ideal results would be foreveryone to recall seeing your PSAand to remember its main point.(Although the recall and main ideacommunication questions will yieldinformation on the other messages inthe commercial sequence, you willdisregard that information whenanalyzing the pretest results.)

Once participants have completedpage 2, the host should finish playingthe videotape to show your PSA asecond time. The host should then askthe main idea question again. (SeePart III.) Usually more respondents will

c rtlk)

be able to describe the main ideafollowing their second exposure, sincethe message is shown by itself and,therefore, is not competing with othermessagef..

In addition to completing the mainidea question after their second expo-sure to the message, participantsshould be asked a series of questionsto gauge their reactions to yourmessage. It is up to you to decidewhich questions from the SampleQuestionnaire you need to ask.

Why 6hould you use the standardquestions?The standard recall and reactionquestions should be incorporated intothe pretest for two reasons. First, thequestions address the most importantmeasures of a message's potentialeffectivenessi.e., whether it attractsaudience attention and whether itconveys its main point.

Also, the use of standard questir,i isallows comparison, because thesequestions were used in many pretestsof television PSAs conducted by theH.'aalth Message Testing Service. Theiesults of these previous pretests canbe used as a guide to assess how well!our message communicates. (Seep3ge 110 for comparisons.)

Why do I nebd to develop additionalquestions?You should develop questions thataddress your special concerns aboutyour PSA. For example, suppose yourmessage asks viewers to call a toll-free number for more information. Youmay want to include a question that

Message Characteristics

Use of music (with or without lyrics)Use of famous spokespersonUse of telephone numberUse of mailing addressRequest for a particular actionInstructions for performing a specific health behaviorPresentation of technical or medical informationPresentation of new informationPromotion of a sponsoring organization or eventCharacters intended to be typical of the target audienceUse of a voice-over announcerPresentation of controversial or unpleasant information

asks, "What action, if any, is themessage recommending that peopletake?" A related question may be, "Didthe telepho^n number appear on thescreen long enough for you to write itdown?"

How do I know what to ask?It is best to develop one or morequestions addressing each messagecharacteristic in your PSA. The chartbelow lists various message character-istics which are commonly found inPSAs. Check these characteristicswhich apply to your test message.

Examples of questions for eachmessage characteristic listed aboveare in Part III of the sample question-naire. These questions are justexamplesadapt the questions toyour needs. Remember that theobjective of pretesting is to uncoverany problems with your PSA prior tofinal production. Any serious concernsyou have about your message shouldbe explored in pretesting.

How do I get the questionnaireready for the pretest?Once you have written your question-naire, you are ready to make copies forthe pretest participants. You will needa cover page that instructs partici-pants not to open their questionnairesuntil they are told to by the meeting

host. Place a cover sheet betweeneach part of the questionnaire thatinstructs participants not to go furtheruntil they are told to do so by themeeting host.

Make extra copies of the question-naire in case more people attend thepretesting session than you expect.For example, if you are expecting 50participants, make 60 to 65 copies ofthe questionnaire.

Step 4: Recruiting Participants for thePretestBefore recruiting respondents, identifythe target audience for your message.These are the people you wdl want torecruit.

How many participants should berecruited?This pretesting methodoiogy is de-signed to provide qualitative, diagnos-tic information Therefore, there is nostatistical procedure for determiningthe sample size The sample sizeshould be large enough, however, togwe you confidence that you havesampled a range of opinions. Areasonable and adequate sample sizeis 50 participants typical of your targetaudience.

If participants are to reflect a gen-eral audience, the sample shouldinclude a mix of men and women,

C. ;)

representing a range in age, educationand income levels. To achieve thisvariation, we recommend that parti-cipants be recruited from at least twodifferent sources. If the message isintended for both a general audienceand a specific target group (e.g.,blacks), you should recruit twosamples of 50 participants each.

To assure that 50 participantsattend, you should over-recruit. That is,for every sample of 50 participants,recruit about 75 people. This isnecessary because there are alwaysparticipants who do not show up.Calling and reminding people the daybefore the test will help the attendancerate and will help you predict anyproblems w lh low attendance. If youchoose a "captive" audience (e.g..seniors enrolled in a senior citizenscenter activities program, children inschool or PTA members at theirmeeting), you may not need to recruitindividuals, but just make arrange-ments with the organization.

An important consideration relatedto recruitment is the type of facilityand the viewing arrangements you willbe using. If you recruit all 50 partici-pants to one test session, you willneed at least two television monitorsso that everyone can see the video-tape. It may be necessary to set uptwo or more test sessions for smallerfacilities. Be sure to choose a locationthat is convenient for your ta getaudience.

How are participants recruited?One technique that has been used:uccessfully to recruit participants forpretesting health messages is to seekthe cooperation of community groupssuch as civic, social o; religiousorganizations whose members aretypical of your target audience.Schools, health clinics and work sitesmay also be contacted for assistancein recruiting participants. Theseorganizations usually have member-ship lists which can be used for

105

recruiting purposes. This is especiallyuseful if you need to recruit a specifictarget audience and you can identifyan organization whose membersmatch your needs. Also, members aremore likely to participate as a gestureof their affiliation or for the benefit oftheir organization. A contribution of$25 to $50 is an attractive incentive forgaining a community group's coopera-tion. in addition, you may want toobtain a token gift from a local mer-chant to raffle as an extra incentive. Oryou may provide small individual giftsor offer refreshments at the pretestingsession.

The following list contains theexamples of community groups withlocal affiliates throughout the countrythat you might consider contacting forrecruiting participants.

Once you have identified potentiallycooperative community groups, con-tact an official within the organization(e.g., the president or program direc-tor). While these initial contacts maybe made by telephone, you shouldfollow up your discussions with aformal written request for cooperation.The written request offers the opportu-nity to outline several impoilant points,including:

o a description of your agency ororganization

0 a description of the PSA and itspurpose ,

0 details regarding the participants tobe recruited

0 an outline of the activities involved inthe pretest

Community Organizations to Contact For Recruiting Participants

WomenYWCAB'nai B'rith WomenJunior LeagueLeague of Women VotersParent Teacher AssociationsNational Council of Catholic WomenChurch or syrogogue groupsWomen's business groupsWork sites or unionsSenior citizen centers

MenYMCAAmerican LegionElksEaglesJayceesKnights of ColumbusLions ClubLoyal Order of MooseRotary ClubVeterans of Foreign WarsLocal business groupsWork sites or unionsSenior citizen centers

106

BlacksNational Council of Negro WomenNational Urban LeagueParent Teacher AssociationsNational Association for the

Advancement of Colored PeopleChurch groupsSenior citizen centersWork sites or unions

TeenagersSchools (auditorium), or test as a

part of a class

2 incentives you are offering theorganization and/or the participants

0 a detailed explanation why theofficial should not reveal the truenature of the pretest to participants

Once you have obtained cooper-ation, decide how participants will berecruited. One possibility is to conductthe pretest as part of the group's regu-larly scheduled meeting. In this case,all members who attend the meetingwill participate in the pretest. Theadvantages of this approach are:

0 relatively little extra effort isexpended in recruiting participants

0 participants may not have to be toldabout the pretest until they arrive atthe meeting, thus minimizing thepossibility of their learning about thetrue nature of the pretest

o their regular meeting facility, aconvenient and familiar location,may be used to conduct the pretest.

If some meeting participants do notmeet your screening requirements,their responses can be discarded later.

The disadvantages of this approachare:

0 there is little control over the numberof people who will come and overthe composition of the group

0 because the pretesting session lastsapproximately one hour, it may bedifficult to place it on the agenda ofa regularly scheduled meeting

0 a number of organizations arrangetheir programs mary months aheadof time (it may be difficult to sched-ule the pretest within a reasonabletime frame).

An alternative s recruiting thegroup's members to a special meeting.In iris instance, members may need tobe contacted and asked to participateeither by someone from your agencyor by someone from the communitygroup. In either case, the membersshould be contacted in the name of oron behalf of their own group.

SummaryRequirements tor Recruiting

o You need at least 50 participants

o For a general audience, recruit a mix of nien and women, with a range inage, education and income levels.

o Over-recruit--75 participants for every 50 desired.

o For a message intended for both a general audience and a target group,recruit two samples.

o Plan two or more test sessions if necessary.

El Reffuit target audiences through civic, social or religious organizations;or through schools, clinics or work sites.

The major advantages of recruitingparticipants independently of sched-uled meetings are:

0 the opportunity to screen partici-pants on relevant characteristicsand to eliminate, in advance of thepretest, health professionals or otherrespondents who should notparticipate

0 since the entire "meeting" will bedevoted to the pretest, part:cipantfatigue is reduced.

Disadvantages of recruiting partici-pants for pretesting apart from ascheduled meeting include:

0 the amount of time and energyinvolved in contacting and remindingparticipants of the pretest

0 the necessity of locating specialfacilities for the pretest (if the regularmeeting facilities are not available).

If participants are recruited by youragency, you will have more controlover what the participants are toldabout the pretest, and you will be ableto screen potential participants tomake certain that they fit any specialaudience characteristics. However,recruiting participants takes a signif-icant amount of time, and it is possiblethat people would be more likely toparticipate if asked by someone whomthey know.

If participants are contacted by themember organization, it is essentialthat you provide the recruiter withdetailed instructions for carrying outthe task. These instructions mustinclude a written description of the"disguised" pretest, which should beread to potential participants verbatim,and a questionnaire to screen partici-pants on relevant characteristics (e.g.,how many children they have, whetherthey have high blood pressure). A setof questions that can be used forscreening participants and foridentifying the characteristics ofparticipants in the test session isincluded in Part IV of the SampleQuestionnaire.

One way to disguise the purpose (Ifthe pretest is to tell participants theyare being invited to screen a newtelevision program and that theirreactions are being sought by theproducers. No reference should bemade to the commercials or to thename of your agency, since this couldbias their response and sensitizeparticipants to the nature of theresearch before they arrive at thepretest session

9 1

In pretesting health PSAs, healthprofessionals should be excluded fromparticipating. In addition, people whoare familiar with your agency'sprogram should not participate in thepretest, since their responses are likelyto be biased.

Conducting the PretestPrior to the pretest session, go overthe checklist below to be sure thatyour session will go as smoothly aspossible. A rehearsal of the pretestingsession at your own office is anexcellent way to avoid any problems.

How do I conduct the owtastsession?The procedures to follow during thepretest session are relatively simple.The keys to a successful testingsession are:

0 having everything organized and inoperating order before the session

o being friendly and courteous toparticipants from the moment theyarrive until they leave (remember tosay "thank you")

o keeping calm and cool headedthroughout the session

o anticipating problems in advance.

The test session should take nomore than an hour and 15 minutes ifyou are organized and well prepared.(See the end of this Appendix for asample script for hosting the session )Encourage participants to take a seatas they arrive, close the doors no laterthan 10 minutes after the scheduledstarting time. When everyone isseated, introduce yourself by nameonly (assuming you are the host). Donot tell participants the name of yourorganization during the session be-cause this might bias their responsesto your test PSA. Thank participants forcoming and assure them that theevening should be enjoyableone inwhich they will have a chance to givetheir views to the producers of tele-vision program material. Discourage

107

SummarySteps for Recruiting Participants

1. Identify community groups representative of your target audience.2. Contact official in writing.3. Decide whether pretest will be held at regular meeting or at special

meeting.4. If pretest will be held at special meeting, recruit participants individually,

preferably through your own agency.

ChecklistPlanning the Pretest

1. Is the pretesting videotape ready for use? Are the video and audioportions clear?

2. Is your videotape equipmentrecorder and television monitorsfunctioning properly?

3. Is the meeting room or other facility set up? Is the room reserved foryou? Are there enough chairs? Are extra chairs available in case morepeople show up than you expect? Do you need another TV monitor sothat everyone will be able to see the program? Is the heat or airconditioning working properly? Do you know where the light switchesare? If a microphone is needed, is it set up and functioning properlY?

4. Have you made all the necessary staffing arrangements? Are yourassistants definitely coming to the session? Do they have transportationand do they have directions for getting there?

5. Have you made enough copies of your pretest questionnaire? Is eachquestionnaire complete (with no pages missing)? Do you have pencilsfor participants? Will they need clipboards or pads?

6. Has participant recruitment taken place as scheduled? Did you call andremind them to attend? Do they have transportation and directions?

7. Has the host rehearsed?

participants from talking to oneanother during the session. Tell themyou are interested in their own par-ticular views and that there are noright or wrong answers. Also,encourage them to write their answersclearly in the space provided.

After your introductory remarks,have your assistants hand out thequestionnaires, pencils and clipboards(if needed). Turn on the video recorderand monitor to begin the test session.

Be attentive and watch for anyproblems with the sound or picture on

108

the monitor. Be sure that the equip-ment is functioning properly through-out the program.

Be prepared to stop the recorderwhen the television program hasended. Introduce the questions,thanking the group for their help thusfar. Tell them to open their question-naires to the questions on the firstpage. Read each question and giverespondents time to fill in theiranswers.

1 t) 4.,

When the participants are throughwith page 1, tell them that before theyleave, you would like to gather theirreactic is to the announcements thatwere shown during the program. Havethem turn to page 2 and instruct themin how to fill out the questions aboutthe advertisements. When they havecompleted these questions, tell thegroup that you have been asked toobtain their reactions to one of the adsin particularthe public serviceannouncement.

Start the recorder. Be sure there isnot too much lead tape before themessage starts, to avoid an awkwardpause in the session's pace. After yourPSA has been played, tell participantsto turn to the next page of the ques-tionnaire and to write their answers tothe remaining questions. Encouragethem to answer every question and toavoid giving more than one answer,except when this option is indicated onthe questionnaire.

It is a good idea to circulate throughthe room to monitor progress and tobe sure participants are not discussingtheir responses. Collect the question-naires as participants finish.

Thank participants for their coopera-tion. If you have token gifts (e.g.,pencils, cent-off coupons, half-pricetickets or other tavors from localmerchants), distribute them to partici-pants as they leave. Otherwise,mention that you hope the groupenjoys the donation you have made totheir organization.

Congratulations on conducting asuccessful pretest!

SummaryConducting the Pretest

1. Be well prepared and organized.2. Introduce yourself by name only, and express your interest in indivi-

dual responses to television program material.3. Pass out questionnaires, pencils and clipboards.4. Play videotaped program, watching for any sound or picture problems.5. Read each question on page 1 of the questionnaire, and allow time for

participants to fill in answers.6. Have respondents fill in page 2, following your instructions.7. Play the PSA.8. Have participants fill in remainder of questionnaire.9. Collect questionnaire.

10. Give out gifts or favors.11. Say thanks and mention your donation to their group.

Analyzing the Pretest ResultsAnalysis of the questionnaires involvestwo steps. First, tabulate or count howmany participants gave each possibleresponse to each question. Next, lookfor patterns in the responses andextract meaning from the numbers.

How do I tabulate closed-endedquestions?Closed-ended questions orceparticipants to select a response fromseveral alternatives. A quick methodfor tabulating or counting the numberof participants who selected eachpossible response to each question isto use a blank questionnaire:

1. Take thP first questionnaire andrecord the answers to each closed-ended question by making a checkmark in the right hand column nextto the appropriate response.

2. Repeat this procedure for everyquestionnaire.

3 Tally the total number of checkmarks and then calculate thepercentage of participants whogave each response.

How do I tabulate open-endedquestions?Tabulating or counting responses toopen-ended questions is more time-consuming. Open-ended questionsallow participants to expressthemselves in their own words. Forexample:

c What did the message say?

0 W`at did the message show'?

0 What was the main idea themessage was trying to get across?

The easiest way to analyze thesequestions is to write each question atthe top of a separate blank page (the"coding sheet"). Since participants areanswering in their own words, the firststep is to group the responses to eachquestion into categories.

Group the responses to each ques-tion into three categories: correct,partially correct and incorrect. Whencategorizing the audience recall andmain idea responses, use the PSAscript and your own statement of themessage's main points to guide theanalysis. For some open-ended ques-tions (e.g., "What, if anything, did youlike about the message?" or "What, ifanything, did you dislike ?"), you maywish to classify the responses asfavorable or unfavorable.

k

How do I interpret the pretestresults?Once you have tabulated the re-sponses to each question, your inter-pretation should be guided by theobjectives you stated for your mes-sage on your pretest planning form

Guidelines for interpreting re-sponses to standard pretestingquestions are listed on page 110.These guidelines are in the form ofhigh, average, and low percentages (orscores) for certain standard pretestingquestions.

The ranges are based on the resultsof 63 television PSA pretests. They canbe helpful in comparing your PSApretest to previous pretest results. Thiscomparison should not be the solebasis of your analysis. Rather, theranges should serve as guidelines.They are designed to help you extractmeaning from the percentages youcompute for your test PSA.

Ideally, your PSA pretest results willfall in the average or the high range,and not the low range. One or two lowscores do not necessarily dictaterevising the PSA. Prirr to drawing anyconclusions or mak.ng recommenda-tions, you should analyze the resultsfor every euesticn.

This book cannot provide ranges orguidelines for the questions whichrelate specifically to your test PSA.Therefore, you will have to interpret thepercentages on your own. Ask yourselfhow critical it is for the majority ofparticipants to respond in a certainmanner to each question. For example,if the test PSA communicates the mainidea information through music, it isvery important that a large majorityreport being able to understand thewords to the music. Otherwise, the testPSA will not meet its communicationobjectives. On the other hand, d themusic in the PSA serves as back-ground sound, then it is not as criticalfor the lyrics to be understood.

It is important to examine how manyrespcnses to open-ended questionsfall into each category (correct,partially correct or incorrect, favorable

109

Table I. Guidelines for Interim** Responses to Standard Pretesting Queetions

High Score Range Average Score Range Low Score Range

Attention, Recall(percent remembering seeing message afterone exposure)

41 or higher 30-40 29 or lower

MaM Idea(percent remembering main idea of messageafter one exposure)


%Nth Remembering(percent indicating "yes")


PersomMyRelevant(percent indicating message is talking tosomeone like themselves)


Anything Confusing(percent indicating "yes")

9 or lower 1020 21 or higher

Believable 91 or higher 75-90 74 or lower

Well Done 66 or higher 50-65 49 or lower

Convincing 71 or higher 55-70 54 or lower

Informative 76 or higher 60-75 59 or lower

Made Its Point 91 or higher 75-90 74 or lower

!Mending 66 or higher 50-65 49 or lower

Pleasant 66 or higher 50-65 49 or lower

or unfavorable). Ideally, the majaity ofresponses will fall into the correctcategory. This pattern tends to confirmthat the test PSA is meeting its objec-tives. On the other hand, if the majorityof responses are categorized as par-tially correct, or incorrect, the PSAmay not be meeting your communica-tion objectives.

At this point, look at the overallresults:

0 What was learned from the pretest?

0 Did the PSA meet its communicationobjectives?

0 What are the PSA's strengths?Weaknesses?

0 Did the PSA receive a favorable andappropriate audience reaction?

110

0 Did any answers to any particularquestion stand out?

0 Should the PSA be revised?

The answers to these questions canbe found in your analysis However, itis no longer appropriate to look ateach individual question

Can I apply any statisticaltechniques to the data?For example, can the findings beanalyzed in terms of males vs.females, or younger participants vs.older ones? The answer is yes butonly if you boost the sample size to 50participants from each group If yourecruit only 50 participants in total, thesample will be too small to bereviewed by subgroup. Comparing

findings between small subgroupscould be misleading and may result infaulty conclusions.

Do not try to make generalizationsfrom the results of this pretest to PSAmessages in general. Each PSApretest you conduct will providedirection for making changes thatmight be necessary prior to finalproduction and for improving thepotential effectiveness of eachparticular message. The sample sizeand the methodology used do notallow for formulating definitiveconclusions about what yourmessage's impact will be after it isdistributed.

SummaryAnalyzing the Pretest

1. Tabulate how many participants gave each possible response to eachquestion.a. Use check marks and percentiges for closed-ended questions.b. Use categories ("correct/partially correct/incorrect" or "favorable/

unfavorable") and jrrcentages for open-ended questions.

2. Compare the results with HMTS pretest results, but use these as guide-lines only.

3. Look for patterns across questions.

4. Compare resuits with your original objectives.

5. Decide whether to recommend changing the PSA or to keep itessentially the same.

Pretest Budget and Timetable

Timetable* Budget**

Out-of.Week Task Pocket Costs

1 Produce rough message and transfer to pretest videotape $1.00012 Contact community organizations for recruiting participants 100***

2 Develop questionnaire2 Begin recruiting participants3 Identify and reserve faciIity for pretesting session 503 Rent video equipment 1504 Pretest and revise questionnaire5 Copy questionnaires 20

-5 Conduct pretesting session(s)6 Analyze results and write report7 Make script changes based on pretesting

---$1,320

*The timetable presented here assumes that a health professional will have the assistance of a secretary and that the communityorganization is handling participant recruitment The variable most likely to affect your pretest timetaPle is the amount of time neededto recruit participants

"The out-of-pocket costs are estimates and do not include the cost of your time working on the pretest Costs will vary dependingupvil your rough message production costs, the number of community organizations you work with, whether or not you need to rent afacility or equipment, and your internal resources

***Reflects a donation of $2 per participant recruited for the pretest

1 ,=. LI

111

Pretest Planning Form

1. What is the communication objective of my PSA?

2. Who is the target audience?

3. What is the main or overall point I am trying to communicate to my audience with this spot?

4. What other important points do I want the viewer to remember from this PSA? (List these in order ofimportance below.)

5 What special concerns do I have about this PSA (e.g., reactions to the characters, music, technical words,setting, etc.)?

10 G 113

Sample PSA Pretest Questionnaire

Part IThank you for watching this program. One of the reasons it was shown tonight was to get your reaction to ittosae what parts you liked and what parts you didn't like.

1. Was there any part of the program that you especially liked?

2. Was there anything about the program that you disliked?

3. Please indicate your overall reaction to the program by circling or,e of the phrases below:

a. A great program, would like to see it againb. A pretty good programc. Just soso, like a million othersd. Another bad program

4. Would you recommend the program to your friends? Why or why not?

Part HFor each commercial that you remember seeing, please write down what the message said and what themessage showed on the dotted lines Write down the main idea each message was trying to get across on thesolid lines.

a) What did the message saY? What was the main idea each message was trying tob) What did the message show? get across?

1. a)

b)

2. a)

b)

115

3. a)

b)

4. a)

b) .

5. a)

b)

Part IIIListed below are the standard reaction questions we recommend you include in your questionnaire

1. This evening you saw a public service announcement. Now that you have seen the message twice, pleasetell us what you think was the mp'n idea of the message?

2. In your opinion, was there anything in particular that was worth remembering about the message?

a. Yesb. No

2a. If yes, what was worth remembering?

3. In your opinion, what type of person was this message talking to?

a. Someone like meb. Someone else, not me

3a. If someone else, why?

4. In your opinion, was there anything in the message that was confusing or hard to understand?

a. Yes

b. No

4a. If yes, what was confusing?

111

108

5 We would now like you to describe thE xiblic service announcement. From each pair of words or phrases,please circle the one which you feel best describes the message.a. Believableb. Not believable

a. Well doneb. Not well done

a. Convincingb. Not convincing

a. Informativeb. Not informative

a. Made its pointb. Didn't make its point

a. Interestingb. Not interesti,a. Pleasantb. Not pleasant

Following are sample questions for each message characteristic discused in Chapter 2. Feel free to adaptthese questions to your needs, or develop your own questions. Make sure your pretest questionnaire covers allaspects of your PSA.

Use of Music:1 Please circle the one answer from each pair which better describes your feelings about the music in the

message.

a. Appropriate to the messageb. Not appropriate to the message

a. Effective in getting the message acrossb. Not effective in getting the message across

a. Could understand the words to the musicb. Could not understand the words to the music

2. Overall, how would you describe the music in the announcement?

a. The music fit the messageb. The music did not fit the messagec. I don't remember the music

Use of Famous Epokesperson:1 Which of the following best describes

pubfic service message?

a. Singerb. Actorr. Comediana. Athletee. Don't know

2 Please circle the one answer from each pair of phrases which better describes your feehngs about theannouncer, (name of spokesperson).a. Believableb. Not believable


a Gets the message acrossb. Does not get the message across

(name of spokesperson), the announcer in the

109 117

Use of Telephone Number or Address:1. The phone number (or address) was on the screen long enough for me ti, ,emember it o write it down.

a. Agreeb. Disagreec. Neither agree nor disagree

Request for a Particular Action:1. What did the announcement ask you to do?

Instructions for Performing a Specific Health Behavior:1. Please circle one answer from each pair of phrases whicib better describes your feelings about the

instructions regarding (fill in behavior) in the announcement.

a. Clear and easy to understandb. Confusing, hard to understand

a. I would be able to perform (fill in behavior) after seeing this arnouncement.

b. I would not be able to perform (fill in oehavior) after seeing tiis

announcer- it.

Presentation of Technical or Medical information:1. The announcement presented technical (or medical) information. Please select one answer from each pair of

pnrases which better describes your feelings about the information.

a. The announcement did a good job of presenting technical information.b. The announcement did a poor job of presenting technical information.

a. I understood all the terms in the announcement.b. I had difficulty understanding the terms in the announcement.

Presentation of New Information:1. How much, if any, of the information in the announcement was new to you?

a. All of itb. Most of itc. Some of itd. None of it

2. Overall, how useful was the information in the announcement to you?

a. Very usefulb. Somewhat usefulc. Not very usefuld. Not usefil at alle. Don'i Know/not sure

118

Promotion of a Sponsoring Organization:1 From among the following choices, please indicate the organization which sponsored this message. (Include

your organization and fill in appropriate alternatives.)

a. CAREb. Baylor College of Medicinec. Save The Children Foundationd. Don't know/Not sure

Characters Who Are Supposed to be Typical of the Target Audience:(The word "characters" in these questions could be substituted with "man," "woman," "family," "children,"etc.)

1. Which of the following statements better describes the characters in the announcement?

a. The characters in the announcement reminded me of people I know.b. The characters in the announcement did not remind me of people I know.

2 Overall, how would you describe the characters in the announcement'? Please select one response fromeach group.

a. Realisticb. Not realistic

a. Helped me to understand the messageb. Did not help me to understand

3 Overall, how would you describ 'he characters in the announcement? Please select one answer from eachgroup.

a. Appealingb. Not appealing

a. Get the message acrossb. Do not get the message across

a. Believableb. Not believable

a. Easy to understandb. Not easy to understand

Use of a Voice-over Announcer:1. Please circle the one answer from each pair of phrases which better descrpes your feelings about the

announcer.

a. Believableb. Not believable


a. Gets the message acrossb. Does not get the message across

a. Easy to understandb. Hard to understand

1 1 1,..

119

Presentation of Controversial or Unpleasant Information:1. Some people have mentioned different feelings they had during or after watching the announcement Please

circle the opinion which comes closest to yours.

a. The announcement made me uncomfortable and I had difficulty paying attention to it.b. The announcement interested me so I paid attention to it.c. I had no particular feeling about the announcement.

2. Overall, how do you think most people would feel about this announcement if they saw it on television at

home?

a. Suitable to show on television at any timeb. Suitable to show, but only at certain timesc. Not suitable to show at any time

Listed below are sample questions to be used for identifying the characteristics of your pretest participantsSome of these questions also may be used as screening questions to recruit participants from a specific targetgroup.

1 What is your sex?

a. Maleb. Female

2. What is your age?

a. Under 18b. 18-24c. 25-34d. 35-44e. 45-49f. 50-54g. 55-60h. Over 60

3. How far did you go in school?

a. Eighth grade or lessb. Some high schoolc. High school graduated. Some collegee. College graduate

120

4. Do you have children?

a. Yes (go to question 4a)b. No (go to question 5)

4a.Please circle the age categories in which your children belong. Circle as many as apply.a. 1-5 years oldb. 6-10 years oldc. 11-15 years oldd. 16-20 years olde. 21 and over

5. Which of the following statements best describes you?

a. I currently smoke.b. I used to smoke, but have now stopped.c. I have never smoked.

6. Have you ever been told by a doctor or a nurse that you have ...a. Heart diseaseb. High blood pressurec. Cancerd. Emphyzemae. Other

7. To the best of yo'ir knowledge, have you ever been exposed to ...a. Asbestosb. Other toxic chemicalsc. Etc.

8. Which of the following best describes your race or ethnic background?a. Whiteb. Blackc. Hispanicd Asiane. Other

1 1 3121

Script for Hosting the Theater Test

IntroductionGood evening. I'm and I'd like to thank you all for coming today (tonight).I think we're all going to have a good time. Just a few more words of introduction and we'll get started. We'veasked you here because we feel that it's very important to get your ,s about new television programmaterial. So consider this your chance to give the TV program prod your opinions ... before the programgets to your home screens.

Keep in mind that what we're interested in is your own personal views. We don't want you to tell us what youthink we want to hear or what your spouse thinks or anyone elsewe need your own opinions.

There are no right answers and no wrong answers. So, please don't discuss the program or your answerswith the people around you.

Also, please make sure that you write your answers clearly in the space provided on the questionnaire we'vegiven you, and be sure that you don't move to each new section of the questionnaire until I tell you to.

Do you have any questior.s about this procedure? I'll answer any other questions you have at the end of thetest session. Okay? Now enjoy the show.

(Play the pretesting program videotape)

Reactions to the ProgramAll right. Now we'd like to find out about your reactions to the programwhat parts you liked and what partsyou didn't like. Now, I'd like you to turn to the first page of the questionnaire and answer the questions. Pleasedon't go on to the next page until I tell you to.

1 1 4 123

Attention. A pretesting measure todescribe a message's ability to attractlistener or viewer attention, this isoften called "recall."

Attitudes. An individual's predis-positions toward an object, person orgroup, which influence his or herresponse to be either positive ornegative, favorable or unfavorable, etc.

Baseline study. The collection andanalysis of data regarding a targetaudience or situation prior to inter-vention.

Central location interceptinterviews. Interviews conducted withrespondents who are stopped at ahighly trafficked location that isfrequented by individuals typical of thedesired target audience.

Channel. The route of messagedelivery (e.g., mass media, community.interpersonal)

Closed-ended questions. Questionsthat provide respondents with a list ofpossible answers from which tochoose, 2' .lo called multiple choicequesticls.

Communication concepts. Rough artwork and statements that convey theidea for a full message.

Communication strategystatement. A written statement thatincludes program objectives, tergetaudiences, an understanding of theinformation needs and perceptions ofeach target audience, what actionsthey should take, the reasons why theyshould act and the benefits to begained. This document provides thedirection and consistency for allprogram messages and materials.

Comparison group. A control grouprandomly selected and matched to thetarget population according tocharacteristics identified in the studyto permit a comparison of changesbetween those who receive theintervention and those who do not.

Comprehension. A pretestingmeasure to determine whethermessages are clearly understood.

Convenience samples. Samples thatconsist of respondents who are typicalof the target audience and who areeasily accessible; not statisticallyprojectable to the entire populationbeing studied.

Diagnostic informrAiIon. Results frompretesting research that indicate thestrengths and weaknesses inmessages and materials.

Focus group interviews. A type ofqualitative research in which anexperienced moderator leads about 8to 10 respondents through adiscussion of a selected topic, allowingthem to talk freely and spontaneously.

Formative evaluation. Evaluativeresearch conducted during programdevelopment. May include state-of-the-art reviews, pretesting messages andmaterials, and pilot testing a programon a small scale before full imple-mentation.

Frequency. In advertising, is used todescribe the average number of timesan audience is exposed to a specificmedia message.

Gatekeeper. Someone you must workwith before you can reach a targetaudience (e.g., a schoolteacher) oraccomplish a task (e.g., a televisionpublic service director)

Goal. The overall improvement theprogram will z,trive to create.

Impact Evaluation. Research de-signed to identify whether and to whateoent a program contributed toaccomplishing its stated goals (here,more global than outcome evalua-tion).

In-depth interviews. A form ofqualitative research consisting ofintensive interviews to find out howpeople think and what they feel abouta given topic.

1 1 5

Intermediaries. Organizations, suchas professional, industrial, civic, socialor fraternal groups, that act aschannels for distributing programmessages and materials to membersof the desired target audience.

Objective. A quantifiable statement ofa desired program achievementnecessary to reach a program goal.

Open-ended questions. Questionsthat allow an individual to respondfreely in his or her own words.

Outcome evaluation. Researchdesigned to account for a program'saccomplishments and effectiveness;also called "impact" evaluation.

Over-recruiting. Recruiting morerespondents than required tocompensate for expected "no-shows."

Polysyllabic words. Words thatconten three or more syllables.

Pretesting. A type of formativeresearch that involves systematicallygathering target audience reactions tomessages and materials before theyare produced in final form.

Probe. Interviewer technique used tosolicit additional information about aquestion or issue. Probe should beneutral (e.g., "What else can you tellme about ?") not directive ("Doyou think the pamphlet was suggestingthat you take a particular stepsuchas changing your diet?")

Process evaluation. Evaluation tostudy the functioning of components ofprogram implementation; includesassessments of whether materials arebeing distributed to the right peopleand in what quantities, whether and towhat extent program activities areoccurnng, and other measures of howand how well the program is working

PSA. Public service announcement,used without charge by the media.

125

Qualitative research. Research thatis subjective in that it involves ob-taining information about feelings andimpressions from small numbers ofrespondents. The information gatheredusually should not be described innumerical terms, and generalizationsabout the target populations should notbe made

Quantitative research. Research de-signed to gather objective informationfrom representative, random samplesof respondents; results are expressedin numerical terms (e.g., 35 Percentare aware of X and 65 percent arenot) Quantitative data are used todraw conclusions about the targetaudience.

Random sample. A sample ofrespondents in which every individualof the population has had an equalchance of being included in thesample.

126

Reach. In advertising, used todescribe the number of differentpeople or households exposed to aspecific media message during aspecific period of time.

Readability testing. Using a formulato predict the approximate readinggrade level a person must haveachieved in order to understandwritten material

Recall. In pretesting, used to describethe extent to which respondentsremember seeing or hearing amessage that was shown in a com-petitive media environment, usuallycenters on main idea or copy pointrecall.

Segmentation. Subdividing an overallpopulation ,nto homogeneous subsetsof target a, diences in order to betterdescribe and understand a segment,predict behavior, and formulate tailoredmessages and programs to meetspecific needs. Segments may bedemographic (eg., age, sex, education,family life cycle), geographic (e.g.,Southeastern, U.S., rural, north side oftown) or psychographic (e.g.,personality, lifestyle, usage patterns,risk factors, benefits sought).

Self-administered questionnaires.Questionnaires that are filled out byrespondents themselves; maileddirectly to the respondent, or filled outby respondents gathered at a centrallocation

V G

Social marketing. A discipline thataddresses an issue with particularregard to those affected by it (thetarget audience), considering theirperspectives and perceived wants andneeds to develop strategies towardchange.

Summative evaluation. Evaluationconducted to identify a program'saccomplishments and effectiveness,also called outcome or impactevaluation.

Target audience. The desired orintended audience for program mes-sages and materials (see segmenta-tion). The primary target avdienceconsists of those individuals theprogram is designed to affect. Thesecondary target audience is thatgroup (or groups) that can help reachor influence the primary audience

Alcohol, Drug Abuse and Mental Health Administration,Communicating with Youth about Alcohol Methods, Messagesand Materials, DHHS Publication No (ADM) 86-1429, 1986

Alexrod, Myril D "10 Essentials for Good Qualitative Research,"Marketing News, March 14. 1975, pp 10-11.

After, William F., "Staying Ahve in the 20th Century," Science85, October 1985, pp 31-41

American Institutes for Research "Six Graphics GuidelinesSimply Stated, 30, October 1982, pp. 1-4.

American Marketing Association, The Marketing ServicesGuide, Chicago, published yearly.

Atkin, Charles, "Research Evidence on Mass Mediated HealthCommunication Campa.gns," in D Nimmo (Ed ), Communica-tion Yearbook 3, New Brunswick, NJ Transaction-InternationalCommunication Association, 1979, pp. 655-668

Bandura, A , Social Foundations of Thought and Action.Englewood Cliffs, NJ Prentice-Hall, Inc , 1986

Basch, Charles E , "Focus Group Interview An UnderutilizedResearch Technique for Improving Theory and Practice inHealth Education," Health Education Quarterly, Winter 1987,14(4), pp 411-448

Bellenger, Danny N., Bernhardt, Kenneth L., and Goldstucker,Jac L , Qualitative Research in Marketing, Chicago. Ar. ericanMarketing Association, Monograph Series #3, 1976.

Bertrand, Jane T, Communications Pretesting, Community andFamily Study Center, Media Monograph 6, University ofChicago, 1978.

Bloom. PN , "Evaluating Social Marketing Programs Problemsand Prospects," in R P Bagozzi et al (Eds ), Marketing in the80's Changes and Challenges, Chicago American MarketingAssociation, 1980, pp 460-463

Burrelle's Media Directories, Burrelle's Special, -)ups MediaDirectory, Update, Livingston, NJ Burrelle's Mecha Directories.updated periodically

Cable TV Publicity Outlets, Cable TV Publicity OutletsNationwide, Washington Depot, CT Cable TV Publicity Outlets,updated regularly

Childers, Terry L., and Ferrell, OC., "Response Rates andPerceived Questionnaire Length in Mail Surveys," Journal ofMarketing Research, 16, August 1979, pp 429-431

Chisnall, Peter M , Marketing Research, New York McGraw Hill,1986.

Covello, Vincent T, Detlof von Winterfeldt, and Slovic, Paul,'Communicating Scientific Information About Health andEnvironmental Risks Problems and Opportunities from a Socialand Behavioral Perspective," Uncertainties in Risk Assessmentand Risk Management, edted by V Covello, A Moghisi, andV.R.R. Uppuluri, New York: Plenum Press, 1986.

Department of Health and Human Services, PromotingHealth/Preventing Disease Objectives for the Nation,Washington, DC, Fall 1980

Dessert, G., More Than You Want to Know About PSA's,Needham, MA The National Broadcast Association forCommunity Affairs, 1982

Doak, Ceciha C., Leonard, G., and Root, Jane H., TeachingPatients with Low Literacy Skills, Philadelphia. J.B. Lippincott,Co., 1985.

Environmental Protection Agency, Be A Better Writer: A Manualfor EPA Employees, March 1980.

Erdos, Paul L., Professional Mail Surveys, Malabar KriegerPublishing Co , 1983

Fink, Arlene and Kosecoff, Jacqueline, An Evaluation Primer,Beverly Hi'ls. Sage Publications, 1978

Fishbein Martin and Nzen, I., Belief, Attitude, Intention, andBehavior, Reading, MA. Addison-Wesley, 1975.

Fitz-Gibbon Carol Taylor and Morris, Lynn Lyons, How to Designa Program Evaluation, Beverly Hills. Sage Publications, 1978.

Flay, Brian F.., "Evaluation of the Development, Disseminationand Effectiveness of Mass Media Health Programming," HealthEducation Research, 2(2), 1987, pp. 123-129.

Fox, Karen F.A. and Philip Kotler, "The Marketing of SocialCauses The First 10 Years," Journal of Marketing, Fall 1980, pp24-33.

Freimuth, Vicki S , "Assessing the Readability of HealthEducation Messages," Public Health Reports, 94, November-December 1979, pp. 568-570.

Freimuth, VS., Greenberg, R.H , Romano, R M., and Wagner, U ,"Health Public Service Announcements. Assessing TheirPotential Effectiveness Through Standardized Pretesting," in D.S.Leathar, G B. Hastings, and J.K. Davies (Eds.) Health Educationand the Media, Oxford. Pergamon Press, 1981.

French, John F., Court C. Fisher, Samuel J. Costa, Jr. (Ed.)Working with Evaluators, A Guide for Drug Abuse PreventionProgram Managers, U.S. Department of Health and HumanServices, Alcohol, Drug Abuse and Mental Health Administra-tion, DHHS Pubhcation No (ADM) 83-1233, 1983

Funkhouser, Judith E., "Before the Cameras Turn. The ResearchBase of the Youth Prevention Campaign," AlcoholHealth and Research World, 2(4) Summer, 1987.

Gannett Newspapers, Media Market Guide, New York. GannettNewspapers, updated regularly.

General Mills, American Family Report 1978-1979. FamilyHealth in an Era of Stress, Minneapolis, MN General Mills,1979.

General Mills, A Summary Report on U.S ConsumerKnowledge, Attitudes and Practices about Nutrition,Minneapolis, MN. General Mills, 1979.

Getting the Word Out, Wisconsin Clearinghouse, 1954 EastWashington Avenue, Madison, WS 53704-5291, 1979.

Goldman, Alfred E and McDonald, Susan Schwartz, The GroupDepth Interview, Principles and Practice, Englewood Cliffs, NJ.Prentice-Hall, Inc , 1985.

127

Gombeski, W.R , Moore, TJ Contant. C F , et al HealthInformation Sources of the Poorly Informed Implications forHealth Educators and Communicators. Health ValuesAchieving High Level Wellness 5(5), 1981, pp 199-206

Green, Lawrence W , Kreuter. Marshall W , Deeds, Sigrid G , andPatridge. Kay B. Health Education Planning A DiagnosticApproach, Palo Alto Mayfield Publishing Company, 1980

Green. Lawrence W . Lewis. Franoes M . Measurement andEvaluatiota la Health Education and Health Promotion. Palo AltoMayfield Publishing Company, 1986

Gross. G M . Active Communities, Healthy Families, Madison,WI Center fo, Public Representation, September 1987

Guraiing. R , "The Fog Index After Twent- Years," The Journal ofBusiness Communication, 6, Winter 1968, pp 313.

Gunning. R , The Technique of Clear Writing (rev ed ), New YorkMcGrawHill, 1968

Hawkins, J David and Britt Nederhood. Handbook forEvaluating Drug and Alcohol Prevention Programs. U SDepartment of Health and Human Services, Alcohol, Drug andMental Health Administration, DHHS Publication No (ADM)87-1512, 1987

Higginbotham. James B , and Cox, Keith K , Focus GroupInierviews a Reader, Chicago American Marketing Association.1979

Holbert. Neil, Advertising Research. Chicago AmericanMarketing Association, Monograph Senes #1, 1975

Institute for Social Researt, The University of Michigan, AnnArbor, Michigan Interviewers Manual, Ann Arbor Institute forSocial Research, 1978

Jamieson, Kathleen Hall and Campbell. Karlyn Kohrs, Interplayof Influence Med'a and Their Publics in Needs. Aoertising andPolitics, Belmont, '',A Wadsworth, 1987

Jordin, Lewis (editor) The New York Times Manual of Style andUsage New York The New York Times Book Company,December 1977

Kotler, Phtlip, and Alan R Andreasen, Strategic Marketing forNonprofit Organizations, Engelwood Cliffs, NJ PrenticeHall,Inc 1987

Kotler, Philip, OC Ferrell and Charles Lamb, Strategic Marketingfor Nonprofit Organizations, Cases and Readings. EnglewoodCliffs, NJ PrenticeHall, Inc . 1987

Lazes, Peter M . Kaplan, Laura H , Gordon, Karen A . TheHandbook of Health Education (2nd ed ), Rockville AspenPublishers, Inc . 1987

Leathar, DS . Hastings, G B , and Davies. J K (Eds ), HealthEducation and the Media, Oxford Pergamon Press, 1981

Lipstein, Benjamin and McGuire, William J , EvaluatingAdvertising A Bibliography of the Communications Frocesc,New York Advertising Research Foundation, 1978

Lovelock, Christopher !-I , Marketing for Public and NonprofitManagers, New York John Wiley & Sons, 1984

128

McAlister, A , Puska. P. Salonen, J I , Tuomilehto J , andKoskela. K., "Theory and Action for Health PromotionIllustrations from the North Karelia Project," American Journalof Public Health 72(1), January 1982, pp 43-50

McGuire W. "Theoretical Foundations of Public CommunicationCampaigns," in R E Rice and WJ Paisley (Eds ), PublicCommunication Campaigns. Beverly Hills Sage Publications,1981. pp 4170

MclOuail, Denis and Sven Wendanl, Communications Models forthe Study of Mass Communications, New York Longmam. Inc ,1982.

Mendelsohn, H.. "lessons from a National Media PreventionCampaigr In DS Leathar, G B Hastings. K M O'Reilly, & J KDavies (Eds ). Health Education and the Media II, New YorkPergamon Press, 1986. pp 133-136.

Merck, Sharp and Dohme, Working with the News Media AGuide for Health Care Organizations, West Point. PA Merck,Sharp and Dohme, 1981,

Miaoules, George and John Bonaguro, "Marketing Strategies inHealth Education," Journal of Health Care Marketing, 1(1),Winter 1980-81. pp 35.44

Morns. Lynn Lyons and Fitz-Gibbon. Carol Taylor, How toMeasure Program Implementation, Beverly Hills SagePublications. 1978

Morris, Lynn Lyons and Fitz-Gibbon, Carol Taylor. How toPresent an Eva uation Report, Beverly Hills Sage Publications,1978

Mullen, Patric a D , Hersey, James C . Iverson, Donald C ."Health Behavior Moaels Compared," Social Science andMedicine, 24(2), 1987. pp 973981

National Cancer Institute, Breast Cancer A Measure ofProgress in Public Understanding. Technical Report, DHHSPublication No (NIH) ol-2291. Bethesda, MD Public HealthService, October 1980

National Cancer Institute, Cancer Prevention Awareness Survey(Technical Report, WAVE II), NIH Publication No 87-2907, July1987

National Digestive Disease Education Program, A NationalSurvey of Public Perceptions of Digestive Health and Disease.1984, contact Dr Brent D Ruben, Dept ot Communication,Rutgers. New Brunswick, NJ 08903

National Heart, Lung, and Blood Institute. Churches as anAvenue to High Blood Pressure Control. NIH Publication No 87-2725, October 1987

National Heart, Lung. and Blood Institute, Community Guide toHigh Blood Pressure Control, NIH Pubhcation No 822333, May19/K

National Heart, Lung, and Blood Institute, Measuring Progressin High Blood Pressure Control, An Evaluation Handbook. NIHPublication No 862647, April 1986

National Heart, Lung. and Blood Institute, Strategies forDiffusing Health Information to Minority Populations, September1987

1 1 '

National Heart, Lung, and Blood Institute. With Every Beat ofYour Heart, An Ideabook for Community Heart HealthPrograms, NIH publication No 872641, September 1987

National Library Service for the Blind and PhysicallyHandicapped, Reaching People, Wasnington, DC Library ofCongress, 1980

Novell!, W.D . "You Can Produce Effective PSA&" PublicRelations Journal 38(5), 1982. pp 30-32

ODPHP Health Information Center, Working with the Print Media(for the Healthy Older People Program), Pubhcation No Y0182,$2.00.

Office of Substance Abuse Prevention. Literature Review onAlcohol and Youth Alcohol Drug Abuse and Mental HealthAdministration, DHHS, November. 1987

Ogilvy. D.. and Raphaelson, J., "Research AdvertisingTechniques That Workand Don't Work:* Harvard BusinessReview, July-Augost 1982. pp 14-18.

Pacific Mutual Life Insurance Company, Health MaintenanceSurvey, Newport Beach, CA Pacific Mutual Life InsuranceCompany, 1978

Palmer. E . "Shaping Persuasive Messages with FormativeResearch:* in RE Rice and WJ Paisley (Eds ), PublicCommunication Campaigns, Beverly Hills Sage Publications,1981. pp 227-338.

Papazian, E., The Media Book, New York The Media Book,1980.

Patton. tv,ichael Quinn. Qualitative Evaluation Methods, BeverlyHills: Sage Publications, 1980

veter Glenn Publications. National Radio Publicity Directory,New York: Peter Glenn Pubhcations. 1982

Rasmuson. Mark R . Renata E. Seidel, William A Smith andElizabeth Mills Booth. Communication for Child Survival, USAgency for International Development, Washington, DC. June1988.

Rice. R E , and Paisley. WJ (Eds ), Public CommunicationCampaigns, Beverly Hills Sage Publications. 1981

Rogers, Everett, M , Diffusion of Innovations, New York FreePress, 1983,

Roman. Kenneth and Maas, Jane. How to Advertise, St Martin'sPress, 175 Fifth Avenue, New York, NY 10010, 1976.

Romano, R.M., 'The Cancer Prevention Awareness ProgramAPproaching Public Understanding with Good News: in DSLeathar.G.B Hastings, K M. O'Reilly. & J K Davies (Eds ),Health Education and the Media II, New York Pergamon Press,1986, pp 93-100.

Ross. Helen S and Mico. Paul R , Theory and Practice in HealthEducation, Palo Alto Mayfield Pubhshing Company. 1980

Rossi, Peter H. and Freeman, Howard E , Evaluation. BeverlyHills Sage Publications, 1985

Schhnger, M J , "The Role of Mass Communications inPromoting Pubhc Health." Advances in Consumer Research, 3,1976. pp. 302-305.

Simmons Market Research Bureau. Simmons 1985 Study ofMedia and Markets, Media Series, Vols 1-13. and ProductSeries, Vols. 1-3G, New York. Simmons Market ResearchBureau, Inc . 1985

Sissors, Jack Z and Surmanek, Jim. Advertising MediaPlanning, Chicago Crain Books. 1976,

Sissors, Jack Z . Lehew. Harry D , and Goodrich, William B .Media Planning Workbook, Chicago. Crain Books. 1979 (3rdprinting).

Spaulding, Seth, "Two Formulas for Estimating the ReadingDifficulty of Spanish," Educational Research Bulletin, 30, May16, 1951, pp 117-124

Squyres, Wendy D . Patient Education and Health Promotion inMedical Care, Palo Alto: Mayfield Publishing Co . 1985

Storer Communications, Broadcasting Yearbook, Hollywood andNew York Broadcasting Pubhcations Inc , updated yearly

Strunk, William, Jr.. and White, E,B . The Elements of Style (3rdedition). New York The Macmillan Company. 1979.

Sudman, Seymour, Greeley, Andrew. and Pinto, Leonard, -TheEffecti..eness of Self-Administered Questionnaires: Journal ofMarketing Research, 2. August 1965, pa 293307

Survey Research Center. Interviewer's Manual, (revised edition).University of Michigan. Ann Arbor Institute for Social Research,1978.

Television Digest. Television Factbook, Washington. DCTelevision Digest, Inc. 1982

TV Pub.ity Outlets. TV Publicity OutletsNationwide.Washington Depot, CT TV Pubhcity Outlets, 1982

Tichy, H J , Effective Writing for EngineersManagersScientists, New York John Wiley & Sons. 1966.

Traub, James, "The World According to Nielson Who WatchesTelevision and Why," Channels, January/February 1985. p 26

US General Accounting Office. Using Structured InterviewingTechniques, Methodology Transfer Paper 5, July, 1985

White. H , How to Produce Effective TV Commercials,Lincolnwood. NTC Business Books. 1986

Willard N "Selling Health Behavior Success and Failures: inDS Leathar, G B Hastings, K M O'Reilly. & J K Davies. (Eds ).Health Education and the Media II, New York Pergamon Press.1986, pp 57-59

Wimmer, R D . & Dominick, J R . Mass Media Research, (2nded ), Belmont. CA Wadsworth, 1987

Working Press of the Nation, TV and Radio Directory The1982-1983 Media Encyclopedia, Chicago The NationalResearch Bureau.

World Health Organization, Health Promotion for AIDSPrevention A Manual for Communicators. Geneva 27,Switzerland. CH-1211, World Health Organization. 1988

1 1 (t 1 129

vv, e would appreciate any comments or suggestionsyou have about this guide; such suggestions willbe used in preparing future editions of the publi-

cation.

1. How much of this book did you read/

All of it Some of it Did not read it

2. If you read some of the workbook, please mark whichchapters you read:

Introduction

The Health Communication Process

Stage 1: Planning and Strategy Selection

Stage 2: Selecting Channels and Pretesting

Stage 3: Developing Materials and Pretesting_ Stage 4: Implementing Your Program

Stage 5: Assessing Effectiveness_ Stage 6: Feedback to Refine Program

3. Did you find the workbook to be

very useful somewhat useful not useful/

4. Please circle those chapters listed above that youfound most useful.

5. How have you used this book? (check as many asaPply)

Personal Reference

StuJent Instruction

Staff Development_ Library Resource

Other

6. Your job title/

7. Affiliation or employer/

8 How might this handbook be improved?

Thank you.

Please send to:

John BurklowOffice of Cancer Communications, NCIBuilding 31, Room 4B43Bethesda, MD 20892

131

DOCUMENT RESUME ED 327 898 AUTHOR Arkin, Elaine Bratic … · 2014-03-24 · DOCUMENT RESUME. CS...

Documents

Transcript of DOCUMENT RESUME ED 327 898 AUTHOR Arkin, Elaine Bratic … · 2014-03-24 · DOCUMENT RESUME. CS...