Completing the post KAP survey

Purpose of post campaign KAP survey

Assess the impact & success of your campaign and determine next steps

How do we achieve that?

By measuring SMART objective attainment and exposure to campaign activities

What data are we looking for ?

• Measure for each step of the results chain for each TA = SMART Objective Questions

• Exposure• Comparability of data • Audience segmentation

Further objectives?

• Collect baseline data• Understand the preferred

media sources• Identify trusted sources • Ground truth assumptions • Identify Barriers/Benefits

to BC.

Objective

By the end of the day you will have: finalized your post campaign KAP

survey questionnaire, sampling methodology & plan.

revised and finalized your SMART objectives & result chains.

The steps

The steps• Step 1: Revise your sampling methodology

and create a survey plan. • Step 2: Check your SMART objectives and

mark them in your survey• Step 3: Revise your campaign survey in word• Step 4: Hide the questions in Survey-Pro

• Step 5 & 6: Implement your survey and carry out data entry

Schedule for the day8-9 am Introduction to shortening KAP survey (AB) 9-10 am Step 1: Revise your sampling methodology and

create a survey plan. Check in with PPM before finalizing

10am -12 pm

Step 2: Check your SMART objectives and mark them in your survey. Check in with PPM before finalizing

1 -2 pm Step 3: Revise your campaign survey in word using track changes

2 - 3 pm Check deletions with PPM3 - 4 pm Step 4: Hide the questions in Survey-Pro that you

deleted from the word version4 - 5 pm Refresher on training enumerators, data entry, top

tips and Q/A (in preparation for Step 5 & 6)

• Step 1: Revise your sampling methodology and create a survey plan.

• Decide which groups you will sample for the post campaign KAP survey and revise your sampling methodology accordingly.

• Create a realistic post-campaign survey plan (to be finalized once you have revised your survey and tested survey length).

Decide which groups you will sample for the post campaign KAP survey and revise your

sampling methodology accordingly

• Keep primary audiences (fishers & community)

• Other groups are optional

• For each TA sampling methodology should be exactly the same as for pre-KAP

Sampling methodology template

A. Define geographic scope of survey and include a map showing area included in survey

B. Sample size calculation & sampling methodologyC. Survey plan

B. Sample sizeN of people Sample size at

95%CL and 3%CI Sample size at 95%CL and 5%CI

Final sample size choice

TA 1 (specify who is included and how many people example: 350 fishermen of Pilar and 50 from Esperanza)

291 196 291

TA 2 (specify who is included and how many people example: 60 people of local enforcement team (35 MPA guards, 10 police, 10 Mancom, 5 LGU officers from Pilar and Esperanza)

57 52 60

TA 3 (specify who is included and how many people) Example: 4000 (3000 Pilar and 1000 Esperanza)

843 351 351

Total 1191 599 702 • Name of TA • Total n of TA (and who it includes with breakdown• Sample size (specify CL and CI)• Final sample size achieved

B. Sampling methodology

• N of TA in each locality• N of samples per TA per location (stratified

sampling)• Method of sampling:

- for pre – KAP: amend to reflect reality- for post KAP – same as amended for Pre

• If there are any differences between pre-and post please take note of these in the table

A. Define geographic scope of survey and include a map showing area included in survey

A. Define geographic scope of survey and include a map showing area included in survey

Survey carried out on in Barangay of Pilar and Esperanza (area shown in rectangle on right side image), Ponson island, Camotes islands, Camotes Sea. Ideally you would show a more detailed map showing boundaries of survey area

C. Create a realistic post-campaign survey plan (to be finalized once you have revised your survey and tested

survey length). Total survey sample size 702 Time for enumerator to do one survey 30 minutes (0.5 hrs) N of hrs enumerator works a day 7.5 N of surveys one enumerator can do a day 15 (=7.5/0.5) N of enumerators available 5 N of surveys done a day by total n of enumerators 75 (=15*5) N of days needed to do survey 10 (=702/75) Additional days needed for travel if any 2 Dates for training of enumerators & final survey testing

1 - 4 February

Beginning and end date of survey 7 - 18 February N of people available to do data entry 3 Beginning date of data entry 11 February Date of completion of data entry 25 February

• Step 2: Check your SMART objectives and mark them in your survey

CHECK THAT:1. Smart objectives are aligned with your results chain2. Smart objectives enable you to measure the success of your

campaign3. There are 1-2 obj. for each section of the ToC, for primary TA

- link to your ideas for BC measurement from yesterday4. Data in the smart objectives matches your pre-campaign results and

the answer options of those results5. Smart objectives targets are realistic (22K, 13A, 28IC, 14BC6. Question numbering in the monitoring plan matches final survey

What’s the issues?Result chain (BR):

Arbor fishers and gleaners actively participate in the strict enforcement of Arbor MPA NTZ regulations by participating in MPA enforcement activities.

Smart Objective:The % of Arbor fishers and gleaners that are able to give at

least one benefit of establishing the Arbor/Granada Marine Sanctuaries will increase from 16% measured in February 2011 to 26% measured in May 2012 (an increase of 10pp, Q5 in KAP survey)

CorrectionResult chain (BR):

Arbor fishers and gleaners actively participate in the strict enforcement of Arbor MPA NTZ regulations by participating in MPA enforcement activities .

Smart Objective:The % of Arbor fishers and gleaners that are able to give at

least one benefit of establishing the Arbor/Granada Marine Sanctuaries Management Committee will increase from 16% measured in February 2011 to 26% measured in May 2012 (an increase of 10pp, Q5 in KAP survey)

Alternative Results chain (BR)

There is a functional 24/7 MPA patrolling and guarding in place at Arbor and Grenada NTZ

SMART objectiveThe % of Arbor fishers and gleaners who agree that there Arbor and Granada MPAs have a functional 24/7 guarding and patrolling system in place ("Yes, in Arbor but not in Granada" AND "Yes, in both Arbor and Granada") will increase from 23 measured in February 2011 to 33 measured in June 2012 (an increase of 10pp, Q29 in KAP survey)

Pre campaign (baseline) Target Desired pp change (specify if increase or decrease)

44% 66% 22pp (K) Knowledge The % of Arbor fishers and gleaners that could name at least one benefit to the local community from having a no-take area nearby nearby, will increase from 36% measured in February 2011 to 58% measured in May 2012 (an increase of 22pp, Q32 in KAP survey)

SMART GoalsBaseline & Target Example: The % of respondents who say... will increase/decrease from XX% measured in

March 2010 to YY% measured in August 2012 (an increase/decrease of YY-XXpp, Qx in KAP survey)

WHAT’S WRONG HERE?

44% 36%

• Step 3: Revise your campaign survey in word

Make sure the survey is your latest versionMake revisions in English version (cross check with vernacular)Use track changes

Guiding principles for shortening surveys

• All questions should remain exactly the same (even if they had a mistake)

• Questions and answer options should remain in the same order

• Reduce survey to 30 minutes – max 40

Should be retained:

1. Socio-economic data - Section 2 2. Questions you used during analysis for cross

tabulations and filters (make sure you check which ones these were. i.e. sampling methodology, which category best describes you)

3. Compulsory cohort questions, selection to be retained (see provided list in Appendix 2)

4. Climate change questions5. Questions needed for your SMART objective

Can be removed :

1. Trusted Sources of Information & Media Access/Exposure- Section 3

2. Flagship sp question - Section 11 3. Any questions that you no longer deem

useful, including cohort questions, provided they are not part of the categories to be retained

To be added: 1. A question asking respondents whether they have taken

this survey before

2. 3-5 answer options to the exposure question “I am going to ask you about a number of ways in which you may or may not have seen or heard about the XXX Sanctuary. For each method, I would like you to tell me whether you remember seeing or hearing that source in the past 6 months.” - Section 10Answer options should relate specifically to campaign events, activities or materials that you implemented over the last 6 month.

Expl: “boat awnings”, “battle of the bands”, or “MPA blessing”retain at least two “dummies” as a measure of “false negative”

Finally please make sure

Under “survey period” there is a “post campaign survey” answer option – Section 1

• Step 4: Hide the questions in Survey-Pro

Open survey proGo to the “questionnaire” tabDouble click on the question you want to hideClick on the “advanced options” tabOn the right select “Hide” for all answer options

The question will still appear in the survey but all answer options will be hidden

For grid questions:The process is the same as above but at n4 above click on “rows”.

• Step 5 : Implement your survey

Choosing & Training enumerators

• Enumerators “neutral” in the community• Should be committed to survey so there is not

a high turnover

• Should receive a day training & practice• Have clear instructions in the survey for

enumerators

Training enumerators agenda

• Intro to campaign => Motivation

• Ethos / Neutral approachable attitude

• Process (one on one; don’t share paper but interview)

• Run through questions and instructions one by one (translated version)

• Keep tally of respondents who don’t want to be interviewed

• Record any changes in sampling methodology• Have feedback system

Training enumerators - agenda

• Go over survey plan Make sure enumerators understand sampling methodologyrandom sampling – (map + dice to select street?)stratified sampling – (create simplified version)

• Provide additional materials needed (MAP for survey question, survey plan with MPAs of area, Community Based Monitoring System (CBMS) map from MPDO; ID)

• Do a practice run & debrief

Training enumerators - agenda

• (55) Please name 3 creatures that live in the sea that you are particularly fond or proud of. (do not read out options but tick box as appropriate and/or fill in the blank for any additional answers)

• FLAGSHIP SP QUESTION - 1st OPTION• [ ] grouper "lapu-lapu" [ ] giant clam [ ] giant

sea fan [ ] XXX ADD OPTIONS AS APPROPRIATE [ ] XXX ADD OPTIONS AS APPROPRIATE [ ] Other ________________ [ ] N/A

Instructions for enumerators

Top Tip

• Laminate survey questionnaire with hole and ring binder

• Create answer sheet

=> Instructions for enumerators in both documents

• Do a pilot survey 3-5 per enumerator(test on people random sample in least important area first)

• Do a debrief from pilot survey

Training enumerators

Training enumerators

• Agree on n of surveys per day and target• Have organizer within group• Safety protocol• At a minimum take part in first 2 days of

surveying• Do data entry of pilot survey with encoders!

Step 6: carry out data entry

KAP Survey TOP TIPS Encoding• Dry run

• Change the structure of the open ended questions before encoding

• Encoders who are willing to work evening/night shifts. Don’t try and do all the encoding yourself: get some help!

• Be strategic about the order in which you encode (and also survey) the target audiences start by encoding fishers and the local community (TA1 &TA2)

• Write the corresponding form number in Survey pro on the paper survey

Data entry

• Bottleneck! Start early• Save every form• System for keeping paper copy

=> BACK UP every day!

Good luck!

Diskarte