The Evaluation of Programs Combating Female Genital Cutting: Data Sources and Options February 6,...
-
Upload
brandon-bain -
Category
Documents
-
view
214 -
download
0
Transcript of The Evaluation of Programs Combating Female Genital Cutting: Data Sources and Options February 6,...
The Evaluation of Programs Combating Female Genital Cutting:
Data Sources and Options
February 6, 2007
P. Stanley YoderMacro
International
Overview of presentation
• Assessing long-term impact
• DHS data
• Single survey data
• Challenges in designing evaluations
Current situation: Anti-FGC programs
• Great variety of program types
- Public awareness media campaigns
- Social mobilization in specific populations
- Lobbying for legal action
- Enlisting help from traditional and political leaders
• Very little empirical evidence of impact
(Diop & Askew, book chapter in R.M. Abusharaf, ed. 2006)
Need for data on FGC program impact
• Statistical evidence for changes in FGC prevalence
• Possible use of DHS data?
• Surveys of sample populations
- Baseline and follow-up surveys
- Comparison of intervention and
control regions
Challenges in collection of relevant data
• Conducting a survey in program areas only
• Need to have data on events 10 years ago
• Temptation to rely on program personnel rather than outside observers to evaluate
• DHS provides data on women 15-49 years old and not on young girls
• Need to describe situation taking account of secular trend
Use of DHS data for FGC evaluations
• Data on prevalence by age group of women
• Data on daughters (most recently cut)
- Age at FGC
- Identity of practitioner
- type of cutting
• Does not provide data on all daughters
DHS data on prevalence by age group
Senegal Kenya Nigeria Tanzania
15-19 24.8% 20.3% 12.9% 9.1%
20-24 28.0% 24.8% 17.0% 13.7%
25-29 28.4% 33.0% 20.8% 15.2%
30-34 30.1% 38.1% 19.4% 16.0%
35-39 30.5% 39.7% 22.2% 16.0%
40-44 30.3% 47.5% 22.2% 18.8%
45-49 30.6% 47.7% 28.4% 22.9%
Total 28.2% 32.2% 19.0% 14.6%
DHS data on distribution of FGC
• Prevalence by region
• Prevalence by ethnicity
• Prevalence by rural/urban residence
• Prevalence by religion (sometimes)
Example of DHS data for Senegal: Ethnicity
FGC prevalence
Percent of total pop.
Wolof 1,6% 40%
Serer 1,8% 16%
Diola 60% 5%
Fulani 62% 25%
Mandinka 74% 5%
Soninke 78% 3%
Limitations in use of DHS data for evaluations
• No complete data on FGC among young girls
• Sample size limits aggregation by small units such as those covered by an FGC program
• Provides data on FGC events 10-15 years ago and more only (women 15-24 years old)
Assessing long-term impact: Stand alone surveys
• Baseline and follow-up surveys
- relatively expensive to design and implement
- results after more than five years
• One survey with intervention & control areas
- much cheaper
- difficult to identify intervention & control areas, for they don’t follow administrative units
- very hard to match intervention & control areas
Example: Tostan evaluation for UNICEF New York
• Commissioned and funded by Child Protection Division of UNICEF New York
• Implemented through contract between UNICEF and Macro International
• Additional funding by USAID
• Two components: Quantitative and Qualitative
- Quantitative: Centre de Recherche pour le Développement Humain (CRDH)
- Qualitative: Population Council, Dakar
Quantitative component: intervention and control areas
• Designed and implemented by the CRDH
• Questionnaire for households and for women 15-49 years old
• Questionnaire similar to DHS
• Questions on knowing about Tostan, participation in Tostan, knowing about a Public Declaration, participation in a PD
Basic questions to answer
• How has Tostan affected:
- average age of marriage?
- cutting of young girls?
• How has participation in a Public Declaration affected:
- average age of marriage?
- cutting of young girls?
Three types of villages where FGC is practiced
Basic comparison:
Data on age of first marriage and rates of FGC in
three types of villages:
• Type A: Tostan operated and population
participated in a Public Declaration (PD)
• Type B: Population participated in PD without Tostan
• Type C: Control villages: No Tostan, no PD
Assessment of program impact
• What are the starting points for rates of FGC and for age of marriage in these three types of villages?
• What is the secular trend in these areas?
• How do the intervention and the control villages differ?
• To what extent can differences be attributed to Tostan activities?
Village type by prevalence of FGC for women
• Type A villages – FGC prevalence 64%
• Type B villages – FGC prevalence 81%
• Type C villages – FGC prevalence 87%
• Implication: The point of departure is different for the assessment of any change that may have occurred
Major challenges for village comparisons
• Matching control and intervention villages by:
- ethnic composition
- practice of FGC
- literacy
- development of social services
• Locating villages near Tostan villages that had no Tostan influence
Reporting on an evaluation: Being clear
• The questions to be answered
• The indicators to be followed
• How and why they are important
• The expectations of the program personnel
• The ways evaluators would assess the program
• The ways evaluators would measure success
Setting up a new evaluation
• Questionnaire that matches program activities
• Sample population that covers program areas plus control areas
• Choosing indicators that show process as well as impact
• Identifying a neutral agency to conduct survey
• Articulating how assessments will be made
• Plan final report that provides all this information