Monitoring Flour Fortification Programs: An Overview · 1 Monitoring Flour Fortification Programs:...
Transcript of Monitoring Flour Fortification Programs: An Overview · 1 Monitoring Flour Fortification Programs:...
![Page 1: Monitoring Flour Fortification Programs: An Overview · 1 Monitoring Flour Fortification Programs: An Overview Second Africa FFI Meeting 26 November 2010 Laird J. Ruth International](https://reader033.fdocuments.us/reader033/viewer/2022043008/5f968b9d97ca8a4f3d2e5763/html5/thumbnails/1.jpg)
1
Monitoring Flour Fortification Programs: An Overview
Second Africa FFI Meeting
26 November 2010
Laird J. Ruth
International Micronutrient Malnutrition and Control
(IMMPaCt) Program
Centers for Disease Control and Prevention (CDC)
www.cdc.gov/immpact
![Page 2: Monitoring Flour Fortification Programs: An Overview · 1 Monitoring Flour Fortification Programs: An Overview Second Africa FFI Meeting 26 November 2010 Laird J. Ruth International](https://reader033.fdocuments.us/reader033/viewer/2022043008/5f968b9d97ca8a4f3d2e5763/html5/thumbnails/2.jpg)
2
Topics
• Monitoring definition and concepts
• Food fortification monitoring system
overview
• General principals for setting up a
monitoring system
• Data sources for monitoring
• Evaluation definition and concepts
• Decisions to make when setting up an
evaluation of a flour fortification program
• Examples
![Page 3: Monitoring Flour Fortification Programs: An Overview · 1 Monitoring Flour Fortification Programs: An Overview Second Africa FFI Meeting 26 November 2010 Laird J. Ruth International](https://reader033.fdocuments.us/reader033/viewer/2022043008/5f968b9d97ca8a4f3d2e5763/html5/thumbnails/3.jpg)
Monitoring & Evaluation (M&E)
• M = An ongoing process of collecting,
analyzing and interpreting data, to compare
how well a program/project is being
executed against expected results
• E = Objective assessment of a program or
project that covers its design,
implementation, impact, efficiency and
sustainability
![Page 4: Monitoring Flour Fortification Programs: An Overview · 1 Monitoring Flour Fortification Programs: An Overview Second Africa FFI Meeting 26 November 2010 Laird J. Ruth International](https://reader033.fdocuments.us/reader033/viewer/2022043008/5f968b9d97ca8a4f3d2e5763/html5/thumbnails/4.jpg)
• Translates objectives into indicators based on
expected results
• Reports progress to managers and alerts them
to problems, in order to implement timely
remedial actions
• Helps sustain successful activities of programs
or projects
Rafael Flores, 2006
Monitoring & Evaluation (M&E)
![Page 5: Monitoring Flour Fortification Programs: An Overview · 1 Monitoring Flour Fortification Programs: An Overview Second Africa FFI Meeting 26 November 2010 Laird J. Ruth International](https://reader033.fdocuments.us/reader033/viewer/2022043008/5f968b9d97ca8a4f3d2e5763/html5/thumbnails/5.jpg)
Goals and Objectives
• Goal: a broad statement of a desired, long-term outcome of the program
• Objectives: statements of desired, specific, realistic,
and measurable program results
SMART
• Specific: identifies concrete events or actions that will take place
• Measurable: quantifies the amount of resources, activity, or change to be expended and achieved
• Appropriate: logically relates to the overall problem statement and desired effects of the program
• Realistic: Provides a realistic dimension that can be achieved with the available resources and plans for implementation
• Time-based: specifies a time within which the objective will be achieved
Source: GAP 2003; Thom Eisele and Joe Keating, Tulane School of Public Health
![Page 6: Monitoring Flour Fortification Programs: An Overview · 1 Monitoring Flour Fortification Programs: An Overview Second Africa FFI Meeting 26 November 2010 Laird J. Ruth International](https://reader033.fdocuments.us/reader033/viewer/2022043008/5f968b9d97ca8a4f3d2e5763/html5/thumbnails/6.jpg)
6
Framework for monitoring flour
fortification programs
![Page 7: Monitoring Flour Fortification Programs: An Overview · 1 Monitoring Flour Fortification Programs: An Overview Second Africa FFI Meeting 26 November 2010 Laird J. Ruth International](https://reader033.fdocuments.us/reader033/viewer/2022043008/5f968b9d97ca8a4f3d2e5763/html5/thumbnails/7.jpg)
7
Process (Program) monitoring
• Inputs extend to the financial, human, and material resources used for a program
• Activities are the specific actions taken or work performed through which inputs, such as funds, technical assistance and other types of resources are mobilized to produce specific outputs.
• Outputs include the products, capital goods and services that result from an intervention, which are relevant to the achievement of outcomes
• Outcomes extend to the likely or achieved effects, or impact of a program in the target population.
![Page 8: Monitoring Flour Fortification Programs: An Overview · 1 Monitoring Flour Fortification Programs: An Overview Second Africa FFI Meeting 26 November 2010 Laird J. Ruth International](https://reader033.fdocuments.us/reader033/viewer/2022043008/5f968b9d97ca8a4f3d2e5763/html5/thumbnails/8.jpg)
8
Logic model of M&E
![Page 9: Monitoring Flour Fortification Programs: An Overview · 1 Monitoring Flour Fortification Programs: An Overview Second Africa FFI Meeting 26 November 2010 Laird J. Ruth International](https://reader033.fdocuments.us/reader033/viewer/2022043008/5f968b9d97ca8a4f3d2e5763/html5/thumbnails/9.jpg)
9
Monitoring and Evaluation pipeline
![Page 10: Monitoring Flour Fortification Programs: An Overview · 1 Monitoring Flour Fortification Programs: An Overview Second Africa FFI Meeting 26 November 2010 Laird J. Ruth International](https://reader033.fdocuments.us/reader033/viewer/2022043008/5f968b9d97ca8a4f3d2e5763/html5/thumbnails/10.jpg)
10
Flour Fortification Monitoring
System Overview
![Page 11: Monitoring Flour Fortification Programs: An Overview · 1 Monitoring Flour Fortification Programs: An Overview Second Africa FFI Meeting 26 November 2010 Laird J. Ruth International](https://reader033.fdocuments.us/reader033/viewer/2022043008/5f968b9d97ca8a4f3d2e5763/html5/thumbnails/11.jpg)
11
Why monitor a
flour fortification program?
1. To ensure that fortified flour meet nutrient content and safety standards
2. To assess access, utilization and coverage of fortified flour by the people (the consumer)
3. To effectively manage and sustain the fortification program to eliminate vitamin and mineral deficiencies
![Page 12: Monitoring Flour Fortification Programs: An Overview · 1 Monitoring Flour Fortification Programs: An Overview Second Africa FFI Meeting 26 November 2010 Laird J. Ruth International](https://reader033.fdocuments.us/reader033/viewer/2022043008/5f968b9d97ca8a4f3d2e5763/html5/thumbnails/12.jpg)
What is an indicator?
• Indicators are signs or markers that
inform the relevant parties whether the
program objectives or being achieved
12
![Page 13: Monitoring Flour Fortification Programs: An Overview · 1 Monitoring Flour Fortification Programs: An Overview Second Africa FFI Meeting 26 November 2010 Laird J. Ruth International](https://reader033.fdocuments.us/reader033/viewer/2022043008/5f968b9d97ca8a4f3d2e5763/html5/thumbnails/13.jpg)
An indicator should be:
• Valid –correctly measures what it is
intended to measure
• Simple and measurable – e.g. the label or
logo on a sack of fortified flour
• Reliable – i.e. provides consistent and
reproducible results on repeat
measurements.
13
![Page 14: Monitoring Flour Fortification Programs: An Overview · 1 Monitoring Flour Fortification Programs: An Overview Second Africa FFI Meeting 26 November 2010 Laird J. Ruth International](https://reader033.fdocuments.us/reader033/viewer/2022043008/5f968b9d97ca8a4f3d2e5763/html5/thumbnails/14.jpg)
An indicator should also be:
• Timely - can be measured at appropriate
intervals to detect the expected change.
• Programmatically important - e.g. legal
monitoring shows that sufficient quality
fortified flour is produced or imported to
meet the needs of the target population.
• Comparable – data are collected using the
same methodology and tools, so that the
results can be compared 14
![Page 15: Monitoring Flour Fortification Programs: An Overview · 1 Monitoring Flour Fortification Programs: An Overview Second Africa FFI Meeting 26 November 2010 Laird J. Ruth International](https://reader033.fdocuments.us/reader033/viewer/2022043008/5f968b9d97ca8a4f3d2e5763/html5/thumbnails/15.jpg)
15
Monitoring system
• Access: is fortified flour available and affordable to the target population?
• Utilization: is fortified flour being purchased by the target households?
• Coverage: is fortified flour being consumed by the target population?
– At what percent?
![Page 16: Monitoring Flour Fortification Programs: An Overview · 1 Monitoring Flour Fortification Programs: An Overview Second Africa FFI Meeting 26 November 2010 Laird J. Ruth International](https://reader033.fdocuments.us/reader033/viewer/2022043008/5f968b9d97ca8a4f3d2e5763/html5/thumbnails/16.jpg)
16
Indicator example #1
Question Measure Indicator
Access:
is fortified
flour available
and affordable
to the target
population?
Increased
production /
importation of
fortified flour
according to
specifications
•Proportion of
fortified /
unfortified flour
produced or
imported
![Page 17: Monitoring Flour Fortification Programs: An Overview · 1 Monitoring Flour Fortification Programs: An Overview Second Africa FFI Meeting 26 November 2010 Laird J. Ruth International](https://reader033.fdocuments.us/reader033/viewer/2022043008/5f968b9d97ca8a4f3d2e5763/html5/thumbnails/17.jpg)
17
Indicator example #2
Question Measure Indicator
Utilization:
is fortified
flour being
purchased by
the target
households?
Increased
purchase of
fortified flour
and by products
•Proportion of
households with
flour ―labeled‖
as fortified
![Page 18: Monitoring Flour Fortification Programs: An Overview · 1 Monitoring Flour Fortification Programs: An Overview Second Africa FFI Meeting 26 November 2010 Laird J. Ruth International](https://reader033.fdocuments.us/reader033/viewer/2022043008/5f968b9d97ca8a4f3d2e5763/html5/thumbnails/18.jpg)
18
. Percent of households with flour specimens
positive with iron spot test,
by type of flour
![Page 19: Monitoring Flour Fortification Programs: An Overview · 1 Monitoring Flour Fortification Programs: An Overview Second Africa FFI Meeting 26 November 2010 Laird J. Ruth International](https://reader033.fdocuments.us/reader033/viewer/2022043008/5f968b9d97ca8a4f3d2e5763/html5/thumbnails/19.jpg)
19
Indicator example #3
Question Measure Indicator
Coverage:
is fortified
flour being
consumed by
the target
population?
Increased
proportion of
non-pregnant
women (15-49)
regularly
consuming
fortified flour
•Proportion of
non-pregnant
women (15-49)
regularly
consuming
fortified flour
![Page 20: Monitoring Flour Fortification Programs: An Overview · 1 Monitoring Flour Fortification Programs: An Overview Second Africa FFI Meeting 26 November 2010 Laird J. Ruth International](https://reader033.fdocuments.us/reader033/viewer/2022043008/5f968b9d97ca8a4f3d2e5763/html5/thumbnails/20.jpg)
Percent of households with fortified flour
and by type of fortified flour, Oman 2004
81.492.2
76.8
48.8
0
10
20
30
40
50
60
70
80
90
100
Households with
FF
White Flour Brown Flour #2 Brown Flour #3
(%)
![Page 21: Monitoring Flour Fortification Programs: An Overview · 1 Monitoring Flour Fortification Programs: An Overview Second Africa FFI Meeting 26 November 2010 Laird J. Ruth International](https://reader033.fdocuments.us/reader033/viewer/2022043008/5f968b9d97ca8a4f3d2e5763/html5/thumbnails/21.jpg)
21
General principals for setting up
a monitoring system
1. Responsibility at each level needs to be clear:
a) For whom are the data collected (stakeholders)?
b) What data are collected (questions and
indicators)?
c) How are the data collected (methodology)?
d) Who collects the data (personnel)?
e) When are the data collected (frequency)?
f) Who analyzes the data?
g) Who reports the data and when?
h) Who does what based on the information?
![Page 22: Monitoring Flour Fortification Programs: An Overview · 1 Monitoring Flour Fortification Programs: An Overview Second Africa FFI Meeting 26 November 2010 Laird J. Ruth International](https://reader033.fdocuments.us/reader033/viewer/2022043008/5f968b9d97ca8a4f3d2e5763/html5/thumbnails/22.jpg)
22
Example: Process monitoring of flour fortification
Is sufficient fortified flour accessible?
1. Analyze
flour industry
production
and sales data
2. Retail
assessment
Milling companies
and distributors;
MoCommerce
Responsibility Method
Annually
(on-going)
Frequency
Is sufficient
fortified flour
available for the
population?
Amount of
fortified flour
(local and/or
imported)
relative to
population
needs
Indicator Question
National
Fortification
Alliance
Determine
how and
who collects
data
Determine
what you
need to
know
Determine
who
reports
data
Determine
how often
to collect
data
Determine
for whom
to collect
data
National
Fortification
Alliance
![Page 23: Monitoring Flour Fortification Programs: An Overview · 1 Monitoring Flour Fortification Programs: An Overview Second Africa FFI Meeting 26 November 2010 Laird J. Ruth International](https://reader033.fdocuments.us/reader033/viewer/2022043008/5f968b9d97ca8a4f3d2e5763/html5/thumbnails/23.jpg)
23
Monitoring implementation
• Do pilot run of monitoring system (data collection, analysis, and reporting process) to:
Correct potential problems
Allow “Stakeholders” to experience the system and:
Their role, level of effort, and importance in the process
The specific kinds of information that would be available to them through the monitoring system
*Parvanta, 2003
![Page 24: Monitoring Flour Fortification Programs: An Overview · 1 Monitoring Flour Fortification Programs: An Overview Second Africa FFI Meeting 26 November 2010 Laird J. Ruth International](https://reader033.fdocuments.us/reader033/viewer/2022043008/5f968b9d97ca8a4f3d2e5763/html5/thumbnails/24.jpg)
24
• Existing data systems
– Health statistics data; anemia from ANC
– Multiple Indicator Cluster Surveys (MICS)
– Reproductive health surveys
– Household Income and Expenditure
Survey (HEIS)
– Other surveys from other sectors (NGOs,
government, Universities, etc)
• New data systems
– Micronutrient Survey
Data Sources for Monitoring
![Page 25: Monitoring Flour Fortification Programs: An Overview · 1 Monitoring Flour Fortification Programs: An Overview Second Africa FFI Meeting 26 November 2010 Laird J. Ruth International](https://reader033.fdocuments.us/reader033/viewer/2022043008/5f968b9d97ca8a4f3d2e5763/html5/thumbnails/25.jpg)
25
38.3
24.5
0
5
10
15
20
25
30
35
40
45
< 1 kg >= 1kg
(%)
Trends in anemia prevalence
screened at first antenatal clinic visit,
Oman
![Page 26: Monitoring Flour Fortification Programs: An Overview · 1 Monitoring Flour Fortification Programs: An Overview Second Africa FFI Meeting 26 November 2010 Laird J. Ruth International](https://reader033.fdocuments.us/reader033/viewer/2022043008/5f968b9d97ca8a4f3d2e5763/html5/thumbnails/26.jpg)
26
• Sentinel monitoring (purposive sampling)
– Schools
– Worksites
– Public health clinics
– Hospitals
• Qualitative research and reports
– Universities
– Industry
Data Sources for Monitoring
![Page 27: Monitoring Flour Fortification Programs: An Overview · 1 Monitoring Flour Fortification Programs: An Overview Second Africa FFI Meeting 26 November 2010 Laird J. Ruth International](https://reader033.fdocuments.us/reader033/viewer/2022043008/5f968b9d97ca8a4f3d2e5763/html5/thumbnails/27.jpg)
Trends in Incidence of Spina
Bifida and other NTDs, Oman
27
![Page 28: Monitoring Flour Fortification Programs: An Overview · 1 Monitoring Flour Fortification Programs: An Overview Second Africa FFI Meeting 26 November 2010 Laird J. Ruth International](https://reader033.fdocuments.us/reader033/viewer/2022043008/5f968b9d97ca8a4f3d2e5763/html5/thumbnails/28.jpg)
28
There Are No Perfect
Monitoring Systems
Only
“Best We Can Do” Ones
Remember ….
*Parvanta, 2003
![Page 29: Monitoring Flour Fortification Programs: An Overview · 1 Monitoring Flour Fortification Programs: An Overview Second Africa FFI Meeting 26 November 2010 Laird J. Ruth International](https://reader033.fdocuments.us/reader033/viewer/2022043008/5f968b9d97ca8a4f3d2e5763/html5/thumbnails/29.jpg)
12/12/2012 29
Program Evaluation
Objective assessment of a program
that covers its need, design,
implementation, impact,
effectiveness, efficiency and
sustainability
![Page 30: Monitoring Flour Fortification Programs: An Overview · 1 Monitoring Flour Fortification Programs: An Overview Second Africa FFI Meeting 26 November 2010 Laird J. Ruth International](https://reader033.fdocuments.us/reader033/viewer/2022043008/5f968b9d97ca8a4f3d2e5763/html5/thumbnails/30.jpg)
Aim of evaluation
• Analyzes why intended impacts were or were not achieved
• Explores unintended results
• Informs practice, decision-making and policy
12/12/2012 30
![Page 31: Monitoring Flour Fortification Programs: An Overview · 1 Monitoring Flour Fortification Programs: An Overview Second Africa FFI Meeting 26 November 2010 Laird J. Ruth International](https://reader033.fdocuments.us/reader033/viewer/2022043008/5f968b9d97ca8a4f3d2e5763/html5/thumbnails/31.jpg)
Evaluation questions
• Does the intervention achieve the intended purpose?
• Can the changes in outcomes be explained by the intervention, or by some other factors occurring simultaneously?
• Do intervention impacts vary across different
groups of intended beneficiaries, regions, and over time?
• Are there any unintended effects of the intervention, either positive or negative?
• How cost-effective is the intervention in comparison with alternative projects?
![Page 32: Monitoring Flour Fortification Programs: An Overview · 1 Monitoring Flour Fortification Programs: An Overview Second Africa FFI Meeting 26 November 2010 Laird J. Ruth International](https://reader033.fdocuments.us/reader033/viewer/2022043008/5f968b9d97ca8a4f3d2e5763/html5/thumbnails/32.jpg)
Steps in designing a flour fortification
monitoring & evaluation system
Centers for Disease Control and Prevention. Framework for Program Evaluation in Public Health. MMWR 1999;48(No. RR-11). CDC. Framework for program evaluation in public health. MMWR Recomm Rep. 1999;48:1 40.
![Page 33: Monitoring Flour Fortification Programs: An Overview · 1 Monitoring Flour Fortification Programs: An Overview Second Africa FFI Meeting 26 November 2010 Laird J. Ruth International](https://reader033.fdocuments.us/reader033/viewer/2022043008/5f968b9d97ca8a4f3d2e5763/html5/thumbnails/33.jpg)
Describing the program:
Logic Model for M&E
12/12/2012 33
![Page 34: Monitoring Flour Fortification Programs: An Overview · 1 Monitoring Flour Fortification Programs: An Overview Second Africa FFI Meeting 26 November 2010 Laird J. Ruth International](https://reader033.fdocuments.us/reader033/viewer/2022043008/5f968b9d97ca8a4f3d2e5763/html5/thumbnails/34.jpg)
Monitoring & evaluation pipeline
12/12/2012 34
![Page 35: Monitoring Flour Fortification Programs: An Overview · 1 Monitoring Flour Fortification Programs: An Overview Second Africa FFI Meeting 26 November 2010 Laird J. Ruth International](https://reader033.fdocuments.us/reader033/viewer/2022043008/5f968b9d97ca8a4f3d2e5763/html5/thumbnails/35.jpg)
Collecting credible data
• Depend on the purpose of the
evaluation
• Can be simple and not costly … or very
complex and expensive
12/12/2012 35
![Page 36: Monitoring Flour Fortification Programs: An Overview · 1 Monitoring Flour Fortification Programs: An Overview Second Africa FFI Meeting 26 November 2010 Laird J. Ruth International](https://reader033.fdocuments.us/reader033/viewer/2022043008/5f968b9d97ca8a4f3d2e5763/html5/thumbnails/36.jpg)
Evaluation expectations
• To reduce the prevalence of nutrient deficiencies >80%
population coverage of quality fortified flour/flour products
must be sustained for at least one year at a time.
• Maximum public health benefits will take multiple years.
• Communication and social marketing critical (must be on-
going).
• The trend in public health impact will depend on the initial
prevalence of nutrient deficiency.
36
![Page 37: Monitoring Flour Fortification Programs: An Overview · 1 Monitoring Flour Fortification Programs: An Overview Second Africa FFI Meeting 26 November 2010 Laird J. Ruth International](https://reader033.fdocuments.us/reader033/viewer/2022043008/5f968b9d97ca8a4f3d2e5763/html5/thumbnails/37.jpg)
Baseline Survey
Impact Survey
TIME (implementation of intervention)
Length depends on nutrient and indicator
Monitoring for Program Management
Ideal Program Monitoring and
Evaluation System
![Page 38: Monitoring Flour Fortification Programs: An Overview · 1 Monitoring Flour Fortification Programs: An Overview Second Africa FFI Meeting 26 November 2010 Laird J. Ruth International](https://reader033.fdocuments.us/reader033/viewer/2022043008/5f968b9d97ca8a4f3d2e5763/html5/thumbnails/38.jpg)
Example with flour fortification and
reduction of iron deficiency in women
• Baseline and survey 2-3
yrs after
• Baseline and survey 2-3
yrs after, looking at
potential confounding
factors
• Baseline + end survey
with control
• Allow to say if there was
a change in iron
deficiency level or not
• Allow to say that impact
may be related to the
program
• Allow to say that the
impact is more likely
due to the program
![Page 39: Monitoring Flour Fortification Programs: An Overview · 1 Monitoring Flour Fortification Programs: An Overview Second Africa FFI Meeting 26 November 2010 Laird J. Ruth International](https://reader033.fdocuments.us/reader033/viewer/2022043008/5f968b9d97ca8a4f3d2e5763/html5/thumbnails/39.jpg)
Choice of indicators
• Effectiveness indicators are related to
outcomes • Change in behaviours
• Consumption of foods/micronutrients
• Biochemical/ physiological/ functional
– For anemia: hemoglobin, serum ferritin,
inflammatory responses (CRP, AGP) and
others if budget allows
12/12/2012 39
![Page 40: Monitoring Flour Fortification Programs: An Overview · 1 Monitoring Flour Fortification Programs: An Overview Second Africa FFI Meeting 26 November 2010 Laird J. Ruth International](https://reader033.fdocuments.us/reader033/viewer/2022043008/5f968b9d97ca8a4f3d2e5763/html5/thumbnails/40.jpg)
How to decide which indicators to
use to determine impact?
Consider: – Can on-going data be collected on biological
indicators (e.g. hemoglobin, serum ferritin or folate) to assess trends in iron status of population?
– Local capacity – laboratory, personnel, field logistics, budgetary resources
– Local culture (e.g. collection of venous vs. capillary blood)
– How are data collected? Are data collected through survey or on-going program based or sentinel monitoring system?
• staff training and assuring data quality
– Will indicator show ―timely‖ change? How often to report to stakeholders?
![Page 41: Monitoring Flour Fortification Programs: An Overview · 1 Monitoring Flour Fortification Programs: An Overview Second Africa FFI Meeting 26 November 2010 Laird J. Ruth International](https://reader033.fdocuments.us/reader033/viewer/2022043008/5f968b9d97ca8a4f3d2e5763/html5/thumbnails/41.jpg)
12/12/2012 41
Data sources for evaluation
• Population based monitoring (GOLD STANDARD):
Periodic national/sub-national cluster surveys
• Program based monitoring:
Health Center based (e.g. 1st trimester pregnant women). Sentinel health centers.
Mothers of children seen in Health Center
School based monitoring. (Sentinel schools)
Large employers of female workforce. (Sentinel worksites)
*adapted from Abe Parvanta, 2003
![Page 42: Monitoring Flour Fortification Programs: An Overview · 1 Monitoring Flour Fortification Programs: An Overview Second Africa FFI Meeting 26 November 2010 Laird J. Ruth International](https://reader033.fdocuments.us/reader033/viewer/2022043008/5f968b9d97ca8a4f3d2e5763/html5/thumbnails/42.jpg)
Prevalence of iron deficiency in women
of childbearing age by per capita/month
consumption of white flour
* P<0.05
* p<0.05
*
![Page 43: Monitoring Flour Fortification Programs: An Overview · 1 Monitoring Flour Fortification Programs: An Overview Second Africa FFI Meeting 26 November 2010 Laird J. Ruth International](https://reader033.fdocuments.us/reader033/viewer/2022043008/5f968b9d97ca8a4f3d2e5763/html5/thumbnails/43.jpg)
Example from flour fortification
program in South Africa
12/12/2012 43 Slide adapted from France Begin, UNICEF
![Page 44: Monitoring Flour Fortification Programs: An Overview · 1 Monitoring Flour Fortification Programs: An Overview Second Africa FFI Meeting 26 November 2010 Laird J. Ruth International](https://reader033.fdocuments.us/reader033/viewer/2022043008/5f968b9d97ca8a4f3d2e5763/html5/thumbnails/44.jpg)
Micronutrient Status of non-pregnant women
of reproductive age before and after
implementation of national flour fortification
Pre-fortification Period (95% CI)
Post-fortification Period (95% CI)
p-value
Serum Folate < 2.5 ng/ml
16.3% 0% 0.001
Red Blood Cell Folate
< 164 ng/ml
26.4% 1.9% 0.000
Serum Ferritin <12.0 µg/ml
25.0% 25.0% 0.74
Hemoglobin <11.0 g/dl
7.5% 5.0% 0.51
Vitamin B12 <145 pg/ml
6.3% 11.3% 0.16
Modjadji SEP., Alberts RL. Folate and iron status of South African non-pregnant women of childbearing age before and after fortification of foods. SAJCN: Vol 20, No 3; 89, 2007.
Slide adapted from France Begin, UNICEF
![Page 45: Monitoring Flour Fortification Programs: An Overview · 1 Monitoring Flour Fortification Programs: An Overview Second Africa FFI Meeting 26 November 2010 Laird J. Ruth International](https://reader033.fdocuments.us/reader033/viewer/2022043008/5f968b9d97ca8a4f3d2e5763/html5/thumbnails/45.jpg)
45
Neural Tube Defects
Surveillance System
Sayed AR., Bourne D., Pattinson R., Nixon J., Henderson B. Decline in the prevalence of neural tube defects following folic acid fortification and its cost-benefit in South Africa. Birth Defects Res A Clin Mol Teratol. 2008 Apr;82(4):211-6.
– NTD surveillance system was established in 2002
– 12 public hospitals in 4 provinces
– 53,000 births/year were monitored since 2002
– Prevalence of NTDs was reduced by 30.5% after mandatory fortification (p<0.05)
Slide adapted from France Begin, UNICEF
![Page 46: Monitoring Flour Fortification Programs: An Overview · 1 Monitoring Flour Fortification Programs: An Overview Second Africa FFI Meeting 26 November 2010 Laird J. Ruth International](https://reader033.fdocuments.us/reader033/viewer/2022043008/5f968b9d97ca8a4f3d2e5763/html5/thumbnails/46.jpg)
Perinatal Mortality Surveillance System
Causes of death up to seven days of age are recorded through 164 sentinel health care facilities
NTD perinatal mortality decreased by 65.9% (P<0.001) As a control, the perinatal mortality rate of hydrocephalus,
unrelated to NTDs, did not change significantly (P=0.77)
0.419
0.143
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
Rate
per
1000 B
irth
s
Pre Fortif ication (2001-2003) Post Fortif ication (2005-2006)
Reduction in Perinatal Mortality Rates from NTDs in South Africa
Sayed AR., Bourne D., Pattinson R., Nixon J., Henderson B. Decline in the prevalence of neural tube defects following folic acid fortification and its cost-benefit in South Africa. Birth Defects Res A Clin Mol Teratol. 2008 Apr;82(4):211-6.
Slide adapted from France Begin, UNICEF
![Page 47: Monitoring Flour Fortification Programs: An Overview · 1 Monitoring Flour Fortification Programs: An Overview Second Africa FFI Meeting 26 November 2010 Laird J. Ruth International](https://reader033.fdocuments.us/reader033/viewer/2022043008/5f968b9d97ca8a4f3d2e5763/html5/thumbnails/47.jpg)
How often to evaluate?
• Done periodically but not frequently
• Elaborates on the information on
program implementation and impact
generated through the ongoing
monitoring system
• it is often targeted to problems identified
through the monitoring process
12/12/2012 47
![Page 48: Monitoring Flour Fortification Programs: An Overview · 1 Monitoring Flour Fortification Programs: An Overview Second Africa FFI Meeting 26 November 2010 Laird J. Ruth International](https://reader033.fdocuments.us/reader033/viewer/2022043008/5f968b9d97ca8a4f3d2e5763/html5/thumbnails/48.jpg)
12/12/2012 48
• Once process monitoring system
indicates:
– Adequate program implementation
• Need regular production and distribution of
fortified product
• Usually after 1 yr, more often after 18-24 mo
– Adequate program coverage for minimum
period (depends on target nutrient)
When to perform an
impact evaluation?
Not Before!
![Page 49: Monitoring Flour Fortification Programs: An Overview · 1 Monitoring Flour Fortification Programs: An Overview Second Africa FFI Meeting 26 November 2010 Laird J. Ruth International](https://reader033.fdocuments.us/reader033/viewer/2022043008/5f968b9d97ca8a4f3d2e5763/html5/thumbnails/49.jpg)
Justifying and sharing
conclusions
• Critical to sustain successful aspects
and adapt program if improvements
required
• Compare data from various sources (if
available)
• Get stakeholders involved to embrace
results and take actions
• Communicate and disseminate
12/12/2012 49
![Page 51: Monitoring Flour Fortification Programs: An Overview · 1 Monitoring Flour Fortification Programs: An Overview Second Africa FFI Meeting 26 November 2010 Laird J. Ruth International](https://reader033.fdocuments.us/reader033/viewer/2022043008/5f968b9d97ca8a4f3d2e5763/html5/thumbnails/51.jpg)
51
Helpful Resources:
• NEW! Intro to Program Evaluation for PH Programs—
A Self-Study Guide: http://www.cdc.gov/eval/whatsnew.htm
• Innovation Network:
http://www.innonet.org/
• W.K. Kellogg Foundation Evaluation Resources: http://www.wkkf.org/programming/overview.aspx?CID=281
• University of Wisconsin-Extension: http://www.uwex.edu/ces/lmcourse/
![Page 52: Monitoring Flour Fortification Programs: An Overview · 1 Monitoring Flour Fortification Programs: An Overview Second Africa FFI Meeting 26 November 2010 Laird J. Ruth International](https://reader033.fdocuments.us/reader033/viewer/2022043008/5f968b9d97ca8a4f3d2e5763/html5/thumbnails/52.jpg)
52
Thanks!
Monitoring and Evaluating
Food Fortification Programs:
General Overview Technical Consultation July 7, 2006
USAID - www.a2zproject.org
Return to Meeting Presentations