A proposal of framework to evaluate community-based obesity

prevention interventions in LAC

Marcia Erazo, Margarita Safdie, Mariana Lazo, Angel Caballero, Richard Visser, Armando Barriguete

PACO IIIWorkshop: EPODE Population based interventions

Background

• In Latin America and the Caribbean (LAC) the obesity prevalence has raised at alarmingly pace.

• Community-based interventions are the most promising interventions because they can reach the entire population.

• These interventions are recommended to address the obesity causes rooted in the society and special needs of the individuals.

Background

• Ideal community interventions combines specific health promotion and evaluation theory.

• In LAC, there is no community interventions probably due to lack of evaluation frameworks.

• Limited data are available to guide the implementation and evaluation of these interventions in LAC.

Rationale

• Standardized and comprehensive evaluation frameworks are key for policy-makers, scientists and stakeholders to evaluate community-based programs

Objective

To thoroughly examine already available evaluation frameworks (WHO, EPODE, and Victoria proposals) and related literature, and to provide a specific framework to evaluate community-based interventions in LAC.

Policy Makers

Practitioners

PublicOther Stakeholders

External Groups

Key stakeholders

WHO Steps Evaluation Health promotion programs

1. Describing the proposed program, policy or initiative1. Program logic2. Identify group to undertake the evaluation3. Baseline data

2. Identify key issues and concerns1. Key outcomes indicators2. Stakeholders interests

3. Design: study, data collection4. Data collection as planned5. Data analysis6. Recommendations7. Dissemination of results8. Take action

Page 55 (28)-56(29)

Wimbush and Watson. Evaluation framework for health promotion

• Common themes in evaluation (Policy and strategic planning)– Outcomes and effectiveness

• Other important components (other stakeholders):– Improvement in quality of program– Improvement in program design

Health Education Board Scotland

Tip of Iceberg

Wimbush and Watson. Evaluation framework for health promotion

• Perspectives:– Policy Makers and Strategic Planners: (make decisions and are held

accountable)• What works?• What are the best buys?

– Program Managers: (role: budget holders, responsible for the delivery)• Success, based on achievement of goals, extent of reacheness and

sustainability

– Practicioners: (role: operation and running of programs)• Feedback from people involved in the collaborative action

– Community/Users:• Avenue to provide feedback, involvement: express concerns, satisfaction• Extent to which program meets their perceived needs

Wimbush and Watson. Evaluation framework for health promotion

• Stages Evaluation– Planning– Design and Pilot– Implementation –early start up– Implementation –establishment– Implementation –fully operational – Dissemination

Evaluation Framework for Health Promotion and Disease Prevention Programs. Victoria, Australia

• Describe the program– Program logic

• Evaluation preview:– Process– Outcomes– Implications for Future programs and policy

• Focus evaluation design:– Agreed or common indicators

• Collect Data• Analyse and Interpret Data• Dissemination of the results

EPODE

• By level:– Central– Local– Settings– Child

• To date, all EPODE programs have initiated continuousmonitoring and evaluation practices at central, local andchild levels, as well as in certain settings such as schools.• Developing consistent practices and collecting comparable data

across communities and programs remains an important challenge.

Obesity Rev 2011

Proposal of a framework to evaluate community-based interventions to prevent child obesity LAC

Public Policy

Policy design Policy process

Conceptual frameworkLogic model

Objectives to achieve

Political concernFocused on the electorate

Objectives not declared

Effectiveness Rating

HIGH QUALITYCitizen’s feelings and technically correct

Good governance

Open: Active communication, accessible language, trust between citizens and the institutions.

Participation: Wide community participation during development,Implementation and evaluation stages to achieve a public policy with high quality, pertinence, efficacy.

Efficacy: Actions must achieve results as planned, evaluating their future impact on the community and considering cumulative experience.

Coherence: Policies and actions must be coherent and easy to understand.

We need to evaluate both, Policy design and its effectiveness and Policy Process

Policy design and its effectiveness Policy process

Through rigorous scientificmethods

Through key questionsThat reflect the politicalEnvironment and supportAnd Population satisfaction

Why to evaluate political process?

Politically strategic and democratic: Responsibility and accountability (social, economic and technical).

Managerial: It helps to improve public management and the quality of the program

Transparence: Strong mechanism to create “public value”. Society needs to know the way the money is spent, the results and the impact of the policies.

Participation: It strengthens the communities and individuals. It also improves control and efficiency of public administration.

Public Policy

Plans

Programs

Projects/Interventions Public services

Complexity

+++

+

Political:Decisions

Agreements

Actions

Problem definitionStakeholders

ObjetivesLogic model

NationwideMonitoring and

Surveillance

Investigation

Policy designWHO Ireland Australia CDC EPODE

Describe proposed program

Common themes in evaluation

Describe program Engage stakeholders Multilevel

Identify key issues and concerns

Engage other stakeholders

Evaluation preview Describe program Multi

stakeholders

Design study and data collection

Consider different perspectives

Evaluation design Evaluation design

Continuos monitoring and

evaluation

Collect dataConsider different

stages of evaluation

Collect data Gather credible evidence

Collecting comparable data

accroos the levels, settings

Data analysis Analyse data Justify conclusions Use logic model

Recomendations DisseminationEnsure use and share lessons

learned

Disemination

Take action

Key common issues

Proposal of a Comprehensive framework LAC

Planning Implementing Results Impact

Need / Problem

Intervention

Objectives Actions Intermediate results Final results Health impact

Diagnose and designEvaluation

ImplementationEvaluation

Results evaluation Impact evaluation

Ex Ante Intermedia Ex post

Beneficiaries/Society

Normativeexistance

Stakeholders ParticipationPolitical commitment

Population feelingCommunity believes

Ex Ante Intermedia Ex post

• To analyse and determine if the intervention is adequate to solve the problem

• It pretends to guarantee that the intervention will achieve the goals

It must evaluate:

• Needs• Program theory• Coverage• Potencial effects, results

and impact

• It is a critical analysis about the implementation and early results.

• It analyses the intervention validity, the pertinence of the objectives, the quality of the managerial and monitoring system.

• It encourages to

improve the implementation in order to achieve the goals

It evaluates:

• The successful of the strategy.

• Its flexibility and its capability to adapt the actions according to reality.

• Its efficacy and efficiency

• Managerial mechanisms

1. “The circumstances external to the implementing agency do not impose crippling constraints” 2.“That adequate time and sufficient resources are made available to the programme”3. That the required combination of resources is actually available”4.”That the policy to be implemented is based upon a valid theory of cause and effect” 5.“That the relationship between cause and effect is direct and that there are few if any, intervening links” 6.“That dependency relationships are minimal” 7.“That there is understanding of, and agreement on, objectives” 8.“That tasks are fully specified in correct sequence”9.“That there is perfect communication and coordination”

10.“That those in authority can demand and obtain perfect compliance”

Key questions

Thank you!!