TAF Impact Evaluation WorkshopSingapore, March 23-26, 2010

What is the program trying to accomplish?

Who/what is the target population?What is the need or gap that the

program meets? What does the program provide?

What is the model for expected change?

Hypotheses or policy questions to be answered through evaluation

How the knowledge generated will be used

What is the intervention, for the purposes of the evaluation?

Target unit/unit of interventionUnit(s) of observation

How will you define “treatment” and “comparison” groups? Or, what is the counterfactual?

Are you concerned with spillovers?Which variables will you measure to

confirm that both groups are similar in the absence of the intervention?

What will you measure? How?What is the expected effect size?

(expected change in outcomes?)How long do you think it will take for

change in outcomes to appear?

What is the sample for the study? And from where/whom will you collect data?

At what time intervals will you make measurements?

What do you think will affect your power to detect a significant change in outcomes? (e.g. number of clusters, cluster size, time required for change to appear, expected size of effect, intra-cluster correlation)

Do you expect any spillovers?

When will you start and end? What needs to be accomplished before you can implement?

What are the steps in implementation of the evaluation?

When do you need to implement the intervention, collect data, etc?

What is your plan for dissemination?

Who are the stakeholders in the evaluation?

Who will manage the evaluation? Who will provide implementation

support? Who will provide data collection and

quality assurance services? Do you need design/statistical support

from technicians? Is training required?

What will drive the cost of the evaluation?

What categories of spending do you expect?

Operational, political, research challenges