Evaluating Advocacy: A Model for Public Policy

Initiatives

Slide 2Innovation Network, Inc.

Who We Are: Innovation Network National nonprofit organization Committed to evaluation as a tool for

empowerment Work with nonprofits at the grassroots level Build evaluation capacity of nonprofits and

funders so they can better serve their communities

Practice a participatory approach

Innovation Network (202) 728-07271625 K St. NW www.innonet.orgWashington, DC 20006 info@innonet.org

Slide 3Innovation Network, Inc.

Overview/Agenda

Discuss challenges inherent to advocacy evaluation

Review traditional evaluation models and their fit for evaluating advocacy

Share lessons learned from our work Evaluation planning Data collection strategies Feedback loops and reflection

Shift in emphasis

Slide 4Innovation Network, Inc.

A Case Study

Coalition for Comprehensive Immigration Reform (CCIR)

Evaluation funded by The Atlantic Philanthropies

Began the evaluation in Fall 2005

Slide 5Innovation Network, Inc.

THE CHALLENGES

Slide 6Innovation Network, Inc.

The Challenges

I continue to find my greatest pleasure, and so my reward, in the work that

precedes what the world calls success.-- Thomas Edison

Slide 7Innovation Network, Inc.

The Challenges

Success is to be measured not so much by the position that one has reached in

life as by the obstacles which he has overcome while trying to succeed.

-- Booker T. Washington

Slide 8Innovation Network, Inc.

The Challenges

Ability is nothing without opportunity.-- Napoleon Bonaparte

Slide 9Innovation Network, Inc.

The Challenges

If you don't risk anything, you risk even more.

-- Erica Jong

Slide 10Innovation Network, Inc.

TRADITIONAL MODELS

Slide 11Innovation Network, Inc.

Continuous Learning Cycle

Logic Model

Analysis, Reflection &Improvemen

t

Evaluation

Planning

Data Collection

Slide 12Innovation Network, Inc.

The Logic Model

Slide 13Innovation Network, Inc.

Chain of Outcomes

Residents improve basic job skills and

identify jobs & career paths of interest

KNOWLEDGE/ ATTITUDE

Participants use post-training services, e.g. job placement, child care, transportation

BEHAVIOR

Closer in Time Easier to Measure

More Attributable to Project

Increased number of community residents obtain and retain jobs that support families

CONDITION

More Distant in TimeHarder to Measure

Less Attributable to Project

Slide 14Innovation Network, Inc.

LESSONS LEARNED

Slide 15Innovation Network, Inc.

Continuous Learning Cycle

Planning

Real-Time Feedback

and Reflection

Data Collectio

n

Slide 16Innovation Network, Inc.

8-Step Advocacy Evaluation Plan1. Locate issue on the

policymaking continuum

2. Identify type of policy change

3. Understand role of the group/network in policymaking

4. Clarify dimensions of success relevant to stakeholders

5. Develop chain of outcomes, leading to the intended change

6. Identify indicators & measurement systems

7. Identify most effective strategies

8. Prioritize

Slide 17Innovation Network, Inc.

Spheres of Influence

Decision

Makers

General Public

Beneficiaries, Stakeholder Communities

Competing Organizations

Organization and Allies

Thinktanks and Research

Institutions

Media

Business and Industry

Key Players:

© Innovation Network, 2006

Slide 18Innovation Network, Inc.

Continuous Learning Cycle

Planning

Real-Time Feedback

and Reflection

Data Collectio

n

Slide 19Innovation Network, Inc.

Data Collection Strategies Mixed-method approach

Deeper focus on “real-time” data Observations/monitoring Frequent opportunities for data collection

De-brief protocol Opportunity windows/intense periods Capturing multiple perspectives

Slide 20Innovation Network, Inc.

Continuous Learning Cycle

Planning

Real-Time Feedback

and Reflection

Data Collectio

n

Slide 21Innovation Network, Inc.

Reporting and Feedback Loops

Frequent, informal reporting Learning briefs; research Approximate “real-time” Transparency/inclusion Face-to-face

Slide 22Innovation Network, Inc.

SHIFT IN EMPHASIS

Slide 23Innovation Network, Inc.

Before…

Outputs

Outcomes

Slide 24Innovation Network, Inc.

Now…

Outputs

Outcomes

Slide 25Innovation Network, Inc.

Finding Balance…

Outputs Outcomes

Slide 26Innovation Network, Inc.

Advocacy Evaluation Project

Online resources Indicators for advocacy Piloting tools Advisory committee

www.innonet.org/advocacy

Slide 27Innovation Network, Inc.

Any Questions?

www.innonet.org

Jennifer Bagnell Stuartjabstuart@innonet.org

Ehren D. Reedereed@innonet.org

Slide 28Innovation Network, Inc.

Thanks for Your Participation!

Measure results. Make informed decisions. Create lasting change.

Innovation Network, Inc.1625 K St. NW, 11th Floor

Washington, DC 20006(202) 728-0727

www.innonet.org