Chapter 5

From Action to Impact

Opening Discussion

• Read the book case about Year Up and answer the following questions: How important was it for Year Up to grow

according to a systematic growth plan? What if the study at the end of Phase One had

not shown a positive effect on wages?

Chapter Outline

• Framework for venture evaluation

• Approaches to social venture effectiveness

• Outcome and impact evaluation

• Monetizing outcome and impact

• Increasing social venture impact: scaling

Chapter Outline

• Framework for venture evaluation

• Approaches to social venture effectiveness

• Outcome and impact evaluation

• Monetizing outcome and impact

• Increasing social venture impact: scaling

Identifying Impact and Org. Effectiveness…• Is this an easy task to identify how well

missions / goals have been fulfilled? For profit organization?

Identifying Impact and Org. Effectiveness…• Is this an easy task to identify how well

missions / goals have been fulfilled? Individual?

Evaluation of Venture Effectiveness

• Early attempts at evaluation usually considered only one measure of performance.

• But overall performance and effectiveness usually depends on a number of factors.

• Evaluation should be linked to the venture’s theory of change.

• This will be tied to the venture’s program logic model.

The Logic Model and Evaluation• The framework for linking evaluation to the

theory of change and logic model

Values-Based Evaluation Models• Competing values model: based on management

values

Values-Based Evaluation Models• The stakeholder satisfaction model can

take the values of a variety of stakeholders into account.

• The model gets more complex as the number and diversity of stakeholders increases.

• Stakeholders are also likely to have competing values.

• The organization can profile the nature and strength of stakeholders’ values.

Multi-Attribute Evaluation Models• The Balanced Scorecard for nonprofits:

mission, via the customer perspective, is key.

Multi-Attribute Evaluation Models

• Example of Dashboard elements

(from Year Up)

Multi-Attribute Evaluation Models

• Example of another Dashboard

Outcome and Impact Evaluation

• Entails measuring what happens to recipients after programs have been delivered

• Becomes more difficult as time goes by due to scope growth, changes in target mkt, etc.

• Outcomes can include recipient’s knowledge, attitudes, skills, intentions, behavior

• Impacts are broad, long-term changes that occur as a result of a program

• Can include changes in a community’s health, income, environment, and so on

Evaluation Research Designs

• Designs differ in the degree to which causation (the program had the desired effect) can be inferred.

• Causality requires Covariation: the behavior changed in tandem

with the program. Proper time order: the program occurred

before the behavior change. Nonspuriousness: behavior change was not

caused by any other factor.

Evaluation Research Designs

• Experiment is the strongest design: has treatment and control groups and before and after tests.

• Quasi-experiment relaxes some of the requirements of the experimental design.

• Nonexperiment may need to relax most of the features of experiments; these could include Before-and-after measures Panel studies Post-program comparisons

Evaluation Research Designs• Comparison of energy use before and after

program implementation

: National Action Plan for Energy Efficiency (p. 4-2).

Monetizing Outcome and Impact

• Assigns monetary value to program outcomes or impacts.

• Provides a number of advantages: Comparison of ventures Combining value of ventures Calculation of efficiency Communication with stakeholders

• Important to note that not all benefits may be monetizable.

Monetizing Outcome and Impact• Cost benefit analysis: requires

comprehensive measurement of cost and program impacts Includes primary, secondary, direct, indirect,

tangible, intangible Net benefits to participant stakeholders and

society

• Social return on investment (SROI): calculates blended value, composed of Enterprise value: revenue from business activities Social purpose value: monetized value of venture

for society

Monetizing Outcome and Impact• Examples of problems in monetization…

• www.robinhood.org – Reducing poverty in NYC

Why is it hard to monetize the above?

What might they be able to monetize?

Scaling Social Impact

• Ventures usually launched in one or a limited number of locations

• After successful launch of venture and evaluation of effectiveness

• Reasons to consider expansion, or scaling, of impact: Chance to accomplish more of the mission Demands from beneficiaries Pressure from funders

Scaling Social Impact

• Caution: poorly planned or unsupported scaling can result in strains in organization or program failure.

• Need to consider organizational capacity and resources to scale, factors such as Staffing Key stakeholders Earnings Market

Framework for Scaling Social Impact• There are a variety of approaches to scaling.

Framework for Scaling Social Impact• Scaling can be indirect or direct.

• Indirect methods Are simpler and faster than direct. Try to elicit others to bring about change. Could lobby, raise public awareness, organize

network.

• Direct methods Provide services in other geographical areas. Require resources and a transferable program

model. Could use promotion, affiliation, or branching.

Discussion

• Consider the points we have made about social venture effectiveness, and answer the following questions: Are some models of effectiveness inherently

better than others? What factors might influence the choice of an

effectiveness model? How might this choice influence the type of

scaling you might choose?