How Logical Is Your Logic Model? Developing Useful and Robust Project Logic Models 2/27/20141.

1

How Logical Is Your Logic Model?

Developing Useful and Robust Project Logic Models

2/27/2014

2

The work of TEAMS is supported with funding provided by the National Science Foundation, Award Number DRL 1238120. Any opinions, suggestions, and

conclusions or recommendations expressed in this presentation are those of the presenter and do not necessarily reflect the views of the National Science

Foundation; NSF has not approved or endorsed its content.

2/27/2014

3

Strengthen the quality of the MSP project evaluation and build the capacity of the evaluators by strengthening their

skills related to evaluation design, methodology, analysis, and reporting

2/27/2014

4

• Online Help-Desk for submitting requests• Website at: teams.mspnet.org • Webinar series targeted to specific

evaluation topics• Tiered technical assistance for differentiated

services• Instrument review and sharing

2/27/2014

5

How Logical Is Your Logic Model?Developing Useful and Robust

Project Logic Models

Dave Weaver, DirectorRMC Research Corporation

Portland, OregonFebruary 27, 2014

2/27/2014

6

Webinar Objectives

• Provide a rationale for the need for a well-defined logic model

• Summarize characteristics of a well-defined theory of action and logic model

• Provide an opportunity to examine your current theory of action and logic model

• Discuss strategies for developing useful and robust logic models

2/27/2014

7

Why Are Logic Models Important?• Common Guidelines for Education Research and

Development– U.S. Department of Education, National Science Foundation.

(August, 2013). – http://ies.ed.gov/pdf/CommonGuidelines.pdf

“The proposal should include a description of the initial concept for the planned investigation, including a well-

explicated theory of action or logic model. The concept and logic model should identify key components of the

intervention and should describe their relationships, theoretically, and operationally.”

2/27/2014

http://ies.ed.gov/pdf/CommonGuidelines.pdf



8

What does it mean?

“well-explicated theory of action or logic model”

2/27/2014

9

It starts with theTheory of Action

A logic model is only as good as the theory of action that it is

based on

2/27/2014

10

What Is a Theory of Action?• Collective belief about causal relationships

between action and desired impacts– Simple:

If . . . Then . . .

– Complex:If . . . and . . . and . . . and . . . Then . . .

• Collaborative interpretation of the literature

2/27/2014

Characteristics of a RobustTheory of Action

• Describes project impact as close to the primary target audience as possible– Example: A description of what students do to learn

• Recognizable when it is going on– Observable

• Defines fidelity of project implementation• Can be a testable hypothesis• Believable– Interpretation of current literature

112/27/2014

Math Example (Common Core State Standards)

If teachers use developmentally appropriate yet challenging tasks and activities that engage students in:• Justifying—Explaining and justifying their reasoning

mathematically• Generalizing—Identifying and verifying conjectures or predictions

about the general case• Representing—Using representations (symbolic, notation, graphs,

charts, tables, and diagrams) to communicate and explore mathematical ideas

• Applying—Applying mathematical skills and concepts to real-world applications

Then student achievement and interest in mathematics will increase.

2/27/2014 12

Does it describe project impact as close to the primary target audience as possible?

If teachers use developmentally appropriate yet challenging tasks and activities that engage students in:• Justifying—Explaining and justifying their reasoning mathematically• Generalizing—Identifying and verifying conjectures or predictions





2/27/2014 13

Is it recognizable when it is going on?






2/27/2014 14

Does it define fidelity of project implementation?






2/27/2014 15

Can it be a testable hypothesis?If teachers use developmentally appropriate yet challenging tasks and activities that engage students in:• Justifying—Explaining and justifying their reasoning mathematically• Generalizing—Identifying and verifying conjectures or predictions





2/27/2014 16

Is it believable?If teachers use developmentally appropriate yet challenging tasks and activities that engage students in:• Justifying—Explaining and justifying their reasoning mathematically• Generalizing—Identifying and verifying conjectures or predictions





2/27/2014 17

Science Example• Students learn science when they:–Articulate their initial ideas,–Are intellectually engaged with important

science content,–Confront their ideas with evidence,– Formulate new ideas based on that

evidence, and–Reflect upon how their ideas have evolved

2/27/2014 18

19

Online Survey 1• Does your project have a theory of action

statement?• If yes, to what extent does it meet the

criteria?– Criteria for a robust theory of action:• Describes impact close to the primary audience• Is recognizable in practice• Defines fidelity of implementation• Can be a testable hypothesis• Is believable—Shared interpretation of literature

2/27/2014

What Is a Logic Model?• A diagram that shows the logical connection

between project resources, activities, outcomes, and expected impacts

• Incorporates a primary theory of action• Can be viewed as a collection of theories of

actions• Answers the question:

2/27/2014 20

Why would the planned activities be expected to have the desired impacts?

Black Box Issues

Basic Logic ModelInputs Activitie

sOutputs Outcome

sImpact

s

Resources

ActivitiesProximal Impacts

Distal Impacts

2/27/2014 21

22

Logic Model Columns

• Resources—What resources are or could reasonably be available?

• Strategies and Activities—What will the activities, events, etc. be?

• Outputs—What are the initial products of these activities?• Short-Term Outcomes—What changes are expected in the

short-term?• Long-Term Outcomes—What changes are wanted after

initial outcomes?• Impacts—What are hoped for changes over long haul?

2/27/2014

23

Characteristics of a Robust Logic Model

• Incorporates a primary theory of action• Shows the logical connection between project

resources, activities, outcomes, and expected impacts– Is a collection of If … Then… statements

• Answers the question: Why would the planned activities be expected to have the desired impacts?

• Serves as a framework for project evaluation• Guides fidelity of implementation

2/27/2014

Making Mathematical Reasoning Explicit (MMRE) Logic Model

Does it incorporate the primary theory of action for the project?

If teachers use rich mathematical tasks combined with purposeful and probing questions to engage students in discourse and the use of the tools (notation, symbolization, graphs, charts, etc.) of mathematics to:• Explain and justify their mathematical

reasoning (justification)• Develop and verify mathematical

generalizationsThen students’ reasoning skills and student achievement in mathematics will increase.

Does it show the logical connections?

Does it show why the planned activities would be expected to have the desired impacts?

Does it serve as a framework for project evaluation?

Does it guide fidelity of implementation?

30

Online Survey 2• Does your project have a logic model?• If yes, to what extent does it meet the

criteria?– Criteria for a robust logic model:• Incorporates the project theory of action• Shows the logical connections between project

resources, activities, outcomes, and expected impacts• Explains why the planned activities would be expected

to have the desired impacts• Serve as a framework for the project evaluation• Guides fidelity of project implementation

2/27/2014

31

Strategies for Logic Model Development

• Engage project stakeholders• Begin by drafting a primary theory of action

statement– Go back to the characteristics of a robust theory

of action• Ask probing questions to help stakeholders

think about why activities should work• Take notes and work toward consensus

2/27/2014

32

Suggested Process• First half-day session with stakeholders– Learn about project activities, objectives, and goals– Introduce logic model development process– Gather input for project theory of action

• Between Sessions– First draft of Theory of Action and Logic Model

• Second half-day session– Gather input on draft– Establish commitment to theory of action statement

• “Do you believe that this statement accurately states your collective belief about why your project will achieve its desired impacts?”

2/27/2014

Go Back to the Characteristics of a Robust Theory of Action

• Describes project impact as close to the primary target audience as possible– Example: A description of what students do to learn

• Recognizable when it is going on– Observable

• Defines fidelity of project implementation• Can be a testable hypothesis• Believable– Interpretation of current literature

332/27/2014

34

Some of My Favorite Questions!

• What would it look like in the classroom if your professional development was effective?

• What kind of cognitive engagement would students be involved in?

• How would you know your project has been successful?

• When this project is over what will you be able to point to as a legacy of the project?

• What is it about your professional development that you think is effective?

2/27/2014

35

Using the Technology• Taking Notes– Capture the language of the stakeholders– Don’t be afraid to paraphrase– Use outliner view

• Drafting the Logic Model– Don’t use a table format—too linear and

constraining– Use graphic software

• Visio• MS Word

2/27/2014

36

Questions!Please use your chat box to submit questions for Dave

2/27/2014

37

John T. Sutton, PI Dave Weaver, Co-PI

RMC Research Corporation633 17th Street Suite 2100 Denver, CO 80202-1620

RMC Research Corporation111 SW Columbia Street Suite 1030 Portland, OR 97201-5883

Phone: 303-825-3636 Toll Free: 800-922-3636 Fax: 303-825-1626 Email: [email protected]

Phone: 503-223-8248 Toll Free: 800-788-1887 Fax: 503-223-8399 Email: [email protected]

2/27/2014

How Logical Is Your Logic Model? Developing Useful and Robust Project Logic Models 2/27/20141.

Documents

Transcript of How Logical Is Your Logic Model? Developing Useful and Robust Project Logic Models 2/27/20141.