Leveraging the KnowHow2GO Network Survey in

Evaluation and Program Improvement Efforts:

The Case of the Washington State College Access

Network

American Evaluation Association

Hilary Loeb hloeb@collegesuccessfoundation.org

Kelly Bay-Meyer kbay-meyer@collegesuccessfoundation.org

Lee Lambert llambert@collegesuccessfoundation.org

October 25, 2012 1

Presentation Overview

Mission and work of Washington

College Access Network (WCAN)

WCAN evaluation planning

Implementing the KnowHow2GO survey

Reflections about institutionalizing the

KnowHow2GO survey

2

WCAN Mission

WCAN strives to improve access to and

preparation for higher education

through a network of community

organizations that collaborates to

promote and support the use of best

practices, leverage training

opportunities and public policies

ensuring the success of each student.

3

WCAN Logic Model

4

WCAN Theory of Change

5

Local college access

networks in every

region of the state.

Short Term Goal

Coordinated delivery of

early college

awareness information

to low-income

students.

Intermediate Goal

Increase in the number

of low-income first

generation student

graduating from

college.

Long Term Goal

• Socioeconomic status no longer a predictor for educational outcomes.

• 60% of Washingtonians with high-quality degrees and credentials by the

year 2025.

WCAN’s Long Term Outcomes

About CSF’s Research and Evaluation

Department

Small team of doctoral-level staff and

university interns

Supported by operational and grant

funding

Influenced by utilization-focused

evaluation and appreciative inquiry

Commitment to mixed-methods inquiry

6

Early Implementation Evaluation

Questions

1. What have been the patterns of growth (membership, network, capacity, etc.) in WCAN since the inception of the Network?

2. How have WCAN and partner organizations built awareness of college access among students and organizations in Washington?

3. What have been the patterns of growth in the College Bound Scholarship since the launch of WCAN?

7

Consultation with external and internal advisory group

was a critical step in designing the study.

Early Implementation Inquiry Methods

and Data Sources

8

Triangulation

across

Data Sources

Focus Groups

Grantee

Reports

College Bound

Scholarship Data

Ongoing Discussion of Findings

KnowHow2GO Surveys

KnowHow2GO Network Survey Design

and Data Collection

Developing customized questions about policy context

Administering over Survey Monkey to selected members

Sending email reminders and making phone calls critical to boosting response rate

Providing opportunity to enter into drawing for small incentive

9

KnowHow2GO Network Survey Analysis

and Reporting Descriptive statistical analysis looking at differences

between 2010 and 2011 administrations

Findings summarized in grant reports and proposals

User-friendly reports shared with team

10

Leveraging the KnowHow2GO Network

Survey

Survey adapted for the District of

Columbia Double the Numbers initiative

2012 process improvement in data

collection to provide weekly updates

about responses

11

Leveraging the KnowHow2GO Network

Survey

2012 survey adapted by WCAN team

and is currently in field

Shortening the survey

Crafting items in which specific progress is

attributed to WCAN

12

Reflections about Institutionalizing the

KnowHow2GO Survey

Survey is critical tool in defining annual progress of newer program at CSF

Need to provide ongoing support to team about technical aspects of administering survey

Triangulation through multiple data sources enhances confidence in larger evaluation findings

Eager to benchmark work with college access networks in other states

13