OnGoing_OpenHSevaluation_05june2014.pdf

7/25/2019 OnGoing_OpenHSevaluation_05june2014.pdf

1/17

Success depends on knowing what works.

Bill Gates


2/17

EVALUATIONQuantifies/Qualifies the

outcome/impact of the OHS program

and measure/determine whetherresources were used efficiently andeffectively.

Did we hit the target?


3/17

OHS EVALUATION QUESTIONS:1. Did it work to achieve

expected outcomes & impacts?2. How did it work?

3. Can it improve?


4/17

Sphere of influence

Outcomes: measure changes intarget groupspractices

Impacts: measure ultimateeffects of program on targetpopulation

Teachers use OHScurriculum as planned

Performance of OHSStudents at par with or

better than regularstudents in majorsubjects

No. of OHS graduateswent to college

No. of OHS graduatesemployed (locally oroverseas)

Education/Long Term Outcomes

Short Term Outcomes


5/17

ImpactsOutcomes

OutputsActivities

Control Influence

M&E Focus

Greater

objective

Contribution

What did you do? What did they do?

How did they

change?

BSE-OHS

PROGRAM INPUTSOHS STUDENTS

& LEARNING

FACILITATORS

MDG/EFA

What happened

beyond the

program scope?


6/17

ImpactsOutcomes

OutputsActivities

Control Influence

OHS M&E Focus

Greater

objective

Contribution

Partic ipat ion Rate

Net Enrolm ent

Rate

Inc rease access &

impro ve qual i ty of

educat ion

No. of OHS Gradu ates

in Col lege and

Employed

Teachers use the

OHS curriculum as

prescribed

Student Learning

Competencies

(OHS vs Regular)

Policies & Trainings

Curriculum

Learning Modules

Learning mode/system

Assessment Methods

GAPS

Did it work?

How did it work?

Can it improve?


7/17

1) Determine the viability of

OHSP as an ADM strategy.Why ADM is a very promising strategy?

What works in the OHS system?

DATA: Strengths and success factors of OHSP as atipping point for secondary education.

OHSP EVALUATION OBJECTIVES


8/17

External factors influence

outcomes

OUTPUT:Training for

OHSteachers

OUTCOME:Home

schooledlearners at

par withregular

students

IMPACT:Home

schooledgraduates

accepted incollege


9/17

2) Investigate the inefficiencies

of OHSP as an ADM system.What is not currently working in OHSP?

How might the efficiency and effectiveness of OHSP be

improved?DATA: Weaknesses, gaps, loopholes/bottlenecks in OHSP

Social norms and influence on demand

Legal & policy framework

National budget available

Strong governance framework

Available supply chain & human resources for O&M

Financial barriers at schools


10/17

Assess barriers from all angles Social norms and influence on demand

Legal & policy framework

National budget available

Strong governance framework

Available supply chain & human resources for O&M

Financial barriers at schools Social, cultural barriers to demand


11/17

3) Establish whether the

students of OHSP arecompetitive and/or at par with

regular students.Whats happening to student learning?

How competitive are ADM students/graduates with

regular students/graduates?DATA: Tracking of student learning outcomes and

OHSP graduates


12/17

4) Develop propositions to

improve and adapt the OHSPfor K to 12.

How best to improve the OHS System to adjust it tothe needs of children with specific learning abilities?

How might OHSP be adapted to align with K to 12reform agenda specifically to Senior High School?

DATA: Action-Recommendations to improve ADMpractice using the OHS system


13/17

Data Gathering Methodology Survey of 300 (out of 855) schools (public and private) with

promising results (adopting OHSP) including schoolswhich did not take off.

Qualitative /in-depth study using appreciative inquiry to

discover the strengths and potentials of OHSP and/or OSTto investigate the weaknesses and barriers to make OHSPsustainable and adaptable for K to 12.

Key Informants: School Administrators, Teachers and

Partner Organizations Survey and/or interview of 400 students (6,179) and

graduates (22,159) of OHSP and the regular High School;select 3 students per school (i.e., independent andinstructional learners)


14/17

Selecting a comparison group Seek similar characteristics as target group

Similar school size, wealth, public/private, similarlocation/climate, similar access to secondary

education Use national census, other data

Typically done in quasi-experimental way (ratherthan pure random selection)

Random selectionwe are making sure oursample represents all in terms of age, gender,SES, and other variables we do not even know tomeasure!


15/17

Implementation Activities Consultation with the BSE from concept note to final

report presentation.

Desk review of literature on OHSP as an innovativeADM solution for improving secondary education.

Preparation of evaluation design with framework,methodologies to be utilized, scope of evaluationtasks, instrument preparation, data gathering, plan for

data analysis; selection of key informants andsecondary schools.

Data gathering in sample schools using appreciativeinquiry, open space technology, focus group discussion

and survey instruments.


16/17

Analytical review of the immediate impact and

changes on secondary education services in terms ofprogram management, teacher and studentperformance, competitive advantage OHSP graduatescompared with regular students and other relevant

performance outcomes. Draft evaluation report for approval.

Preparation of final evaluation report.

Packaging and printing of approved OHSP Evaluation

Report. Forum to present the OHSP Evaluation findings and

action-recommendations to DepED-BSE officials andother key project stakeholders.


17/17

TIMELINEDELIVERABLES TIMEFRAME

Consultation Meetings beginning July 24, 2012

Desk Review August, 2012

Evaluation Design &

Instruments

September 2012

Data Gathering October to December 2012

Data Consolidation and

Analytical Review

January 2013

Draft Evaluation Report EO January 2012

Final Evaluation Report EO March 2013

Packaging & printing EO April 2013

Forum on OHSP Viability for

K to 12

EO June 2013

OnGoing_OpenHSevaluation_05june2014.pdf

Documents

Transcript of OnGoing_OpenHSevaluation_05june2014.pdf