Program Review Advisory BoardOctober 24, 2008

Ronna Vanderslice, Ed.D.Cameron University

rvanderslice@cameron.edu

PRAB

• Provide consultation • Conduct reviews for non-NCATE

institutions and non-affiliated areas each semester

• Conduct reviews for new programs• Monitor changes in state and

national standards and make recommendations for modifications

PRAB Compensation

• Annual stipend of up to $800 + travel– $200 per semester– $200 for each review session attended

• Additional compensation may be provided for reviews outside the semester review session

• Chair stipend $1500 per semester; vice-chair $1000 per semester

The Program Review Process

• Submitted to NCATE (via the web) or OCTP(via CD) a year prior to the visit– Due on September 15 in the fall– Due on February 1 in the spring

• Reviewers prepare independent reports• Lead reviewer compiles final report• BOE access the report for the on-site

visit

The Program Review Process for New Programs• Submit rationale, governing board approval

documentation, and program review to OCTP 6-12 months prior to offering courses

• Review is conducted—recognition decision is determined. Results forwarded to Program Accreditation Committee (PAC)

• If approved by PAC and the full commission, the approval decision becomes effective the next semester, and the State Department of Education is notified of the certification program’s status so that candidate credentials can be appropriately issued

Our State Partnership

• Our state partnership determines the program review process

• The program review process is a critical part of the partnership agreement

• Our current partnership protocol remains in effect until December 2009

Program Areas affiliated with NCATE

• Early childhood education • Elementary education • English/language arts

education • Foreign language education • Mathematics education • Reading specialists and

supervisors • Physical Education

• School administrators • School library media

specialists • School psychologists • Science education • Social studies education • Special education • Teaching English to

speakers of other languages

Oklahoma needs national reviewers on PRAB in each of these areas!

Non-Affiliated Areas

• Agriculture education • Technology education • Career and technology

business • Marketing education • Speech/Drama/Debate • Instrumental/General Music• Vocal/General Music

• Journalism• Driver/Safety education• Art education• Dance education• School Psychometry• School Counseling• Gifted and Talented• Family and Consumer

Sciences

Standards can be located at http://www.sde.state.ok.us/Teacher/ProfStand/pdf/Competencies.pdf

Special Situations

• Programs accredited by another accrediting agency do not submit program reports in those areas

• Dormant Programs

• Small programs

Unit Review vs. Program Review: the Connection

• Standard 1-Candidate Knowledge, Skills, and Professional Dispositions

• Standard 2-Assessment System and Unit Evaluation

The Process is more Similar than Different

• 6 Required Assessments• Alignment is critical

– Standards to assignments– Assignments to Rubrics

• Data summarized • Use of data for program

improvement

How Much Data is Needed?

• Fall 2008 and Spring 2009– 2 years

• Fall 2009 and beyond– 3 years

Overview of what to look for in the Program Report

• Section I—Contextual Information– Faculty qualifications– Course of Study– Relationship of assessments to unit

assessment– Concerns, strengths, or deficiencies

should be noted in Section F of the Recognition Report

Overview of what to look for in the Program Report

• Section II—Assessment Chart• Section III—Standards Assessment

Chart• Section IV—Evidence for Meeting

Standards– are the assessments appropriate for the

cited standards?– do the candidate data demonstrate that

most candidates meet the standard?

Overview of what to look for in the Program Report• Section V—Use of assessment

results to improve candidate achievement and program performance in content, professional KSD– Have they summarized the findings

from the evidence?– Have they interpreted the findings?– Have they made program changes

based on the findings?

Assessment 1

State licensure examinations– there must be an indication of the content

of those exams and how well they are aligned with SPA standards

– The most recent year of data must include the mean and range of total scores and sub-scores on the licensure test

– Our alignment matrices are located at http://www.ok.gov/octp/Program_Accreditation/CEOE_Correlation_Tables/

Assessment 2

• Content Assessment• Grades are now acceptable

– Expectations include a chart listing courses and which SPA standards are addressed in the course with either the course title or brief (1-3 sentence) description of the course

• Portfolio Projects, Comprehensive Exams, etc.

Assessment 3

• Instructional Plan (Unit Plan)

Assessment 4

• Clinical Practice Assessment• Generic instruments that are not

clearly related to the standard are not enough

• Options include using a content addendum, coding the generic instrument to the SPA standards, or design an instrument that includes SPA standards

Assessment 5

• Demonstrates candidates’ ability to impact student learning or provide a supportive learning environment for student learning

• Often TWS• Should address SPA standards

Assessment 6, 7, and 8

• Additional assessment that addresses the standards

• Must have at least one (#6)

• Possibilities include case studies, follow-up studies, employer data, etc.

What Does the Reviewer Do?

• Judge alignment of assessment and candidate data with SPA standards

• Clearly communicate strengths and weaknesses in relation to the standards

• Make as objective an assessment as possible about the degree to which a given program meets the SPA standards, based on candidate performance evidence.

Recognition Decisions

• Recognized• Recognized with conditions• Further Development

Required/Recognized with probation (currently called Decision Deferred)

• Not recognized

The Recognition Report

• Use the Reviewer Worksheet• Check alignment

– Content– Range

• Check for challenging assignments • Check scoring guides– are they aligned to

the standards and are performance expectations clear?

• Are the qualifiers used in the scoring guides distinguishable between levels?

• Determine if data indicate candidate mastery

The Recognition Report• Ensure introductory information

matches their cover sheet• Below each standard, list the

assessments identified to meet that standard and indicate whether they do

• If the standard is not met, include a comment to explain why

• Cite strengths• Evaluate program evidence in Part C• Evaluate their use of assessment results• Add strengths, comments, and concerns

Frequently Asked Questions

• What if there is not adequate data for each assessment?

• What if the unit doesn’t have an 80% pass rate on the OSAT?

• What if a program offers several different levels of programs in the same discipline (e.g. programs at both the undergrad and grad level)?