UK’s Program Review Process for Administrative Units: Academic Support

1

UK’s Program Review Processfor

Administrative Units: Academic Support

Mia Alexander-Snow, PhD

Director, Office for Planning and Institutional Effectiveness

2012-2013 Program Review Orientation

2

Topics to Cover Institutional Effectiveness & Assessment program review overview UK’s program review schedule & current process administrative units participating in 2011-2012 cycle program review components: self study, external

review, and implementation plan role of Planning, Assessment, and Institutional

Research sources that inform review 2012-2013 program review calendar contacts questions


2012-2013 Program Review Orientation 3

What is Institutional Effectiveness?

“The institution engages in ongoing, integrated, and institution-wide research-based planning and evaluation processes that incorporate a systematic review of programs and services that (a) results in continuing improvement and (b) demonstrates that the institution is effectively accomplishing its mission” (SACS Core Requirement 2.5).


What is Assessment?Assessment is the process by which …

“the institution identifies expected outcomes for its educational programs and its administrative and educational support services; assesses whether it achieves these outcomes; and provides evidence of improvement based on analysis of those results” (SACS Comprehensive Standard 3.3.1).


What is Assessment at UK?

“Assessment is used to monitor the University’s effectiveness in achieving its mission, vision, and goals. The University and its units shall demonstrate an explicit use of assessment results to facilitate resource allocation and budgeting decisions in support of their strategic plans and to ensure quality enhancement” (UK AR 1:4)


What assessment activities monitor UK’s Institutional Effectiveness efforts?

Strategic Planning

• Identifies and prioritizes the actions the University and its units can take to help it best accomplish the University’s goals and fulfill its mission (AR 1:4)

6



Annual Progress Reporting

Dynamic process for reviewing, updating and revising strategic planning efforts over a 3-5 year period

Answers the following questions in a systematic and thorough way :“How are we doing? ---Actual Results“What things are working? ---Reflection and Analysis“What needs to happen next?” ---Improvement Action



6 yr Periodic Program Review : the primary vehicle for assessment of educational and administrative units and for documentation of institutional effectiveness (AR 1:4).

• 424 Units participate in Program Review• 77 Administrative and Educational Support units• 18 Colleges and Schools• 307 Academic Departments and degree programs• 22 Research Centers

8


The Role of the Office of Planning in Program Review

Planning

Learning Assessme

nt

Institution

al Research

10

Program Review Overview

Background: Program Reviews in Kentucky

Governing Regulation IX-I

Administrative Regulations 1:4

required every 5-7 years for all academic and administrative units (exceptions may be negotiated to align with specialized accreditation cycle)

Slide reference citation: University of Kentucky Administrative Regulations (AR)1:4:11/15/06


11

UK’s Program Review Schedule Schedule & Current Year Progress Updates

Purpose: communicate to organizational entities the full 6-yr review cycle and when

units can expect to undergo program review

Goals: provide the transparent and accurate maintenance of the review schedule

for the university’s educational (academic) and administrative units; and monitoring unit progress

Administrative Units participating in 2011-2012 Cycle President: 2 units Provost: 3 units EVPFA: 3 units



12

Program Review Overview, continued

What is the purpose and goal of program review?

to improve the quality and effectiveness of teaching and learning, research, public service, and operations; and

to develop recommendations leading to organizational improvement based on internal evaluation with appropriate input from external experts

Slide reference citation: University of Kentucky Administrative Regulations (AR)1:4:11/15/062012-2013 Program Review Orientation

13

Program Review Overview, continued

Who is responsible for satisfying program review? President, provost and executive vice presidents

deans, vice presidents, associate vice presidents, associate and vice provosts, department chairpersons, directors, and other administrators

Vice President for Institutional Research, Planning, and Effectiveness

unit/area faculty, staff, and/or appropriate personnel

Slide reference citation: University of Kentucky Administrative Regulations (AR)1:4:11/15/06 2012-2013 Program Review Orientation


What are the components of UK’s program review process?

14

• Unit/Program Self –Study• (internal Review)

SWOT Analysis

• External Review• within UK• outside UK

Recommendations

• Implementation Plan

Quality Enhancement

Agenda

• Annual Progress Report

Documentation of enhancement

results & Informs Strategic

Planning & Budget

15

Program Review ComponentsI.*Self-Study Report (include as appropriate):

program documents resources input from affected constituents adherence to policies and procedures evaluation of quality and productivity analysis of strengths and recommendations for quality

enhancement Elements evaluated:

centrality competitive /comparative advantage cost effectiveness demand quality distinctiveness

*Slide reference citation: University of Kentucky Administrative Regulations (AR)1:4:11/15/062012-2013 Program Review Orientation



16


SWOT Analysis


Recommendations


Quality Enhancement

Agenda




Planning & Budget

17

Program Review Components, continued

II.i. External Review (completed by External Review Committee)examine the self-study report;use appropriate data collection techniques to

assure objectivity;assess validity of conclusions reached in self-

study; identify additional strengths and

recommendations for quality enhancement; andprepare a final report—report made available to

faculty, staff employees, and students


Slide Reference citation: University of Kentucky Administrative Regulations (AR)1:4:11/15/06


18

Program Review Components, continuedII.ii. Administrative External Review Committee

appointed and charged by administrator to whom the unit head reports

consists of 4-5 members—stakeholders and constituencies affected by the unit program and services

4-5 faculty and staff employees, or students from outside the unit 1-2 ex-officio members, appointed to support external review committee

following external review, meets with unit and its leadership to discuss preliminary findings and writes report



19


II.iii. External Review Committee Report considers the unit’s…

Program documents: strategic plan (i.e. mission statement, goals, and objectives, & criteria for measuring progress); primary contributions to UK’s mission and vision, organizational chart or structure, & annual progress reports

Resources: adequacy of budget, facilities, equipment, personnel, including faculty and staff numbers demographics, and support from other university units essential to effective operations (e.g., research, engagement, development, alumni affairs, human resources, facilities management, financial units, & information technology)

Input from Affected Constituents: evaluation data from faculty, staff, and students affected by the delivery of program and services to the unit.



20


II.iii. External Review Committee Report considers the unit’s…

Adherence to Policies and Procedures: evidence of adherence to university policies and procedures (e.g., registration, student activity fees, hiring practices, etc.)

Evaluation of Quality and Productivity: evidence of quality of the collegial culture and climate Faculty and staff employees, communications and interactions; Orientation, advising, and other student services programs; Learning outcomes; Customer or client satisfaction; Business and operating procedures;

Slide reference citation: University of Kentucky Administrative Regulations (AR)1:4:11/15/06 2012-2013 Program Review Orientation


Accreditation and the External Review ProcessFully Accredited Programs/Units: Accreditation

Self-Study, Accreditation Review and Accreditation Report May Substitute for:

UK’s self-study UK’s External Review, and UK’s External Review Committee Report

Will NOT Substitute for: Program Review Implementation Plan

Partially Accredited Programs/Units: Accreditation Review and Accreditation Report May ONLY Substitute for:

External Reviewer (s) for the UK External Review



22


SWOT Analysis


Recommendations


Quality Enhancement

Agenda




Planning & Budget

23

Program Review Components, continuedIII. Implementation Plan

Sets agenda for change and quality enhancement over the next 5-7 year cycle;

unit faculty, staff, and/or students under the leadership of unit head define unit agenda based on self-study and external review report/recommendations;

must be approved by unit head’s supervisor;

used by unit to document future plans and resource needs for consideration in budgetary decision-making; and

supports annual progress reporting



24

Sources that inform Review unit website peer benchmarking and “best practices” last unit self-study reports (2005-06, or 2006-07) annual progress reports (past 3 years) most recent accreditation or certification results and

recommendations formative and summative assessments

qualitative– focus groups, interviews, etc. quantitative—satisfaction surveys, employer surveys, etc.

Institutional data (provided by Office of Institutional Research) for examples see: http://www.uky.edu/IRPE/ir.html


http://www.uky.edu/IRPE/ir.html


The Role of Institutional Research in Program Review

Planning

Student Learning

Institutional Resear

ch


What IR sources inform Program Review?Student-Related Data Freshman Profile (ACT Scores & High School GPAs)

Headcount Enrollments (10 years)

Degrees Conferred (10 years)

Retention and Graduation Rates

Surveys Senior Alumni NSSE Campus Climate


Institutional Data Sources Institutional data compiled by the UK Office of

Institutional Research can be found at:

www.uky.edu/IRPE/ir.html

www.uky.edu/IRPE/ie/supportdata.htmlwww.uky.edu/IRPE/colleges.html

http://www.uky.edu/IRPE/ir.html

http://www.uky.edu/IRPE/ie/supportdata.html

http://www.uky.edu/IRPE/colleges.html


The Role of Assessment in Program Review

Planning

StudentLearnin

g Assessment

Institutional Research


Student Learning Outcomes Assessment Process

Student learning outcomes

Assessmentmethods

Datacollection

Programimprovement

Timeline/schedule

Unit staff achieve consensus on learning outcomes to be assessed, and map outcomes to curriculum

Unit staff choose assessment methods appropriate to the outcomes, purpose of assessment, and student population

Unit staff develop timeline for gathering assessment data, both

formative and summative

Unit staff work together to gather

formative and summative

assessment data and compile for review

Unit staff collaborate to interpret assessment data and develop strategies to improve student learning

outcomes

Continuous improvement

cycle

30

Program Review Calendar

*Calendar

Purpose: communicates steps and timeline for completing

program review; and ensures timely completion

*Refer to administrative calendar for 2012-2013


31

Additional Program Review Questions

General Program Review Process Mia Alexander-Snow, PhD

Director, Planning and Institutional EffectivenessEmail: [email protected] phone: 257-2873

6-Year Schedule Connie Vaughn

Program Planning CoordinatorEmail: [email protected] phone: 257-7915



Presentation Contact InformationMia Alexander-SnowOffice for Planning & Institutional Effectiveness Website: http://www.uky.edu/IRPE/ie.html

Tara RoseOffice of AssessmentWebsite: http://www.uky.edu/IRPE/assessment.html

Roger SugarmanOffice of Institutional ResearchWebsite: http://www.uky.edu/IRPE/welcome.html

http://www.uky.edu/IRPE/ie.html

http://www.uky.edu/IRPE/assessment.html

http://www.uky.edu/IRPE/welcome.html

UK’s Program Review Process for Administrative Units: Academic Support

Documents

Transcript of UK’s Program Review Process for Administrative Units: Academic Support