A US Model that is the Same and Different from the UKZN QPA Model Planning for Improvement Victor M....
47
A US Model that is the Same and Different from the UKZN QPA Model Planning for Improvement Victor M. H. Borden, Ph.D. Professor of Educational Leadership and Policy Studies Indiana University Bloomington
-
date post
19-Dec-2015 -
Category
Documents
-
view
215 -
download
0
Transcript of A US Model that is the Same and Different from the UKZN QPA Model Planning for Improvement Victor M....
A US Model that is the Same and Different from the UKZN QPA Model
Planning for Improvement
Victor M. H. Borden, Ph.D.Professor of Educational Leadership and Policy Studies
Indiana University Bloomington
Overview
۞ Description of the IUPUI – Planning & Institutional Improvement model► And related T&L supports
۞ IUPUI Frameworks for Excellence۞ More general approaches from the US۞ Comparisons with UKZN’s QPA model
► A Discussion
Planning for Improvement
Planning & Budgeting 1. Mission, Vision, Goals developed 2. Unit goals aligned 3. Annual reports on web 4. Programs based on assessable goals,
with performance indicators 5. Biennial planning/budgeting hearings
conducted
Implementation
(Everyone on campus implements goals)
Evaluation
1. Academic and administrative program reviews
2. Evaluation of process effectiveness 3. Assessment of learning outcomes
in major in general education
4. Course evaluations 5. Student assessment 6. Constituent surveys 7. Management information and analysis 8. Program cost analysis 9. Web-based evaluation tools 10. Annual campus performance report 11. NCA accreditation
Assessable
Outcomes
Inst
rum
enta
tion
Data Collection
Analysis
App
licat
ion
of F
indi
ngs Culture
OfEvidence
Improvement 1. Reporting to internal constituents
2. Demonstrating accountability to external stakeholders
3. Applying findings in campus improvement initiatives
4. Proposing improvement initiatives
5. Improving assessment methods · Web-based data
· Electronic portfolios
Planning for Improvement
Planning & Budgeting 1. Mission, Vision, Goals developed 2. Unit goals aligned 3. Annual reports on web 4. Programs based on assessable goals,
with performance indicators 5. Biennial planning/budgeting hearings
conducted
Planning for Improvement
Evaluation1. Academic and administrative program reviews2. Evaluation of process effectiveness3. Assessment of learning outcomes
• In major• In general education
4. Course evaluations5. Student assessment6. Constituent surveys7. Management information and analysis8. Program cost analysis9. Web-based evaluation tools10.Annual campus performance report11.General and specialized accreditation
Planning for Improvement
Improvement1. Reporting to internal constituents2. Demonstrating accountability to
external stakeholders3. Applying findings in campus
improvement initiatives4. Proposing improvement initiatives5. Improving assessment methods
• Web-based data collection/reporting• Electronic portfolios
IUPUI – Planning & Institutional Improvement
۞ PAII Office۞ Information Management and Institutional
Research۞ Testing Center۞ Institutional Effectiveness۞ Economic Modeling Office
PAII Processes
۞ Academic program review۞ Process Improvement۞ Assessment of learning outcomes۞ Annual performance report & PIs۞ Annual national assessment conference۞ Annual budget/planning process۞ Economic modeling۞ Student evaluations۞ Testing/instrumentation۞ Management reports۞ Student, faculty and staff surveys
IUPUI Frameworks for Excellence
۞ Performance Objectives & Indicators۞ Principles of Undergraduate Learning۞ RISE to the IUPUI Challenge۞ Centers for T, R, S & L
IUPUI Performance Objectives۞ Teaching & Learning
► Attract and support a well-prepared and diverse student population► Support and enhance effective teaching► Enhance undergraduate student learning and success► Provide effective professional and graduate programs
۞ Research, Scholarship and Creative activity► Conduct world-class research, scholarship, and creative activity
relevant to Indianapolis, the state, and beyond► Provide support to increase scholarly activity and external funding ► Enhance the infrastructure for research, scholarship, and creative
activity ۞ Civic Engagement
► Enhance capacity for civic engagement► Enhance civic activities, partnerships, and patient and client services ► Intensify commitment and accountability to Indianapolis, Central
Indiana, and the state
Principles of Undergraduate Learning
۞ Core Communication and Quantitative Skills۞ Critical Thinking۞ Integration and Application of Knowledge۞ Intellectual Depth, Breadth, and Adaptiveness۞ Understanding Society and Culture۞ Values and Ethics
Frameworks for Commonality
۞ RISE to the IUPUI Challenge► Research ► International Experience► Service Learning► Experiential Learning
Centers for…
۞ Teaching and Learning۞ Research and Learning۞ Service and Learning
Approaches to Evaluation and Improvement
۞ Simple assessment models۞ Quality improvement۞ Program review
Simple Models of Assessment
۞ Advantages► Easy to communicate, use, and learn from► Can be built into everyday work► Helps build and maintain culture of evidence
۞ Models► The evaluation cycle (or spiral)► The assessment matrix template
The Planning-Evaluation-Improvement Cycle
Plan
Implement
Assess
ImprovePlan
Check
Do
Act
Adapted from Norman Jackson
Toward a Spiral of Improvement
2. ENGAGE WITH THE PROBLEM CALLED HOW DO WE IMPROVE DIVERSITY?
5. EVALUATE IMPACT ON DIVERSITY
* did it work as I intended?
* how did people respond?
* what were the results?
6. PLAN TO IMPROVE
1. THINK ABOUT DIVERSITY ISSUES
3. DEVELOP RESOURCES/ STRATEGIES TO IMPROVE DIVERSITY
4. IMPLEMENT INTERVENTIONS
* experiment
Bac
k to
the
dra
win
g bo
ard
On
to s
om
ethi
ng
els
e
Core Evaluation Cycle Questions
What are you trying to achieve?
Needs assessment
What are you doing to achieve it?
Process assessment
How will you know when you get there
Outcomes assessment
What can you do with the results?
Improvement
The Assessment Matrix
1. What
general outcome are you
seeking?
2. How
would you know it if you saw
it?
3. How are
you trying to achieve
the outcome?
4. How could
you measure what you hope to
see?
5. What
have you found?
6. What are you doing about it?
The Limits of Simple Models
۞ Often overly simplistic relative to problems۞ Actual measures can be misguided۞ Implementation can be inconsistent across
units۞ Not always easy to link outcome measures to
“responsible” processes► doing the right thing vs. doing it right
Are we conducting the intervention effectively and efficiently?
GO
Yes No
Yes
Is it
an
effe
ctiv
e in
terv
en
tion
?
No
Improve
Quality Improvement Models
۞ Advantages► Focus on process provides best chances for
identifying points of improvement► Collaborative teams empower staff and help
improve communication across units► Formulaic method and external staff support
help guide and keep on track۞ Sample methods
► Penn State’s Fast Track and IMPROVE► U of Wisconsin Accelerated Improvement
Penn State University Office of Planning and Institutional Assessment
http://www.psu.edu/president/pia/innovation/index.htm
Fast Track
IMPROVE Process
Results Achieved to DateApril 2001: CORED -- The Commission on Racial/Ethnic Diversity (CORED) was appointed in 1989 as a University-wide advisory body to the PSU President in matters relating to racial and ethnic diversity. CORED also serves as a resource for all University departments and individuals seeking to improve Penn State's climate for diversity. Accomplishments include: 1) annual meeting with the President of Penn State, 2) published terminology guidelines and recommendations, 3) analysis of Penn State's policies regarding exit interviews for departing minority faculty, staff and students, 4) study on the treatment of diversity in the University's curriculum, 5) organized and sponsored a national conference for 400 entitled, "Best Practices in Diversity," 6) developed a CORED web site to inform the community about the commission's activities. April 2001: Equal Opportunity Planning Committee -- EOPC's mission is to promote greater equity for historically underrepresented groups witin the University and/or those groups that have been historical targets of discrimination. EOPC was formed in 1983 as a response to a desegregation mandate issued by the U.S. Department of Education to all public institutions within the Commonwealth of PA.
Seven challenges articulated in "A framework to Foster Diversity at Penn State: 1998-2003" comprise the funding categories of EOPC: 1) developing a shared and inclusive understanding of diversity, 2) creating a welcoming campus climate, 3) recruiting and retaining a diverse student body, 4) recruiting and retaining a diverse workforce, 5) developing a curriculum that supports the goals of the University's general education plan, 6) diversifying University leadership and management, and 7) coordinating organizational change to support diversity goals. April 2001: The Lesbian, Gay, Bisexual, and Transgendered Support Network is sponsored by the Commission on Lesbian, Gay, Bisexual, and Transgendered Equity and the EOPC.
Goals: 1) to identify and mobilize a network of people empathetic and knowledgeable about lesbian, gay, bisexual, and transgendered issues and concerns; 2) to provide evidence of that support by displaying a sign as a visible symbol,; 3) to reduce the fear of reprisal and discrimination of lesbian, gay, bisexual and transgendered persons within the Penn State community.
UWisc Accelerated ImprovementDefine Goals and measures of success
Document processUnderstand customer needsCheck/refine goals
Design Develop potential solutionsAnalyze solutions/optionsFinalize solution develop implementation plan
Implement Inform affected peopleConduct training, if neededExecute action plans w/timeline
Follow-up Collect data to track improvementReview and refine process changesIssue final report with results
Limits of QI Models
۞ Academicians wary of “business” models۞ Focus on process emphasizes doing it right
over doing right thing۞ Can be episodic rather than continuous
Program Review
۞ Program self-study, site visit by “peers”۞ Common method for academic programs
► Increasing use for administrative programs۞ Fits well with accreditation framework۞ Guidelines shape tone and tenor
► Content standards► Review team composition
۞ Flexibility accommodates range of inquiry orientations
Limits of Program Review
۞ Expensive and time-consuming۞ Can be done with little participation
► Or with a lot۞ Results not always directly useful for change
► Memorandum of understanding helpful۞ Episodic nature not responsive to changing
environment
More Complex Models
۞ Advantages► Handle true complexity► Provide in-depth insight into context► Academicians respect the scholarship
(although not necessarily the particular approach)
۞ Examples (from EMU’s evaluation checklist)► CIPP► Constructivist Evaluation
More Complex Models
۞ The Evaluation Center
Eastern Michigan University
http://www.wmich.edu/evalctr/checklists/► CIPP Model► Constructivist Evaluation► Deliberative Democratic Evaluation► Key Evaluation Checklist► Qualitative Evaluation► Utilization-Focused Evaluation
The CIPP Model
1. Contractual Agreements2. Context Evaluation3. Input Evaluation4. Process Evaluation5. Impact Evaluation6. Effectiveness Evaluation7. Transportability Evaluation8. Sustainability Evaluation9. Metaevaluation10. The Final Synthesis Report
Constructivist Evaluation
۞ Guba & Lincoln (2001)۞ Two-stage process
► Discovery - effort to describe “what’s going on here,” the “here” being the evaluand and its context
► Assimilation - effort to incorporate new discoveries into the existing construction or constructions …so that the new construction will fit, work, demonstrate relevance, and exhibit modifiability.
Limits of Complex Models
۞ Too complex to be practical۞ Expensive۞ They require an…
► “evaluation unit as a staff operation at a high level of the organization in order to help insulate the unit from inappropriate internal influences and enhance its influence on decision making .”
Daniel J. Stufflebeamhttp://www.wmich.edu/evalctr/checklists/institutionalizingeval.htm
