Accountability in Higher Education Data Driven Schools A Presentation to the Educational Approval...
-
Upload
nichole-lemons -
Category
Documents
-
view
212 -
download
0
Transcript of Accountability in Higher Education Data Driven Schools A Presentation to the Educational Approval...
![Page 1: Accountability in Higher Education Data Driven Schools A Presentation to the Educational Approval Board State of Wisconsin.](https://reader034.fdocuments.us/reader034/viewer/2022051819/551630b3550346a2308b5fe0/html5/thumbnails/1.jpg)
Accountability in
Higher Education Data Driven Schools
A Presentation to the
Educational Approval Board State of Wisconsin
![Page 2: Accountability in Higher Education Data Driven Schools A Presentation to the Educational Approval Board State of Wisconsin.](https://reader034.fdocuments.us/reader034/viewer/2022051819/551630b3550346a2308b5fe0/html5/thumbnails/2.jpg)
Presentation Outline Institutional Effectiveness
Importance in Higher Education ACCSCT’s Journey Newer thinking – a paradigm shift The ACCSCT outcomes model The Habits of Accountable (data-driven) schools The Pay-off Thoughts about the future
![Page 3: Accountability in Higher Education Data Driven Schools A Presentation to the Educational Approval Board State of Wisconsin.](https://reader034.fdocuments.us/reader034/viewer/2022051819/551630b3550346a2308b5fe0/html5/thumbnails/3.jpg)
Education In the U.S. U.S. ranks 16th in the World behind most
of Europe in its high school graduation rate at 73%
Of all first-time college students who entered a community college in 1995, only 36% earned a degree within six years.
Completion rates for low-income students and minorities are even lower.
![Page 4: Accountability in Higher Education Data Driven Schools A Presentation to the Educational Approval Board State of Wisconsin.](https://reader034.fdocuments.us/reader034/viewer/2022051819/551630b3550346a2308b5fe0/html5/thumbnails/4.jpg)
Education in the U.S. Almost 20% of traditional-aged community
college students never complete even 10 credits
Only 6% of the lowest income college entrants obtain a degree at all.
![Page 5: Accountability in Higher Education Data Driven Schools A Presentation to the Educational Approval Board State of Wisconsin.](https://reader034.fdocuments.us/reader034/viewer/2022051819/551630b3550346a2308b5fe0/html5/thumbnails/5.jpg)
Statistics suggest a need for thinking differently about accountability.
Older view:An effective institution is one that provides resources (faculty, library, services) so learning can occur
Students have responsibility for learning Accreditation/ assessment ensures that appropriate
resources are provided by institutions
![Page 6: Accountability in Higher Education Data Driven Schools A Presentation to the Educational Approval Board State of Wisconsin.](https://reader034.fdocuments.us/reader034/viewer/2022051819/551630b3550346a2308b5fe0/html5/thumbnails/6.jpg)
No Child Left Behind Shifts the emphasis from teaching to
learning Holds institutions accountable for student
learning Uses standard (although perhaps
imperfect) methodologies to test learning Will influence HEA reauthorization and
Department of Education proposals
![Page 7: Accountability in Higher Education Data Driven Schools A Presentation to the Educational Approval Board State of Wisconsin.](https://reader034.fdocuments.us/reader034/viewer/2022051819/551630b3550346a2308b5fe0/html5/thumbnails/7.jpg)
Accountability in Post-secondary Education
Discussions regarding institutional effectiveness began before “No Child Left Behind.”
1998 Amendments to the Higher Education Act signaled Congress’ expectation that outcomes measures would be used to determine institutional effectiveness
![Page 8: Accountability in Higher Education Data Driven Schools A Presentation to the Educational Approval Board State of Wisconsin.](https://reader034.fdocuments.us/reader034/viewer/2022051819/551630b3550346a2308b5fe0/html5/thumbnails/8.jpg)
Early regulation/discussion Focused on “vocational education” Thought of performance outcomes as one
way to measure the effectiveness of institutions in light of mission
Demanded more from accrediting commissions that accredited colleges and schools with clear career objectives (Default rate concerns)
![Page 9: Accountability in Higher Education Data Driven Schools A Presentation to the Educational Approval Board State of Wisconsin.](https://reader034.fdocuments.us/reader034/viewer/2022051819/551630b3550346a2308b5fe0/html5/thumbnails/9.jpg)
Accrediting Agency Response All institutional accrediting commissions
adopted standards related to measurements of institutional effectiveness
Some institutional accrediting commissions adopted quantitative standards, setting rates of completion, placement, licensure and other outcomes.
![Page 10: Accountability in Higher Education Data Driven Schools A Presentation to the Educational Approval Board State of Wisconsin.](https://reader034.fdocuments.us/reader034/viewer/2022051819/551630b3550346a2308b5fe0/html5/thumbnails/10.jpg)
Accrediting Commission Response
Accrediting commissions adopted different methodologies for assessing institutional effectiveness Setting specific rates
By program or by institution (aggregate) Changing rates based on averages
By program or by institution (aggregate)
![Page 11: Accountability in Higher Education Data Driven Schools A Presentation to the Educational Approval Board State of Wisconsin.](https://reader034.fdocuments.us/reader034/viewer/2022051819/551630b3550346a2308b5fe0/html5/thumbnails/11.jpg)
Accrediting Commission Response Some commissions explored alternative
methodologies for assessing quality
HLC of North Central Association: AQUIP SACS adopts similar model and standards for all
institutions
![Page 12: Accountability in Higher Education Data Driven Schools A Presentation to the Educational Approval Board State of Wisconsin.](https://reader034.fdocuments.us/reader034/viewer/2022051819/551630b3550346a2308b5fe0/html5/thumbnails/12.jpg)
Newer thinking about accountability after 10 years: Workforce Preparation is an important goal of
education Student learning is a shared responsibility Successful student achievement is itself an
important outcome in addition to being a measure of institutional effectiveness (feedback loop)
Higher Education must be accountable to the public for student learning Reflected in H.R. 609 Cost, Accountability and
Transparency Provisions as public policy
![Page 13: Accountability in Higher Education Data Driven Schools A Presentation to the Educational Approval Board State of Wisconsin.](https://reader034.fdocuments.us/reader034/viewer/2022051819/551630b3550346a2308b5fe0/html5/thumbnails/13.jpg)
Commission on the Future of Higher Education Goals World –class higher-education system that
creates new knowledge, contributes to economic prosperity and global competitiveness and empowers citizens
Accessible to all Americans Affordable high quality instruction Adaptable workplace skills Innovative: adaptable to changing technology and
demographics
![Page 14: Accountability in Higher Education Data Driven Schools A Presentation to the Educational Approval Board State of Wisconsin.](https://reader034.fdocuments.us/reader034/viewer/2022051819/551630b3550346a2308b5fe0/html5/thumbnails/14.jpg)
ACCSCT’s Journey
![Page 15: Accountability in Higher Education Data Driven Schools A Presentation to the Educational Approval Board State of Wisconsin.](https://reader034.fdocuments.us/reader034/viewer/2022051819/551630b3550346a2308b5fe0/html5/thumbnails/15.jpg)
ACCSCT Promulgated specific outcomes standards for
completion, placement and licensure in July, 1998 Standards Require “Acceptable” Student Achievement
Completion Placement State licensing examination pass rates
Establish bright line indicators Allow for extenuating circumstances
External factors which have a bearing on student achievement
![Page 16: Accountability in Higher Education Data Driven Schools A Presentation to the Educational Approval Board State of Wisconsin.](https://reader034.fdocuments.us/reader034/viewer/2022051819/551630b3550346a2308b5fe0/html5/thumbnails/16.jpg)
ACCSCT Completion and Placement Chart
Cohort Reporting Defined Timeframe and Parameters
Individual Tracking Defined Individual Classifications
Starting Point Allowance for Mitigating Factors
![Page 17: Accountability in Higher Education Data Driven Schools A Presentation to the Educational Approval Board State of Wisconsin.](https://reader034.fdocuments.us/reader034/viewer/2022051819/551630b3550346a2308b5fe0/html5/thumbnails/17.jpg)
Outcomes Data Collection
ACCSCT Outcomes Trends: 1997-2001 Annual Reporting Process Verification Requirements Completion & Placement v. Retention Calculating Student Achievement
Standards
![Page 18: Accountability in Higher Education Data Driven Schools A Presentation to the Educational Approval Board State of Wisconsin.](https://reader034.fdocuments.us/reader034/viewer/2022051819/551630b3550346a2308b5fe0/html5/thumbnails/18.jpg)
COMPLETION RATES BY REPORTING YEAR
68.9%70%71.6%73%
20.5%19.9%18.8%14%
0
10
20
30
40
50
60
70
80
1998 1999 2000 2001
AVERAGE
STANDARDDEVIATION
![Page 19: Accountability in Higher Education Data Driven Schools A Presentation to the Educational Approval Board State of Wisconsin.](https://reader034.fdocuments.us/reader034/viewer/2022051819/551630b3550346a2308b5fe0/html5/thumbnails/19.jpg)
85.6%86.0%86.0%85%
15.2%14.0%14.0%15%
0102030405060708090
100
1998 1999 2000 2001
AVERAGE
STANDARDDEVIATION
PLACEMENT RATES BY REPORTING YEAR
![Page 20: Accountability in Higher Education Data Driven Schools A Presentation to the Educational Approval Board State of Wisconsin.](https://reader034.fdocuments.us/reader034/viewer/2022051819/551630b3550346a2308b5fe0/html5/thumbnails/20.jpg)
Annual Reporting Process Completion and Placement Charts required
annually – for each program Data entry Process & Screening:
Review of each submission for accuracy Additional data requested as required
Supporting Documentation Required Verifiable records of all classifications
Employment, Further Education Schools must keep supporting documentation on file
![Page 21: Accountability in Higher Education Data Driven Schools A Presentation to the Educational Approval Board State of Wisconsin.](https://reader034.fdocuments.us/reader034/viewer/2022051819/551630b3550346a2308b5fe0/html5/thumbnails/21.jpg)
Verification Required
Completion and placement rates are verified as part of the on-site evaluation process
Random samples of completion and placement data are selected for verification
All submissions of Outcomes reports require supporting documents
![Page 22: Accountability in Higher Education Data Driven Schools A Presentation to the Educational Approval Board State of Wisconsin.](https://reader034.fdocuments.us/reader034/viewer/2022051819/551630b3550346a2308b5fe0/html5/thumbnails/22.jpg)
Completion and Placement v. Retention Completion and Placement track separate
cohorts of students through graduation Retention measures the extent to which
students remain enrolled in and/or graduate from an institution (snap shot)
Both measures can be used to evaluate successful student achievement (longer programs)
![Page 23: Accountability in Higher Education Data Driven Schools A Presentation to the Educational Approval Board State of Wisconsin.](https://reader034.fdocuments.us/reader034/viewer/2022051819/551630b3550346a2308b5fe0/html5/thumbnails/23.jpg)
Programs are organized according to a single program commonality - program length
The completion rates for groupings of programs with common lengths are averaged & one standard deviation is calculated.
Raw data & calculations are verified by a third party.
Determining the Student Achievement Standard for Completion
![Page 24: Accountability in Higher Education Data Driven Schools A Presentation to the Educational Approval Board State of Wisconsin.](https://reader034.fdocuments.us/reader034/viewer/2022051819/551630b3550346a2308b5fe0/html5/thumbnails/24.jpg)
ProgramLength #ofPrograms AvgCompletion<1 4 100%1 83 93%2 63 92%3 95 92%4 69 86%5 94 83%6 134 84%7 257 73%8 272 72%9 245 72%
10 165 70%11 101 66%12 348 66%13 48 66%14 202 61%15 195 61%16 74 65%17 83 58%18 188 58%19 28 58%20 31 61%21 60 65%22 21 56%23 13 41%
Ascending Program Length InMonths
Completion rates
decline as programs lengthen.
Groupings of program length are evident.
Calculating the Student
Achievement Standard
![Page 25: Accountability in Higher Education Data Driven Schools A Presentation to the Educational Approval Board State of Wisconsin.](https://reader034.fdocuments.us/reader034/viewer/2022051819/551630b3550346a2308b5fe0/html5/thumbnails/25.jpg)
Determining the Student Achievement Standard for Placement
Different program characteristics do not yield any commonality relative to placement.
Data collected over five years did not show that geography, enrollment numbers, program length, or occupational area consistently had any effect on placement.
Some relevancy to economic conditions, extenuating circumstances to be proven by individual institution.
![Page 26: Accountability in Higher Education Data Driven Schools A Presentation to the Educational Approval Board State of Wisconsin.](https://reader034.fdocuments.us/reader034/viewer/2022051819/551630b3550346a2308b5fe0/html5/thumbnails/26.jpg)
Data and calculations are reviewed by Dr. Morgan Lewis, Statistician, Ohio
State University
STUDENT ACHIEVEMENT STANDARD
Based on 2001 Annual Report Data
Completion Rates Placement Rate
Length in Months
Mean1 Std.
Deviation
Mean Less 1 Standard Deviation
Mean1 Std.
Deviation
Mean Less 1 Standard Deviation
1-688% 13% 75% 85% 15% 70%
7-972% 15% 57%
10-1665% 18% 47%
17 and over 56% 22% 34%
(Based on 3009 programs reported)
![Page 27: Accountability in Higher Education Data Driven Schools A Presentation to the Educational Approval Board State of Wisconsin.](https://reader034.fdocuments.us/reader034/viewer/2022051819/551630b3550346a2308b5fe0/html5/thumbnails/27.jpg)
School As A Whole
Recruitment
Admissions
Student Progress
Market Needs
Educational Resources
Faculty Qualifications
Program Length
Student Services
Student Outcomes
On-Going Assessment
& Evaluation
Accreditation and Quality Assessment Model
![Page 28: Accountability in Higher Education Data Driven Schools A Presentation to the Educational Approval Board State of Wisconsin.](https://reader034.fdocuments.us/reader034/viewer/2022051819/551630b3550346a2308b5fe0/html5/thumbnails/28.jpg)
Habits of Accountable Institutions Commitment to strong Leadership
Schools with experienced leaders tend to have better student outcomes (McComis, 2005). But, leadership in our industry requires commitment on the part of owners/investors.
School that establish operations based on best practice, not just because a rule or standard requires it, tend to be more successful. In this way, compliance becomes a natural extension of school operations. For example, the use of PACs because it is a best practice, not because an ACCSCT standard requires it.
![Page 29: Accountability in Higher Education Data Driven Schools A Presentation to the Educational Approval Board State of Wisconsin.](https://reader034.fdocuments.us/reader034/viewer/2022051819/551630b3550346a2308b5fe0/html5/thumbnails/29.jpg)
Habits of Accountable Institutions: those that care about completion Act at all times in the best interest of students Take attendance React quickly to lack of attendance/progress Take responsibility for student learning and
achievement Listen mightily to student feedback Respond enthusiastically to student needs and
concerns Respond immediately to a change in completion
trends
![Page 30: Accountability in Higher Education Data Driven Schools A Presentation to the Educational Approval Board State of Wisconsin.](https://reader034.fdocuments.us/reader034/viewer/2022051819/551630b3550346a2308b5fe0/html5/thumbnails/30.jpg)
Habits of Accountable Schools: those that care about placement Involve employers in the development of programs Work feverishly at the school’s reputation in the
community including the workforce community Pursue programmatic accreditations Pursue articulation and consortia agreements with other
institutions Make a strong commitment to continuous improvement Constantly evaluate student achievement data and seek
to improve program content and delivery.
![Page 31: Accountability in Higher Education Data Driven Schools A Presentation to the Educational Approval Board State of Wisconsin.](https://reader034.fdocuments.us/reader034/viewer/2022051819/551630b3550346a2308b5fe0/html5/thumbnails/31.jpg)
Pay –off: Happier, more successful students
0
2
4
6
8
10
12
14
16
18
20
JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC
2004
2005
2006
![Page 32: Accountability in Higher Education Data Driven Schools A Presentation to the Educational Approval Board State of Wisconsin.](https://reader034.fdocuments.us/reader034/viewer/2022051819/551630b3550346a2308b5fe0/html5/thumbnails/32.jpg)
What have we learned? Verification is critical to any accountability
model Differences in accountability models make
comparable data elusive Diversity in higher education makes one
common model unlikely
![Page 33: Accountability in Higher Education Data Driven Schools A Presentation to the Educational Approval Board State of Wisconsin.](https://reader034.fdocuments.us/reader034/viewer/2022051819/551630b3550346a2308b5fe0/html5/thumbnails/33.jpg)
What is yet to learn? What is the goal of higher education?
Learning? Workforce preparation? Global conscience? Some combination?
If we know the goal, how will we know when we have attained it? What’s good enough performance? Does striving to be average improve higher education?
![Page 34: Accountability in Higher Education Data Driven Schools A Presentation to the Educational Approval Board State of Wisconsin.](https://reader034.fdocuments.us/reader034/viewer/2022051819/551630b3550346a2308b5fe0/html5/thumbnails/34.jpg)
How has our thinking changed? Continuous monitoring of program viability a
useful tool for schools and accrediting commissions
Integrating outcomes assessment as part of institutional improvement planning and program advisory committee review is a good practice
Data collection useful for trend analysis and longitudinal studies
![Page 35: Accountability in Higher Education Data Driven Schools A Presentation to the Educational Approval Board State of Wisconsin.](https://reader034.fdocuments.us/reader034/viewer/2022051819/551630b3550346a2308b5fe0/html5/thumbnails/35.jpg)
The Future . . . Need for further refinement and continuing
study Continued congressional and Department of
Education interest in accountability Increased public disclosure of information on student
achievement Need to ensure accurate disclosure of information by
institutions and accrediting commissions Greater transparency of the accreditation process Greater emphasis on methodology and verification
(perhaps a common approach for similar institutions)
![Page 36: Accountability in Higher Education Data Driven Schools A Presentation to the Educational Approval Board State of Wisconsin.](https://reader034.fdocuments.us/reader034/viewer/2022051819/551630b3550346a2308b5fe0/html5/thumbnails/36.jpg)
The Future . . . Continuing questions regarding how to judge the relative
quality of education programs (cost/consumer protection issue)
Continuing debate on the limits of federal involvement in determining measures of accountability
Concern over ability of U.S. higher education institutions to compete globally where other countries are investing significantly in research and disclosure of outcomes information
Continuing questions about the cost of higher education and the value added.
![Page 37: Accountability in Higher Education Data Driven Schools A Presentation to the Educational Approval Board State of Wisconsin.](https://reader034.fdocuments.us/reader034/viewer/2022051819/551630b3550346a2308b5fe0/html5/thumbnails/37.jpg)
Accountability In Higher EducationA Presentation for the Wisconsin Educational Approval
Board
Elise ScanlonAccrediting Commission of Career Schools and Colleges
of Technology2101 Wilson Blvd., Suite 302Arlington, VA [email protected]