Measuring Up on College-Level Learning
description
Transcript of Measuring Up on College-Level Learning
![Page 1: Measuring Up on College-Level Learning](https://reader035.fdocuments.us/reader035/viewer/2022062322/56814943550346895db68d2f/html5/thumbnails/1.jpg)
Measuring Up on Measuring Up on College-Level College-Level
LearningLearning
Margaret Miller, Project Director
September 2003
![Page 2: Measuring Up on College-Level Learning](https://reader035.fdocuments.us/reader035/viewer/2022062322/56814943550346895db68d2f/html5/thumbnails/2.jpg)
Measuring Up 2000
Measuring Up 2000
![Page 3: Measuring Up on College-Level Learning](https://reader035.fdocuments.us/reader035/viewer/2022062322/56814943550346895db68d2f/html5/thumbnails/3.jpg)
Learning in the States: Incomplete
[Add state map on incomplete]
![Page 4: Measuring Up on College-Level Learning](https://reader035.fdocuments.us/reader035/viewer/2022062322/56814943550346895db68d2f/html5/thumbnails/4.jpg)
State Efforts to Measure Learning
(taxonomy: Peter Ewell, Change magazine)• Certification of individual students
– E.g., Texas’s TASP, Florida’s CLAST• Institutional assessment for
improvement– E.g., Tennessee's performance
measures– Missouri’s accountability program– Campus-based assessment
• Institutional assessment for accountability– E.g., S. Dakota and Arkansas
![Page 5: Measuring Up on College-Level Learning](https://reader035.fdocuments.us/reader035/viewer/2022062322/56814943550346895db68d2f/html5/thumbnails/5.jpg)
National Attention to College-Level Learning
• Pew’s Quality of Undergraduate Education and writing assessment projects
• American Association of Colleges and Universities’ general education assessment project
• Council on Higher Education Accreditation’s project on institutional effectiveness
• Secretary's Commission on Achieving Necessary Skills (SCANS) skills
• Equipped for the Future• National Skills Standards Board
![Page 6: Measuring Up on College-Level Learning](https://reader035.fdocuments.us/reader035/viewer/2022062322/56814943550346895db68d2f/html5/thumbnails/6.jpg)
Key Questions
What do the state’s college-educated citizens know and what can they do that contributes to the social good? What kind of educational capital do they represent?
and
![Page 7: Measuring Up on College-Level Learning](https://reader035.fdocuments.us/reader035/viewer/2022062322/56814943550346895db68d2f/html5/thumbnails/7.jpg)
Key Questions (cont.)
How well do the state’s public and private, two- and four-year colleges and universities collectively contribute to that capital? What do those whom they educate know, and what can they do?
![Page 8: Measuring Up on College-Level Learning](https://reader035.fdocuments.us/reader035/viewer/2022062322/56814943550346895db68d2f/html5/thumbnails/8.jpg)
Key Decisions
• Whose learning will we measure?• What learning will we measure?• How will we use the information?• What strategies will we pursue?
![Page 9: Measuring Up on College-Level Learning](https://reader035.fdocuments.us/reader035/viewer/2022062322/56814943550346895db68d2f/html5/thumbnails/9.jpg)
Whose Learning
The college-educated in the states
and
college students
• Whose learning• What learning• The policy uses for the information• Assessment strategies
![Page 10: Measuring Up on College-Level Learning](https://reader035.fdocuments.us/reader035/viewer/2022062322/56814943550346895db68d2f/html5/thumbnails/10.jpg)
What Learning
National Education Goal 6:
“By the year 2000, every adult American will be
literate and will possess the knowledge and skills
necessary to compete in a global economy and exercise
the rights and responsibilities of
citizenship”
• Whose learning• What learning• The policy uses for the information• Assessment strategies
![Page 11: Measuring Up on College-Level Learning](https://reader035.fdocuments.us/reader035/viewer/2022062322/56814943550346895db68d2f/html5/thumbnails/11.jpg)
What Learning (cont.)
National Goal 6, objective for college education:
“By the year 2000, every adult American will be
literate and will possess the knowledge and skills
necessary to compete in a global economy and exercise
the rights and responsibilities of
citizenship”
• Whose learning• What learning• The policy uses for the information• Assessment strategies
![Page 12: Measuring Up on College-Level Learning](https://reader035.fdocuments.us/reader035/viewer/2022062322/56814943550346895db68d2f/html5/thumbnails/12.jpg)
Policy Purposes
Higher education policyand
K-12 education + economic development +
adult literacy policy
• Whose learning• What learning• The policy uses for the information• Assessment strategies
![Page 13: Measuring Up on College-Level Learning](https://reader035.fdocuments.us/reader035/viewer/2022062322/56814943550346895db68d2f/html5/thumbnails/13.jpg)
Direct Strategies
• National Assessment of Adult Literacy
• Graduate-admissions and licensing exams
• General intellectual skills tests
• Whose learning• What learning• The policy uses for the information• Assessment strategies
![Page 14: Measuring Up on College-Level Learning](https://reader035.fdocuments.us/reader035/viewer/2022062322/56814943550346895db68d2f/html5/thumbnails/14.jpg)
National Assessment of Adult Literacy (NAAL)
concludes12/03 Disadvantages:• Labor-intensive,
expensive • Decadal federal
survey --timing• National sample
only, except in 6 states
• Not what colleges think they teach
Advantages:• Advanced literacy
levels of a good measure of educational capital
• Assesses general population
• Comparison group of non-college-educated
• Household survey – respondent motivation high
![Page 15: Measuring Up on College-Level Learning](https://reader035.fdocuments.us/reader035/viewer/2022062322/56814943550346895db68d2f/html5/thumbnails/15.jpg)
Existing Exams• Graduate-admissions exams
– Dental– Graduate Management– Graduate Record– Law School, – Medical College – Optometry– Pharmacy
• Licensing exams– Clinical Pathology– Dental Hygiene– Occupational Therapy– Physical Therapy– Physician Assistant– Nursing– Respiratory Therapy– Teaching
• Whose learning• What learning• The policy uses for the information• Assessment strategies
![Page 16: Measuring Up on College-Level Learning](https://reader035.fdocuments.us/reader035/viewer/2022062322/56814943550346895db68d2f/html5/thumbnails/16.jpg)
Existing Examsdata gathered by 03/04
Disadvantages:• Selection bias• Uneven coverage
by discipline• Variable (and
sometimes small) numbers of test- takers in each state
• Most in health professions
Advantages:• Established,
credible instruments
• Highly motivated test-takers
• Admissions tests assess general intellectual abilities
• Availability• Low cost
![Page 17: Measuring Up on College-Level Learning](https://reader035.fdocuments.us/reader035/viewer/2022062322/56814943550346895db68d2f/html5/thumbnails/17.jpg)
General Intellectual Skills Tests
administered fall 03
• WorkKeys to a sample of two-year students in each state– Applied Math– Locating Information– Reading for Information– Business Writing
• Collegiate Learning Assessment (CLA) to a sample of four-year students in each state
![Page 18: Measuring Up on College-Level Learning](https://reader035.fdocuments.us/reader035/viewer/2022062322/56814943550346895db68d2f/html5/thumbnails/18.jpg)
WorkKeys and CLA
Disadvantages:
• Institutional motivation
• Test-taker motivation
• Expense
Advantages:• Excellent tests
of general & functional intellectual skills
• Can impart useful information to student and school
![Page 19: Measuring Up on College-Level Learning](https://reader035.fdocuments.us/reader035/viewer/2022062322/56814943550346895db68d2f/html5/thumbnails/19.jpg)
Indirect MeasuresNSSE/CCSSE co-administered with
testsCRS summer through fall, 03
• National Survey of Student Engagement (NSSE)
• Community College Survey of Student Engagement (CCSSE)
• College Results Survey (CRS)
• Whose learning• What learning• The policy uses for the information• Assessment strategies
![Page 20: Measuring Up on College-Level Learning](https://reader035.fdocuments.us/reader035/viewer/2022062322/56814943550346895db68d2f/html5/thumbnails/20.jpg)
Surveys
Disadvantages:• Not direct
learning measures
• Not yet cross-correlated with direct measures
Advantages:• Excellent and
recently developed instruments
• Process measure could lead to improvement
• Both have face validity
• Respondent motivation good
![Page 21: Measuring Up on College-Level Learning](https://reader035.fdocuments.us/reader035/viewer/2022062322/56814943550346895db68d2f/html5/thumbnails/21.jpg)
Challenges
• Political instability in states: gubernatorial, SHEEO
• Personnel changes among key players
• Institutional skepticism• Faculty resistance• Data-collection hurdles• Test-taker motivation
![Page 22: Measuring Up on College-Level Learning](https://reader035.fdocuments.us/reader035/viewer/2022062322/56814943550346895db68d2f/html5/thumbnails/22.jpg)
General Timeline
• Measuring Up 2002: model tested with incomplete data from Kentucky
• 2002-2004: Five-state pilot to test assessment model: IL, KY, NV, OK, SC
• Measuring Up 2004: publish the results of the pilot
• Measuring Up 2006: if enough states adopt the model, grade states on learning
![Page 23: Measuring Up on College-Level Learning](https://reader035.fdocuments.us/reader035/viewer/2022062322/56814943550346895db68d2f/html5/thumbnails/23.jpg)
Reasons to Act
• It is the right thing to do.• We can determine how to
do it right.• This initiative will generate
information useful to states, institutions, and students.
• State-level analysis can promote collaborations to serve underachieving subpopulations or regions of the state.
• State resources can be effectively targeted.
![Page 24: Measuring Up on College-Level Learning](https://reader035.fdocuments.us/reader035/viewer/2022062322/56814943550346895db68d2f/html5/thumbnails/24.jpg)
http:///collegelevellearning.org