Focus on 2011-2012. All of these vocabulary words will increase testing achievement.
Testing, Academic Achievement, and the Future of College ... · Testing, Academic Achievement, and...
Transcript of Testing, Academic Achievement, and the Future of College ... · Testing, Academic Achievement, and...
![Page 1: Testing, Academic Achievement, and the Future of College ... · Testing, Academic Achievement, and the Future of College Admissions. #NACAC17 Measuring Success: Testing, Grades, and](https://reader033.fdocuments.us/reader033/viewer/2022053001/5f059ef57e708231d413debd/html5/thumbnails/1.jpg)
#NACAC17
B12:Testing, Academic Achievement, and the Future of College Admissions
![Page 2: Testing, Academic Achievement, and the Future of College ... · Testing, Academic Achievement, and the Future of College Admissions. #NACAC17 Measuring Success: Testing, Grades, and](https://reader033.fdocuments.us/reader033/viewer/2022053001/5f059ef57e708231d413debd/html5/thumbnails/2.jpg)
#NACAC17
Measuring Success:Testing, Grades, and the Future of College Admissions
Jack BuckleyAmerican Institutes for Research, Washington DC
![Page 3: Testing, Academic Achievement, and the Future of College ... · Testing, Academic Achievement, and the Future of College Admissions. #NACAC17 Measuring Success: Testing, Grades, and](https://reader033.fdocuments.us/reader033/viewer/2022053001/5f059ef57e708231d413debd/html5/thumbnails/3.jpg)
#NACAC17
Measuring Success: Two Key Questions
• More heat than light in the debate over standardized testing in admissions
• My co-editors and I wanted to produce a volume that brings together the best
empirical thinking around two questions:
– To what extent should standardized tests play a role in college and university
admissions?
– How should an institution of higher education best assess prospective student talent in
a manner that promotes fairness?
• Result is the forthcoming volume from Johns Hopkins University Press!
![Page 4: Testing, Academic Achievement, and the Future of College ... · Testing, Academic Achievement, and the Future of College Admissions. #NACAC17 Measuring Success: Testing, Grades, and](https://reader033.fdocuments.us/reader033/viewer/2022053001/5f059ef57e708231d413debd/html5/thumbnails/4.jpg)
#NACAC17
Overview
• Chapter 1: Eight Myths about Standardized Admissions Testing
– Paul Sackett and Nathan Kuncel, University of Minnesota
• Chapter 2: The Core Case for Testing: The State of Our Research Knowledge
– Emily Shaw, The College Board
• Chapter 3: Grade Inflation and the Role of Standardized Testing
– Michael Hurwitz, The College Board and Jason Lee, Tennessee Higher Education Commission
• Chapter 4: Merit-Based Scholarships in Student Recruitment and the Role of Standardized Tests
– Jonathan Jacobs, Jim Brooks, and Roger Thompson, University of Oregon
![Page 5: Testing, Academic Achievement, and the Future of College ... · Testing, Academic Achievement, and the Future of College Admissions. #NACAC17 Measuring Success: Testing, Grades, and](https://reader033.fdocuments.us/reader033/viewer/2022053001/5f059ef57e708231d413debd/html5/thumbnails/5.jpg)
#NACAC17
Overview
• Chapter 5: When HSGPA and Test Scores Disagree: Implications for Test-
Optional Policies
– Edgar Sanchez and Krista Mattern, ACT
• Chapter 6: The Rise of Test-Optional Admissions Practices
– Jerome Lucido, University of Southern California
• Chapter 7: Going Test-Optional: A Case Study
– Eric Maguire, Franklin & Marshall College
• Chapter 8: Test Scores and High School Grades as Predictors
– William Bowen (Mellon), Matthew Chingos (Urban), Michael McPherson (Spencer)
![Page 6: Testing, Academic Achievement, and the Future of College ... · Testing, Academic Achievement, and the Future of College Admissions. #NACAC17 Measuring Success: Testing, Grades, and](https://reader033.fdocuments.us/reader033/viewer/2022053001/5f059ef57e708231d413debd/html5/thumbnails/6.jpg)
#NACAC17
Overview
• Chapter 9: How Do Percent Plans and Other Test-Optional Admissions Programs
Affect the Academic Performance and Diversity of the Entering Class?
– Rebecca Zwick, Educational Testing Service
• Chapter 10: The Test-Optional Movement at America’s Selective Liberal Arts
Colleges: A Book for Equity or Something Else?
– Andrew Belasco (College Transitions), Kelly Rosinger (Penn State), and James Hearn
(University of Georgia)
• Chapter 11: The Effect of Going Test-Optional on Diversity and Admissions: A
Propensity Score Matching Analysis
– Kyle Sweitzer, A. Emiko Blalock, and Dhruv Sharma, Michigan State University
![Page 7: Testing, Academic Achievement, and the Future of College ... · Testing, Academic Achievement, and the Future of College Admissions. #NACAC17 Measuring Success: Testing, Grades, and](https://reader033.fdocuments.us/reader033/viewer/2022053001/5f059ef57e708231d413debd/html5/thumbnails/7.jpg)
#NACAC17
Teaser from Chapter 3
![Page 8: Testing, Academic Achievement, and the Future of College ... · Testing, Academic Achievement, and the Future of College Admissions. #NACAC17 Measuring Success: Testing, Grades, and](https://reader033.fdocuments.us/reader033/viewer/2022053001/5f059ef57e708231d413debd/html5/thumbnails/8.jpg)
#NACAC17
When HSGPA and Test Scores Disagree: Implications for Test-Optional Policies
Edgar Sanchez & Krista MatternACT, Iowa
![Page 9: Testing, Academic Achievement, and the Future of College ... · Testing, Academic Achievement, and the Future of College Admissions. #NACAC17 Measuring Success: Testing, Grades, and](https://reader033.fdocuments.us/reader033/viewer/2022053001/5f059ef57e708231d413debd/html5/thumbnails/9.jpg)
#NACAC17
Test Optional Policies
• Test-optional policies come in many shapes and sizes– No standardized test scores are considered for any applicant
– Standardized test scores are still required, but applicants may choose from a list that includes the ACT and SAT and various tests such as AP, IB, or SAT II subject tests.
– Standardized test scores may or may not be required depending upon other criteria such as program being applied to, home-schooling, out-of-state residency, or HSGPA or class rank minimum
– Placement tests or school-specific admissions exams rather than ACT/SAT
– Standardized test used only for placement
– Even colleges that offer to consider an application without test scores may require additional proof of achievement, such as a graded writing sample or an on-campus interview
![Page 10: Testing, Academic Achievement, and the Future of College ... · Testing, Academic Achievement, and the Future of College Admissions. #NACAC17 Measuring Success: Testing, Grades, and](https://reader033.fdocuments.us/reader033/viewer/2022053001/5f059ef57e708231d413debd/html5/thumbnails/10.jpg)
#NACAC17
Perceived Benefits of Test Optional Policies
• Address perceived concerns with Standardized Testing– Improper use of tests
– Equity in access to test prep/coaching
– Fairness/Bias
– Predictive validity of test scores
• Increase diversity
![Page 11: Testing, Academic Achievement, and the Future of College ... · Testing, Academic Achievement, and the Future of College Admissions. #NACAC17 Measuring Success: Testing, Grades, and](https://reader033.fdocuments.us/reader033/viewer/2022053001/5f059ef57e708231d413debd/html5/thumbnails/11.jpg)
#NACAC17
Researched Responses to Perceived Benefits
• Little scientific research to suggest prep/coaching has a significant impact on test scores. Hard work in school and rigorous course taking has a strong impact on test scores.
• Gaps cited as concerns in test scores are also seen in all other academic achievement indicators including HSGPA. There are many studies that document the subgroup gaps, but rather than highlight bias they seem to highlight population differences resulting from educational experiences.
• There are many studies that highlight the predictive power of test scores. While HSGPA is typically the strongest predictor of postsecondary outcomes, test scores provide unique and valuable additional information.
• Research has shown mixed results in regards to diversity outcomes. Additionally, test optional policies are often implemented in conjunction with other awareness and recruitment initiatives making it difficult to isolate the effect of test optional policies on campus diversity.
![Page 12: Testing, Academic Achievement, and the Future of College ... · Testing, Academic Achievement, and the Future of College Admissions. #NACAC17 Measuring Success: Testing, Grades, and](https://reader033.fdocuments.us/reader033/viewer/2022053001/5f059ef57e708231d413debd/html5/thumbnails/12.jpg)
#NACAC17
Discrepant ACT and HSGPA
• Prior research has documented the relationship between academic preparation, as measured by high school grade point average (HSGPA) and admissions test scores, and successful college outcomes.
• Test scores and grades often send a consistent message about the academic achievement level of students.
• There are instances, however, when test scores and grades send mixed messages about the academic performance of students.
![Page 13: Testing, Academic Achievement, and the Future of College ... · Testing, Academic Achievement, and the Future of College Admissions. #NACAC17 Measuring Success: Testing, Grades, and](https://reader033.fdocuments.us/reader033/viewer/2022053001/5f059ef57e708231d413debd/html5/thumbnails/13.jpg)
#NACAC17
Heat map of ACT Composite score and HSGPA for the ACT-tested graduating class of 2016 (N= 1,709,659)
Relationship between ACT Composite and HSGPA
![Page 14: Testing, Academic Achievement, and the Future of College ... · Testing, Academic Achievement, and the Future of College Admissions. #NACAC17 Measuring Success: Testing, Grades, and](https://reader033.fdocuments.us/reader033/viewer/2022053001/5f059ef57e708231d413debd/html5/thumbnails/14.jpg)
#NACAC17
Implications to Test Optional Colleges
• The impact of discrepant performance on college outcomes is mixed making an even stronger case for considering both pieces of information when considering an applicants' likelihood of future success.
• More precise diagnoses of a student's level of college readiness is useful not only for admissions but also for providing tailored interventions for at-risk students.
![Page 15: Testing, Academic Achievement, and the Future of College ... · Testing, Academic Achievement, and the Future of College Admissions. #NACAC17 Measuring Success: Testing, Grades, and](https://reader033.fdocuments.us/reader033/viewer/2022053001/5f059ef57e708231d413debd/html5/thumbnails/15.jpg)
#NACAC17
Implications to Test Optional Colleges
• Compensatory nature of test scores and HSGPA– Higher grades can offset lower test scores and vice versa in terms of future
college outcomes.– High school grades encompass students’ non-cognitive factors such as
scholastic engagement, self-regulation, discipline, or habits of inquiry, in addition to providing information on cognitive skills.
– Standardized tests, on the other hand, provide a more narrow assessment focusing on cognitive skills in a standardized manner.
• This compensatory relationship is lost when institutions elect to institute test-optional policies.
![Page 16: Testing, Academic Achievement, and the Future of College ... · Testing, Academic Achievement, and the Future of College Admissions. #NACAC17 Measuring Success: Testing, Grades, and](https://reader033.fdocuments.us/reader033/viewer/2022053001/5f059ef57e708231d413debd/html5/thumbnails/16.jpg)
#NACAC17
• Multiple pieces of information presents a clearer picture of students’ strengths and weaknesses and their likelihood of future success emerges.
• Empirical evidence of the validity of test scores for use in college admissions underscores the value of multiple measures.
– This is particularly true when the various pieces of information are inconsistent. These are the very students who are mostly likely to be attracted to test-optional institutions.
• While test-optional policies promote discarding useful information, we stand behind the belief that more information is better than less information.
Implications to Test Optional Colleges
![Page 17: Testing, Academic Achievement, and the Future of College ... · Testing, Academic Achievement, and the Future of College Admissions. #NACAC17 Measuring Success: Testing, Grades, and](https://reader033.fdocuments.us/reader033/viewer/2022053001/5f059ef57e708231d413debd/html5/thumbnails/17.jpg)
#NACAC17
0.90
0.82
0.51
0.87
0.77
0.69
0.88
0.69 0.67
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1Pr
obab
ility
of E
nrol
lmen
t
Example: Probability of Enrollment by Discrepancy Group
![Page 18: Testing, Academic Achievement, and the Future of College ... · Testing, Academic Achievement, and the Future of College Admissions. #NACAC17 Measuring Success: Testing, Grades, and](https://reader033.fdocuments.us/reader033/viewer/2022053001/5f059ef57e708231d413debd/html5/thumbnails/18.jpg)
#NACAC17
Example: Probability of Persistence by Discrepancy Group0.81
0.69
0.57
0.73
0.60 0.60
0.76
0.670.65
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
Prob
abili
ty o
f Per
sist
ence
![Page 19: Testing, Academic Achievement, and the Future of College ... · Testing, Academic Achievement, and the Future of College Admissions. #NACAC17 Measuring Success: Testing, Grades, and](https://reader033.fdocuments.us/reader033/viewer/2022053001/5f059ef57e708231d413debd/html5/thumbnails/19.jpg)
#NACAC17
The Rise and Relevance of Test Optional Admission Policies
Jerome A. Lucido, Professor and Exec. Director
USC Center for Enrollment Research, Policy, and Practice
![Page 20: Testing, Academic Achievement, and the Future of College ... · Testing, Academic Achievement, and the Future of College Admissions. #NACAC17 Measuring Success: Testing, Grades, and](https://reader033.fdocuments.us/reader033/viewer/2022053001/5f059ef57e708231d413debd/html5/thumbnails/20.jpg)
#NACAC17
Charge for the Chapter
Analysis of test-optional from a practitioner/scholar:
• What are its stated purposes and claims
• What does research on the matter reveal
• Structured interviews
• Policy analysis
![Page 21: Testing, Academic Achievement, and the Future of College ... · Testing, Academic Achievement, and the Future of College Admissions. #NACAC17 Measuring Success: Testing, Grades, and](https://reader033.fdocuments.us/reader033/viewer/2022053001/5f059ef57e708231d413debd/html5/thumbnails/21.jpg)
#NACAC17
Guidelines for Policy and Practice
• Commentary on the way forward for those who wish to:
- Consider policy options,
- Examine their results, and
- Draw upon the latest thinking in this realm
![Page 22: Testing, Academic Achievement, and the Future of College ... · Testing, Academic Achievement, and the Future of College Admissions. #NACAC17 Measuring Success: Testing, Grades, and](https://reader033.fdocuments.us/reader033/viewer/2022053001/5f059ef57e708231d413debd/html5/thumbnails/22.jpg)
#NACAC17
Guidance for the Profession
• Places test-optional practices within the evolution of the
admission and enrollment management profession
• Examines the role of testing and test-optional policies in
promoting college access.
• Considers the potential of both to further influence thinking
and practice.
![Page 23: Testing, Academic Achievement, and the Future of College ... · Testing, Academic Achievement, and the Future of College Admissions. #NACAC17 Measuring Success: Testing, Grades, and](https://reader033.fdocuments.us/reader033/viewer/2022053001/5f059ef57e708231d413debd/html5/thumbnails/23.jpg)
#NACAC17
A Very Brief History of Test-Optional
• Bowdoin (1969)
• Atkinson and the University of California (2001)
• Hiss and Franks at Bates (2004) (2014)
• NACAC Commission: Report on the Use of Standardized
Tests in Undergraduate Admission (2008)
• Foundation in liberal arts colleges
![Page 24: Testing, Academic Achievement, and the Future of College ... · Testing, Academic Achievement, and the Future of College Admissions. #NACAC17 Measuring Success: Testing, Grades, and](https://reader033.fdocuments.us/reader033/viewer/2022053001/5f059ef57e708231d413debd/html5/thumbnails/24.jpg)
#NACAC17
Purposes and Objectives of T-O
• Concerns about testing and the by-products of testing:
- Misuse or narrow use of results in admissions
- Test prep industry
- Stress of testing
- Concerns about efficacy for underrepresented
populations
![Page 25: Testing, Academic Achievement, and the Future of College ... · Testing, Academic Achievement, and the Future of College Admissions. #NACAC17 Measuring Success: Testing, Grades, and](https://reader033.fdocuments.us/reader033/viewer/2022053001/5f059ef57e708231d413debd/html5/thumbnails/25.jpg)
#NACAC17
• Student Benefit
- Addresses fears
- Opens minds to possibility
- Brings rhetoric into balance with practice
Purposes and Objectives of Test-Optional
![Page 26: Testing, Academic Achievement, and the Future of College ... · Testing, Academic Achievement, and the Future of College Admissions. #NACAC17 Measuring Success: Testing, Grades, and](https://reader033.fdocuments.us/reader033/viewer/2022053001/5f059ef57e708231d413debd/html5/thumbnails/26.jpg)
#NACAC17
• Institutional Benefit
- Enrollment and rankings strategy
- Diversity enhancement
- T-O does not operate in a vacuum
Purposes and Objectives of Test-Optional
![Page 27: Testing, Academic Achievement, and the Future of College ... · Testing, Academic Achievement, and the Future of College Admissions. #NACAC17 Measuring Success: Testing, Grades, and](https://reader033.fdocuments.us/reader033/viewer/2022053001/5f059ef57e708231d413debd/html5/thumbnails/27.jpg)
#NACAC17
• Conflicting Research
- Belasco, et al
- Hiss and Franks
- The criticality of local studies
How Do T-O Policies Play Out in Practice?
![Page 28: Testing, Academic Achievement, and the Future of College ... · Testing, Academic Achievement, and the Future of College Admissions. #NACAC17 Measuring Success: Testing, Grades, and](https://reader033.fdocuments.us/reader033/viewer/2022053001/5f059ef57e708231d413debd/html5/thumbnails/28.jpg)
#NACAC17
• Self limiting application decisions
• Narrow admission heuristics
• Overreliance on profile
• Discouragement of underrepresented prospects
• The intimidation factor
Barriers Addressed by Test-Optional
![Page 29: Testing, Academic Achievement, and the Future of College ... · Testing, Academic Achievement, and the Future of College Admissions. #NACAC17 Measuring Success: Testing, Grades, and](https://reader033.fdocuments.us/reader033/viewer/2022053001/5f059ef57e708231d413debd/html5/thumbnails/29.jpg)
#NACAC17
• A view of the test-optional adopter as
pragmatic idealist emerges from this
investigation
• Not actually test “opponents”
The People Behind Test-Optional
![Page 30: Testing, Academic Achievement, and the Future of College ... · Testing, Academic Achievement, and the Future of College Admissions. #NACAC17 Measuring Success: Testing, Grades, and](https://reader033.fdocuments.us/reader033/viewer/2022053001/5f059ef57e708231d413debd/html5/thumbnails/30.jpg)
#NACAC17
• Neither panacea nor the only way
• The future is not one or the other
• Toward enlightened practice
• No substitute for expertise and information
Reconciling Testing and Test-Optional
![Page 31: Testing, Academic Achievement, and the Future of College ... · Testing, Academic Achievement, and the Future of College Admissions. #NACAC17 Measuring Success: Testing, Grades, and](https://reader033.fdocuments.us/reader033/viewer/2022053001/5f059ef57e708231d413debd/html5/thumbnails/31.jpg)
#NACAC17
• While T-O seeks to mitigate negative impacts of
testing and test misuse (among other things), the
need for sound information in the admission
decision remains.
Conclusion
![Page 32: Testing, Academic Achievement, and the Future of College ... · Testing, Academic Achievement, and the Future of College Admissions. #NACAC17 Measuring Success: Testing, Grades, and](https://reader033.fdocuments.us/reader033/viewer/2022053001/5f059ef57e708231d413debd/html5/thumbnails/32.jpg)
#NACAC17
• Test results, in the hands of thoughtful practitioners,
adds value to admission decisions. Can wise
decisions be made without them? Years of test-
optional practice now suggests that they can.
Whether they “should” is a matter of institutional
choice, mission, and local research.
Conclusion
![Page 33: Testing, Academic Achievement, and the Future of College ... · Testing, Academic Achievement, and the Future of College Admissions. #NACAC17 Measuring Success: Testing, Grades, and](https://reader033.fdocuments.us/reader033/viewer/2022053001/5f059ef57e708231d413debd/html5/thumbnails/33.jpg)
#NACAC17
• Testing and T-O together hold the seeds of better
and more highly informed practice. This includes
mitigating the negative impact of test shortcomings
and misuse while enlightening admission decision
makers with information that can lead to better
admission decisions, better academic advising, and
better results for students.
Conclusion
![Page 34: Testing, Academic Achievement, and the Future of College ... · Testing, Academic Achievement, and the Future of College Admissions. #NACAC17 Measuring Success: Testing, Grades, and](https://reader033.fdocuments.us/reader033/viewer/2022053001/5f059ef57e708231d413debd/html5/thumbnails/34.jpg)
#NACAC17
• Jerome A. Lucido, Professor and Director
• USC Center for Enrollment Research, Policy and
Practice
http://cerpp.usc.edu/
Thank You!