Assessment in language learning classrooms:More questions than answers
-
Upload
robert-dickey -
Category
Education
-
view
458 -
download
2
Transcript of Assessment in language learning classrooms:More questions than answers
Assessment in language learning classrooms:More questions than
answers
Robert J. DickeyKeimyung University, S. Korea
Defining Assesment…
?
Defining Assesment…• the wide variety of methods that educators use to evaluate, measure, and
document the academic readiness, learning progress, and skill acquisition of students
• the use of empirical data on student learning to refine programs and improve student learning
• the process of gathering and discussing information from multiple and diverse sources in order to develop a deep understanding of what students know, understand, and can do with their knowledge as a result of their educational experiences; the process culminates when assessment results are used to improve subsequent learning
• the systematic basis for making inferences about the learning and development of students --. the process of defining, selecting, designing, collecting, analyzing, interpreting, and using information to increase students’ learning and development
• the systematic collection, review, and use of information about educational programs undertaken for the purpose of improving student learning and development
Opening Quiz!Rank the following in determining student grades.__ performance (as indicated in formal tests, quizzes, etc)__ (oral) participation in class__ improvement over the course__ behavior in class__ effort__ motivation__ assignments__ punctuality and attendance__ intuitive, informal assessments of class performance (Most important = 1, Least/Not important = 9)
Who are we testing?
?Elicitation is both testing and lesson confirmation
Who are we testing?
• The Students
• The Teacher
What are we testing?
?
What are we testing?
• Overall English– competence? proficiency? skill?
• Mastery of course content• Needs of students• Improvements during the course• Teaching performance• Coursebook / Curriculum
Learning Objectives
Bloom’s Taxonomy
Learning Objectives
When do we test?
?
When do we test?
• Pre-test• Midterms• Finals• Quizzes• More Frequent > > >
– Elicitation– Homework/Assignments– “heat of the action”
Why do we test?
?
Why do we test?
• Required (from above)• Duty (teacher’s sense)• Tradition (always been testing)• Expectation (parents, others)• Feedback (students, others) (request)• Ease (following coursebook pattern)
How do we test?
?
How do we test?
• 4 skills• “forms” : vocabulary,
grammar, spelling…• fluency/accuracy• paper & pencil• multiple-choice• fill-the-blank• open-ended• interview
• group-tests• rubrics• standardized• textbook-source• commercial (generic)• curriculum-based• standards-based• teacher-created• computer-based
Types of tests
• Proficiency (regardless of training)• Achievement (final and progress)• Diagnostic• Formative / Pedagogic• Placement• Selection/Gate-keeping• Aptitude• Encouragement !!!
Oral & Written test forms
Types of tests
Traditional & Alternative• One-shot, standardized exams• Timed, multiple-choice format• Decontextualized test items
• Scores suffice for feedback
• Norm-referenced scores• Focus on the "right" answer• Summative• Oriented to product• Non-interactive performance• Fosters extrinsic motivation
• Continuous long-term assessment• Untimed, free-response format• Contextualized communicative
tasks• Individualized feedback and
washback• Criterion-referenced scores• Open-ended, creative answers• Formative• Oriented to process• Interactive performance• Fosters intrinsic motivation
Other Assessment Instruments
• Portfolio / Journal / Notebook• Projects• Systematic set of observations• Communicative tests through real-
world tasks• “personal chats” with instructor
How do we score?
?
How do we score?
• Normative• Relative• “pure” scores
• “Good Job!” / “good scolding”
“The Curve”
“The Curve”
How do we feedback scores?
?
How do we feedback scores?
• Public / Private• Personalized / Relative to the class• Commentary / Numbers• Constructive
Principles in Assessment
• Practicality• Reliability• Validity
Practicality in Assessment
• Expense• Within time constraints
– preparation– implementation– grading and review
• Scoring system is specific and time-efficient
Reliability in Assessment
• Consistent & Dependable• Students (a bad day?)• Rater (inter-rater, intra-rater)• Conditions (noise, heat, etc)• Is a timed-test more fair? To whom?• More than one possible answer?• Too easy/difficult (grade distinctions)
Validity in Assessment• Rampantly misunderstood !1.Face Validity2.Content Validity3.Criterion Validity (compare to other test)
(concurrent, predictive)
concurrent, predictive4.Construct Validity (test assumptions match
matter under investigation)(speaking test must include vocabulary?)
Face ValidityMostly Under-appreciated•“looks right” to ALL users•Familiar tasks•Familiar format•Clearly do-able within allowed time•Clear and uncomplicated items•Clear directions•Difficulty is “a reasonable challenge”
Authenticity
No one agrees exactly what this means•Language as natural as possible•Contextualized testing•Meaningful topics•Thematic organization of materials•Tasks approximate “real-world” tasks
Washback
• Pedagogic (formative) testing INTENDS a positive washback
• Information flows to students and/or teacher
• Affects future studies:– Lesson plan designs– Student motivation and focus
Choices we make…
Aims of Assessment
• Assessment for Learning
• Assessment as Learning
• Assessment of Learning
Assessment for Learning• to give teachers information• to modify the learning activities• to differentiate & focus how individual students
approach their learning• emphasis is on teachers using the information
from carefully-designed assessments• to determine what students know• to gain insights on how, when, & whether
students use what they know• so instruction can be targeted
Assessment as Learning• a process of developing & supporting
metacognition for students• focuses on the role of the student as the critical
connector between assessment and learning– Students, as active, engaged and critical assessors
make sense of information, relate it to prior knowledge, and use it for new learning
• the vehicle for helping students develop, practice and become comfortable with reflection and with critical analysis of their own learning
Assessment of Learning• to confirm what students know• to demonstrate whether or not the students have
met the standards and/or show how they are placed in relation to others
• Focus on creating accurate and sound statements of proficiency or competence for students, so that the recipients of the information can use the information to make reasonable and defensible decisions
Integrating Assessment
A quick survey
• Instructors for English Conversation courses
• Universities in S. Korea• June 2015• 58 respondents• Online collection (solicited through
Facebook groups)
Included in final course grade• attendance 94.83%• participation 91.38%• final examination 91.38%• midterm examination 86.21%• assignments 53.45%• presentation (oral) 41.38%• quiz 36.21%• report (written) 12.07%• notebook 5.17%• Other includes
-- online assignments -- TOEIC test-- transcription of a conversation -- video assignment-- writing lab
Included in final examination• Oral-based open response 61.11%• paper-based multiple-choice 40.74%• paper-based vocabulary assessment 40.74%• paper-based fill-in the blank/complete the sentence 37.04%• paper-based grammar assessment 35.19%• oral-based vocabulary assessment 33.33%• paper-based open response 31.48%• oral-based grammar assessment 29.63%• listening (accuracy) 29.63%• pronunciation 24.07%• oral-based taught phrases 24.07%• oral-based "content" (non-linguistic materials) 24.07%• paper-based taught phrases 14.81%• reading (accuracy) 11.11%• paper-based "content" (non-linguistic materials) 9.26%• oral-based fill-in the blank/complete the sentence 3.70%• other
Questions?
File available at http://tinyurl.com/q7pm5vt