Post on 30-Jul-2015
Top 5 pitfalls to avoid in English CEFR progress assessment
Brano Pokrivcak
About TrackTest.eu
• established in 2012 in Slovakia
• users from 140 countries
• customers in 60 countries
• companies, schools, individuals
TrackTest coverage
Do you like maths?
Reliable language assessment
http://en.wikipedia.org/wiki/Glicko_rating_system
Top 5 pitfalls to avoid in English CEF progress assessment
1. Choosing the right test format
One test vs. level tests ?
One test: Pros:– One test = one score– Fast and comparable– Must for placement tests.
Cons:– Assumes that your knowledge is linear– Not wide enough– Problem how to match to CEFR
- Adaptive test
Single test vs. level tests ?
Level test: Pros:– Different skills can be tested (less multi-choice grammar
questions, more reading comprehension in higher levels)– Questions adequate to the level– More detailed assessment of topics for the level
Cons:– Difficult to decide which level to start– Progress between levels
Assessments systems CEFR Cambridge ESOL IELTS TOEFL TOEICC2 Proficiency (CPE) 8 - 9
C1 Advanced (CAE), BEC Higher 7 110-120 880
B2 First Certificate (FCE), BEC Vantage 5 - 6 87-109
B1Preliminary (PET), BEC Preliminary 4 57-86 550
A2 KEY (KET)
Source: blog.tracktest.eu/2013/12/tracktest-cef-levels-compared-with.html
2. How to measure progress between levels ?
2. How to measure progress between CEF levels ?
3. Reliability of test
Same difficulty for everybody (with one test or generated test)
• heavily organized assessments (e.g. maturita)- one set of questions
• Diversified environments (corporations and/or multiple locations) require more flexibility
Challenge: Generated test needs to match the difficulty & topics coverage
What is cheating ?
• Copying answers from classmates ?
• Googling for the right answer ?
• Dictionary ?
• PC spellchecker ?
4. Stress factor
4. Stress factor
- Time
- Computer vs. pen&paper- Generation X vs Millenials
5. Motivate
Motivate
a) Feedback
During the test- preliminary score (How am I doing)
After the completion- analysis of mistakes (What should I improve)
19 Topics
Adjectives/adv ArticlesPrepositionsComparatives ConjunctionsPronounsQuantifiersModal verbs
Infinitive/ing Past tensesPresent tenses Future tenses Conditionals Questions
Idioms VocabularyPhrasal verbs Reading Listening
Slovakia
Adjectives/adv ArticlesPrepositionsComparatives ConjunctionsPronounsQuantifiersModal verbs
Infinitive/ing Past tensesPresent tenses Future tenses Conditionals Questions
Idioms VocabularyPhrasal verbs Reading Listening
Slovakia
Adjectives/adv ArticlesPrepositionsComparatives ConjunctionsPronounsQuantifiersModal verbs
Infinitive/ing Past tensesPresent tenses Future tenses Conditionals Questions
Idioms VocabularyPhrasal verbs Reading Listening
Slovakia
Adjectives/adv ArticlesPrepositionsComparatives ConjunctionsPronounsQuantifiersModal verbs
Infinitive/ing Past tensesPresent tenses Future tenses Conditionals Questions
Idioms VocabularyPhrasal verbs Reading Listening
Slovakia
Adjectives/adv ArticlesPrepositionsComparatives ConjunctionsPronounsQuantifiersModal verbs
Infinitive/ing Past tensesPresent tenses Future tenses Conditionals Questions
Idioms VocabularyPhrasal verbs Reading Listening
Vietnam
Adjectives/adv ArticlesPrepositionsComparatives ConjunctionsPronounsQuantifiersModal verbs
Infinitive/ing Past tensesPresent tenses Future tenses Conditionals Questions
Idioms VocabularyPhrasal verbs Reading Listening
United Kingdom
Adjectives/adv ArticlesPrepositionsComparatives ConjunctionsPronounsQuantifiersModal verbs
Infinitive/ing Past tensesPresent tenses Future tenses Conditionals Questions
Idioms VocabularyPhrasal verbs Reading Listening
5. Motivate
a) Feedback
b) Elements of gamification
Gamification in assessment
Key elements of gamification (B-P-L):
1. Badges
2. Points
3. Leaderboards
Badges
Points
Leaderboards ?
1. Choose the right format2. Different test, same difficulty3. Progress between levels4. Avoid stress5. and motivate !
Top 5 pitfalls to avoidin CEF English progress assessment