Rick Chan Frey, PhD University of California, Berkeley rick@mustardseedbooks
25
Perspectives on Word Difficulty Using Item Response Theory Rick Chan Frey, PhD University of California, Berkeley [email protected]
description
Perspectives on Word Difficulty Using Item Response Theory. Rick Chan Frey, PhD University of California, Berkeley [email protected]. Perspectives on Word Difficulty Using Item Response Theory. Rick Chan Frey - LRA 2013 [email protected]. The Amazonification of Knowledge - PowerPoint PPT Presentation
Transcript of Rick Chan Frey, PhD University of California, Berkeley rick@mustardseedbooks
Rethinking the Role of Decodable Texts in Early Literacy Instruction
Perspectives on Word Difficulty Using Item Response TheoryRick Chan Frey, PhDUniversity of California, [email protected] The Amazonification of KnowledgeHow does Amazon know what books youll enjoy reading?Amazon has no theoretical framework to distinguish between hundreds of potential recommendations based on genre or style. Analyzing the data from millions of sales with similar reading patternsPerspectives on Word Difficulty Using Item Response TheoryRick Chan Frey - LRA 2013 [email protected] percentage of the area of the square is covered by the circle?Perspectives on Word Difficulty Using Item Response TheoryRick Chan Frey - LRA 2013 [email protected]
How to Know Something Without Knowing Anything About ItRemember back to your 5th grade math, calculating the area of a square, circle and calculating a ratio.If you know the right formula, its easy and a mathematician could even explain why it works.Perspectives on Word Difficulty Using Item Response TheoryRick Chan Frey - LRA 2013 [email protected] to Know Something Without Knowing Anything About ItBut pretend for a moment you didnt know the formulas, didnt know how to calculate ratios.Is there another way to solve the problem?Perspectives on Word Difficulty Using Item Response TheoryRick Chan Frey - LRA 2013 [email protected] the Answer with ScratchDraw a square 4 x 4Draw a circle 2 diameter w/different colorHave the computer draw random spritesCount what colors they land onPerspectives on Word Difficulty Using Item Response TheoryRick Chan Frey - LRA 2013 [email protected] on Word Difficulty Using Item Response TheoryRick Chan Frey - LRA 2013 [email protected]
Perspectives on Word Difficulty Using Item Response TheoryRick Chan Frey - LRA 2013 [email protected] is IRT?
First and foremost, IRT (as with classical test theory) is about assessment, not instruction. Adaptive testing - some questions are rated easier or harder than others, so the test has a better ability to differentiate student ability than a generic test.Most developed statistical model for interactively measuring item difficulty and student ability.Perspectives on Word Difficulty Using Item Response TheoryRick Chan Frey - LRA 2013 [email protected] on Word Difficulty Using Item Response TheoryRick Chan Frey - LRA 2013 [email protected] - Response - Theory
Item questions on a survey, math problems, multiple choice questions on SATResponse easily quantifiable, usually right/wrong, can have partial credit.Theory hypothesizing a construct (e.g. intelligence, love of art, etc.) that can be measured.
Perspectives on Word Difficulty Using Item Response TheoryRick Chan Frey - LRA 2013 [email protected] by example - Are you a tech junkie?
Do you own a computer?Do you own a smart phone?Do you own two or more tablets?Have you ever tried Google glasses?
Perspectives on Word Difficulty Using Item Response TheoryRick Chan Frey - LRA 2013 [email protected] tried Google glassesBill GatesgrandmaAverage teenagerOwn a computerOwn a smart phoneOwn 2+ tabletsLudditeTech junkiePerspectives on Word Difficulty Using Item Response TheoryRick Chan Frey - LRA 2013 [email protected] could IRT offer reading researchers?
Quantitative measure of word difficultyImproved accuracy in determining student reading ability from assessmentsReliability measures (item and test level)
Perspectives on Word Difficulty Using Item Response TheoryRick Chan Frey - LRA 2013 [email protected] do researchers estimate word difficulty?Ehri (2005) Earliest and simplestBeck, McKeown & Kucan (2002) Word TiersHiebert, Stewart & Uzicanin (2010) Word Features and Word Recognition
Fountas & Pinnell Guided ReadingMetaMetrics Lexile Scoring SystemPerspectives on Word Difficulty Using Item Response TheoryRick Chan Frey - LRA 2013 [email protected] are the basics we know?
Shorter words easier than longer wordsFewer syllables are easierHigher frequency words are easierHighly imageable words are easierA quantitative measure of word difficulty could be highly usefulPerspectives on Word Difficulty Using Item Response TheoryRick Chan Frey - LRA 2013 [email protected] the IdeaOne school, two 1st grade classes, (n=75)Data from two years of DIBELS assessments Oral Reading Fluency (ORF) assessmentFirst 40 words of four different ORF assessmentsSimple IRT analysis marking incorrectly read words as incorrect responses (no partial credit)Perspectives on Word Difficulty Using Item Response TheoryRick Chan Frey - LRA 2013 [email protected] measures of word difficulty and student reading abilityStrong correlations (r between .54 and .72) for basics of word difficultyPotential measure for text reliabilityIdentifies words that defy expectationsProvides potential model for analyzing the impact of context on word difficultyPerspectives on Word Difficulty Using Item Response TheoryRick Chan Frey - LRA 2013 [email protected] Word Difficulty
TextWordLtrsDifficultyORF 1B-36 waiting 7 3.66 ORF 1A-6 outside 7 3.41 ORF 1B-3 mind 4 2.93ORF 2A-21 spot 4 1.81 ORF 2A-29 got 3 0.28ORF 1B-32 fish 4 -1.65ORF 1B-1 I 1 -3.12 Perspectives on Word Difficulty Using Item Response TheoryRick Chan Frey - LRA 2013 [email protected] Strange Cases
First we picked a spot far from the big waves -- guess the word with the highest item difficulty score?3 letters, fairly easy phonetically, but easily confused with for of the hardest 20 words had 3-4 letters, one syllable and were phonetically regularYellow, castle, anymore, all easier than traits would indicatePerspectives on Word Difficulty Using Item Response TheoryRick Chan Frey - LRA 2013 [email protected] Context Effect
A six letter, two syllable word with an unusual spelling pattern is missed by only 2 students, difficulty score -1.01We built a giant sand castle at the beach.Built scored 2.85 difficult and giant was one of the top 5 hardest words at 3.34Fish tank and rocky road, no such luckPerspectives on Word Difficulty Using Item Response TheoryRick Chan Frey - LRA 2013 [email protected] Context Possibilities Compare instances of reading same words in different contexts (starting a sentence, in subordinate clause, etc.)Compare instances of words with suffixes (lick, licks, licked, licking)Compare the effects of background knowledge on word difficulty scoresPerspectives on Word Difficulty Using Item Response TheoryRick Chan Frey - LRA 2013 [email protected] Text Reliability
Use of Chronbachs alpha to measure reliability of a given text for assessing student reading abilityEach of the four passages scored .97 or higherFew words across the four passages had poor fitPerspectives on Word Difficulty Using Item Response TheoryRick Chan Frey - LRA 2013 [email protected] provides researchers with a quantitative method for assessing word difficulty that can be used in a wide variety of research designs.IRT offers useful information for text designers attempting to design and redesign increasingly complicated texts that comply with common core standards.IRT offers a window into the brave new world of big data, suggesting new ideas about literacy development we don't necessarily understand but would be wise to consider.Perspectives on Word Difficulty Using Item Response TheoryRick Chan Frey - LRA 2013 [email protected] Texts 3.0 -- www.mustardseedbooks.org
Perspectives on Word Difficulty Using Item Response TheoryRick Chan Frey - LRA 2013 [email protected] a copy of the presentation or questions contact:
Rick Chan Frey, [email protected]
www.mustardseedbooks.org
