Assessing Reading Skills in Young Children: Assessing Reading Skills in Young Children: The TBALL...
-
Upload
ashley-daisy-terry -
Category
Documents
-
view
219 -
download
3
Transcript of Assessing Reading Skills in Young Children: Assessing Reading Skills in Young Children: The TBALL...
![Page 1: Assessing Reading Skills in Young Children: Assessing Reading Skills in Young Children: The TBALL Project (Technology Based Assessment of Language and.](https://reader030.fdocuments.us/reader030/viewer/2022020219/56649e9a5503460f94b9cc83/html5/thumbnails/1.jpg)
Assessing Reading Skills Assessing Reading Skills in Young Children:in Young Children:The TBALL Project
(Technology Based Assessment of Language and
Literacy)*
CRESST Conference, UCLA
Sept 9th, 2004
Elaine AndersenUSC
Patti Price PPRICE Speech and
Language Technology
*Funding by NSF is gratefully acknowledged (Grant No. 0326214).
![Page 2: Assessing Reading Skills in Young Children: Assessing Reading Skills in Young Children: The TBALL Project (Technology Based Assessment of Language and.](https://reader030.fdocuments.us/reader030/viewer/2022020219/56649e9a5503460f94b9cc83/html5/thumbnails/2.jpg)
TBALL Overview
• TBALL people and goals
• Challenges (with examples)
• Approach and progress
• Plans
![Page 3: Assessing Reading Skills in Young Children: Assessing Reading Skills in Young Children: The TBALL Project (Technology Based Assessment of Language and.](https://reader030.fdocuments.us/reader030/viewer/2022020219/56649e9a5503460f94b9cc83/html5/thumbnails/3.jpg)
Schools:
28th street
UES
Hoover
Para los Ninos
Esperanza
TBALL Team
EE
GSEIS
CS
EE
Linguistics
Psychology
Education
PPRI CE Speech and Language
Technology
![Page 4: Assessing Reading Skills in Young Children: Assessing Reading Skills in Young Children: The TBALL Project (Technology Based Assessment of Language and.](https://reader030.fdocuments.us/reader030/viewer/2022020219/56649e9a5503460f94b9cc83/html5/thumbnails/4.jpg)
TBALL Team• UCLA: Abeer Alwan (PI), Eva Baker (Co-PI), Alison Bailey, Christy Boscardin, Margaret Heritage, Dick Muntz, Carlo Zaniolo• Berkeley: David Pearson (Co-PI)• USC: Elaine Andersen, Shrikanth Narayanan (Co-PI)• Consultant: Patti Price • Students: 11 Graduate Students and 3 Undergraduates • Teachers (RETs): Six teachers from K-12 schools• Advisory Board: 5 internationally known experts in speech technology, reading, and education
![Page 5: Assessing Reading Skills in Young Children: Assessing Reading Skills in Young Children: The TBALL Project (Technology Based Assessment of Language and.](https://reader030.fdocuments.us/reader030/viewer/2022020219/56649e9a5503460f94b9cc83/html5/thumbnails/5.jpg)
TBALL Specific Aims
Develop assessment system and tools
• Helpful for teachers
• Test mono and multi-lingual students consistently
• Automatically score, analyze K-2 children
Investigate emerging literacy measures that are reliable indicators of later academic performance
![Page 6: Assessing Reading Skills in Young Children: Assessing Reading Skills in Young Children: The TBALL Project (Technology Based Assessment of Language and.](https://reader030.fdocuments.us/reader030/viewer/2022020219/56649e9a5503460f94b9cc83/html5/thumbnails/6.jpg)
Why Technology-Based Assessment?
• Teacher time constraints
• Teacher knowledge constraints
• Attractive activity for children
• Assessment tailored to individual students needs
• Valid, reliable information about students’ progress and needs
![Page 7: Assessing Reading Skills in Young Children: Assessing Reading Skills in Young Children: The TBALL Project (Technology Based Assessment of Language and.](https://reader030.fdocuments.us/reader030/viewer/2022020219/56649e9a5503460f94b9cc83/html5/thumbnails/7.jpg)
Components of Assessment
Analyze results
Collect responses
Select and present test materials
![Page 8: Assessing Reading Skills in Young Children: Assessing Reading Skills in Young Children: The TBALL Project (Technology Based Assessment of Language and.](https://reader030.fdocuments.us/reader030/viewer/2022020219/56649e9a5503460f94b9cc83/html5/thumbnails/8.jpg)
Sample Challenges
What materials to present and how?
How to adapt speech recognition to children’s speech?
How to diagnosis discrepancies arising from
• Pronunciation differences
• Language exposure differences
How to detect distinct learner profiles?
![Page 9: Assessing Reading Skills in Young Children: Assessing Reading Skills in Young Children: The TBALL Project (Technology Based Assessment of Language and.](https://reader030.fdocuments.us/reader030/viewer/2022020219/56649e9a5503460f94b9cc83/html5/thumbnails/9.jpg)
What Material to Present?
Many different aspects of reading skills
Phonemic Awareness
Letter-sound knowledge, Blending, Spelling
Word Recognition, Rate and Accuracy
Morphology, Syntax, Comprehension
How to diagnostically assess all aspects within the focus span of a young child?
![Page 10: Assessing Reading Skills in Young Children: Assessing Reading Skills in Young Children: The TBALL Project (Technology Based Assessment of Language and.](https://reader030.fdocuments.us/reader030/viewer/2022020219/56649e9a5503460f94b9cc83/html5/thumbnails/10.jpg)
.. And how to present it??
Children’s demonstration of language & cognitive skills is highly variable across contexts
Researchers need to be sensitive to ecological validity of procedures
How will our collection technique affect the data?
Will it disadvantage some children in the measures?
![Page 11: Assessing Reading Skills in Young Children: Assessing Reading Skills in Young Children: The TBALL Project (Technology Based Assessment of Language and.](https://reader030.fdocuments.us/reader030/viewer/2022020219/56649e9a5503460f94b9cc83/html5/thumbnails/11.jpg)
Example of Presentation Differences: Hecht’s Results
Pretest Object Game Posttest
Per
cen
t co
rrec
t u
se
20
40
60
80
100
Percentage appropriate use of plural in each task
![Page 12: Assessing Reading Skills in Young Children: Assessing Reading Skills in Young Children: The TBALL Project (Technology Based Assessment of Language and.](https://reader030.fdocuments.us/reader030/viewer/2022020219/56649e9a5503460f94b9cc83/html5/thumbnails/12.jpg)
Speech Recognition Challenges
Shorter vocal tract lengths, higher pitch
Significant intra- and inter-speaker variability
Significant variability
• Different linguistic backgrounds
• Misarticulations
• Signal to noise ratio
Significant progress in speech recognition
Basic method is statistical pattern matching
This task is easy in some ways, but…
![Page 13: Assessing Reading Skills in Young Children: Assessing Reading Skills in Young Children: The TBALL Project (Technology Based Assessment of Language and.](https://reader030.fdocuments.us/reader030/viewer/2022020219/56649e9a5503460f94b9cc83/html5/thumbnails/13.jpg)
Reading Error or Pronunciation Difference?
How do we know that reading is correct? /k aw/• A misreading of ‘car’ (saw first letter and guessed)• Or, a misarticulation/idiolect (can’t say ‘r’)• Or, possibly a dialect/accent issue (/jh eh s/ for ‘yes’)
We don’t know what the word is unless we know something about the system
![Page 14: Assessing Reading Skills in Young Children: Assessing Reading Skills in Young Children: The TBALL Project (Technology Based Assessment of Language and.](https://reader030.fdocuments.us/reader030/viewer/2022020219/56649e9a5503460f94b9cc83/html5/thumbnails/14.jpg)
What marks “Hispanic accent” in English?
In Spanish, compared to English
Phonetics p t k closer to Eng b d g than to p t k
s z n t d: tongue on teeth, not behind them
Sounds missing: th, oy, etc.
Phonology s+ptkbdg only across syllables
Distinctions like ‘bit-beat’ not made
Literacy Words spelled ‘y’ pronounced ‘j’, (by some)
Words spelled ‘i’ pronounced ‘ee’, etc.
Exposure May be more likely to hear much BEV
Age of exposure, time of exposure are highly variable
![Page 15: Assessing Reading Skills in Young Children: Assessing Reading Skills in Young Children: The TBALL Project (Technology Based Assessment of Language and.](https://reader030.fdocuments.us/reader030/viewer/2022020219/56649e9a5503460f94b9cc83/html5/thumbnails/15.jpg)
Time
Wo
rd R
eco
gn
itio
n
5 6 7 8 9 10 11 12
0
Time
Wo
rd R
eco
gn
itio
n
5 6 7 8 9 10 11 12
0
Learner Profiles
Individual data is ‘messy’, but the ‘average’ line hides the two distinct groups of learners.
![Page 16: Assessing Reading Skills in Young Children: Assessing Reading Skills in Young Children: The TBALL Project (Technology Based Assessment of Language and.](https://reader030.fdocuments.us/reader030/viewer/2022020219/56649e9a5503460f94b9cc83/html5/thumbnails/16.jpg)
The Biggest Challenge
Multidisciplinary collaboration
To solve these challenges requires• Engineering
• Psychology, linguistics, psycholinguistics
• Experts in reading, assessment, datamining
Starting from such different points of view• Difficult to integrate into one coherent view
• Also the biggest opportunity
• And probably essential
![Page 17: Assessing Reading Skills in Young Children: Assessing Reading Skills in Young Children: The TBALL Project (Technology Based Assessment of Language and.](https://reader030.fdocuments.us/reader030/viewer/2022020219/56649e9a5503460f94b9cc83/html5/thumbnails/17.jpg)
Target Rating, Explanation
put
watch
cold
full
Target
put Wrong, confuses letter b and letter p
Wrong, not paying attention
Right, Hispanic accent
watch Wrong, doesn’t know –tch
Right, Hispanic accent
cold Wrong, confuses short and long vowels
Right, just child’s way of pronouncing word
full Wrong, confuses short and long vowels
Right, just child’s way of pronouncing word
Samples For You To Rate!
![Page 18: Assessing Reading Skills in Young Children: Assessing Reading Skills in Young Children: The TBALL Project (Technology Based Assessment of Language and.](https://reader030.fdocuments.us/reader030/viewer/2022020219/56649e9a5503460f94b9cc83/html5/thumbnails/18.jpg)
Components
(Query-based datamining: monitor progress, compare, experiment)
(Displays for teachers to combine data to help make decisions)
(Resources for teacher development)
* Present auditory, text, graphical stimuli
Did this student improve?Which improved most?Which data set performs best?Who is teacher C?
* Measure decoding, comprehension skills
* Score, analyze, and adapt to responses
![Page 19: Assessing Reading Skills in Young Children: Assessing Reading Skills in Young Children: The TBALL Project (Technology Based Assessment of Language and.](https://reader030.fdocuments.us/reader030/viewer/2022020219/56649e9a5503460f94b9cc83/html5/thumbnails/19.jpg)
Development Process
• Task specifications
• Write items
• Teacher review
• Try with students
• Design interface
• Try interface
• Teacher review
• Displaying results
![Page 20: Assessing Reading Skills in Young Children: Assessing Reading Skills in Young Children: The TBALL Project (Technology Based Assessment of Language and.](https://reader030.fdocuments.us/reader030/viewer/2022020219/56649e9a5503460f94b9cc83/html5/thumbnails/20.jpg)
Sampling Domain
Oral Language
Decoding
Fluency
Comprehension
Core that all do, sampling of rest
Focus on high frequency items
Name letters
Say Sound of letters
Hear sound, point to letter
Rhyming, blending
Reading words, timed and not
Naming images, timed and not
Reading sentences, and pointing to image matching word
![Page 21: Assessing Reading Skills in Young Children: Assessing Reading Skills in Young Children: The TBALL Project (Technology Based Assessment of Language and.](https://reader030.fdocuments.us/reader030/viewer/2022020219/56649e9a5503460f94b9cc83/html5/thumbnails/21.jpg)
Fall Battery (K example)
Letter Names b, k, y, s, j, z + 6 randomReading (LTS) d, a, i, s, j + 5 randomSpelling (STL) p ih v iy z + 5 randomBlending z+oo, t+ub, s+ix, ch+ick,
thr+ee + 5 random
Reading Words 5 fixed, 5-35 random, hi freq words sorted by decodability
Naming Pictures As above, but with imagesRapid Naming Words and images, timed
![Page 22: Assessing Reading Skills in Young Children: Assessing Reading Skills in Young Children: The TBALL Project (Technology Based Assessment of Language and.](https://reader030.fdocuments.us/reader030/viewer/2022020219/56649e9a5503460f94b9cc83/html5/thumbnails/22.jpg)
Speech Recognition Approach
Speaker adaptation techniques
Pronunciation modeling
Noise robust (front end and/or back end)
Source and vocal tract parameter estimation
![Page 23: Assessing Reading Skills in Young Children: Assessing Reading Skills in Young Children: The TBALL Project (Technology Based Assessment of Language and.](https://reader030.fdocuments.us/reader030/viewer/2022020219/56649e9a5503460f94b9cc83/html5/thumbnails/23.jpg)
Sorting/Analyzing Data
Database design allocates a place to put the collected data and its context, e.g.,
• Demographic info from parent, date, time, type of test
• Data from test
Later the data can used for computations, e.g.,
• Words in isolation correct: 21/51 = 41%
• Words in connected text: 20/36 = 55%
• 75% of native speakers do better in connected text…
![Page 24: Assessing Reading Skills in Young Children: Assessing Reading Skills in Young Children: The TBALL Project (Technology Based Assessment of Language and.](https://reader030.fdocuments.us/reader030/viewer/2022020219/56649e9a5503460f94b9cc83/html5/thumbnails/24.jpg)
Growth Mixture Modeling can reveal unobserved heterogeneity in the model
Time Point
1 2 3 4
-3-2
-10
12
Kindergarten Growth (Five Classes)full7wtrnl.out
Time Point
5 6 7 8 9 10 11 12
-3-2
-10
12
First and Grade 2 Growth (Five Classes)full7wtrnl.out
K1
K2
K3
K4
K5
W1
W2
W3
W4
W5
Different developmental trajectories are accurately estimated
Students who are most at-risk for reading problems can be identified
Sorting Analyzing Data
Time Point
1 2 3 4
-3-2
-10
12
Kindergarten Growth (Five Classes)full7wtrnl.out
Time Point
5 6 7 8 9 10 11 12
-3-2
-10
12
First and Grade 2 Growth (Five Classes)full7wtrnl.out
K1
K2
K3
K4
K5
W1
W2
W3
W4
W5
Data from Christy Kim Boscardin
![Page 25: Assessing Reading Skills in Young Children: Assessing Reading Skills in Young Children: The TBALL Project (Technology Based Assessment of Language and.](https://reader030.fdocuments.us/reader030/viewer/2022020219/56649e9a5503460f94b9cc83/html5/thumbnails/25.jpg)
Plans
Refine assessment content and tasks based on feedback
Address validity, utility, and impact for different language groups
Pilot studies on comprehension and reading in context
Train teachers, deploy in more classrooms each year
Further evaluate and refine the ASR system
Get feedback from teachers
![Page 26: Assessing Reading Skills in Young Children: Assessing Reading Skills in Young Children: The TBALL Project (Technology Based Assessment of Language and.](https://reader030.fdocuments.us/reader030/viewer/2022020219/56649e9a5503460f94b9cc83/html5/thumbnails/26.jpg)
TBALL Specific Aims (Revisited)
Reading assessment, K-2
Helpful for teachers
Test consistently
Automate
Diagnosis
Prediction
Integrate early
Native speakers and not
SR, datamining, displays
Diverse measures
Growth mixture modeling
http://diana.icsl.ucla.edu/Tball/