1 Language, Learning, and Assessment: Improving Validity Eva L. Baker UCLA Graduate School of...
-
Upload
hillary-malone -
Category
Documents
-
view
218 -
download
2
Transcript of 1 Language, Learning, and Assessment: Improving Validity Eva L. Baker UCLA Graduate School of...
![Page 1: 1 Language, Learning, and Assessment: Improving Validity Eva L. Baker UCLA Graduate School of Education & Information Studies National Center for Research.](https://reader035.fdocuments.us/reader035/viewer/2022062717/56649e3f5503460f94b301c5/html5/thumbnails/1.jpg)
1
Language, Learning, and Assessment: Improving
Validity
Eva L. Baker
UCLA Graduate School of Education & Information StudiesNational Center for Research on Evaluation,Standards, and Student Testing (CRESST)
26th Annual Language Testing Research ColloquiumOf the International Language Testing Association
Temecula, CA
March 26, 2004
![Page 2: 1 Language, Learning, and Assessment: Improving Validity Eva L. Baker UCLA Graduate School of Education & Information Studies National Center for Research.](https://reader035.fdocuments.us/reader035/viewer/2022062717/56649e3f5503460f94b301c5/html5/thumbnails/2.jpg)
2
Today’s Focus
Language and identity—a personal note
Quick assessment policy tour
Need for cognitive model-based assessment to link learning to external outcome measures
Examples
Research agenda involving language and assessment interplay
![Page 3: 1 Language, Learning, and Assessment: Improving Validity Eva L. Baker UCLA Graduate School of Education & Information Studies National Center for Research.](https://reader035.fdocuments.us/reader035/viewer/2022062717/56649e3f5503460f94b301c5/html5/thumbnails/3.jpg)
3
Assessment Policy Tour©
1965 F
an
tasy R
ecord
s
![Page 4: 1 Language, Learning, and Assessment: Improving Validity Eva L. Baker UCLA Graduate School of Education & Information Studies National Center for Research.](https://reader035.fdocuments.us/reader035/viewer/2022062717/56649e3f5503460f94b301c5/html5/thumbnails/4.jpg)
4
Assessment Policy Tour (Cont’d)
Accountability is pushing testing practice
Soon to have greater role in postsecondary
Assessment and testing high priority in K-12, language certification, and adult and professional learning
Students’ language background a major area
![Page 5: 1 Language, Learning, and Assessment: Improving Validity Eva L. Baker UCLA Graduate School of Education & Information Studies National Center for Research.](https://reader035.fdocuments.us/reader035/viewer/2022062717/56649e3f5503460f94b301c5/html5/thumbnails/5.jpg)
5
K-12 Accountability Requirements Goals (standards) to be linked to instruction,
classroom assessments
External examinations are to be lined up with goals and instruction
Few examples of alignment, unless goals are restricted
Lack procedures and measures to document level of practice
Early growth likely not sustainable
Transfer of learning a major issue
![Page 6: 1 Language, Learning, and Assessment: Improving Validity Eva L. Baker UCLA Graduate School of Education & Information Studies National Center for Research.](https://reader035.fdocuments.us/reader035/viewer/2022062717/56649e3f5503460f94b301c5/html5/thumbnails/6.jpg)
6
Description:Extra comfort for senior dogs. Our popular orthopedic pet bed, made extra thick for aging dogs. A full 4" of medical grade convoluted foam supports bones and joints, and the elevated headrest provides proper neck and spine alignment.
http://www.petdiscounters.com/dog/beds/cu_orthopedic.html
![Page 7: 1 Language, Learning, and Assessment: Improving Validity Eva L. Baker UCLA Graduate School of Education & Information Studies National Center for Research.](https://reader035.fdocuments.us/reader035/viewer/2022062717/56649e3f5503460f94b301c5/html5/thumbnails/7.jpg)
7
http://www.powerofyoga.com/
Joyful manifestation of the heart’s desire.
![Page 8: 1 Language, Learning, and Assessment: Improving Validity Eva L. Baker UCLA Graduate School of Education & Information Studies National Center for Research.](https://reader035.fdocuments.us/reader035/viewer/2022062717/56649e3f5503460f94b301c5/html5/thumbnails/8.jpg)
8
Cognitive Model-Based Assessment
Cross-curricular cognitive families define core constructs
Embed in content, like linguistic rules in AI language understanding systems
Derives from work in writing assessment and AI systems
Evidence that it works across subject matters, types of learning, and students
![Page 9: 1 Language, Learning, and Assessment: Improving Validity Eva L. Baker UCLA Graduate School of Education & Information Studies National Center for Research.](https://reader035.fdocuments.us/reader035/viewer/2022062717/56649e3f5503460f94b301c5/html5/thumbnails/9.jpg)
9
Intellectual Capital Cognitive Families
ContentUnderstanding
ProblemSolving
Teamwork andCollaboration
MetacognitionLearning to LearnCommunication
Learning
![Page 10: 1 Language, Learning, and Assessment: Improving Validity Eva L. Baker UCLA Graduate School of Education & Information Studies National Center for Research.](https://reader035.fdocuments.us/reader035/viewer/2022062717/56649e3f5503460f94b301c5/html5/thumbnails/10.jpg)
10
Cognitive Demands Examples
Factual knowledge Definitions, propositions (e.g., statements about events,
causes, relations; “math facts”; formulas), etc. Terms and symbols (beyond arithmetic)
Identify domain by letter: A for algebra, G for geometry, etc.
Conceptual knowledge Knowledge of “more complex, organized knowledge
forms,” including classification schemes, principles and generalizations, theories, models, and knowledge structures
Procedural knowledge Knowledge about how to do something, including
mathematical skills and algorithms, techniques and methods
Knowledge Categories
![Page 11: 1 Language, Learning, and Assessment: Improving Validity Eva L. Baker UCLA Graduate School of Education & Information Studies National Center for Research.](https://reader035.fdocuments.us/reader035/viewer/2022062717/56649e3f5503460f94b301c5/html5/thumbnails/11.jpg)
11
Cognitive Demands Examples (Cont.)
Recall information
Apply information
Interpret figures, charts, or tables
Transform representations Change or move from one representation to another
(e.g., language to diagram, text to equation, graph to equation)
Multi-step reasoning Requires “students to generate an intermediate image,
construct, or sub-problem before solving the original problem”
Process Categories
![Page 12: 1 Language, Learning, and Assessment: Improving Validity Eva L. Baker UCLA Graduate School of Education & Information Studies National Center for Research.](https://reader035.fdocuments.us/reader035/viewer/2022062717/56649e3f5503460f94b301c5/html5/thumbnails/12.jpg)
12
From Science to Models to Templates
DOMAIN-INDEPENDENT
COGNITIVEPRINCIPLES
CONTENT CONTENT CONTENT
CBA
TEMPLATE TEMPLATE TEMPLATE
MODEL
SCIENTIFICFINDINGS
COGNITIVE DEMANDS
SCIENTIFICFINDINGS
SUBJECT MATTERSPECIFIC MODELS
CONDITIONSTOPICS OF TASKSSPECIFIC SCORING FEATURES
![Page 13: 1 Language, Learning, and Assessment: Improving Validity Eva L. Baker UCLA Graduate School of Education & Information Studies National Center for Research.](https://reader035.fdocuments.us/reader035/viewer/2022062717/56649e3f5503460f94b301c5/html5/thumbnails/13.jpg)
13
From Templates to Tasks
CBA
TEMPLATE TEMPLATE TEMPLATE
TASK TASKTASK
TASKTASK
TASK
TASK
TASKTASK
TASK
TASK TASK
![Page 14: 1 Language, Learning, and Assessment: Improving Validity Eva L. Baker UCLA Graduate School of Education & Information Studies National Center for Research.](https://reader035.fdocuments.us/reader035/viewer/2022062717/56649e3f5503460f94b301c5/html5/thumbnails/14.jpg)
14
Templates for Model of Content Understanding
Primary source materials in each domain
Student required to integrate prior knowledge and principles to succeed
Scored by using expert model in subject matter
![Page 15: 1 Language, Learning, and Assessment: Improving Validity Eva L. Baker UCLA Graduate School of Education & Information Studies National Center for Research.](https://reader035.fdocuments.us/reader035/viewer/2022062717/56649e3f5503460f94b301c5/html5/thumbnails/15.jpg)
15
Content UnderstandingTemplate #1 Explanation
An array of primary source materials
A prompt that asks for an explanation in context
Constructed (written) answer
Evaluated by means of a scoring rubric
![Page 16: 1 Language, Learning, and Assessment: Improving Validity Eva L. Baker UCLA Graduate School of Education & Information Studies National Center for Research.](https://reader035.fdocuments.us/reader035/viewer/2022062717/56649e3f5503460f94b301c5/html5/thumbnails/16.jpg)
16
Excerpts from U.S. HistoryPrimary Source Documents
Judge Douglas made two points upon my recent speech at Springfield. Hesays they are to be the issues of this campaign. The first one of these pointshe bases upon the language in a speech which I delivered at Springfield, whichI believe I can quote correctly from memory. I said there that “we are now farinto the fifth year since a policy was instituted for the avowed object, and withthe confident promise, of putting an end to slavery agitation; under theoperation of that policy, that agitation had not only not ceased, but hadconstantly augmented.” “I believe it will not cease until a crisis shall havebeen reached and passed. ‘A house divided against itself cannot stand.’ Ibelieve this Government cannot endure permanently, half slave and half free.” “I do not expect the Union to be dissolved”—I am quoting from my speech—“I do not expect the house to fall, but I do expect it will cease to be divided.It will become all one thing or the other. Either the opponents of slavery willarrest the spread of it and place it where the public mind shall rest, in the belief that it is in the course of ultimate extinction, or its advocates will pushit forward until it shall become alike lawful in all the States, North as well asSouth.” [text continues]
ABRAHAM LINCOLN
![Page 17: 1 Language, Learning, and Assessment: Improving Validity Eva L. Baker UCLA Graduate School of Education & Information Studies National Center for Research.](https://reader035.fdocuments.us/reader035/viewer/2022062717/56649e3f5503460f94b301c5/html5/thumbnails/17.jpg)
17
U.S. History WritingAssignment: Civil War Era
Be sure to show the relationships among your ideas and facts.
Your essay should be based on two major sources:
1. The general concepts and specific facts you know about United States history, and especially what you know about the history of the Civil War.
2. What you have learned from the readings yesterday.
Imagine that it is 1858 and you are an educated citizen living in Illinois.Because you are interested in politics and always keep yourself well-informed, you make a special trip to hear Abraham Lincoln and Stephen A.Douglas debating during their campaigns for the Senate seat representingIllinois. After the debates you return home, where your cousin asks youabout some of the problems that are facing the nation at this time.
Write an essay in which you explain the most important ideas and issuesyour cousin should understand.
![Page 18: 1 Language, Learning, and Assessment: Improving Validity Eva L. Baker UCLA Graduate School of Education & Information Studies National Center for Research.](https://reader035.fdocuments.us/reader035/viewer/2022062717/56649e3f5503460f94b301c5/html5/thumbnails/18.jpg)
18
Excerpts from Hawaiian HistoryPrimary Source Documents
LILIUOKALANI
For many years our sovereigns had welcomed the advice of American residents who had established industries on the Islands. As they becamewealthy, their greed and their love of power increased. Although settledamong us, and drawing their wealth from resources, they were alien to usin their customs and ideas, and desired above all things to secure their own personal benefit.
Kalakaua valued the commercial and industrial prosperity of his kingdomhighly. He sought honestly to secure it for every class of people, alien ornative. Kalakaua’s highest desire was to be a true sovereign, the chiefservant of a happy, prosperous, and progressive people.
And now, without any provocation on the part of the king, having maturedtheir plans in secret, the men of foreign birth rose one day en masse, calleda public meeting, and forced the king to sign a constitution of their ownpreparation, a document which deprived [him] of all power and practically took away the franchise from the Hawaiian race.
![Page 19: 1 Language, Learning, and Assessment: Improving Validity Eva L. Baker UCLA Graduate School of Education & Information Studies National Center for Research.](https://reader035.fdocuments.us/reader035/viewer/2022062717/56649e3f5503460f94b301c5/html5/thumbnails/19.jpg)
19
Content Knowledge Prompt:Hawaiian History Writing
Assignment—BayonetConstitution
Be sure to show the relationships among your ideas and facts.
Your essay should be based on two major sources:
1. The general concepts and specific facts you know about Hawaiian history, and especially what you know about the period of the Bayonet Constitution.
2. What you have learned from the readings yesterday.
Imagine you are in a class that has been studying Hawaiian history. One ofyour friends, who is a new student in the class, has missed all the classes.Recently, your class began studying the Bayonet Constitution. Your friend isvery interested in this topic and asks you to explain everything that you havelearned about it.
Write an essay explaining the most important ideas you want your friend tounderstand. Include what you have already learned in class about Hawaiianhistory, and what you have learned from the texts you have just read. Whileyou write, think about what Thurston and Liliuokalani said about the BayonetConstitution, and what is shown in the other materials.
![Page 20: 1 Language, Learning, and Assessment: Improving Validity Eva L. Baker UCLA Graduate School of Education & Information Studies National Center for Research.](https://reader035.fdocuments.us/reader035/viewer/2022062717/56649e3f5503460f94b301c5/html5/thumbnails/20.jpg)
20
Content Knowledge Prompt (Cont’d)
*From Hawaii’s Story by Hawaii’s Queen, Liliuokalani (Boston: Lee and Shepard Publishers, 1898).
It may be asked, “Why did the king give them his signature?” I answerwithout hesitation, because he had discovered traitors among his mosttrusted friends and because the conspirators were ripe for revolution, andhad taken measures to have him assassinated if he refused.
It has been known ever since that day as “The Bayonet Constitution,” and the name is well-chosen; for the cruel treatment received by the king from the military companies. [text continues]
Explain to your friend who missed class the reasons and differences for the Queen and the Senator’s approach to Hawaii’s future.
Scoring Rubric •General impression (on task)•Principles and themes•Prior knowledge•Relevant concrete examples•Avoidance of misconceptions
![Page 21: 1 Language, Learning, and Assessment: Improving Validity Eva L. Baker UCLA Graduate School of Education & Information Studies National Center for Research.](https://reader035.fdocuments.us/reader035/viewer/2022062717/56649e3f5503460f94b301c5/html5/thumbnails/21.jpg)
21
History Template
![Page 22: 1 Language, Learning, and Assessment: Improving Validity Eva L. Baker UCLA Graduate School of Education & Information Studies National Center for Research.](https://reader035.fdocuments.us/reader035/viewer/2022062717/56649e3f5503460f94b301c5/html5/thumbnails/22.jpg)
22
Benefits
Architecture for design of classroom instruction, assessments, and external tests, not multipurpose test, but multipurpose architecture
Transfer of learning
Flexibility of teaching
Vertical coherence
Specific procedures
Evidence base
![Page 23: 1 Language, Learning, and Assessment: Improving Validity Eva L. Baker UCLA Graduate School of Education & Information Studies National Center for Research.](https://reader035.fdocuments.us/reader035/viewer/2022062717/56649e3f5503460f94b301c5/html5/thumbnails/23.jpg)
23
Measuring Transfer of Learning
Not teach x, measure y: but describe variations in conditions, task content, criteria that permit students to demonstrate their flexibility
Learning to learn measurement
Metacognition and self-regulation
Time to learn new domains or tasks
Safeguard against teaching to the test as a preferred intervention
![Page 24: 1 Language, Learning, and Assessment: Improving Validity Eva L. Baker UCLA Graduate School of Education & Information Studies National Center for Research.](https://reader035.fdocuments.us/reader035/viewer/2022062717/56649e3f5503460f94b301c5/html5/thumbnails/24.jpg)
24
How Language Testing Influences Achievement Testing
Rubrics and procedures for assessing open ended performance
Linguistic analysis as accommodations
Linguistic descriptors as key task dimension
Arguments about language, content, and context
![Page 25: 1 Language, Learning, and Assessment: Improving Validity Eva L. Baker UCLA Graduate School of Education & Information Studies National Center for Research.](https://reader035.fdocuments.us/reader035/viewer/2022062717/56649e3f5503460f94b301c5/html5/thumbnails/25.jpg)
25
Toward Cognitively Oriented Constructs for English Language
Development Exams (K-12) Ironies
Expedience and science
Social and academic constructs
Measure of beginning skill or transfer
Relationships to other external exams not clear
Characterizing text Readability
Lexiles
LEARNOME
![Page 26: 1 Language, Learning, and Assessment: Improving Validity Eva L. Baker UCLA Graduate School of Education & Information Studies National Center for Research.](https://reader035.fdocuments.us/reader035/viewer/2022062717/56649e3f5503460f94b301c5/html5/thumbnails/26.jpg)
26
Towards Common Language Across Domains
Genome: “Complete set of instructions”
LEARNOME: “Complete map of features in learning domain”
![Page 27: 1 Language, Learning, and Assessment: Improving Validity Eva L. Baker UCLA Graduate School of Education & Information Studies National Center for Research.](https://reader035.fdocuments.us/reader035/viewer/2022062717/56649e3f5503460f94b301c5/html5/thumbnails/27.jpg)
27
Preliminary Exploration of the LEARNOME
Fundamental and generalizable research on identification of components
Cognitive demands, e.g., problem solving
Linguistic requirements, i.e., discourse, syntax, lexicon
Task, e.g., context, constraints
Content (ontologies)
Descriptive language—granularity
Modeling and representation of results
Empirical verification
![Page 28: 1 Language, Learning, and Assessment: Improving Validity Eva L. Baker UCLA Graduate School of Education & Information Studies National Center for Research.](https://reader035.fdocuments.us/reader035/viewer/2022062717/56649e3f5503460f94b301c5/html5/thumbnails/28.jpg)
28
Research Interactions: Language Testing and Assessment
Model-based R&D
EFF Adult Literacy
Questions about differential validity and fairness
![Page 29: 1 Language, Learning, and Assessment: Improving Validity Eva L. Baker UCLA Graduate School of Education & Information Studies National Center for Research.](https://reader035.fdocuments.us/reader035/viewer/2022062717/56649e3f5503460f94b301c5/html5/thumbnails/29.jpg)
29
EFF A
ssessm
en
t Con
sortiu
m: S
RI In
tern
atio
nal &
Cen
ter fo
r Litera
cy
Stu
die
s. (20
04
, Jan
uary
). EFF sta
nd
ard
s an
d p
erfo
rman
ce le
vel
descrip
tors fo
r: Read
ing
, writin
g, m
ath
, speakin
g, a
nd
Listen
ing
. Wash
ing
ton
, DC
: Natio
nal In
stitute
for Lite
racy
.
![Page 30: 1 Language, Learning, and Assessment: Improving Validity Eva L. Baker UCLA Graduate School of Education & Information Studies National Center for Research.](https://reader035.fdocuments.us/reader035/viewer/2022062717/56649e3f5503460f94b301c5/html5/thumbnails/30.jpg)
30
EFF A
ssessm
en
t Con
sortiu
m: S
RI In
tern
atio
nal &
Cen
ter fo
r Litera
cy
Stu
die
s. (20
04
, Jan
uary
). EFF sta
nd
ard
s an
d p
erfo
rman
ce le
vel
descrip
tors fo
r: Read
ing
, writin
g, m
ath
, speakin
g, a
nd
Listen
ing
. Wash
ing
ton
, DC
: Natio
nal In
stitute
for Lite
racy
.
![Page 31: 1 Language, Learning, and Assessment: Improving Validity Eva L. Baker UCLA Graduate School of Education & Information Studies National Center for Research.](https://reader035.fdocuments.us/reader035/viewer/2022062717/56649e3f5503460f94b301c5/html5/thumbnails/31.jpg)
31
EFF Assessment Consortium: SRI International & Center for LiteracyStudies. (2004, January). EFF standards and performance levelDescriptors for: Reading, writing, math, speaking, andListening. Washington, DC: National Institute for Literacy.
![Page 32: 1 Language, Learning, and Assessment: Improving Validity Eva L. Baker UCLA Graduate School of Education & Information Studies National Center for Research.](https://reader035.fdocuments.us/reader035/viewer/2022062717/56649e3f5503460f94b301c5/html5/thumbnails/32.jpg)
32
EFF A
ssessm
en
t Con
sortiu
m: S
RI In
tern
atio
nal &
Cen
ter fo
r Litera
cy
Stu
die
s. (20
04
, Jan
uary
). EFF sta
nd
ard
s an
d p
erfo
rman
ce le
vel
descrip
tors fo
r: Read
ing
, writin
g, m
ath
, speakin
g, a
nd
Listen
ing
. Wash
ing
ton
, DC
: Natio
nal In
stitute
for Lite
racy
.
![Page 33: 1 Language, Learning, and Assessment: Improving Validity Eva L. Baker UCLA Graduate School of Education & Information Studies National Center for Research.](https://reader035.fdocuments.us/reader035/viewer/2022062717/56649e3f5503460f94b301c5/html5/thumbnails/33.jpg)
33
EFF A
ssessm
en
t Con
sortiu
m: S
RI In
tern
atio
nal &
Cen
ter fo
r Litera
cy
Stu
die
s. (20
04
, Jan
uary
). EFF sta
nd
ard
s an
d p
erfo
rman
ce le
vel
descrip
tors fo
r: Read
ing
, writin
g, m
ath
, speakin
g, a
nd
Listen
ing
. Wash
ing
ton
, DC
: Natio
nal In
stitute
for Lite
racy
.
![Page 34: 1 Language, Learning, and Assessment: Improving Validity Eva L. Baker UCLA Graduate School of Education & Information Studies National Center for Research.](https://reader035.fdocuments.us/reader035/viewer/2022062717/56649e3f5503460f94b301c5/html5/thumbnails/34.jpg)
34
EFF A
ssessm
en
t Con
sortiu
m: S
RI In
tern
atio
nal &
Cen
ter fo
r Litera
cy
Stu
die
s. (20
04
, Jan
uary
). EFF sta
nd
ard
s an
d p
erfo
rman
ce le
vel
descrip
tors fo
r: Read
ing
, writin
g, m
ath
, speakin
g, a
nd
Listen
ing
. Wash
ing
ton
, DC
: Natio
nal In
stitute
for Lite
racy
.
![Page 35: 1 Language, Learning, and Assessment: Improving Validity Eva L. Baker UCLA Graduate School of Education & Information Studies National Center for Research.](https://reader035.fdocuments.us/reader035/viewer/2022062717/56649e3f5503460f94b301c5/html5/thumbnails/35.jpg)
35
EFF A
ssessm
en
t Con
sortiu
m: S
RI In
tern
atio
nal &
Cen
ter fo
r Litera
cy
Stu
die
s. (20
04
, Jan
uary
). EFF sta
nd
ard
s an
d p
erfo
rman
ce le
vel
descrip
tors fo
r: Read
ing
, writin
g, m
ath
, speakin
g, a
nd
Listen
ing
. Wash
ing
ton
, DC
: Natio
nal In
stitute
for Lite
racy
.
![Page 36: 1 Language, Learning, and Assessment: Improving Validity Eva L. Baker UCLA Graduate School of Education & Information Studies National Center for Research.](https://reader035.fdocuments.us/reader035/viewer/2022062717/56649e3f5503460f94b301c5/html5/thumbnails/36.jpg)
36
EFF A
ssessm
en
t Con
sortiu
m: S
RI In
tern
atio
nal &
Cen
ter fo
r Litera
cy
Stu
die
s. (20
04
, Jan
uary
). EFF sta
nd
ard
s an
d p
erfo
rman
ce le
vel
descrip
tors fo
r: Read
ing
, writin
g, m
ath
, speakin
g, a
nd
Listen
ing
. Wash
ing
ton
, DC
: Natio
nal In
stitute
for Lite
racy
.
![Page 37: 1 Language, Learning, and Assessment: Improving Validity Eva L. Baker UCLA Graduate School of Education & Information Studies National Center for Research.](https://reader035.fdocuments.us/reader035/viewer/2022062717/56649e3f5503460f94b301c5/html5/thumbnails/37.jpg)
37
EFF A
ssessm
en
t Con
sortiu
m: S
RI In
tern
atio
nal &
Cen
ter fo
r Litera
cy
Stu
die
s. (20
04
, Jan
uary
). EFF sta
nd
ard
s an
d p
erfo
rman
ce le
vel
descrip
tors fo
r: Read
ing
, writin
g, m
ath
, speakin
g, a
nd
Listen
ing
. Wash
ing
ton
, DC
: Natio
nal In
stitute
for Lite
racy
.
![Page 38: 1 Language, Learning, and Assessment: Improving Validity Eva L. Baker UCLA Graduate School of Education & Information Studies National Center for Research.](https://reader035.fdocuments.us/reader035/viewer/2022062717/56649e3f5503460f94b301c5/html5/thumbnails/38.jpg)
38
EFF A
ssessm
en
t Con
sortiu
m: S
RI In
tern
atio
nal &
Cen
ter fo
r Litera
cy
Stu
die
s. (20
04
, Jan
uary
). EFF sta
nd
ard
s an
d p
erfo
rman
ce le
vel
descrip
tors fo
r: Read
ing
, writin
g, m
ath
, speakin
g, a
nd
Listen
ing
. Wash
ing
ton
, DC
: Natio
nal In
stitute
for Lite
racy
.
![Page 39: 1 Language, Learning, and Assessment: Improving Validity Eva L. Baker UCLA Graduate School of Education & Information Studies National Center for Research.](https://reader035.fdocuments.us/reader035/viewer/2022062717/56649e3f5503460f94b301c5/html5/thumbnails/39.jpg)
39
EFF A
ssessm
en
t Con
sortiu
m: S
RI In
tern
atio
nal &
Cen
ter fo
r Litera
cy
Stu
die
s. (20
04
, Jan
uary
). EFF sta
nd
ard
s an
d p
erfo
rman
ce le
vel
descrip
tors fo
r: Read
ing
, writin
g, m
ath
, speakin
g, a
nd
Listen
ing
. Wash
ing
ton
, DC
: Natio
nal In
stitute
for Lite
racy
.
![Page 40: 1 Language, Learning, and Assessment: Improving Validity Eva L. Baker UCLA Graduate School of Education & Information Studies National Center for Research.](https://reader035.fdocuments.us/reader035/viewer/2022062717/56649e3f5503460f94b301c5/html5/thumbnails/40.jpg)
40
Research Interactions: Language Testing and Assessment (Cont’d)
Technology
TBALL project—NSF education, linguistics, electrical engineering—automated testing of ELL children
Essay scoring—towards propositional rather than regression solutions
Authoring systems with smarts
Simulation architecture and narrative structures
![Page 41: 1 Language, Learning, and Assessment: Improving Validity Eva L. Baker UCLA Graduate School of Education & Information Studies National Center for Research.](https://reader035.fdocuments.us/reader035/viewer/2022062717/56649e3f5503460f94b301c5/html5/thumbnails/41.jpg)
41
![Page 42: 1 Language, Learning, and Assessment: Improving Validity Eva L. Baker UCLA Graduate School of Education & Information Studies National Center for Research.](https://reader035.fdocuments.us/reader035/viewer/2022062717/56649e3f5503460f94b301c5/html5/thumbnails/42.jpg)
42
Research Interactions: Language Testing and Assessment (Cont’d)
Validity inferences for different groups
Comparability among tasks, paths
Alternative to norms for comparisons among groups
![Page 43: 1 Language, Learning, and Assessment: Improving Validity Eva L. Baker UCLA Graduate School of Education & Information Studies National Center for Research.](https://reader035.fdocuments.us/reader035/viewer/2022062717/56649e3f5503460f94b301c5/html5/thumbnails/43.jpg)
43
Priorities
Linking context, purposes, and evidence to an appropriate grain-sized framework
Document growth patterns associated with different instructional paths
Explore validity interpretations for ELL groups
![Page 44: 1 Language, Learning, and Assessment: Improving Validity Eva L. Baker UCLA Graduate School of Education & Information Studies National Center for Research.](https://reader035.fdocuments.us/reader035/viewer/2022062717/56649e3f5503460f94b301c5/html5/thumbnails/44.jpg)
44
CRESST Web Site
http://www.cresst.org