IRT Models for Polytomous Response Data · PDF fileNominal Response Data • Nominal...
-
Upload
vuongthuan -
Category
Documents
-
view
227 -
download
1
Transcript of IRT Models for Polytomous Response Data · PDF fileNominal Response Data • Nominal...
IRT Models for Polytomous Response Data
University of KansasItem Response Theory
Stats Camp ‘07
Overview• General Theory
• Polytomous Data Types:– Graded Response– Partial Credit– Nominal Response
• Polytomous IRT Models:– Graded Response– Partial Credit– Nominal Response
General Theory of Polytomous Response Models
• Item response models for polytomous data extend the general underlying IRT premise…– Modeling item response behavior as a function of
(possibly multiple) latent traits.
• …to data types that are not only limited to binary responses.– The type of model is dependent on the type of data
that have been collected.
A Polytomous Starter Example
• Imagine you have the following data from a set of “examinees”:– The number of baskets
made by each examinee out of 10 shots from various locations on a basketball court.
Basketball Data
• The idea of the number of successfully made baskets out of a set number of attempts could possibly follow a well known distribution.– The Binomial Distribution:
( ) xnx ppxn
xXP −−⎟⎟⎠
⎞⎜⎜⎝
⎛== 1)(
Binomial, Continued…
Binomial Distribution Parameters• n is the number of trials (10 for each location in our
example).• x is the number of made baskets for each location.• p is the probability of a made basket for each trial
(assumed constant).
( ) xnx ppxn
xXP −−⎟⎟⎠
⎞⎜⎜⎝
⎛== 1)(
Linking IRT and the Data
• If we could adapt IRT for these data, here is what IRT could provide for the basketball example:– A measure of the tendency for an “examinee”
to hit shots from around the court (the examinee’s latent variable).
• Could be thought of as an examinee’s shooting ability.
– A measure of the relative difficulty of each shot location (the model item parameters).
Linking, Continued• The idea behind polytomous IRT models is to
adapt the IRT portion of the model so that the data are modeled appropriately.
• In this case, the binomial model can be adapted by using the binary IRT model response function in place of the binomial model p parameter.
• See: – Safrit, Costa, and Costa (1989)– Spray (1990)
Binomial IRT Model
• A Two-Parameter Logistic Binomial IRT model would then be:
( )
( )jij
jij
bDa
bDa
ij eeP −
−
+= θ
θ
θ1
)(
• Where:
( ) ijijij xnij
xij
ij
ijiijij PP
xn
xXP −−⎟⎟⎠
⎞⎜⎜⎝
⎛== )(1)()|( θθθ
Binomial IRT Response Function
Binomial IRF
0
0.2
0.4
0.6
0.8
1
-4-3
.4-2
.9-2
.4-1
.9-1
.4-0
.8-0
.3 0.2
0.72
1.24
1.76
2.28 2.
83.
323.
84
Theta
P(X=
x|Th
eta) P(X=0|Theta)
P(X=1|Theta)P(X=2|Theta)P(X=3|Theta)P(X=4|Theta)
Open “binomial response demo.xls” for file demo
Graded Response Models
(AKA Ordered Categorical Responses)
Graded Responses
• One of the more straight-forward polytomous data types is that of the graded-response format.
• Graded response data consists of a score that is an ordinal number, typically ranging from 0 to M.– 0, 1, 2,…, M+1
• Higher scores represent better performance on the item.
Example Graded Response ItemFrom the 2006 Illinois Standards Achievement Test (ISAT):
www.isbe.state.il.us/assessment/pdfs/Grade_5_ISAT_2006_Samples.pdf
ISAT Scoring Rubric
Additional Example Item• Cognitive items are not the only ones where graded
response data occurs.
• Likert-type questionnaires are commonly scored using ordered categorical values.– Typically, these ordered categories are treated as continuous
data.• Methods of analysis typically include Factor Analysis.
– More on this during Friday’s discussion.
• Consider the following item from the Satisfaction With Life Scale (e.g. SWLS, Diener, Emmons, Larsen, & Griffin, 1985)…
SWLS Item #1
I am satisfied with my life.1. Strongly disagree.2. Disagree.3. Slightly disagree.4. Neither agree nor disagree.5. Slightly agree.6. Agree.7. Strongly agree.
Graded Response Model
• An extension of the 2PL model for graded response data is Samejima’s Graded Response Model (1969).
• The graded response model specifies the likelihood that an examinee of a given ability will respond provide a response that receives a grade of xij (xij = 0,…,M).
Graded Response Model
• Samejima’s Graded Response Model:
( ) ( )( )( )ijxij
ijxij
ij bDa
bDa
iijijixe
exXPP −
−
+=≥= θ
θ
θθ1
|*
• Where:
( ) ( ) ( )ixixiijij ijijPPxXP θθθ *
1*| +−==
Graded Response Model Parameters
( ) ( )( )( )ijxij
ijxij
ij bDa
bDa
iijijixe
exXPP −
−
+=≥= θ
θ
θθ1
|*
• For each item j: one discrimination parameter aj.• For M-1 difficulty parameters bm.
– b1 < … < bm
• P(Xij ≥ 0) = 1.0• P(Xij ≥ M) = 0.0
Graded Response Function
Graded Response IRF
0
0.2
0.4
0.6
0.8
1
-4-3
.4-2
.9-2
.4-1
.9-1
.4-0
.8-0
.3 0.2
0.72
1.24
1.76
2.28 2.
83.
323.
84
Theta
P(X=
x|Th
eta) P(X=0|Theta)
P(X=1|Theta)P(X=2|Theta)P(X=3|Theta)P(X=4|Theta)
Open “graded response demo.xls” for file demo
Nominal Response Models
Nominal Response Data
• Nominal Response Models (e.g. Bock, 1972) are models for polytomous data where item responses are not numeric values.– Rather, item responses are in the form of nominal
categories.
• Information gained from the use of such models can be useful for detecting which distracter options are better than others in multiple choice tests.
Nominal Response Model Features
• The nominal response model is a model for the categorical responses possible within an item of a test.
• The model specifies the probability that an examinee with a given value of the latent trait selects response option m.
Example Nominal Response Item
Additional Item Types
• Non-cognitive tests can also contain differing item types that could be modeled using a Nominal Response Model.
• For example, consider an item from a questionnaire about political attitudes…
Example Nominal Response Item
Which political party would you identify yourself with?
1. Democrat2. Republican3. Independent4. Green5. Unaffiliated
Nominal Response Model
• An extension of the 2PL model for nominal response data is Bock’s Nominal Response Model (1972).
• The nominal response model specifies the likelihood that an examinee of a given ability will selection option kj of item j.
Nominal Response Model
• Bock’s Nominal Response Model:
( )( )
( )∑=
−
−
== m
h
ba
ba
iijjhijh
jkijk
e
ekXP
1
|θ
θ
θ
Nominal Response Model Parameters
( )( )
( )∑=
−
−
== m
h
ba
ba
iijjhijh
jkijk
e
ekXP
1
|θ
θ
θ
• For each level k of item j: one discrimination parameter ajk and one difficulty parameter bjk.
• No additional constraints on the parameter values.
Nominal Response FunctionOpen “nominal response demo.xls” for file demo
Nominal Response IRF
0
0.2
0.4
0.6
0.8
1
1.2
-4
-3.3
-2.7 -2
-1.4
-0.8
-0.1
0.52
1.16 1.8
2.44
3.08
3.72
Theta
P(X=
m|T
heta
)
P(X=a|Theta)P(X=b|Theta)P(X=c|Theta)P(X=d|Theta)
Conclusion
• Numerous polytomous IRT models exist.– Each extends the basic philosophy of IRT to a
a data type that is not binary.• Some differ with respect to how each model is
parameterized for the same data.• Some differ with respect to the differing types of
data to be modeled.
• Each polytomous IRT model specifies the behavior of an examinee as a function of a latent trait (often representing ability).
Next…
• Estimation of Parameter for IRT Models–Estimate person parameters
when item parameters are known
–Joint estimation of person and item parameters