Calibration of Response Data Using MIRT Models with Simple and Mixed Structures Jinming Zhang...
-
Upload
christine-barker -
Category
Documents
-
view
216 -
download
0
Transcript of Calibration of Response Data Using MIRT Models with Simple and Mixed Structures Jinming Zhang...
![Page 1: Calibration of Response Data Using MIRT Models with Simple and Mixed Structures Jinming Zhang Jinming Zhang University of Illinois at Urbana-Champaign.](https://reader033.fdocuments.us/reader033/viewer/2022051417/56649ef35503460f94c05858/html5/thumbnails/1.jpg)
Calibration of Response Data Using MIRT Models
with Simple and Mixed Structures
Jinming ZhangUniversity of Illinois at Urbana-Champaign
![Page 2: Calibration of Response Data Using MIRT Models with Simple and Mixed Structures Jinming Zhang Jinming Zhang University of Illinois at Urbana-Champaign.](https://reader033.fdocuments.us/reader033/viewer/2022051417/56649ef35503460f94c05858/html5/thumbnails/2.jpg)
2
Multidimensionality
![Page 3: Calibration of Response Data Using MIRT Models with Simple and Mixed Structures Jinming Zhang Jinming Zhang University of Illinois at Urbana-Champaign.](https://reader033.fdocuments.us/reader033/viewer/2022051417/56649ef35503460f94c05858/html5/thumbnails/3.jpg)
3
Structure
• Simple structure test (SST)– items belong to one and only one dimension
• Mixed structure test (MST)– items belong to one or more dimensions
• Approximate simple structure test (ASST)– most of the items belong to one and only one
dimension with a little amount of items cross more than one dimensions
![Page 4: Calibration of Response Data Using MIRT Models with Simple and Mixed Structures Jinming Zhang Jinming Zhang University of Illinois at Urbana-Champaign.](https://reader033.fdocuments.us/reader033/viewer/2022051417/56649ef35503460f94c05858/html5/thumbnails/4.jpg)
4
Data Analysis
• Unidimensional approach(consecutive unidimensional approach)– calibrate each subtest separately– then calculate the correlations
• Unidimensional approximation approach(unidimensional approach)– treat the whole test as unidimensional
• Multidimensional approach– take into account the structure/collateral information– generate correlation/covariance as by-product
![Page 5: Calibration of Response Data Using MIRT Models with Simple and Mixed Structures Jinming Zhang Jinming Zhang University of Illinois at Urbana-Champaign.](https://reader033.fdocuments.us/reader033/viewer/2022051417/56649ef35503460f94c05858/html5/thumbnails/5.jpg)
5
M3PL
• It is a compensatory model.• M3PL for multiple-choice items and M2PL for
dichotomously scored constructed-response item.
![Page 6: Calibration of Response Data Using MIRT Models with Simple and Mixed Structures Jinming Zhang Jinming Zhang University of Illinois at Urbana-Champaign.](https://reader033.fdocuments.us/reader033/viewer/2022051417/56649ef35503460f94c05858/html5/thumbnails/6.jpg)
6
Simple Structure
• When and in other dimension, the M3PL model become two separate U3PL model.
![Page 7: Calibration of Response Data Using MIRT Models with Simple and Mixed Structures Jinming Zhang Jinming Zhang University of Illinois at Urbana-Champaign.](https://reader033.fdocuments.us/reader033/viewer/2022051417/56649ef35503460f94c05858/html5/thumbnails/7.jpg)
7
Item Parameter Estimation
• JMLE– abilities are treated as fixed unknown parameters– theoretically, same parameter estimates for unidimensional and
multidimensional approach– Estimates are not consistent
• MMLE– abilities are treated as random variable with a prior distribution– involving the correlation matrix– Did not consistent for unidimensional and multidimensional
approach
![Page 8: Calibration of Response Data Using MIRT Models with Simple and Mixed Structures Jinming Zhang Jinming Zhang University of Illinois at Urbana-Champaign.](https://reader033.fdocuments.us/reader033/viewer/2022051417/56649ef35503460f94c05858/html5/thumbnails/8.jpg)
8
Simulation Study with SS
• Purpose: The accuracy of item parameter estimates obtained from unidimensional and multidimensional approaches in MMLE method.
• Three factors:– The number of items: 30, 46 or 62– Sample size: 500, 1000, 2000, 3000, 4000, 5000– The correlation coefficient: 0, .5 or .8
• Total 54 combinations which implemented by ASSEST with 100 replications.
![Page 9: Calibration of Response Data Using MIRT Models with Simple and Mixed Structures Jinming Zhang Jinming Zhang University of Illinois at Urbana-Champaign.](https://reader033.fdocuments.us/reader033/viewer/2022051417/56649ef35503460f94c05858/html5/thumbnails/9.jpg)
9
![Page 10: Calibration of Response Data Using MIRT Models with Simple and Mixed Structures Jinming Zhang Jinming Zhang University of Illinois at Urbana-Champaign.](https://reader033.fdocuments.us/reader033/viewer/2022051417/56649ef35503460f94c05858/html5/thumbnails/10.jpg)
10
Evaluation• RMSE for each item parameter and correlations
• RMSE for item response function (IRF)
• Percentage of counts for the superior approach
![Page 11: Calibration of Response Data Using MIRT Models with Simple and Mixed Structures Jinming Zhang Jinming Zhang University of Illinois at Urbana-Champaign.](https://reader033.fdocuments.us/reader033/viewer/2022051417/56649ef35503460f94c05858/html5/thumbnails/11.jpg)
11
![Page 12: Calibration of Response Data Using MIRT Models with Simple and Mixed Structures Jinming Zhang Jinming Zhang University of Illinois at Urbana-Champaign.](https://reader033.fdocuments.us/reader033/viewer/2022051417/56649ef35503460f94c05858/html5/thumbnails/12.jpg)
12
![Page 13: Calibration of Response Data Using MIRT Models with Simple and Mixed Structures Jinming Zhang Jinming Zhang University of Illinois at Urbana-Champaign.](https://reader033.fdocuments.us/reader033/viewer/2022051417/56649ef35503460f94c05858/html5/thumbnails/13.jpg)
13
![Page 14: Calibration of Response Data Using MIRT Models with Simple and Mixed Structures Jinming Zhang Jinming Zhang University of Illinois at Urbana-Champaign.](https://reader033.fdocuments.us/reader033/viewer/2022051417/56649ef35503460f94c05858/html5/thumbnails/14.jpg)
14
![Page 15: Calibration of Response Data Using MIRT Models with Simple and Mixed Structures Jinming Zhang Jinming Zhang University of Illinois at Urbana-Champaign.](https://reader033.fdocuments.us/reader033/viewer/2022051417/56649ef35503460f94c05858/html5/thumbnails/15.jpg)
15
![Page 16: Calibration of Response Data Using MIRT Models with Simple and Mixed Structures Jinming Zhang Jinming Zhang University of Illinois at Urbana-Champaign.](https://reader033.fdocuments.us/reader033/viewer/2022051417/56649ef35503460f94c05858/html5/thumbnails/16.jpg)
16
![Page 17: Calibration of Response Data Using MIRT Models with Simple and Mixed Structures Jinming Zhang Jinming Zhang University of Illinois at Urbana-Champaign.](https://reader033.fdocuments.us/reader033/viewer/2022051417/56649ef35503460f94c05858/html5/thumbnails/17.jpg)
17
Mixed Structure
![Page 18: Calibration of Response Data Using MIRT Models with Simple and Mixed Structures Jinming Zhang Jinming Zhang University of Illinois at Urbana-Champaign.](https://reader033.fdocuments.us/reader033/viewer/2022051417/56649ef35503460f94c05858/html5/thumbnails/18.jpg)
18
Approximate simple structure
Reference composite:
Correlation coefficients between reference composite:
![Page 19: Calibration of Response Data Using MIRT Models with Simple and Mixed Structures Jinming Zhang Jinming Zhang University of Illinois at Urbana-Champaign.](https://reader033.fdocuments.us/reader033/viewer/2022051417/56649ef35503460f94c05858/html5/thumbnails/19.jpg)
19
Simulation Study with MS
• Purpose: The consequences of the violation of simple structure.
• Three factors:– The number of items: 30– Sample size: 1000, 3000, or 5000– The correlation coefficient:.8
• Total 3 combinations which implemented by ASSEST with 100 replications.
![Page 20: Calibration of Response Data Using MIRT Models with Simple and Mixed Structures Jinming Zhang Jinming Zhang University of Illinois at Urbana-Champaign.](https://reader033.fdocuments.us/reader033/viewer/2022051417/56649ef35503460f94c05858/html5/thumbnails/20.jpg)
20
![Page 21: Calibration of Response Data Using MIRT Models with Simple and Mixed Structures Jinming Zhang Jinming Zhang University of Illinois at Urbana-Champaign.](https://reader033.fdocuments.us/reader033/viewer/2022051417/56649ef35503460f94c05858/html5/thumbnails/21.jpg)
21
Conclusion and Discussion
• For SST, when MMLE is applied, when the number of items is mall, multidimensional approach is superior; otherwise, unidimensional approach is superior.
• For MT (ASST), the correlation will be overestimated while the items were mis-specified.