An Algorithm for Testing Unidimensionality and Clustering Items in Rasch Measurement
description
Transcript of An Algorithm for Testing Unidimensionality and Clustering Items in Rasch Measurement
![Page 1: An Algorithm for Testing Unidimensionality and Clustering Items in Rasch Measurement](https://reader035.fdocuments.us/reader035/viewer/2022062500/56815aa4550346895dc83492/html5/thumbnails/1.jpg)
AN ALGORITHM FOR TESTING UNIDIMENSIONALITY AND CLUSTERING ITEMS IN RASCH MEASUREMENT
Rudolf Debelak & Martin Arendasy
![Page 2: An Algorithm for Testing Unidimensionality and Clustering Items in Rasch Measurement](https://reader035.fdocuments.us/reader035/viewer/2022062500/56815aa4550346895dc83492/html5/thumbnails/2.jpg)
Outline
1. Aims of this study2. PCA and Parallel analysis based on
tetrachoric correlations3. The proposed algorithm
1. Procedures2. Statistical test
4. Simulation study5. Empirical study6. Discussion
![Page 3: An Algorithm for Testing Unidimensionality and Clustering Items in Rasch Measurement](https://reader035.fdocuments.us/reader035/viewer/2022062500/56815aa4550346895dc83492/html5/thumbnails/3.jpg)
Aims of Study
Clustering items: exploratory approach to identify items scales with strict criterion
Testing unidimensionality: confirmatory approach to test a unidimensional item set whether yielding a single cluster
![Page 4: An Algorithm for Testing Unidimensionality and Clustering Items in Rasch Measurement](https://reader035.fdocuments.us/reader035/viewer/2022062500/56815aa4550346895dc83492/html5/thumbnails/4.jpg)
Literature Review
Commonly used procedures to test unidimensionality: PCA and EFA Applying to binary data -> based on
tetrachoric correlations Correct number of components/factors ->
parallel analysis Cluster analysis
![Page 5: An Algorithm for Testing Unidimensionality and Clustering Items in Rasch Measurement](https://reader035.fdocuments.us/reader035/viewer/2022062500/56815aa4550346895dc83492/html5/thumbnails/5.jpg)
Cluster analysis (from Wikipedia)
Cluster analysis or clustering is the task of assigning a set of objects into groups (called clusters) so that the objects in the same cluster are more similar (in some sense or another) to each other than to those in other clusters.
Hierarchical Cluster Analysis is based on the core idea of objects being more related to nearby objects than to objects farther away.
Measure of similarity (distance)
![Page 6: An Algorithm for Testing Unidimensionality and Clustering Items in Rasch Measurement](https://reader035.fdocuments.us/reader035/viewer/2022062500/56815aa4550346895dc83492/html5/thumbnails/6.jpg)
The Basic Structure of the Procedure
Test Item Triplets
Expand Item Set
O3
A3
On
Function f
Maximum
NOT Maximum
An+1
On+1
Function f
NOT Maximum
Maximum
Ok
p less than 0.5
![Page 7: An Algorithm for Testing Unidimensionality and Clustering Items in Rasch Measurement](https://reader035.fdocuments.us/reader035/viewer/2022062500/56815aa4550346895dc83492/html5/thumbnails/7.jpg)
Assessing the Model Fit
“a function f” is a global fit statistics in this study
The test can be used to evaluate whether the set of items, as a whole, fits the model. ( Suarez-Falcon & Glas, 2003) First-order statistics (R1): violation of the
property of monotone increasing and parallel item characteristic curves
Second-order statistics (R2): violation of the assumptions of unidimensionality and local independence
![Page 8: An Algorithm for Testing Unidimensionality and Clustering Items in Rasch Measurement](https://reader035.fdocuments.us/reader035/viewer/2022062500/56815aa4550346895dc83492/html5/thumbnails/8.jpg)
R1C Statistics
R1C can be regarded as being asymptotically chi-square distribution with (k – 1)(k – 2) degrees of freedom.
![Page 9: An Algorithm for Testing Unidimensionality and Clustering Items in Rasch Measurement](https://reader035.fdocuments.us/reader035/viewer/2022062500/56815aa4550346895dc83492/html5/thumbnails/9.jpg)
Simulation Study
Aim: whether able to detect and reconstruct subsets of items that fit the Rasch model.
Two subsets of items which fit the Rasch model
Six variables were manipulated (next slide)
10,000 replications were carried out with eRm package which employed the CML estimation method.
![Page 10: An Algorithm for Testing Unidimensionality and Clustering Items in Rasch Measurement](https://reader035.fdocuments.us/reader035/viewer/2022062500/56815aa4550346895dc83492/html5/thumbnails/10.jpg)
Variables
1. The distribution of the item parameters (normal, uniform)
2. The standard deviations of the item and person parameters
3. The size of the person sample (250, 500, 1000)
4. The size of the item set (10, 30, 50)
5. The correlation between the person parameters (0.0, 0.5)
Type Person
Item
A 1.0 0.5
B 1.5 0.5
C 2.0 1.5
D 2.5 1.5
![Page 11: An Algorithm for Testing Unidimensionality and Clustering Items in Rasch Measurement](https://reader035.fdocuments.us/reader035/viewer/2022062500/56815aa4550346895dc83492/html5/thumbnails/11.jpg)
Data Analysis
The proposed algorithm The PCA based on tetrachoric
correlations with parallel analysis (95th percentile eigenvalues)
![Page 12: An Algorithm for Testing Unidimensionality and Clustering Items in Rasch Measurement](https://reader035.fdocuments.us/reader035/viewer/2022062500/56815aa4550346895dc83492/html5/thumbnails/12.jpg)
Results (Proposed Method)
![Page 13: An Algorithm for Testing Unidimensionality and Clustering Items in Rasch Measurement](https://reader035.fdocuments.us/reader035/viewer/2022062500/56815aa4550346895dc83492/html5/thumbnails/13.jpg)
Results (PCA with Parallel Analysis)
![Page 14: An Algorithm for Testing Unidimensionality and Clustering Items in Rasch Measurement](https://reader035.fdocuments.us/reader035/viewer/2022062500/56815aa4550346895dc83492/html5/thumbnails/14.jpg)
Sample Size in PCA
Sample size small than 250 (test length = 10 items) would result in large numbers of indefinite matrices of tetrachoric correlations, making the application of PCA impossible. (Parry & McArdle, 1991; Weng & Cheng, 2005)
![Page 15: An Algorithm for Testing Unidimensionality and Clustering Items in Rasch Measurement](https://reader035.fdocuments.us/reader035/viewer/2022062500/56815aa4550346895dc83492/html5/thumbnails/15.jpg)
Empirical Study
The Basic Intelligence Functions (IBF; Blum et al., 2005)
Subtests and Items: verbal intelligence functions (2; 12+15), numerical intelligence functions (2), long-term memory (1; 15), visualization (1; 13).
Between 281 and 284 persons
![Page 16: An Algorithm for Testing Unidimensionality and Clustering Items in Rasch Measurement](https://reader035.fdocuments.us/reader035/viewer/2022062500/56815aa4550346895dc83492/html5/thumbnails/16.jpg)
Data Analysis
Using Raschcon for scale-construct with the proposed algorithm.
Andersen likelihood ratios (eRm), fit statistics and PCA on residuals (Winsteps) were calculated.
PCA and parallel analysis of tetrachoric correlations were performed.
all subtests were analyzed separately
![Page 17: An Algorithm for Testing Unidimensionality and Clustering Items in Rasch Measurement](https://reader035.fdocuments.us/reader035/viewer/2022062500/56815aa4550346895dc83492/html5/thumbnails/17.jpg)
Results
Proposed algorithm: all subtests identical to the respective subtest; fit to Rasch model.
Andersen tests: fit at .01 level; 3 out of 4 subtests unfit at .05 level.
Mean square in/outfit: all ranged [1.33, 0.65]
PCA on residuals: long-term memory(2.0), others (<1.4)
PCA: long-term memory (2 components)
![Page 18: An Algorithm for Testing Unidimensionality and Clustering Items in Rasch Measurement](https://reader035.fdocuments.us/reader035/viewer/2022062500/56815aa4550346895dc83492/html5/thumbnails/18.jpg)
Discussion
A new algorithm was presented and compared with another method PCA of tetrachoric correlations.
R1c statistics: when sample size is large and correlation between latent traits was low, and better in small scales, large variances of item and person parameters.
More preferable than PCA of tetrachoric when sample size is small and scales are large.
![Page 19: An Algorithm for Testing Unidimensionality and Clustering Items in Rasch Measurement](https://reader035.fdocuments.us/reader035/viewer/2022062500/56815aa4550346895dc83492/html5/thumbnails/19.jpg)
Further Studies
1. Systematic comparison with PCA of tetrachoric
2. Involve more tests for model assumptions
3. Compare test statistics for the fit of Rasch model
4. Conduct other IRT models
For the conditions of higher correlation and small sample size, is it possible to find a cut-point (correction) to improve the use of this method?