Item Analysis & Evaluation

Item Analysis & Evaluation At the heart of instrument development is the evaluation of the items. Only with a careful evaluation of items can one develop a measure with the appropriate psychometric properties.

To conduct item level analyses your data should be entered at the item level. Be sure to assign some sort of ID to each survey so that you can refer back to an individual's responses at a later date if necessary. Also be sure to enter in variable labels and values for each of your numeric entries. In SPSS this means going to the variable view which looks like the following:

When looking at item correlations it is important to reverse score those items that are negatively phrased. This is also important when considering what your overall score on the scale represents.

In SPSS this can be accomplished by using the "Recode" subcommand under the "Transform" command. Be sure to choose "Recode into new variable" so that you do not write over your original data. Call your new variable rcq1, or something similar.

Note that you will have to tell SPSS how to recode your variables. Once you have recoded your variables be sure to use the recoded variables in all subsequent analyses.

One of the desirable qualities of an item is that it is highly correlated to other items on the scale, or subscale. This is a direct result of how we calculate reliability. However, having highly correlated items may detract from validity. Another desirable quality of the scale is that the items are highly correlated to the total score on the scale, or subscale. This is a measure of the discrimination of an item. The corrected item-scale correlation excludes the item being studied which can inflate the correlation coefficient,

The uncorrected item scale correlation includes the item being studied.

It is better to use the corrected item-scale correlation.

Yet another desirable attribute for an item is that it has relatively high variance. Consider the effect of all respondents answering an item in an identical manner.

It is also important that items have a mean close to the center of the range of the construct. If items have a mean that is near one of the extremes then they are not able to detect certain levels of the construct because the items are not worded strongly enough or are worded too strongly.

Finally, it is important to observe what would happen to your estimate of reliability if an item were deleted from the scale. An item that detracts from the reliability of your scale or subscale is probably measuring some other construct than the rest of the items on your scale. All of the item level statistics that need to be considered can be found under the "Scale" subcommand, under "Analyze" in SPSS.

Be sure to conduct your analyses at the subscale level, as well as the overall level, if you have conceptualized your scale as consisting of subscales.