Chapter 3Introduction to

Scaling

1

LEVELS OF MEASUREMENTStevens (1946) NOIR

• Variables are measured at one of these four levelsVariables are measured at one of these four levels

• Qualities of one level are characteristic of the next level upQualities of one level are characteristic of the next level up

• The more precise (higher) the level of measurement, the more The more precise (higher) the level of measurement, the more accurate is the measurement processaccurate is the measurement process

  

   

Level of Level of Measurement Measurement

DataDataFor ExampleFor Example Quality of LevelQuality of Level

RatioRatio Rachael is 5Rachael is 5’’ 10 10”” and Gregory is 5 and Gregory is 5’’ 55””

Absolute zeroAbsolute zero

IntervalInterval Rachael is 5Rachael is 5”” taller than Gregory taller than Gregory An inch is an inch is an inchAn inch is an inch is an inch

OrdinalOrdinal Rachael is taller than GregoryRachael is taller than Gregory Greater thanGreater than

NominalNominal Rachael is tall and Gregory is shortRachael is tall and Gregory is short Different fromDifferent from

2

NOMINAL SCALE DATA

Qualities Example What You Can Say

What You Can’t Say

Assignment Assignment of of labelslabels

Gender—Gender— ((male ormale or femalefemale))Preference—Preference— ((like or like or dislikedislike))Voting recordVoting record—— (for or(for or against)against)

Each Each observatioobservation belongs n belongs in its own in its own categorycategory

An An observation observation represents represents “more” or “more” or “less”“less” than than another another observationobservation

3

ORDINAL SCALE DATA

Qualities Example What You Can Say

What You Can’t Say

Assignment Assignment of values of values along some along some underlying underlying dimension dimension (order)(order)

Rank Rank in in collegecollegeOrderOrder of of finishing a finishing a racerace

One One observatioobservation is n is ranked ranked above or above or belowbelow another.another.

The amount The amount that one that one variable is variable is more or lessmore or less than anotherthan another

4

INTERVAL SCALE DATA

Qualities Example What You Can Say

What You What You Can’t SayCan’t Say

Equal Equal distances distances between between pointspointsArbitrary or Arbitrary or artificial zeroartificial zero

Number of Number of words spelled words spelled correctlycorrectlyIntelligence Intelligence testtest scores scoresTemperatureTemperature

One score One score differs differs from from another another on some on some measure measure that has that has equally equally appearing appearing intervalsintervals

The amount The amount of difference of difference is an exact is an exact representatiorepresentation of n of differences of differences of the variable the variable being studiedbeing studied

5

RATIO SCALE DATA

Qualities Example What You Can Say

What You Can’t Say

Meaningful Meaningful and and non-non-arbitrary or arbitrary or absolute absolute zerozero

AgeAgeWeightWeightTimeTime

One value ne value is is twice as twice as muchmuch as as another or another or no no quantity quantity of that of that variable variable can existcan exist

Not muchNot much!

6

• Ascending Order Ascending Order Numbers are said to be in ascending order Numbers are said to be in ascending order when they are arranged from the when they are arranged from the smallest to smallest to the largest the largest number. number. E.g. 5, 9, 13, 17 and 21 are arranged in E.g. 5, 9, 13, 17 and 21 are arranged in ascending order.ascending order.

Descending OrderDescending OrderNumbers are said to be in descending order Numbers are said to be in descending order when they are arranged from the when they are arranged from the largest to the largest to the smallest smallest number.number.

E.g. 25, 21, 17, 13 and 9 are arranged in E.g. 25, 21, 17, 13 and 9 are arranged in descending order.descending order.

7

*Transformation of Data: is about a relationship between a set of data.X and Y are a set of data. Types of Transformation:

• Nominal Nominal DataData------Isomorphic Isomorphic TransformationTransformation

• Ordinal Ordinal DataData --- ---Monotonic Monotonic TransformationTransformation

• Interval Interval DataData------Linear Linear TransformationTransformation

• Regression or Linear FormulaRegression or Linear Formula Y= bX+a Y= bX+a b=2 a=1 Y=2X+1b=2 a=1 Y=2X+1

• Ratio Ratio DataData------linear linear TransformationTransformation

• Y= bX+a Y= bX+a

• b=2 a= 0 alwaysb=2 a= 0 always

• Y=bXY=bX or or Y=cX Y=cX

8

Isomorphic Transformation• Any Any set of numbers set of numbers (DATA)(DATA)

• The numbers are usually The numbers are usually isolated, isolated, no relation between no relation between the numbers, and use the numbers, and use nominal nominal scale (data). scale (data). Ex. 6, 87, 49, 105, Ex. 6, 87, 49, 105, 21, 0, 1008, 63121, 0, 1008, 631

9

Monotonic Transformation They are rank ordered rank ordered numbers (DATA), and use ordinal ordinal scale(data) scale(data) examples; 1, 2, 3, 4, or 2, 4, 6, 8, 10

• Spearman Correlation Spearman Correlation can be used to measure the degree of degree of MONOTONIC RELATIONSHIP MONOTONIC RELATIONSHIP between two variables.

10

MONOTONIC RELATIONSHIP MONOTONIC RELATIONSHIP

• AA MONOTONIC MONOTONIC RELATIONSHIP RELATIONSHIP is a relationship that does one of is a relationship that does one of the following: the following: (1) (1) as the value of as the value of one variable increases, so does one variable increases, so does the value of the other variable; or the value of the other variable; or (2)(2) as the value of one variable as the value of one variable increases, the other variable increases, the other variable value decreases.(Ex. + or – value decreases.(Ex. + or – correlations)correlations) 11

Linear Transformation

• We use We use linear formula linear formula for for straight line straight line Y=Y=bbX+X+aa, if , if b=2b=2 and and a=1a=1 then then Y= Y=22X+X+11

This formula This formula Y=Y=22X+X+1 1 is used is used for for Interval DATA/ScaleInterval DATA/Scale. Ex. . Ex. If X=3 the Y=7If X=3 the Y=7

12

Linear Transformation• We use We use linear formula linear formula for straight line for straight line

Y=Y=bbX+aX+a

• If a=0 then;If a=0 then;

• The formula for The formula for ratio scale (data) is ratio scale (data) is Y=bXY=bX

• if if b=2, X=3b=2, X=3 and a=0 then and a=0 then Y=6Y=6

• Ex. if X=7 then Y=14Ex. if X=7 then Y=14

• Exercise 1 p.64 next slideExercise 1 p.64 next slide

13

Ranking• Score Rank position Final Rank

• 3 1 1.5

• 3 2 1.5

• 5 3 3

• 6 4 5

• 6 5 5

• 6 6 5

• 12 7 7 14

*Exercise 1 p.64*Exercise 1 p.64• 1. Identify the 1. Identify the type of transformation type of transformation that that

follows and the follows and the type of data type of data (scales of (scales of measurement)measurement) to which such transformations to which such transformations may be legitimately applied.may be legitimately applied.

• A.A. X X YY

37 37 7575

26 26 5353

54 54 109109

28 28 5757

3232 6565

15

Exercise 1 p.64 cont..

• B. B. X X YY

1212 55

1515 44

1818 33

2121 00

2424 22

1515 44

16

Exercise 1 p.64 cont..

C. C. X X YY

2222 8787

2525 102102

1919 1919

6 6 5 5

17

Exercise 1 p.64 cont..

C. C. X X YY

2222 55

2525 1919

1919 102102

6 6 87 87

18

Exercise 1 p.64 ContExercise 1 p.64 Cont• A.A. X X YY

18 18 3636

16 16 3232

55 55 110110

22 22 4444

3939 7878

19

**Scaling Approaches in Scaling Approaches in Test Test Development Development by W.S.by W.S.Torgerson Torgerson (1958)(1958)

• 1. 1. SubjectSubject-Centered Methods-Centered Methods

• 2. 2. StimulusStimulus-Centered Methods-Centered Methods

• 3. 3. ResponseResponse-Centered Methods-Centered Methods

20

1- 1- Subject-Subject-Centered MethodsCentered Methods

• We are interested in the We are interested in the subject’s score.subject’s score.

• Find the location of an Find the location of an individual’s individual’s score score on a on a quantitative continuum quantitative continuum with respect with respect to a particular psychological construct to a particular psychological construct i.e., Depression, Anxiety, Stress, IQ, etc. i.e., Depression, Anxiety, Stress, IQ, etc.

• 0______________________________1000______________________________100

• Usually uses Usually uses an an interval scale interval scale e.g., from 0 to ???e.g., from 0 to ???

21

2- Stimulus-Centered Methods

The concentration is on The concentration is on stimulistimuli

•What is stimuli? What is stimuli? Usually the items on an Usually the items on an

inventory or instrument, but it may be a physical stimuli. inventory or instrument, but it may be a physical stimuli.

•SADSAD

22

Light Therapy ResearchLight Therapy Research

• 3 groups of SAD patients 3 groups of SAD patients

• Group 1---Group 1--- 200 W light 200 W light

Group 2 ---Group 2 ---300 W light300 W light

Group 3-----Group 3-----400 W light400 W light

23

200 W 300 W 400W

24

2- Stimulus-Centered Methods

• JNDJND:: In psychophysics a In psychophysics a JJust ust NNoticeable oticeable DDifference,ifference, is the smallest detectable is the smallest detectable difference between a starting (primary) difference between a starting (primary) and secondary level of a particular and secondary level of a particular sensory stimulus. It is also known as the sensory stimulus. It is also known as the Difference Limen Difference Limen or the or the Differential Differential ThresholdThreshold. This rule was first discovered . This rule was first discovered by by Ernst Heinrich Weber

• It is called It is called the the Weber’s Law Weber’s Law

• Weber’s Law Weber’s Law is based on is based on JNDJND25

2- Stimulus-Centered Methods• Weber’s Law Weber’s Law is based on JNDis based on JND

An equal stimulus An equal stimulus (300 W (300 W light bulb) light bulb) Produces an equal Produces an equal magnitude in the participants magnitude in the participants sensory response sensory response (feeling (feeling good).good).

26

The Weber’s Law• Weber’s law, Weber’s law, also called also called Weber–Fechner lawWeber–Fechner law, ,

historically important psychological law historically important psychological law quantifying the perception of change in a quantifying the perception of change in a given stimulus. given stimulus. The law states that the The law states that the change in a stimulus that will be just change in a stimulus that will be just noticeable is a constant noticeable is a constant ratioratio of the original of the original stimulus stimulus (see the (see the ratioratio on the next slide). on the next slide).

• KEY TERMSKEY TERMS

• Threshold: Threshold: the point at which a the point at which a physiologicalphysiological or or psychologicalpsychological effect begins to be produced. effect begins to be produced.

• Intensity: Intensity: The magnetite of a quantity such as The magnetite of a quantity such as energy per unit, or the energy per unit, or the degree of strengthdegree of strength. . 27

The Weber LawThe Weber Law

28

2- Stimulus-Centered Methods• Louis L.Louis L.ThurstonThurston (1929) (1929) Law of Law of

Comparative JudgmentComparative Judgment. . EstimatesEstimates the the scale values for each stimulus scale values for each stimulus by using Z-by using Z-Score.Score.

• See p.43 problem #8 next sideSee p.43 problem #8 next side

29

2- Stimulus-Centered MethodsExercise 8 p.43

30

31

3- 3- Response-Response-CenteredCentered MethodsMethods• The concentration is on The concentration is on

subject/participant subject/participant responseresponse

• TheThe goalgoal is to locate the is to locate the stimuli and the stimuli and the individual.individual. Ex. (where do you stand on Ex. (where do you stand on an issue).an issue).

• Louis Louis GuttmanGuttman (1950) (1950) Scalogram Scalogram

• AnalysisAnalysis, see , see next slide next slide

• Coefficient of Reproducibility=CCoefficient of Reproducibility=C

• C=1- [Total #of errors/total number C=1- [Total #of errors/total number

• of responses]of responses]• See Exercise 2 p. 64See Exercise 2 p. 64

32

StatementsStrikes by Public Employees (Issue)• A.A. ClericalClerical personnel in personnel in public public

employment employment should have the should have the right to right to strike. strike.

• B.B. Public school Public school teachersteachers should have the should have the right to strike.right to strike.

• C.C. NursesNurses in state and city hospitals in state and city hospitals should have the should have the right to strike.right to strike.

• D.D. City City firefightersfirefighters should have the should have the right right to strike.to strike.

33

KeyKey, , oror Allowable Response Patterns Allowable Response Patterns to the Guttman Scale to the Guttman Scale of of Four Attitude Items Four Attitude Items on on Strikes by Public EmployeesStrikes by Public Employees

StatementsStatements

Allowable PatternsAllowable Patterns A B C DA B C D

11 + + + ++ + + +

22 + + + -+ + + -

33 + + - -+ + - -

44 + - - -+ - - -

55 - - - -- - - -

34

*P2 p64: Consider the following responses of 10 subjects to theP2 p64: Consider the following responses of 10 subjects to the four four

items items on on strikes by public employees strikes by public employees (see allowable response (see allowable response patterns).patterns).

Subject Subject Items or Statements Items or Statements

A B C DA B C D

11 + + + ++ + + +

22 + - + ++ - + +

33 + + - -+ + - -

44 + - - -+ - - -

55 + + - -+ + - -

66 - - - - - - - -

77 + - + - + - + -

88 - + + -- + + -

99 + - - - + - - -

1010 - + - -- + - - 35

*P2 p64: Consider the following responses of 10 subjects to theP2 p64: Consider the following responses of 10 subjects to the four four

items items on on strikes by public employees strikes by public employees (see allowable response (see allowable response patterns).patterns).

Subject Items or StatementsSubject Items or Statements

A B C DA B C D

11 + + + ++ + + +

22 + - + ++ - + +

33 + + - -+ + - -

44 + - - -+ - - -

55 + + - -+ + - -

66 - - - - - - - -

77 + - + - + - + -

88 - + + -- + + -

99 + - - - + - - -

1010 - + - -- + - - 36

Ex.2 p64: Consider the following responses of 10 Ex.2 p64: Consider the following responses of 10 subjects to the subjects to the four items four items on on strikes by public strikes by public employees employees (see allowable response patterns).(see allowable response patterns).

37

•THERE IS AN THERE IS AN ORDERED ORDERED SCALESCALE

38

3- 3- Response-Response-CenteredCentered MethodsMethods• The goal is to locate the stimuli and the The goal is to locate the stimuli and the

individual scale (individual scale (II scale) scale) on the on the Joint Scale Joint Scale ((JJ scale) scale)

• C. H. C. H. CoombsCoombs (1950) (1950) Unfolding TechniqueUnfolding Technique

J scale J scale and and II scale scale• See p. 56-59See p. 56-59

39

40

Response-Centered MethodsResponse-Centered Methods• Subject (J scale) Location Subject (J scale) Location Response Patterns (I scales) Response Patterns (I scales)

• Region 1 Region 1 ABCDABCD

• Region 2 Region 2 BACDBACD

• Region 3 Region 3 BCADBCAD

• Region 4 Region 4 BCDABCDA

• Region 5 Region 5 CBDACBDA

• Region 6 Region 6 CDBACDBA

• Region 7 Region 7 DCBADCBA

41

Chapter 4*Process of Test Construction

1-Identifying purposes of test scores use1-Identifying purposes of test scores use

•2-Identifying 2-Identifying behaviorsbehaviors to represent the to represent the constructconstruct

•3- Preparing test specification 3- Preparing test specification

•4- Item construction4- Item construction

•5- Item Review5- Item Review

•6- Preliminary item tryouts6- Preliminary item tryouts

•7- Field test7- Field test

•8- Statistical Analysis8- Statistical Analysis

•9- Reliability and Validity9- Reliability and Validity

•10- Guidelines10- Guidelines42

1-Identifying Purpose(s) of Test Scores Use

• For variety of reasons e.g., use For variety of reasons e.g., use depression tests to assist us depression tests to assist us with diagnosiswith diagnosis

Use SAT or GRE-Use SAT or GRE-----College College AdmissionAdmission

43

2-Identifying Behavior(s) to Represent a Construct

• Behavior(s)Behavior(s)=Symptoms or =Symptoms or characteristics of a characteristics of a disorderdisorder or a or a constructconstruct

• Ex. If the construct is depression, Ex. If the construct is depression, then behaviors are; hypersomnia, then behaviors are; hypersomnia, or hyposomnia, lack of energy, or hyposomnia, lack of energy, social withdrawal, poor social withdrawal, poor concentration, loss of appetiteconcentration, loss of appetite etc. etc.

44

• *6 Different *6 Different ActivitiesActivities for Test Developers for Test Developers

Test developers should engage in one or more Test developers should engage in one or more of these activitiesof these activities

• 1-Content Analysis1-Content Analysis

• 2-Review of Research2-Review of Research

• 3-Clinical Incidents3-Clinical Incidents

• 4-Direct Observations4-Direct Observations

• 5-Expert Judgment5-Expert Judgment

• 6-Instruction Objectives6-Instruction Objectives 45

2-Identifying Behavior(s) to Represent a Construct

*6 Different ActivitiesActivities for Test Developers

• 1-Content Analysis1-Content Analysis It is about It is about what goes in the test.what goes in the test.

• Ex. If you are creating a Ex. If you are creating a self-self-esteem test for childrenesteem test for children, you , you need to know which items need to know which items should select to measure the should select to measure the children’ self-esteem children’ self-esteem p.68 i.e., p.68 i.e., children describing themselves. children describing themselves.

46

*6 Different *6 Different ActivitiesActivities for for Test DevelopersTest Developers

• 2-Review of Research2-Review of Research Review Review different related research and different related research and use different statistical use different statistical procedures procedures (Meta Analysis, (Meta Analysis, Item Item AnalysisAnalysis, and , and Factor AnalysisFactor Analysis)) to learn about a behavior which to learn about a behavior which has been frequently studied. has been frequently studied.

47

*6 Different *6 Different ActivitiesActivities for Test for Test DevelopersDevelopers

• 3-Clinical Incidents3-Clinical Incidents Are a list of Are a list of “critical behaviors” “critical behaviors” for rating a for rating a behavior or a job performance e.g., behavior or a job performance e.g., in in job evaluation, job evaluation, # of days absent, # of days absent, being on time etc.being on time etc.

• 4-Direct Observations4-Direct Observations Identifying Identifying a behavior by direct observation.a behavior by direct observation.

48

*6 Different *6 Different ActivitiesActivities for Test for Test DevelopersDevelopers

5-Expert Judgment5-Expert Judgment Receiving Receiving input from individuals who have input from individuals who have first hand experience with the first hand experience with the construct of interest. construct of interest. Ex. If you Ex. If you want to want to evaluateevaluate the staff nurses the staff nurses in a hospital then, you need to talk to in a hospital then, you need to talk to their their supervisorssupervisors to to identify the identify the type of performance type of performance that should be that should be included in your included in your evaluation.evaluation.

49

*6 Different *6 Different ActivitiesActivities for for Test DevelopersTest Developers

• 6-Instruction Objectives6-Instruction Objectives Develop a Develop a set of instructional objectives for set of instructional objectives for your test (inventory). your test (inventory). What are the What are the objectives of your test? Find them objectives of your test? Find them and follow them.and follow them. The test developer The test developer should focus on the should focus on the specific content specific content and the tasks that examinees should and the tasks that examinees should be able to performbe able to perform. . e.g. know the e.g. know the basic basic stats stats to calculate reliability and validity in test to calculate reliability and validity in test construction exam.construction exam. 50

**Empirical Empirical CriterionCriterion Keying KeyingMMPI II and BDI II were developed on the basis of MMPI II and BDI II were developed on the basis of EECCK K

Group 1 Group 1 Criterion Groups Criterion Groups Group 2Group 2

ClinicalClinical Non-clinicalNon-clinical

The responses on these tests gathered The responses on these tests gathered from from 2 2 populations which differentiates populations which differentiates between these groups. See BDI II Test between these groups. See BDI II Test items next slide.items next slide.

51

Empirical Criterion KeyingEmpirical Criterion KeyingMMPI II and BDI II were developed on the basis MMPI II and BDI II were developed on the basis of ECK of ECK

AA. Sadness. Sadness

0 I don’t feel sad0 I don’t feel sad

11I feel sad much of the timeI feel sad much of the time

22I am sad all the timeI am sad all the time

33I am so sad or unhappy that I can’t stand itI am so sad or unhappy that I can’t stand it

BB. Loss of Energy. Loss of Energy

0 I have as much energy as ever0 I have as much energy as ever

11I have less energy that I use to haveI have less energy that I use to have

22I don’t have enough energy to do very muchI don’t have enough energy to do very much

33I don’t have enough energy to do anythingI don’t have enough energy to do anything

2s and 3s are from the Clinical Group 2s and 3s are from the Clinical Group and 0s and 1s are from nonclinical group and 0s and 1s are from nonclinical group

52

*Empirical Criterion Keying• Empirical Criterion Empirical Criterion

keyingkeying is the process of is the process of using using criterion groups criterion groups to to develop test items, develop test items, wherein the wherein the keying keying (scoring) (scoring) of items has of items has been demonstrated been demonstrated empiricallyempirically to to differentiate among differentiate among groups of test takers. groups of test takers. p. 347 53

2-Identifying Behaviors to Represent the Construct Cont..

Key Terms:Key Terms:

1. 1. CCriterion riterion RReferenced eferenced MMeasurementeasurement

2. 2. IItem tem DDomainomain

3. 3. DDomain omain SSamplingampling

These 3 terms are related These 3 terms are related 54

*C*Criterionriterion R Referencedeferenced M Measurementeasurement (CRM) (CRM)

• According to According to Glaser, glā-zər (1963) Glaser, glā-zər (1963) The term criterion The term criterion referenced measurement applies to referenced measurement applies to test scores test scores that that derive their meaning from thederive their meaning from the examinees’ levels of examinees’ levels of performance on different test items performance on different test items (See Item Analysis handouts)

• Item Domain (ID)

• ID is established From CRM.

It is a pool of questions.

• Domain Sampling

Selecting items from an Selecting items from an ItemItem Domain Domain is called is called ““Domain Domain Sampling.” Sampling.”

• First, we establish item First, we establish item domain domain from from (CRM), (CRM), then, we then, we select a select a sample of items from sample of items from this this Item Item DomainDomain.. 55

*3- Preparing Test Specification • Bloom Taxonomy (1956) see p. 73-74 used in Florida Teacher

Certification

56

• KnowledgeKnowledge: : knowledge of knowledge of domain domain (job) work(job) work, , topic, recognize, relate, recall, and repeat. topic, recognize, relate, recall, and repeat.

• Understanding/Understanding/ComprehensionComprehension: : translation, translation, interpretationinterpretation, classify, , classify, describe, discuss, describe, discuss, explain, express, identify, indicate, recognize, explain, express, identify, indicate, recognize, report, restate, review, select, translate, report, restate, review, select, translate,

• ApplicationApplication: : solving problems by using solving problems by using familiar principalsfamiliar principals, generalization, , generalization, applyapply, , demonstrate, dramatize, demonstrate, dramatize, employ, illustrate, employ, illustrate, interpret, interpret,

57

• AnalysisAnalysis: : breaking down breaking down a problem into a problem into pieces, analyze, appraise, calculate, pieces, analyze, appraise, calculate, categorize, compare, contrast, criticize, categorize, compare, contrast, criticize, differentiate, differentiate,

• SynthesisSynthesis: : combining elements combining elements into into whole, whole, Meta analysis, Meta analysis, arrange, assemble, arrange, assemble, construct, create, design, develop, construct, create, design, develop, formulate, manage, organize,formulate, manage, organize,

• EvaluationEvaluation: : Making critical judgments in Making critical judgments in terms of accuracy, appraise, terms of accuracy, appraise, assess, assess, judge, judge, predictpredict, rate, select, support, , rate, select, support, evaluate evaluate

58

59

*4- Item Construction

• *B.F. Lindquist (1936)*B.F. Lindquist (1936)• He developed a He developed a 5 step procedure 5 step procedure for test construction’s format and designing, for test construction’s format and designing,

scoring, reporting, and interpreting the results.scoring, reporting, and interpreting the results.

• 1-Selecting an appropriate 1-Selecting an appropriate item format item format (next slide)(next slide)

• 2-Verifying that the proposed format is feasible for 2-Verifying that the proposed format is feasible for the intended examineesthe intended examinees

• 3-Selecting and training the item writers3-Selecting and training the item writers

• 4-Writing the items4-Writing the items

• 5-Monitoring the progress of the item writers and 5-Monitoring the progress of the item writers and the quality of items the quality of items

• e.g. AP Psychology ETSe.g. AP Psychology ETS 60

4- Item ConstructionCont…

• Two Major different types of Item format:

• 1-Item Formats for Optimal Performance (Optimal Performance (Tests)

• 1-Alternate Choice

• 2-Multiple Choice (most effective in measuring comprehension, application, and other higher order objectives of instruction.

• 3-Matching

• 2-Item Formats for Typical Performance (Inventories)Typical Performance (Inventories)

• 1-Agree-Disagree Format by Thurston (1928), see p.78

• 2-Likert Format, see p.78

• 3-Bipolar Adjective Checklist, see p.80 See handouts61

Likert FormatLikert Format

• 1= strongly unfavorable to the 1= strongly unfavorable to the concept concept

• 2= somewhat unfavorable to the 2= somewhat unfavorable to the concept concept

• 3= undecided 3= undecided

• 4= 4= somewhat favorable to the somewhat favorable to the concept concept

• 5= strongly 5= strongly favorable to the concept favorable to the concept 62

multiple-choice(completion) itemmultiple-choice(completion) item

• The type of The type of objective item objective item that is most effective in that is most effective in measuring comprehension, measuring comprehension, application, and other higher application, and other higher order objectives of order objectives of instruction.instruction.

63

*5- Item Review

• Why should the items be reviewed before Why should the items be reviewed before they administered? they administered?

• Items should be reviewed by a panel of Items should be reviewed by a panel of experts: 6 stepsexperts: 6 steps

• 1. 1. accuracy, accuracy,

• 2. 2. relevance to test specifications, relevance to test specifications,

• 3. 3. Technical item-construction flaws Technical item-construction flaws

64

*5- Item Review

• Why should the items be reviewed before they administered?

• Items should be reviewed by a panel of experts : 6 steps

• 4. grammar,

• 5. potential for offensiveness or gender bias

• 6. readability.

65

*6- Preliminary Item Tryouts*6- Preliminary Item Tryouts

• Items should be administered to a Items should be administered to a small sample small sample of examinees of examinees (n=15-30) (n=15-30) for tryouts to check for for tryouts to check for timingtiming and and instructionsinstructions ofof the test the test

or, to identify or, to identify ambiguous or poorly ambiguous or poorly written written itemsitems

Then, make revisions.Then, make revisions.

66

*7- Field Test*7- Field Test

• Test a larger sample of Test a larger sample of participants about participants about n=200, which is a good n=200, which is a good representative of representative of population.population.

67

*8- Statistical Analysis

• Such as Item Analysis to eliminate unnecessary items in your test

68

9- 9- ReliabilityReliability and and ValidityValidity• For For consistency/dependability consistency/dependability and and accuracy/trueness accuracy/trueness of your testof your test..

69

10- Develop Guidelines10- Develop Guidelines

• For administrating, scoring, For administrating, scoring, and interpretation of your and interpretation of your test scores such as norms test scores such as norms and cutoff scoresand cutoff scores. .

70

Psychological Measurements are Psychological Measurements are about Quantifying a Behaviorabout Quantifying a Behavior

71

72

73

74

intellectually disabled

75

• Responses on a seven-category bipolar rating scale are scored

-3, -2, -1, 0, 1, 2, 3.-3, -2, -1, 0, 1, 2, 3.

76