TABLE OF SPECIFICATIONSIN NURSING

RODOLFO B. CUDAL, BSAED, BSN, MAT, Ed.D.

Introduction

The aims of nursing education principally center on the transmission of nursing knowledge, and assisting nursing students to acquire the necessary skills and attitudes associated with nursing practice. As with professional preparation generally, nursing education encompasses the three domains of learning, the cognitive, the affective, and the psychomotor.

Clinical instructors have always been concerned with measuring and evaluating the progress of their students. As the goals of nursing education have become more complex and as the number of students has increased enormously, this task has become much more difficult.

Clinical instructors have an obligation to provide their students with the best evaluation. This implies that they must have some procedures whereby they can reliably and validity evaluate how effectively their students have been taught. The classroom achievement is one such tool.

After completing this seminar workshop the participants are expected to:

1. Prepare a table of specifications for a particular subject;2. Give the importance of a table of specifications in preparing a test;3. Apply the different principles in preparing test items;4. Manifest the skills a good item writer should possess.

I. What is a Table of Specifications?

Is a devise for describing test items in terms of the content and the process dimensions. That is, what a student is expected to know and what he or she is expected to do with the knowledge. It is described by combination of content and process in the table of specification.

A two-way chart which describes the topics to be covered by a test and the number of items or points which will be associated with each topic.

II. The purpose of a Table of Specification

1. To identify the achievement domains being measured.2. To ensure that a fair and representative sample of questions appear on the test.3. To allows the teacher to construct a test which focuses on the key areas and

weights those different areas based on their importance.4. To provides the teacher with evidence that a test has content validity, that it

covers what should be covered.

III. Designing a Table of Specifications

TOS typically are designed based on the list of course objectives, the topics covered in class, the amount of time spent on those topics, textbook chapter topics and the emphasis and space provided in the text. In some cases a great weight will be assigned to a concept that is extremely important, even if relatively little class time was spent on the topic.

IV. Three Steps are Involved in Creating a TOS

1. Choosing the measurement goals and domain to be covered.2. Breaking the domain into key or fairly independent parts – concepts, terms,

procedures, applications.3. Constructing the table.

V. How Can the Use of a TOS Benefit Your Students, including Special Need?

1. It improves the validity of teacher-made tests.2. It can improve student learning as well.

VI. How do you prepare a TOS for different types of test for a teacher made test?

You have to determine what content you want to cover and what percent of the test items should there be for each content. You need to decide what depth of understanding you want for your test items, like what percent of the questions should be synthesis etc.

TABLE 1

Sample of One Way TOS in Community Health Nursing

Topics CoveredNumber of Class

SessionsNumber of

ItemsTest Item

Distribution1. Family Health Nursing

a. Bag Techniqueb. Home Visit

2 4 1-4

2. Primary Health Care and Community Organizing

2 4 5-8

3. Community Diagnosis 2 4 9-124. Health Statistics 2 4 13-165. Public Health Programs 3 6 17-226. Communicable Diseases 4 8 23-307. Integrated Management of

Childhood Illnesses5 10 31-40

TOTAL 20 40 40

Number of Items = Number of class sessions x desired total number of itemsTotal number of class sessions

Example: Number of items for the topic “Family Health Nursing”Number of class sessions = 2Desired number of items = 40Total number of class sessions = 20

Number of Items = Number of class sessions x desired total number of itemsTotal Number of Class

= 2 x 40 20

Number of Items = 4

TABLE 2

Sample of Two Way Table of Specification forCommunity Health Nursing

Topics Covered ClassHours

Know-ledge

Compre-hension

Appli-cation

Analysis Synthesis Evaluation Total

1. Family Health Nursinga. Bag Techniquesb. Home Visits

2 1 1 1 1 4

2. Primary Health Care and Community Organizing 2 1 1 1 2 1 6

3. Community Diagnosis 24. Health Statistics 2 1 1 1 1 45. Public Health Programs 3 1 1 1 1 1 1 66. Communicable Diseases 4 1 1 2 1 1 2 87. Integrated Management of

Childhood Illnesses 5 1 1 3 4 1 2 12

TOTAL 20 6 6 8 9 3 8 40

TABLE 3

Table of Specification for NCM3Alterations in Human Functioning

BehaviorContents

Knowledgeof

SpecificFacts

Understandingof basic

concepts and principles

Skill in

Item placement

Percent of items

Observing and

describing objects

Comparing,classifying

objects based on

observable character-

istics

Making inferences

from observation

Total number of items

Disturbance in oxygenation 10 7 3 4 24 1-24 24%

Disturbance in metabolic and

endocrine functioning

17 10 7 5 3 42 25-65 42%

Disturbances in Elimination 7 14 7 4 2 34 66-100 34%

Number of Items 34 31 17 13 5 100 100

Percent of Items 34 31 17 13 5 100%

TABLE 4

Table of Specifications

Subject : Emergency Nursing (Basic Life Support)Degree Program : BSNAcademic Level : 4th YearCompetency : Demonstrate knowledge, skills and attitude necessary in the

performance of Cardio-Pulmonary

Instructional ObjectivesLearning Outcome

MotorSkills

DeclarativeKnowledge

ProceduralKnowledge

StrategicKnowledge

Attitude Total

I. Safe and Quality Nursing CareA. Demonstrates knowledge base on the

status of the individual and the emergency situation.

1. Identifies emergency situation prompting to implementation of basic life support procedure.

2. Identifies individual subject to basic life support during emergency situations.

Pencil paper test: Selection Type Multiple Choice

(5pts)

5

B. Provides sound decision making in the emergency care f the individual/ group in the emergency situation.

1. Identifies the problem2. Gathers Primary Survey data

related to the individual in the emergency situation

Pencil paper test: Selection Type Multiple Choice

(1pt)

Pencil paper test: Selection Type Multiple Choice

(3pts)

Pencil paper test: Selection Type Multiple Choice (1pt) 5

Formats:1.

TopicsCovered

Levels of LearningRemembering Understanding Applying Analyzing Evaluating Creating Total PercentTP IP TP IP TP IP TP IP TP IP TP IP

Legend: TP = Total Points IP = Item Placement

DF = PUG + PLG

2

VII. Selecting the Appropriate Item Format

Decision as to item format should be governed by such factors as: the purpose of the test the time available to prepare and score the test the number of pupils to be tested the physical facilities available for reproducing the test the skill in writing the different types of items

ITEM ANALYSIS

Item Analysis refers to the process of examining the student’s response to each item in test According to Abubakar S. Asaad and William M. Hailaya (Measurement and Evaluation Concepts & Principles) Rex Bookstore (2004 Edition), there are two characteristics of an item. These are desirable and undesirable characteristics. An item that has desirable characteristics can be retained for subsequent use and that with undesirable characteristics is either be revised or rejected.

Three criteria in determining the desirability and undesirability of an item.

a. difficulty of an itemb. discriminating power of an itemc. measures of attractiveness

Difficulty Index (DF) refers to the proportion of the number of students in the upper and lower groups who answered an item correctly. In a classroom achievement test, the desired indices of difficulty not lower, than 0.20 nor higher than 0.80. The average index of difficulty from 0.30 or 0.40 to a maximum of 0.60.

PUG = proportion of the upper group who got an item rightPLG = proportion of the lower group who got an item right

Level of Difficulty of an Item

Index Range Difficulty Level0.00 – 0.20 Very Difficult0.21 – 0.40 Difficult0.41 – 0.60 Moderately Difficult0.61 – 0.80 Easy0.81 – 1.00 Very Easy

Discrimination Index is the difference between the proportion of high performing students who got an item right. The high and low performing students usually defined as the upper 27% of the students based on the total examination score. Discrimination Index is the degree to which the item discriminates between high performing group and low performing group in relation of scores on the total test. Index of discrimination are classified into positive discrimination, negative discrimination and zero discrimination. Positive Discrimination if the proportion of students who got an item right in the upper performing group is greater than the proportion of the low performing group. Negative Discrimination if the proportion of students who got an item right in the low performing group is greater than the students in the upper performing group. And Zero Discrimination if the proportion of the students who got an item right in the upper performing group and low performing group are equal.

Discrimination Index Item Evaluation0.40 and up Very good item0.30 – 0.39 Reasonably good item but possibly subject to improvement0.20 – 0.29 Marginal item, usually needing and being subject to improvementBelow 0.19 Poor item, to be rejected or improved by revision

Maximum Discrimination is the sum of the proportion of the upper and lower groups who answered the item correctly. Possible maximum discrimination will occur if the half or less of the sum of the upper and lower groups answered an item correctly.

Discriminating Efficiency is the index of discrimination divided by the maximum discrimination.

Notations:

PUG = proportion of the upper group who got an item rightPLG = proportion of the lower group who got an item rightDi = discrimination indexDM = maximum discriminationDE = discriminating efficiency

Formula:

Di = PUG – PLG

DE = Di / DM

DM = PUG +PLG

Example: Eighty students took an examination in Algebra, 6 students in the upper group got the correct answer and 4 students in the lower group got the correct answer for item number 6. Find the Discriminating efficiency.

Given:

Number of students took the exam = 8027% of 80 = 21.6 or 22, which means that there are 22 students in the upper

performing group and 22 students in the lower performing group.

PUG = 6/22 = 27%PLG = 4/22 = 18%Di = PUG – PLG

Sample item data and the resulting indices derived from the procedures described in this chapter are presented below.

Item Group

Answers Total No. of

Correct Answers

DifficultyIndex

H-LDiscrimination

IndexA B C D

1 HL

2020

310

147

23

10

21 52.5 7 0.35

2 HL

2020

00

03

189

28

27 67.5 9 0.45

3 HL

2020

410

82

44

44

10 25.0 6 0.30

4 HL

2020

32

34

410

104

14 35.0 6 0.30

5 HL

2020

151

210

24

15

16 40.0 14 0.70

* Italicized numbers indicate correct answers Number of Students Tested = 60Using Information About Index of Difficulty

The difficulty index of a test item is important because it tells a teacher something meaningful about the comprehension of or performance on, material or task contained in an item. A closer look at the sample item data presented above reveals that item 2 is easy as its difficulty index is 67.5%. The same is true with item 1, with an estimated difficulty index of 52.5%.

The foregoing points to one thing – that the higher the value of the difficulty index, the easier is the item. This is because difficulty index represents the percentage of the total number of students answering an item correctly. Thus, there is an inverse relationship between the magnitude of the index and what it purports to represent. For an item to be considered a good item, its difficulty index should be 50%. An item with 50% difficulty index is neither easy not difficult. If an item has a difficulty index of 67.5%, this means that it is 67.5% easy and 32.5% difficult. Information on the index of difficulty of an item can help a teacher decide whether a test item should be revised, retained, or modified.

Exercises:

Analyze the summarized results of a hypothetical test item analysis. Supply the necessary details to complete the table. Decide whether to retain; reject; or modify each item. Write your answer on the column for decision a teacher should make.

Item Index ofDifficulty

Level ofDifficulty

Index ofDiscriminatio

n

Type ofItem

Decision

1 91% 0.112 48% 0.363 69% 0.234 50% 0.665 77% 0.44

Do an item analysis with the scores using the procedures cited in this chapter.

Item Group Answers*A B C D

Total No.of Correctanswers

DifficultyIndex

H-L DiscriminationIndex

1 HL

3030

810

147

410

43

2 HL

3030

06

1013

183

28

3 HL

3030

420

184

44

42

4 HL

3030

59

59

1010

102

5 HL

3030

154

310

214

102

* Italicized numbers indicate correct answers Numbers of Students Tested = 90

INSTRUCTIONAL OBJECTIVES

Instructional objectives play a very important role in the instructional process and the evaluation process. It serves as a guides for teaching and learning., communicate the intent of the instruction to others and it provide a guidelines for assessing the learning of the students. Instructional objectives also known as behavioral objectives or learning objectives are statement which clearly describe an anticipated learning outcome.

Characteristics of well-written and useful instructional objectives

1. Describe a learning outcome.2. Be student oriented-focus on the learner not on the teacher.3. Be observable or describe an observable product.4. Be sequentially appropriate.5. Be attainable within a reasonable amount of time.6. Be developmentally appropriate

BLOOMS TAXONOMY OF EDUCATIONAL OBJECTIVES

1. COGNITIVE DOMAIN call for outcomes of mental activity such as memorizing, reading problem solving, analyzing, synthesizing and drawing conclusions.

2. AFFECTIVE DOMAIN refers to a person’s awareness and internalization of objects and stimulation, it focus on emotions.

3. PSYCHOMOTOR DONMAIN it focus on the physical and kinesthetic skills of the learner. This domain is characterized by the progressive levels of behaviors form observation to mastery of physical skills.

Blooms Cognitive Taxonomy

Bloom identified six levels within the cognitive domain from simple recall of facts as the lowest level through increasingly more complex and abstract mental level, to the highest level that can be classified as evaluation. Verb samples for stating specific learning outcomes that represent intellectual activity on each level are presented here.

1. Knowledge – recognizes students’ ability to use rote memorization and recall facts. Verb samples: define, name, recognize, repeat, list, label, memorize, select, cite, reproduce, state.

2. Comprehension – involves students’ activity to read subject matter, extrapolate and interpret important information and out other ideas to their own words. Verb samples: describe, classify, explain, discuss, express, identify, translate, restate, review, give examples, interpret, summarize. Test questions should focus on the use of facts, rules and principles.

3. Application – students take new concept and apply the another situation. Verb samples: construct, arrange, compute, discover, show, relate, produce, prepare, predict, solve, dramatize, interpret.Test questions focus on applying facts or principles.

4. Analysis – students have the ability to take new information and break it down into parts to differentiate between them. Samples verbs: determine, differentiate, distinguish, estimate, point out, discriminate, categorize, compare, criticize, examine, experiment, debate. Test questions focus on separation of a whole into components and parts.

5. Synthesis – creating a pattern where one did not previously exist. Sample verbs: assemble, compose, create, formulate, plan, prepare, design, reorganize, rewrite, rearrange, propose, set up.

6. Evaluation – involves students’ ability to look at someone else’s ideas or principles and see the worth of the work and the value of the conclusion.

Sample verbs: conclude, justify, criticize, assess, judge, predict, rate, evaluate, select, choose, support, compare, argue, appraise.Rest questions focus on developing opinions, judgment or decisions.

Krathwohl’s Affective Taxonomy refers to a person’s awareness and internalization of objects and stimulation.

Anderson and Krathwohl (2001) revised the Bloom’s original taxonomy by combining both the cognitive process and knowledge dimensions. From lowest level to highest level.

1. Receiving – listens to ideasVerb samples: identify, select, give, listen to ideas

2. Responding – answers questions about ideasVerb samples: rest, select, tell, write, assist, present

3. Valuing – thinks about how to take advantage of ideas, able to explain them well Verb samples: explain, follow, initiate, justify, propose

4. Organizing - commits to using ideas, incorporate them to activity.Verb samples: prepare, follow, explain, relate, synthesize, integrate, join, generalize

5. Characterizing – incorporate ideas completely into practice, recognized by the use of them Verb samples: solve, verify, propose, modify, practice, qualify

Psychomotor Domain

This domain is characterized by the progressive levels of behaviors from observation to mastery of physical skills. From lowest to highest level.

1. Observing – active mental attending of a physical level.2. Imitating – attempted copying of a physical behavior.3. Practicing – trying a specific physical activity over and over.4. Adapting – fine tuning, making minor adjustment in the physical activity in order

to perfect it.

Factors to Consider when Constructing Good Test Items

A. VALIDITY is the degree to which the test measures what is intended to measure. It is the usefulness of the test for a given purpose. A valid test is always reliable.

B. RELIABILITY refers to the consistency of score obtained by the same person when retested using the same instrument or one that is parallel to it.

C. ADMINISTRABILITY the test should be administered uniformly to all students so that the scores obtained will not vary due to factors other than differences of the students’ knowledge and skills. There should be a clear provision for instruction for the student, proctors and even the one who will check the test or the scorer.

D. SCORRABILITY the test should be easy to score; directions for scoring is clear, provide the answer sheet and the answer key.

E. APPROPRIATENESS the test item that the teacher constructed must assess the exact performances called for in the learning objectives. The test item should require the same performance of the student as specified in the learning objectives.

F. ADEQUACY the test should contain a wide sampling of items to determine the educational outcomes or abilities so that the resulting scores are representatives of the total performance in the areas measured.

G. FAIRNESS the test should be biased to the examinees. It should not be offensive to any examinee subgroups. A test can only be good if it is also fair to all test takers.

H. OBJECTIVITY represents the agreement of two or more raters or test administrators concerning the score of a student. If the two raters who assess the same student on the judge is valid, thus lack of objectivity reduces test validity in the same way that lack reliability influence validity.

Factors affecting the Validity of a test Item

1. The test itself.2. The administration and scoring of a test.3. Personal factor influencing students’ response to the test.4. Validity is always specific to a particular group.

Ways to Reduce the Validity of the Test Item

1. Poorly constructed test items2. Unclear directions3. Ambiguous items4. Reading vocabulary too difficulty5. Complicated syntax6. Inadequate time limit7. Inappropriate level of difficulty8. Unintended clues9. Improper arrangement of items

Test Design to Improve Validity

1. What is the purpose of the test?2. How well do the instructional objectives selected for the test represent the

instructional goals?3. Which test items format will best measure achievement of each objectives?4. How many test items will be required to measure the performance adequately

on each objective?5. When and how will the test be administered?

Reliability of a Test

Reliability refers to the consistency of measurement, that is, how consistent test results or other assessment results from one measure to another. We can say that a test is reliable when it can be determined by means of Pearson Product Correlation Coefficient, Spearman-Brown Formula and Kuder-Richardson Formula .

Factors Affecting the Reliability of a Test

1. Length of the lest 2. Moderate items difficulty3. Objective scoring4. Heterogeneity of the student group5. Limited time

Four Methods of Establishing Reliability

1. Test-retest Methods. A type of reliability determined by administering the same test twice to the same group of students with any time interval between test. The results of the scores are correlated using the Pearson Product Correlation Coefficient (r) and this correlation coefficient provides a measure of reliability. This indicates how stable the test over a period of time.

2. Equivalent-Form Method. A type of reliability determined by administering two different but equivalent forms of the test (also called parallel or alternate forms) to the same group of students in close succession. The equivalent forms are constructed to the same set of specifications that is similar in content, type of items and difficulty. The result of the test scores are correlated using the Pearson Product Correlation Coefficient (r) and this correlation coefficient provides a measure of the degree to which generalization about the performance of students from one assessment to another assessment is justified. It measures the equivalence of the tests.

3. Split-Half Method. Administer test once. Score two equivalent halves of the test. To split the test into halves that are equivalent, the usual procedure is to score the even-numbered and the odd-numbered separately. This provide two score for each student the result of the test scores are correlated using the Spearman-Brown formula and this correlation coefficient provides a measure of internal consistency. It indicates the degree to which consistent results are obtained from two halves of the test.

4. Kuder-Richardson Formula. Administer the test once. Score total test and apply the Kuder-Richardson formula. The Kuder-Richardson formula is applicable only in situation where students responses are scored dichotomously and therefore is most useful with traditional test items that are scored as right or wrong. KR-20 estimates of reliability that provide information about the degree to which the items in the test measure that same characteristics, it is an assumption that all items are of equal difficulty. (A statistical procedure used to estimate coefficient alpha, a correlation coefficient is given.

Descriptive Statistics of Test Scores

Statistics play a very important role in describing the test scores of students. Teachers should have a background on the statistical techniques in order for them to analyze and describe the results of measurement obtained in their own classroom; understand the statistics used in the test and research reports; interpret the types of scores used in testing.

Descriptive Statistics – is concerned with collecting, describing and analyzing a set of data without drawing conclusions or inferences about a large group of data in terms of tables, graphs, or single number (example average score of the class in a particular test).

Inferential Statistics – is concerned whit the analysis of a subset of data leading to prediction or inferences about the entire set of data or population.

We shall discussed statistical techniques used in describing and analyzing test results.

1. Measures of Central Tendency (Averages)2. Measures of Variability (Spread of Scores)3. Measures of Relationship (Correlation)4. Skewness

Measures of Central Tendency it is a single value that is used to identify the center of the data, it is taught as the typical value in a set of scores. It tends to lie within the center if it is arranged from lowest to highest or vice versa. There are three measures of central tendency commonly used; the man, the median and mode.

REVISED BLOOM’S TAXONOMY (RBT)

Changes in terminology between two versions are perhaps the most obvious differences and can also cause the most confusion. Basically, Bloom’s six major categories were changed from noun to verb forms. Additionally, the lowest level of the original, knowledge was renamed and became remembering. Finally, comprehension and synthesis were retitled to understanding and creating. In an effort to minimize the confusion, comparison images appear below.

Caption: Terminology changes “The graphic is a representation of the NEW verbage associated with the long familiar Bloom’s Taxonomy. Note the change from Nouns to Verbs [e.g., Application to Applying] to describe the different levels of the taxonomy. Note that the top two levels are essentially exchanged from the Old to the New Version.” (Schultz, 2005) (Evaluation moved from the top to Evaluating in the second from the top, synthesis moved from second on top to the top as creating). Source: http://www.odu.edu/educ./11schult/bloomstaxonomy.htm

Old taxonomy – evaluation was the highest level, higher than synthesis. In the past, educators have accepted that critical thinking was higher than creative thinking.

Revised Taxonomy – to create which corresponds to synthesis, is considered the highest level of the cognitive process. This means that the ability to create new knowledge is now believed to pose on anyone the greatest cognitive demand.

Cognitive process, their definitions and examples

Cognitive Process Definition ExampleRemembering Retrieving, recognizing and

recalling relevant knowledge from long-term memory

Vocabulary, terms, reference, terminology, meaning(s), facts, factual information

Understanding Constructing meaning from oral, written, and graphic messages through interpreting, exemplifying, classifying, summarizing, inferring, comparing and explaining

Meaning(s) abstractions, representations, words, phrases, and compare

Applying Carrying out or using a procedure through executing or implementing

Principles, laws, conclusions, methods theories, abstractions, generalization, process and select

Analyzing Breaking material into constituent parts, determining how the parts relate to one another and to an overall structure or

Hypotheses, conclusions, assumptions, relationships, cause-effects, attribute

purpose through differentiating, organizing, and attributing

Evaluating Making judgments based on criteria and standards through differentiating, organizing and attributing

Accuracy, consistency, reliability, critique

Creating Putting elements together to

Design an appropriate sampling procedure for a given research proposal

BLOOM’S TAXONOMY

The Cognitive Process DimensionKnowledge Dimension

Remember Understand Apply Analyze Evaluate Create

Factual knowledge

List Summarize Classify Order Rank Combine

Conceptual Knowledge

Describe Interpret Experiment Explain Assess Plan

Procedural Knowledge

Tabute Predict Calculate Differentiate Conclude Compose

Meta-Cognitive Knowledge

Appropriate use

Execute Construct Achieve Action Actualize

Samples of Objective-Test Item Relationship

Knowledge LevelObjective : The student will be able to identify various types of schizophreniaTest Item : Which of the following is not a type of schizophrenia?

a. hebephrenic b. catatonic c. paranoid d. autistic

ComprehensionObjective : The student will be able to provide examples of several psychological

concepts.Test Item : Which of the following would be an instance of compliance?

a. moving away from the group norm regardless of prior opinionsb. maintaining one’s belief in the face of group pressure to change those

beliefs.c. following group norms in overt behavior but not cognitivelyd. seeking group consensus without carefully considering all possible

arguments.

Application LevelObjective : The student will be able to calculate the expected date of delivery.Test Item : If Cathy’s cycle is regular and her last menstrual period is April 20, 2010,

when is the expected date of delivery?a. January 20, 2011b. January 27, 2011c. January 13, 2011d. January 30, 2011

Analysis Level

Objective : The student will be able to make generalizations about the research results.

Test Item : Nurse Bryan has carried out a number of experiments on the bargaining process. Generalizing from his results, we can conclude that

a. The more threatening the weapons that bargainers acquire, the more likely they care to compete.

b. The more threatening the weapons that bargainers have, the more likely they are to cooperate.

c. Threatening weapons do not influence bargaining as much as saving face does.

d. We cannot accurately conclude any of the above

Evaluation LevelObjective : The student can recognize the values and points of view used in a

particular judgment of a work.Test Item : Which of the statements can correctly be made about Organization and

Management.

a. An organization (or company) is people. Values make people persons; value give vitality, meaning and direction to a company. As the people

of an organization value, so the company becomes.b. Management is the process by which administration achieves its

mission, goals and objective.c. Management effectiveness can be measured in terms of

accomplishment of the purpose of the organization while management efficiency is measured in terms of the satisfaction of individual motives.d. Management principles are universal therefore, one need not be

concerned about peoples, culture, values, traditions and human relations.