Course Design: Student Assessment/Evaluation · 2020. 6. 15. · 9 Ref: Dormans JP. (2005)...

7/5/2017

1

Assessment and Evaluation of Student Learning

Constructing Examinations that Identify Learning--

1

Rebecca Porter, PT, [email protected]

Interim Dean School of Health and Rehabilitation Sciences

NOT Test Taking Skills

Course Design: Student Assessment/Evaluation

Why do we test students?

O To assess student learning—O Or at least a sample of their learning

O To promote student learningO To motivate students to studyO To communicate to students what material is

important

2continued

7/5/2017

2

Why do we test students?

O To identify areas of deficiency O To determine gradesO To determine the effectiveness of our

teachingO To identify areas where

course/curriculum is weak

3

Do tests promote learning?

4

Kar

pick

eJD

, Blu

nt J

R. R

etrie

val P

ract

ice

Pro

duce

s M

ore

Lear

ning

than

E

labo

rativ

e S

tudy

ing

with

Con

cept

Map

ping

. Sci

ence

201

1 Ja

n 20

. [E

pub

ahea

d of

prin

t]

7/5/2017

3

Where should test items come from?

O The items should be derived from/linked to:O Course session (lecture/lab/learning experience) objectives

which are linked to—

O Course objectives which are linked to—

O CAPTE standards // NPTE content outline

O APTA Minimum Required Skills of PT(A) Graduates at Entry Level // FSBPT Practice Analysis

O Program mission statement

5Do you see the connections with what we have talked about in previous sections?

How do you decide what type of exam format to use?

O Number of studentsO Learning objectives being tested

O Describe, explain O Synthesize, interpretO Perform

O Formative or Summative evaluationO Other

6

7/5/2017

4

What should be tested?O Exam content should match course objectives

O Table of specifications

7

What should be tested?O Exam content should match course objectives

O Important topics weighted more heavily (more points)

O Testing time devoted to topic should reflect relative importance of topic

8

7/5/2017

5

Questions that ask ‘Why’ O Which of the following statements is the

BEST explanation of why tumbling is contraindicated for children with Down syndrome?

a. Down syndrome is associated with atlantoaxialinstability.

b. Down syndrome is associated with hip dysplasia.

c. Tumbling requires too much coordination for a child with Down syndrome.

d. Tumbling may lead to gastroesophageal reflux in children with Down syndrome. 9

Ref: Dormans JP. (2005) Pediatric Orthopaedics. 1st Ed. Elsevier Mosby. 287-288

O When assessing exercise intensity in an individual who is taking a beta-blocker, why would a rating of perceived exertion be a better method than changes in heart rate?

a. Baseline heart rate may be depressed.b. Baseline heart rate may be elevated.c. Heart rate response to exercise may be

increased.d. Heart rate response to exercise may be

reduced.10

7/5/2017

6

Questions that predict outcomes/consequences

O A patient with ____ dysfunction is treated with _____ intervention. Which of the following outcomes is MOST likely to occur?

a. xxxxxb. xxxxxc. xxxxxd. xxxxx

Question could be constructed either based on an appropriate matching of the dysfunction and intervention or an incorrect selection of the intervention.

11

Questions that require assessment of info in the stem

O Description of a patient. Which of the following statements is the BEST assessment of the information gained from these observations?

a. xxxxxb. xxxxxc. xxxxxd. xxxxx

12

7/5/2017

7

O Select items that permit you to make inferences beyond items on the exam

O Consider relative importance of the information being tested

O Selection of sample determines reproducibility and validityO To be reproducible, need to sample broadly

13

Avoid ambiguity based on which aspect of the question is the focus of the test taker. O Example

O A patient had a stroke. Which of the following approaches is the BEST method of practice? O If focus on principles of practice, the

answer could be random practice. O If focus on the patient and decide that the

patient recently had a stroke, the answer could be blocked practice. 14

7/5/2017

8

Advice from the Web

OIf opposite answers are given as choices, one of them is often the right answer.

O http://712educators.about.com/cs/assessment/qt/mctips.htm

15

Advice from the Web

O Law of Minimum VariationO ‘Typically, if you have 4 choices for a

multiple choice question, there will be one answer that is way out. That answer is wrong. The three remaining answers will be somewhat similar. However, contextually, one of them will be a "median" of the other two. That is the correct answer typically.’

http://nerdparadise.com/humor/xe/testtakingtips/ (Page no longer available)

16

7/5/2017

9

Advice students receive on strategies to answer difficult questions

O Give each option of a question the "true-false test”O This may reduce your selection to the best answerO What is the issue with T/F options?

O Question options that grammatically don't fit with the stem

O Question options that are totally unfamiliar O Question options that contain negative or

absolute words


O If the answers are numbers O Toss out the high and low and consider the

middle range numbersO Look alike options

O Probably one is correct; choose the best but eliminate choices that mean basically the same thing, and thus cancel each other out

Adapted from http://www.studygs.net/tsttak3.htm

7/5/2017

10


O Double negativesO Create the equivalent positive statement

O Echo optionsO If two options are opposite each other,

chances are one of them is correct O Carefully consider options that contain

qualifiers

Adapted from http://www.studygs.net/tsttak3.htm

O Parsimony-‘Which of the following signs will MOST likely NOT be present in respiratory alkalosis?’ or ‘Which of the following signs will MOST likely to be present in a patient who has respiratory alkalosis?’

O Answers are a mixture of signs and symptoms while question asks for a clinical sign.

O The sensation of dizziness could be experienced during an anxiety attack.

20

A physical therapist (assistant) is caring for an adult that has recently been diagnosed with respiratory alkalosis. Which of the following clinical signs would most likely not be present?

A: Anxiety attacksB: Blurred vision C: DizzinessD: Hyperventilation cyanosis

7/5/2017

11

A male patient reports the development of continuous urine flow. Referral to a physician is MOST appropriate if the patient has a history of which of the following conditions?

a. Prostate surgeryb. Hip replacement surgeryc. Diabetes mellitusd. Aortic aneurysm

The problem is that the patient should be referred regardless of the prior history.

21

2. A patient with right hemiplegia reports right shoulder pain with movement. The humeral head is displaced inferiorly. Which of the following modalities would be MOST appropriate? a. Direct Current Stimulation

The wording change makes all 4 options into types of stimulation. b. Functional Electrical Stimulation

FES is directed toward reducing the displacement which is the underlying source of the pain.

c. Interferential Current Stimulation ICS would not address the underlying displacement.

d. Transcutaneous Electrical Nerve Stimulation While TENS does treat pain, it does not address the underlying problem of the displacement of the humeral head.

22

REVISED

7/5/2017

12

3. A (adolescent female) patient reports knee pain following a twisting injury with the foot planted. The patient’s knee is swollen and the patient is unable to jump or run. Which of the following tests is MOST appropriate?a. Lachman’s test (for anterior cruciate ligament) b. Posterior drawer test (for posterior cruciate ligament)c. Active Lachman’s test (for anterior cruciate ligament) d. External Rotation Recurvatum Test (for posterior cruciate

ligament)

O Correct answer is 1. Note that 1 and 3 are similar which makes it more likely that it is one of these 2 options.

O Is there enough information in the stem to answer the question? Does the sport need to be included? Is the information in the parentheses necessary?

O Do the ligaments need to be specified in the options?

23

4. In preparation for a body building contest, a patient has been working on muscle definition of the upper arm. The patient has been performing multiple repetitions of elbow flexion with supination using a heavy weight. Which of the following structures is being exercised?a. Biceps brachiib. Brachioradialisc. Brachialisd. Bicepital tendonThe scenario is window dressing.Which of the following structures performs elbow

flexion with supination?Sometimes there are only 3 logical choices. A

throwaway option adds nothing.

24

7/5/2017

13

O OLD--The angiospastic disorder that affects the fingers is called:

5. Which of the following conditions is an angiospastic disorder affecting the fingers?

a. Endarteritis obliteransb. Raynaud diseasec. Thromboangiitis obliteransd. Thrombophlebitis

O Options organized alphabeticallyO If you use italics to indicate the key, it is easy to

remove when you generate the test form 25

6. A patient reports jaw pain when awakening and difficulty opening the mouth until the jaw muscles loosen up in the morning. Headaches in the morning are frequent. Which of the following actions is the MOST appropriate next step (for the physical therapist)?a. Ask the patient to describe the pain. b. Instruct the patient in proper jaw mechanics.c. Measure the patient’s jaw opening.d. Refer the patient to a dentist.

26

Revised

7/5/2017

14

7. Which of the following approaches is MOST appropriate to provide directions to a patient with global aphasia?a. Ask the patient to repeat the instructions.

(‘Avoid using verbal instructions’--as originally written, it is not mutually exclusive with other options)

b. Speak loudly and slowly.c. Use visual and tactile cues.d. Write down the instructions.

27

Revised

One possibility

O A patient reports problems with getting to the toilet in time after experiencing the need to urinate. Which of the following interventions would be MOST appropriate (to address the underlying urinary problem)?

O What level is this question?O Need to know what type of urinary

incontinence is describedO Need to know the appropriate intervention

for that type of incontinence

28

7/5/2017

15

Cluster Multiple Choice items Scenario presented which is followed

by several multiple choice items Construct carefully to avoid ◦ Cueing◦ Hinging

This is the format used by the Practice Review Tool◦ https://www.fsbpt.org/OurServices/LicenseeServices

/PracticeReviewTool(PRT).aspx

29

Cluster Multiple Choice Item Example

O An unresponsive individual was admitted to the intensive care unit after collapsing. The family reports the patient felt well in the morning, but developed a headache that progressively worsened during the day. The patient has a history of hypertension and artrial fibrilliation. Medications are an oral anticoagulant and an antihypertensive drug. Blood pressure is 220/130 mm Hg. Respiratory pattern is apnea alternating with hyperpnea. The patient’s only response is to noxious stimuli which results in extensor posturing involving the right arm and leg. Papilledema is present on the left. Pupils are 3.0/7.0 (R/L) with no reaction to light on the left. Left gaze preference is present. There is diffuse hyperreflexia (R>L). Babinski’s sign is present bilaterally.

30

7/5/2017

16

item 1O The dilated, unreactive left pupil is

MOST consistent with injury to which of the following structures on the left?

a. Lateral genigulate nucleusb. Oculomotor nerve c. Optic nerved. Optic tracte. Superior colliculus

31

item 2O The extensor posturing on the right is

MOST consistent with injury to which of the following structures on the left?

a. Diencephalonb. Medullac. Midbraind. Ponse. Telencephalon

32

7/5/2017

17

item 3O Which of the following terms BEST

describes the patient’s respiratory pattern?

a. Apneusticb. Ataxicc. Central neurogenic hyperventilationd. Cheyne-Stokese. Normal

33

item 4O Which of the following herniation

syndromes is MOST consistent with the clinical presentation?

a. Brainstem through the tentorial notchb. Cerebellar tonsils through the foramen

magnumc. Cingulate gyrus beneath the falxd. Diencephalon through the tentorial notche. Temporal lobe uncus across the tentorium

34Adapted from http://www.nbme.org/PDF/2001iwg.pdf

7/5/2017

18

Scenario 1Gender: MaleAge: 77

Presenting Problem/Current Condition: Patient reports intermittent and variable lower extremity pain starting at the

buttocks and the posterior calf (pain in the right side greater than in left). The lower extremity pain began gradually, and patient denies any incident.

Past Medical History: Diagnosis of chronic pulmonary disease (emphysema) in 2001. Patient stopped smoking in 2001 after 50 years of smoking cigarettes (60

pack-years). Mild heart attack in 1999 with subsequent revascularization Back surgery in 1975; L4-L5 segments were fused. Currently taking medications for hypertension and high cholesterol.

Other Information: Patient is a retired auto mechanic. Married Lives in two-story home with bedroom and bath on first floor.

Physical Therapy Examination: No pain while sitting or lying down (0/10). Pain worsens while walking (7/10).

Which of the following medications would MOST likely be prescribed to the patient for the lower extremity symptoms?

A. Cilostazol (Pletal)B. Naprosyn (Naproxen)C. Atenolol (Tenormin)D. Isosorbide mononitrate

(Imdur)








Which of the following tests and measures would be BEST to perform to assess the patient’s lower extremity vascular status?

A. Ankle-brachial indexB. Blood pressureC. Auscultation for abnormal

heart soundsD. Capillary nail refill test

7/5/2017

19








Which of the following results from measurement of the ankle-brachial index would explain the patient’s lower extremity symptoms?

A. 0.70B. 0.95C. 1.00D. 1.20


Presenting Problem/Current Condition: Patient reports intermittent and variable lower extremity pain starting at

the buttocks and the posterior calf (pain in the right side greater than in left).

The lower extremity pain began gradually, and patient denies any incident.

Past Medical History: Diagnosis of chronic pulmonary disease (emphysema) in 2001. Patient stopped smoking in 2001 after 50 years of smoking cigarettes

(60 pack-years). Mild heart attack in 1999 with subsequent revascularization Back surgery in 1975; L4-L5 segments were fused. Currently taking medications for hypertension and high cholesterol.



Positive results from which of the following tests performed during an examination would support a positive result from the ankle-brachial index test for the patient?

A. Rubor of dependencyB. Mediate percussion of the

chestC. Jugular vein distentionD. Pulse oximetry

7/5/2017

20


Presenting Problem/Current Condition: Patient reports intermittent and variable lower extremity pain starting at

the buttocks and the posterior calf (pain in the right side greater than in left).

The lower extremity pain began gradually, and patient denies any incident.





Which of the following diagnoses would BEST describe the patient’s symptoms after obtaining positive results from rubor of dependency and ankle-brachial index tests?

A. Peripheral arterial diseaseB. Spinal stenosisC. Piriformis syndromeD. Venous insufficiency








Which of the following interventions with the goal of decreasing symptoms would be BEST to initiate for the patient’s report of lower extremity pain?

A. Treadmill walkingB. Prone press-upsC. Water aerobicsD. Resistive Training

7/5/2017

21

Cluster Multiple Choice Items

O See additional examples in the Candidate Handbook for the Canadian Physiotherapist Competency ExaminationO http://www.alliancept.org/wp-

content/uploads/2016/08/CAPR_PCE-EssentialCandidateGuide_EN.pdf

41

Canadian Physiotherapist Competency Examination

O An 84-year-old man, who resides in a long term care facility and who ambulates for short distances with supervision, falls one evening. A staff member notes that the resident’s hip is in external rotation and that there is significant bruising around the hip. Investigation reveals a fracture of the neck of the right femur, and a hip arthroplasty is performed via posterior approach.

42

7/5/2017

22

1. Following the surgery, the physiotherapist is asked to assist the staff on the ward to sit the client on the edge of the bed. Which of the following progressions should the physiotherapist use?

2. Which symptom should the physiotherapist expect when this client sits on the edge of the bed for the first time?

3. While the client remains on bed rest following surgery, which of the following activities would he be encouraged to do on his own?

43

4. The physiotherapist wishes to ensure that the client slides his right thigh forward in the chair when attempting to move from sitting to standing in the parallel bars. Which of the following rationales should the physiotherapist give for this precaution?

5. Four days post-surgery, the client reports right calf pain. What would be the most likely cause of this pain?

6. Upon discharge to the long term care facility, what equipment would the physiotherapist most likely recommend for this client?

44

7/5/2017

23

‘Cloning’

O After you write a question and while you still have the source material in front of you, consider what other questions can be written on the same or related concept.

45

Strategy‐Flip Stem Info into Options or Key into the Stem

O Patient scenario with a specified condition/disease/impairment. Which of the following interventions is most appropriate?

O Reverse thought process by describing the intervention and asking which disease/condition/impairment the intervention would address.

46

7/5/2017

24

Strategy—Swap Out Info in StemO Patient scenario with a specified

condition/disease/impairment. Which of the following interventions is most appropriate?

O What other related condition/disease/ impairment could be substituted for the condition/disease/impairment in the original question

47

Strategy—Expand Questions from One Scenario

O Patient scenario with specified condition/disease/ impairment—O Question on examination/data collectionO Question on the condition or diseaseO Question on differential diagnosis or prognosis O Question on intervention/ interventionO Question on assistive deviceO Question on modalityO Etc.

48

7/5/2017

25

Strategy—New Correct Answer

O Patient scenario which indicates that the patient requires strengthening exercises Which of the following exercises is MOST appropriate?O Example of a strengthening exerciseO Example of an endurance exerciseO Example of a flexibility exerciseO Example of a stability exercise

O Change a component in the stem so that the next iteration requires one of the other options.

49

Strategy—Change Item FormatO After developing the 4 questions in the prior

strategy --O ImpairmentstrengtheningO ImpairmentenduranceO ImpairmentflexibilityO Impairmentstability

O Develop a matching questionO With matching, always have one more option than

is needed to complete the match

50

7/5/2017

26

Is this a visual item?O The test

demonstrated in the photograph is utilized to assess which of the following structures?

51

http://www.medceu.com/tests/anklesprain.htm

Visual ItemsO Advantages

O Takes fewer words to present the information in the stem

O May be easier for the test taker to understand/interpret

O Can increase the validity of the examinationO May be more discriminatingO Increases the scope and range of questionsO Can use single graphic for multiple

questions

52

7/5/2017

27

Visual ItemsO Disadvantages

O Difficult in obtaining good imagesO Need for obtaining permission to photographO Images from textbooks may be subject to

imprintingO Essential characteristics

O Large enough to seeO Contain information needed to answer the

questionO Sufficient clarity to identify necessary

information

53

4.While abducting the shoulder, the patient in the photograph denies pain while moving the arm through the range indicated in photograph A, but reports increasing pain severity as the arm moves into the range indicated by photograph B. Which of the following disorders is MOSTlikely present?

a. Acromioclavicular joint lesion b. Subacromial bursitis c. Infraspinatus tendinopathy d. Partial tear of the supraspinatus

54

https://www.fsbpt.org/ForCandidatesAndLicensees/NPTE/CandidateHandbook/SampleQuestions/

7/5/2017

28

Student Reports of Cheating

Eighty-four percent of students at a public research university believe students who cheat should be punished, yet two of every three admit to having cheated themselves.

Most of the cheating students admit to involves homework, not tests, and they see academic misconduct applying differently to those two kinds of work.

Read more at: http://www.insidehighered.com/news/2012/03/16/arizona-survey-examines-student-cheating-faculty-responses#ixzz1pWCodMYLInside Higher Ed

55

Incidence of CheatingO About 6 percent of students who took an online final

exam proctored by Examity last fall broke the rules…O 3/30/16 Inside Higher Ed

O A Wall Street Journal analysis of data from more than a dozen large U.S. public universities. One case [of cheating] for every 100 domestic undergraduate students.O Colleges See More Cheating With Foreign Students , June 6, 2016, Inside Higher Ed

O Ohio State University found 85 students in the veterinary medicine college guilty of cheating, The alleged cheating involved sharing answers in online take-home exams for which group work was not permitted. O June 8, 2016 Inside Higher Ed 56

7/5/2017

29

Incidence of CheatingO Canadian Alliance of Physiotherapy Regulators found an

individual who was sharing exam questions on the internet. A 5 year ban on retesting was imposed. 5/4/17

O A student at the University of Kentucky climbed through ceiling ducts and dropped down into a faculty member's office to steal an exam. Earlier in the semester, he had succeeded in doing so for another test. This time, the student again reached the office and unlocked it to let in another student. But early Wednesday morning, while the students were still in the office, the faculty member returned. He had been working late and stepped out to get something to eat when the students broke into his office. The students fled, but then returned and confessed. https://www.insidehighered.com/quicktakes/2017/05/04/student-drops-ceiling-steal-exam

57

ANALYZING TEST items

58

7/5/2017

30

Approaches to Analysis

OQualitativeO Questions during the examO Questions written on test bookletO Questions raised after the exam

OQuantitative

59

Components of a Typical Computer Scored Reports

60

Individual Item Information

7/5/2017

31

Typical Computer Scored Reports

O Item DiscriminationO Biserial or Point Biserial Correlation

Discrimination Index O The ability of an item to differentiate among

students on the basis of how well they know the material being tested

O An estimate of the degree to which an individual item is measuring the same thing as the rest of the itemsO A high scoring test taker should do well

selecting the key (+ pbs or pbr) and a low scoring test taker should not (- pbs or pbr)

61

Typical Computer Scored Reports

Item Discrimination (point biserial) Simplified

O Correct answer should have a (high) positive correlation

O Distractors should have a (high) negative correlation

62

7/5/2017

32

9. How would you explain the negative point biserial for this question?

O Difficulty pt biseriala. 0.018 0.009b. 0.000 ---c. 0.000 ---d. *0.982 -0.009

O Negative pt biserial is due to the cluster of high and low performers on the one answer. Only 1 person chose ‘a’.

63

10. What is your interpretation of this item analysis information?

O Difficulty pt biseriala. 0.218 -0.214b. 0.200 -0.076c. 0.273 0.042d. *0.309 0.216

O A very hard question that discriminates well. How will your students react?

64

7/5/2017

33

65

A. This option is very attractive to poorly performing students. 50% of the low performers selected the option and the pbr is strongly negative

B. This option was attractive to high performers (46%). The pbr is positive which is concerning and the highest percent of students selected this option. This option should be looked at closely to be sure it is clearly wrong.

C. The pbr is strong even though only 25% of students selected the key.D. Only 2 individuals selected the option and they were

high performers which resulted in the just barely positive pbr.

Overall this was a very hard question which may have 2 answers that are correct or there is confusion in the class on the underlying concept.

12. Based the review of the item analysis, the decision was made to keep the question. What do you think?

Difficulty pt biseriala. 0.218 0.286b. 0.018 -0.285c. 0.200 -0.248d. * 0.564 0.039O High performers selected ‘a’ resulting in the high

pbr. Looking at the question and determining why high performers selected ‘a’ is warranted.

66

7/5/2017

34

Difficulty Pt BiserialA 0.018 0.009B 0.000 ‐‐C 0.000 ‐‐D * 0.982 ‐0.009Omitted 0.000 ‐‐‐

67

13. Would you keep the following question? Why?

The negative pbr occurs because only one person selected the incorrect answer.

Difficulty Pt BiserialA* 0.164 0.004B 0.764 0.054C 0.036 0.066D 0.036 ‐0.197Omit 0.000 ‐‐‐

68

14. Your colleague asks your advice about dropping this item, but giving one point for the students who selected ‘a’. What do you think?

The stats indicated that B was selected by more students and some of the high performers. Overall, this question has problems and it is questionable if ‘a’ is clearly the correct answer. Was the information covered in the course? What is the source of the confusion?

7/5/2017

35

Item Analysis Report

No.Correct Group Response Response Frequency

TotalUpper 27%

Lower 27%

Point Biserial A B C D E

15 94.74% 100% 80% 0.48 0 0 *18 1 0

69

15. If an individual has uncontrolled hypertension, total peripheral resistance is likely to ______ and cardiac output will ______.

a. decrease; decreaseb. decrease; increasec. increase; increased. increase; decrease

70

Note:•No one selected A or B•One person selected D

A B *C D E

Pt-Biserial 0.00 0.00 0.48 -0.48 0.00

Frequency % 0.00 0.00 94.74 5.26 0.005.26% = 1 student

7/5/2017

36

Item 9‐11 Detailed Item Analysis

No. A B C D E

16 Pt-Biserial 0.00 -0.06 *0.65 -0.65 0.00

Frequency % 0.00 5.26 63.16 31.58 0.00

17 Pt-Biserial *-0.28 0.00 0.00 0.28 0.00

Frequency % 63.16 0.00 0.00 36.84 0.00

18 Pt-Biserial 0.00 0.00 -0.15 *0.15 0.00

Frequency % 0.00 0.00 36.84 63.16 0.00

71

16. A physical therapist (assistant) is treating a patient that is two days post a hemorrhagic stroke to the left middle cerebral artery. The patient is 40 years old and has no significant medical history. The patient currently exhibits some active motion at the right shoulder and elbow, but no movement of the right hand. What is the patient’s MOST likely prognosis for the ability to complete the activity of upper extremity dressing in three months?

a. Dependent and requires a caregiver to completeb. Independent using the right upper extremity onlyc. Independent using the left upper extremity onlyd. Moderate Assist for management of the right upper extremity

72

No. A B C D E

16 Pt-Biserial 0.00 -0.06 *0.65 -0.65 0.00

Frequency % 0.00 5.26 63.16 31.58 0.00

Is option d clearly wrong?How would you improve the wording of the stem?

7/5/2017

37

16. A patient two days post hemorrhagic stroke with no significant medical history exhibits some active motion at the right shoulder and elbow, but no movement of the right hand. What is the MOST likely prognosis for the ability to complete upper extremity dressing in three months?

a. Minimal Assist for management of left upper extremity

b. Independent using the right upper extremity onlyc. Independent using the left upper extremity onlyd. Moderate Assist for management of the right

upper extremity

73

Possible Rewording with Room for Improvement

17. When a patient falls on an outstretched hand, one possible injury is a Colles’ fracture. What x-ray view would BEST show this fracture?a. Lateral of the wristb. Oblique of the handc. PA of the handd. PA of the wrist

74

No. A B C D E

20 Pt-Biserial *-0.28 0.00 0.00 0.28 0.00

Frequency % 63.16 0.00 0.00 36.84 0.00

•Need to determine why stronger performers selected D•Need to strengthen options B and C•You could feel good that the students all know that a Colles’ fracture is associated with the wrist

7/5/2017

38

18. This patient has been attending outpatient PT for two weeks. She is PWB R LE and still utilizes an assistive device to ambulate. Prolonged changes in her “normal” gait pattern and use of an assistive device have caused some moderate low back pain. You have determined that her pain is muscular in origin. What type of modality or procedure would be the most appropriate?

a. Flourimethane spray or lumbar mechanical tractionb. Hydrotherapy in the hubbard tank or iontophoresisc. Lumbar mechanical traction or electrical stimulationd. Moist heat or ultrasound

75

No. A B C D E

18 Pt-Biserial 0.00 0.00 -0.15 *0.15 0.00

Frequency% 0.00 0.00 36.84 63.16 0.00

•The use of ‘or’ makes this an unnecessarily complex and confusing question. Would using ‘and’ be appropriate?•A and B need to be more attractive.

Item Analysis Report

No.Correct Group Response Response Frequency

TotalUpper 27%

Lower 27%

Point Biserial A B C D E

15 94.74% 100% 80% 0.48 0 0 *18 1 0

16 63.16% 100% 20% 0.65 0 1 *12 6 0

17 63.16% 60% 100% ‐0.28 *12 0 0 7 0

18 63.16% 60% 40% 0.15 0 0 7 *12 0

19 89.47% 100% 100% ‐0.04 1 1 0 *17 0

76

7/5/2017

39

19. If the patient presents to an outpatient clinic 6 weeks following her lung transplantation, which of the following is the most appropriate therapeutic exercise program to target cardiovascular conditioning that also targets the post operative complications related to medication use?

a. Exercise in the pool, including walking at target heart rate

b. Seated open chain resistive therapeutic exercise c. Supine therapeutic resistive exercise on matd. Walking on treadmill at target heart rate x 15-20

minutes77

A B C *D E

Pt-Biserial 0.03 0.03 0.00 -0.04 0.00

Frequency% 5.26 5.26 0.00 89.47 0.00

5.26% = 1 student

78

20.Transection of the nerve fibers in which of the following locations is MOST likely to result in loss of motor functions with preservation of sensation?A. Dorsal rootB. Peripheral nerveC. Spinal nerve D. Ventral root

Omit A B C D

Number 0 1 0 2 34

Pt-Biserial -- -0.147 -- -0.342 +0.371

Frequency % 0.00 0.03 0.00 0.05 0.92

SO—is this a ‘good’ or a ‘bad’ question?

7/5/2017

40

O Item analysis data does not determine item validity

O Items that perform poorly statistically might be retained if they correspond to specific instructional objectives

O Discrimination index (point biserial) is not always a measure of item quality

O Item analysis data are tentative

79

REQUEST FOR REVIEW OF EXAMINATION QUESTION

Requests for review of examination questions must be submitted within 5 working days of the date that examination results are available to students.

O Write your name on the reverse side of this paper.O Date of Examination__________O Date of Submission___________O Question # _______O Correct Answer_______O Your Answer_________O Explain why your answer is the BEST answer (if the

question is multiple choice) or why your answer is correct.

O Provide a reference for your answer. Complete citation is required. You may use textbooks, articles, or the class notes provided for the class.

80

Course Design: Student Assessment/Evaluation · 2020. 6. 15. · 9 Ref: Dormans JP. (2005)...

Documents

Transcript of Course Design: Student Assessment/Evaluation · 2020. 6. 15. · 9 Ref: Dormans JP. (2005)...