Formative and summative eAssessment

Christina Keing

Computer OfficerPaul LamAssistant Professor

29 March 2007

Assessment

FormativeAssessment used as a learning tool: students

realize what they know or don’t know through answering questions

SummativeAssessment used as a student learning

evaluation tool and is typically used to assign students a course grade

Formative assessment “Formative evaluation takes place while a class is

ongoing. It is the indispensable part of assessment that provides a way for us to continuously monitor our students’ learning and our own teaching practices.” (Nuhfer, 1996; p.385)

Summative assessment

Assessment must be matched to the desirable learning outcomes

Suggested approach to

course planning

Student learning needs

1. Aims/ desired learning

outcomes

2. Content/ fundamental

concepts

2. Learning activities

3. Assessment

Actual learning

outcomes

4. Feedback for evaluation

Formative eAssessement Traditional eAssessment

Engagement No time Out of class practice

Feedback Not immediate

Not individualized

Immediate

Can be individualized

Workload High Auto-marking

Summative eAssessmentTraditional eAssessment

Fairness Fair Also fair

Question types Varied question types

Closed-ended

Multi-media

Workload High in marking Auto-marking

High in setting questions

Case 1

To engage students in extended learning through quizzes Weekly (Molecular Biology)

Competitive and game-like

For details, please contact Prof Lam Honming, Biology, CUHK

Case 2

Train students’ decision-making with cases (Chinese Medicine)

Self-learning resources Rich in information, pictures and feedback

For details, please contact Prof Lin, Chinese Medicine, CUHK

Case 3

Realistic assessment on the ability of students to defend main concepts of the course in interview-like situations (Investment banking)

Multimedia and resemble real-life situation Timed

For details, please contact Mr Joseph Mohan, School of Commerce, City University of Hong Kong

Sample student answers

…What is the difference between Investment Banking and Financial Services?

“I would like to use a special example to answer this question. Have you ever know about the gardener and the flowers? What will people focus on? The answer is without doubt, the flowers. However, the gardener plays an important role; he provides nourishment to the flowers, and takes care of them. Without the gardener, flowers cannot grow well, and may even die. I think the relationship between the investment banking and HK’s financial services industry is like that. Investment banking helps to maintain the financial services’ performance. For example, it helps in the merger and acquisition, making the loan or even help the companies/ countries to manage their risks/ debts...”

Case 4

Summative assessment as a tool for a large cohort of students (Common English Foundation)

900 students in 45 classes taught by 21 ELTU teachers

40 MC, and 40 short-answer questions

1 hour of duration Large question bank 20 % final mark

Why eAssessment?

Large group of students, heavy workload in marking examination papers.

45 classes, scheduled to take the exam at different time slots in one week. Security is a concern.

Teachers wanted to analyze the performance of students on each question and make necessary adjustment.

The assessment will be progressively reused.

What are the benefits?

The CEF examinations were conducted in four computer labs. The 45 classes were assigned to different examination time slots within one week. During the peak period, there were about 150 students working on the examinations at the same time.

For MC and short-answer questions, the platform supported auto-marking which would significantly save teachers’ time.

It was possible to randomly select questions from a question bank and compile a different examination paper for each student even during the same examination session.

It would be very convenient for teachers to do item analysis and performance analysis.

What are the limitations?

eAssessments is relatively weak in auto-marking open-ended questions.

It could help to collect students’ answers to the open-ended questions electronically but teacher may not like marking it electronically.

There are quite some efforts in importing questions to the database and checking the format of the input questions.

Demonstration

Various type of questions Individual timing Immediate marking and feedbacks (optional) Submission summary

Number of trials Records of each trial Score Time spent

Detail item analysis Summary statistics

Demonstration Adding a question

Marks and comments

Creating a quiz Assigning questions random select from different set?

Setting up a quizNumber of attemptsAvailabilitySecurity

Students’ comments

Students' acceptance of the new method was also not unanimous.

The responses to the survey question about “I prefer the online test to a pen-and-paper test” are represented below.

0

10

20

30

40

50

60

70

strongly agree agree cannot decide disagree stronglydisagree

Students’ comments

The open-ended questions showed some positive remarks concerning the new method: “It is more convenient than a pen-and-paper test.” “Not so tiring as a pen-and-paper test.” “Not as nervous as doing a paper test.” “Easy to see which question I have answered.” “I can amend my answer without using a correction pen.” “The timers on computers are accurate.”

Students’ comments

There are also negative remarks concerning eAssessment: “It is better to use the traditional pen-and-paper test.” “Too tiring for eyes.” “I can’t collect my thoughts in front of a computer.” “The quality of the mouse and the computer has an effect on

performance.” “It would be unfair for those who type words slowly and wrongly.” “It is not fair that the questions are not the same for all students.

To make the test fair, all students should answer the same set of questions.”

More tips

1. Make the purpose of each quiz clear. Is it for learning processes? Does it focus on revision of terms? Is it for application of concepts to new contexts?

2. There are many sites which provide technical rules for writing good items. Some useful URLs arehttp://www.park.edu/cetl/quicktips/writingtest.html http://www.au.af.mil/au/afiadl/curriculum/icwguide/unit_2.htm

3. Remember to think about how readable the items will be onscreen.

4. Competition can be fun.

Some more tips

1. Release quizzes progressively during the course. 2. The feedback you provide for each item is essential to

making quizzes into learning experiences.3. Good items evolve over time. Ask a colleague to

review them for you, or give them to a few previous students or higher degree students.

4. If you are using multiple-choice items, do an item analysis on the results. Which alternatives were popular? Which were not at all appealing, and hence need to be replaced? How difficult was each item?

5. Run a few quiz items in class to show students how useful the resource can be.

Reference

Nuhfer, E. B. (1996). The place of formative evaluations in assessment and ways to reap their benefits. Journal of Geoscience Education, 44, 385-394.

Thank you