The potential Impact of Computer Aided

Assessment Technology in Higher Education

Ernest Azwindini Tshibalo

University of South Africa,

Pretoria, South Africa

NADEOSA 10th Anniversary Conference

23-24 August 2006

Overview• Introduction• Computer Aided Assessment (CAA)• Rationale for using CAA• Potential benefits and limitations of

CAA• CAA and its potential impact on

student learning• Strategies for developing effective CAA

technology in Higher Education• Conclusion

Introduction

• Distance learning physically separates students from educators – interventions necessary

• Increased number of students in Higher Education • Increased time spent by staff on assessment • E-learning courses & assessment• How can technology assist in this area?

An overhaul of traditional assessment procedures in higher education is long overdue, as present

methods are ill-suited to the digital age in which using information is more important than

remembering it (Mason 1998).

Computer Aided Assessment - CAA

… the use of computers to deliver, mark and analyse assignments or examinations, to keep a record of assessment and to provide feedback.

(Greenberg 1998; Lingered 2000; McKenna & Bull 2000)

“…any instance in which some aspects of computer technology is deployed as part of assessment process.”

(Atkinson & Davies 2000:2)

Computer technology and its role in assessment

• Online completion of interactive exercises and tests

• The use of revision software• Electronically mailed coursework• Marking and feedback• Web pages to set tasks for students and

provide tutor support• Software to detect plagiarism etc.

Rationale for using CAA in HE

• Increasing academic workload in HE• Too many students• Not enough teachers • Students need many assessment

opportunities

• The increase in student numbers: • At Unisa, in some first year modules can

now reach as many as 30 000 students• eg, more than 20 000 in Economics 1.

Rationale continues…• Paper-based versus computer-

based assessmentTraditional assessment methods – paper

based• Certain skills cannot be assessed e.g.

listening• CAA diversifies assessment tasks• CAA provides flexible assessment

• ODL institutions need to change assessment practice to meet these needs.

Rationale continues• CAA and students with learning

disabilitiesThe nature of computers as information

processing tools, and their possibility of creating instructional tools to meet individual student needs, make computers a potential powerful technology for assessment (Kumar, 1994,4).

• Lack of the computer technology – prevents disabled students to access HE

• Lack of CAA ‘closes’ the ‘openness’ of ODL

Potential benefits and limitations

• It is generally agreed that assessment plays a central role in student learning irrespective of the mode.

• Computer based technology challenges students to learn new skills and ways of learning.

• Devlin (2002,2)argues that online assessment teaches skills that is not possible to gain from a normal essay

Some potential benefits of CAA technology

• Use of multimedia, for example, photographs, diagrams, maps, audio, video, and animations can be used as part of the question and or answer.

• Automatic, immediate, and detailed, feedback to all students.

• Objective and consistent standard during marking, however in traditional hand marking – subjectivity..

• Information is more accessible to disabled students E.g.: Digital Audio Integration System – listen to

text, Optical Character Recognition – scan text

Some limitations of CAA technology

• African students – technologically less advantaged• CAA may be unfair to students who are unfamiliar

with computer operations.• CAA is unable to assess skills such as constructive

argument, writing, presentation and interpersonal• Testing High order cognitive skills such as synthesis

and analysis – Bloom’s revised taxonomy can solve• Student authentication, cheating and plagiarism –

can be solved by invigilation, plagiarism detection software, and biometrics (finger print measurement)

CAA and its impact on student learning

• CAA is very good for formative assessment in distance education

• Formative assessment is designed to assist the learning process by providing feedback to learners.

• Feedback motivates students and inform them how well they are performing.

CAA and its impact continues..

Example 1. Online Marking and Results system (OMAR) OMAR is a web-based software that

permits academics to customise online templates for performing their marking and offer feedback to students

• Its introduction at the University of Melbourne led to improved student learning (McKenzie, 2004,1-8)

• ODL institutions can benefit

CAA and its impact continues..

• Example 2. Birkbeck College, University of London

• Baggott and Rayne (2001) used frequent computer-based assessments (CBA) to test strategies for improving learning support for mature, part-time students in a first year Biology Curriculum.

• Extensive feedback was given, areas of need were identified and assistant given.

• Results: Mean CBA score = 65% compared to 50% of none participants

CAA and its impact continues..

• Conclusions made from the use of CBA• CBA provided student support –as

evidenced by benefits gained from formative tests

• CBA promoted learning as evidenced by the fact that the median score was maintained in open and closed book summative assessment

• Extensive feedback – student success

CAA impact on student learning continues..

• Example 3. Research studies in Norway

• Lysne and Jostein (2000) used WebOrama computer tool to involve distance learning students in their learning

• WebOrama is a system where video clips, sounds and texts are integrated

• Its advantages: engagement, work & guidance.

• Results: ODL students compared to internal students

Impact continues

• Scores of the final exams revealed the following:• Distance students scored equally or even

better than internal students• Conclusion: Increased emphasis on

active students, and problem based working methods made for better scores

• Computer based technology has the potential to actively engage distance students in their learning and improve their scores

Strategies for developing effective CAA in HE

• Devlin (2002) introduced strategies for developing effective CAA.

• His strategies are summarised into three checklist:

• An access and usage checklist• A quality of teaching and learning

checklist• A technical and administrative

checklist

Strategies continues..• 1. Access and usage checklist

• Make sure all students have access to computers and that computer skills are developed

• Technologically disadvantaged students should receive support, African students?

• Unisa students for example, should have access to the main and regional campuses

Strategies continues

• 2. Quality of teaching and learning

• Assessment method should relate to the learning outcomes of the course

• Identify where CAA might be beneficial• High order cognitive skills should be considered• It must be clear what is being assessed and

why• Rapid feedback from both students and

academics• Appropriate technology for students with

learning disabilities essential

Strategies continues..

3.Technical and administrative checklist

• Computer used by both students and academics should be compatible

• Computer system used should deal with problems related to password, access and usage

• Adequate technical support staff during the development and use of CAA

• Emergency backup facilities should be put in place

Strategies continues..

• Start up guidelines:

• Start with clear learning outcomes

• Start small

• Start were success is most likely

Conclusion

• Many issues should be explored before embarking on the use of CAA

• In this paper the following were explored:• Definition, rationale, benefits and

limitations, impact on student learning and development strategies.

Conclusion continues..

• In conclusion. • CAA cannot be an answer to all challenges

related to assessment in HE• It can be considered where it best benefits

teaching and learning.• Where the technology is available,

institutions are encourages to optimise its use for the benefit of all

CAA should not be considered as a complete replacement of all other methods

of assessment…

It should be used where it best benefits teaching and learning.

Contact details

Azwindini Ernest TshibaloEducation ConsultantInstitute for Curriculum & Learning DevelopmentUniversity of South Africa, PretoriaTel: +27-12-429-6569tshibae@unisa.ac.za