Assessment of Learning Outcomes in Engineering - an International Perspective Mona Dahms...

Assessment of Learning Outcomes in Engineering

- an International Perspective

Mona [email protected]

UCPBL, Aalborg University

aloe conference 2008 2

Presentation Outline

1. Curriculum Development and Assessment

2. An International Perspective

3. The Situation in Denmark


Definitions

• Learning outcomes: ”sets of competences, expressing what the student will know, understand or be able to do after completion of a process of learning” (ECTS Guide 2005); describes the end product of learning

• Assessment: Judgement of students’ performance, i.e. judgement of whether students have achieved the learning outcomes

• Evaluation: Judgement of quality of study programs and activities


Curriculum Development (CD) – a sequential process

Design of Nth sem.

Competenceprofile of graduate

Design of study program

Design of 1st sem.

Design of 2nd sem.

Design of 3rd sem.

Design of 4th sem.

Study activity 1

Study activity 2

Study activity 3

Study activity 4

Study activity 5

5

Teaching, learning and assessment

Competence profile of graduate

Design of study program

Design of semester program

Design of study activity

Evaluation of study activity

Evaluation of semester program

Evaluation of study program

Evaluation of graduate

competences

This is where assessment is carried out!!!

CD – a top-down process

aloe conference 2008 6Cowan, 2004

= learning outcomes

The Cowan Curriculum Development model


Constructive alignment

Learning outcomes

Assessment Learning/teaching“Alignment is about trust in the relationship between student and teacher” (Derick)

8

What is assessment?

• setting criteria (LOs)

• collecting evidence

• making judgement

and possibly

• providing feed-back for improvement

Example: The customer tastes the food served and ......

• setting criteria

• collecting evidence

• making judgement

and always

• providing feed-forward for further improvement

Example: The chef tastes the food to be served and .........

Formative Summative


Assessment

Important questions to consider are:

– WHO sets criteria? – WHEN and WHERE is evidence prepared?– HOW is evidence presented? – WHO makes the judgement? – WHO provides feed-forward/feed-back?


Changing competences…

remembering

understanding

applying

analysing

evaluating

creating

Lower level competences

Higher level competences

Declared learning outcomes

Learning outcomes assessed

…necessitates changes in assessment

aloe conference 2008 11Fail Pass

Fai

lP

ass

Exam

Traditional assessment

Jakobsen and Rump, 1998

Good understandingFailed examination:0 %

Good understandingPassed examination:25 %

Poor understandingFailed examination:25 %

Poor understandingPassed examination:

50 % !!!!

Understanding


The International Perspective

• Malaysia (2006?) and • South Africa (1997) have both introduced• Outcomes Based Education (OBE) at all levels


Malaysian Qualification Framework (MQF) Learning Outcomes

1. knowledge of discipline areas 2. practical skills 3. social skills and responsibilities 4. value, attitude and professionalism 5. communication, leadership and team skills 6. problem solving and scientific skills 7. information management and life-long learning skills 8. managerial and entrepreneurial skills

“The challenge to us is in drafting assignments, projects and exams as means of evaluating the MQF learning outcomes.”


South African Critical Cross-Field Learning Outcomes

1. Identify and solve problems and make decisions using critical and creative thinking

2. Work effectively with others as members of a team, group, organisation and community

3. Organise and manage themselves and their activities responsibly and effectively

4. Collect, analyse, organise and critically evaluate information5. Communicate effectively using visual, symbolic, and/or language skills

in various modes6. Use science and technology effectively and critically showing

responsibility towards the environments and the health of others7. Demonstrate an understanding of the world as a set of related systems

by recognising that problem-solving contexts do not exist in isolation


Achievements in SA ?

• A study of critical thinking abilities of 1st year education students documented deficient critical thinking abilities

• Possible reasons: Teaching practice; educational change; socio-cultural environment; language ability (Lombard & Grosser 2008)


MoHE - Denmark

Uni 1 Aalborg University Uni 3

HUM FAC

Uni 2

SOC FACTECH FAC

Study Brd. Civil Eng. Study Brd. Elec. Eng. Study Brd. Mec. Eng,

Study Prg. 1 - Team Study Prg. 2 -Team Study prg. 3 - Team Study Prg. 4 - Team

Course 1 - Lecturer Course 2 - LecturerProject - Facilitator

Other study brds


Skills To what extent do you feel that the following skills were evaluated at the exam? The students’ ability to a high or some degree:

0% 20% 40% 60% 80% 100%

Quickly answer theasked questions?

Rememberconcepts anddefinitions?

Relate differentconcepts to each

other?

Students: groupexam 2006 (N=455)

Staff: group exam2006 (N=159)

External examiners:group exam 2006(N=130)Students: individualexam 2007 (597)

Staff: individualexam 2007 (N=113)

External examiners:individual exam2007 (N=63)

Kolmos and Holgaard 2007


SkillsTo what extent do you feel that the following skills were evaluated at the exam? The students’ ability to a high or some degree:

0% 20% 40% 60% 80% 100%

Argue formethodological

choices?

Assume atheoreticaloverview?

Apply analyticalskills?

Students: groupexam 2006(N=455)


Externalexaminers: groupexam 2006(N=130)Students:individual exam2007 (597)

Staff: individualexam 2007(N=113)

Externalexaminers:individual exam2007 (N=63)



SkillsTo what extent do you feel that the following skills were evaluated at the exam? The students’ ability to a high or some degree:

0% 20% 40% 60% 80% 100%

Argue for and against asolution to a problem?

Transfer knowledgegained in projects to

other situations?

Communicateknowledge through a

presentation?

Students: groupexam 2006 (N=455)



Staff: individualexam 2007 (N=113)

External examiners:individual exam2007 (N=63)



Skills To what extent do you feel that the following skills were evaluated at

the exam? The students’ ability to a high or some degree:

0% 20% 40% 60% 80% 100%

Enter intodialogue andcooperation?

Complementand expand on

othersanswers?

Participate in ateamwork?

Students: group exam2006 (N=455)

Staff: group exam 2006(N=159)


Staff: individual exam2007 (N=113)

External examiners:individual exam 2007(N=63)



Statements on team based exams• “I dread a transition to an individual exam because I

believe that this would result in a significant decline in the academic level. Besides I am of the opinion that the team based exam is the type of exam which best relate to the working environment which the students will be in later on.” (External examiner)

• “I think it is important that the members of the group are tested in each others’ co-presence and in a collaborative process. It opens the possibility for partially evaluating their team play and co-operation, ability to stand out in a group, ability to follow up and to have an effect on a discourse.” (External examiner)



Conclusion

• Individual assessment does not test complex knowledge and collaborative competences (Kolmos and Holgaard 2007)

Thank you for your attention!

Comments and questions are very welcome at:

[email protected]

Assessment of Learning Outcomes in Engineering - an International Perspective Mona Dahms...

Documents

Transcript of Assessment of Learning Outcomes in Engineering - an International Perspective Mona Dahms...