1 Curriculum Design 9 Approaches to evaluation Instructor: Prof. Mavis Shang Erin 9610002M 97/06/12.

1

Curriculum Design

9 Approaches to evaluation

Instructor: Prof. Mavis Shang

Erin 9610002M

97/06/12

2

Introduction＊ Curriculum evaluation:• Is the curriculum achieving its goals?• What is happening in classrooms and schools where it is

being implemented?• Are those affected by the curriculum (e.g., teachers,

administrators, students, parents, employers) satisfied with

the curriculum?• Have those involved in developing and teaching a

language course done a satisfactory job?• Does the curriculum compare favorably with others of its

kind?

3

Introduction

＊ Evaluation may focus on many different aspects :

• curriculum design• the syllabus and program content• classroom processes• materials of instruction• the teachers• teacher training• the students

4

Introduction

＊ Evaluation may focus on many different aspects :

• monitoring of pupil progress• learner motivation• the institution• learning environment • staff development• decision making

5

Introduction

＊ The scope of evaluation :

Focus from test results to the need to collection

information and make judgments about all

aspects of the curriculum, from planning to

implementation

6

Purposes of evaluation

＊ Weir and Roberts (1994):

• Program accountability :

to examine the effects of a program or a project

• Program development:

to improve the quality of a program or a project

7


1. Formative evaluation

To find out what is working well, and what is not,

and what problems need to be addressed

Ongoing development and improvement of the

program

Some typical questions (p. 288)

To address problems and to improve

8



Example 1:

* situation: task-oriented communicative methodology

* problem: teachers are resorting to a teacher-dominated

drill and practice mode of teaching during the

implementation

* solution: provide videos to model teaching strategies

9



Example 2:

* situation: to implement integrated skills

* problem: different perceptions of what the major

points in the course (after few weeks)

* solution: to held meetings to review teachers’

understanding and to clarify the weighting as

well as to provide peer observation

10



Example 3:

* situation: implement conversation skills for low-level,

and the pronunciation is not a major element

of the course

* problem: students have serious pronunciation problem

(after four weeks)

* solution: refocus one section of the course which

includes pronunciation component

11


2. Illuminative evaluation

To find out the different aspects of the program and to

provide a deep understanding of the processes of the

teaching and learning without any change


12



Example 1:

* situation:

1) teach many reading skills for students

2) teachers are interested in what the students’ major

focus of the course

* evaluated way: conduct a short questionnaire

=> understand students’ opinion and need

13



Example 2:

* situation: teacher is interested in teacher-student

interaction learning

* evaluated way: to invited colleague to do the

classroom observation => assess what happened

during the course

14



Example 3:

* situation: teacher wants to know how students carry

out group work and prepare students for

group-work task

* evaluated way: record and review the recordings

=> know what kind of roles for each member in a

group and make sure students participate actively

15



classroom action research or teacher inquiry

Block (1998) : interview learners regularly

Richards and Lockhart (1994): do classroom action

research (classroom observation, learner journals, and

interviews) => useful to confirm and make explicit

some things

16


3. Summative evaluation

To determine the effectiveness of a program, its

efficiency, and to some extent with its acceptability

Used “after” a program


17

Purposes of evaluation Different measures of a course’s effectiveness and

each one can be used for different purposes

a) Mastery of objectives:

“How far have the objectiveness been achieved?”

ex: speaking skills

Objective: In group discussions students will listen to and

respond to the opinions of others in their group

BUT it does not provide the whole picture of them

effectiveness of a course

18

Purposes of evaluation b) Performance on tests:

◎ Formal tests

◎ Weir (1995): it is helpful for teaching and learning

◎ Brindley (1989): informal tests

BUT Weir (1995) stated that summative evaluation and

progress-sensitive performance tests for use are

necessarily

19

Purposes of evaluation c) Measures of acceptability:

◎ satisfactory achievement of the objectives and

good levels of performance on exit tests

≠teachers and learners’ opinion

◎ should be considered some factors: time-tabling,

class size, choice of materials, or teachers’

teaching styles

20


d) Retention rate or reenrollment rate:

◎ whether students continue in the course

◎ the percentage of students who reenroll for

another course at the end

21


e) Efficiency of the course:

◎ How to develop and implement the course efficiently

◎ some problems may occur during the course:

* the time (planning and course development),

* the needs (material and teacher training),

* the time needed (consultations and meetings)

22

Issues in Program Evaluation Weir and Robert (1994, 92) proposed:

a need for both insider and outsider commitment and involvement to ensure adequate evaluation

a central interest in improvement, as well as the demonstration of the “product value” of a program or project or their components

An associated commitment to a deeper professional understanding of the processes of educational change, as well as the results of that change

23

Issues in Program Evaluation Weir and Robert (1994, 92) proposed:

Systematic documentation for evaluation purposes both during implementation and at the beginning and end of a program or project’s life

A willingness to embrace both qualitative and quantitative methodology appropriate to the purpose of the evaluation and the context under review

24

Issues in Program Evaluation1. The audience for evaluation

◎ to identify who the different audiences are and what

kind of information they are most interested in

Ex: new textbooks

* Officers in the ministry: money

* Teachers: the sufficient materials

* Outside consultant: interaction and language practices

* Vocational training centers: school leavers’ English

◎ Evaluation has to satisfy all interested parties.

25


◎ For example

* Students:

What did I learn? Do I need another course?...

* Teachers:

How well did I teach? What did my students learned?...

* Curriculum developers:

Is the design of the course and materials appropriate?...

26


◎ For example

* Administrators:

Was the time frame of the course appropriate?...

* Sponsors:

Was the cost of the course justified?...

27


◎ Three audience are identifiable for all summative

evaluation of language courses (Shaw and Dowsett, 1986):

* other teachers in the program

* managers of the institution or program

* the curriculum support or development

◎ carefully identified different audience and the

result should be appropriate for each audience

28

Issues in Program Evaluation2. Participants in the evaluation process

◎ Two types of participants:

* insiders: teachers, students, and anyone else

Ex: formative evaluation: teachers

summative evaluation: students

Why insiders are important?

=> They are the direct participants for a curriculum

29

Issues in Program Evaluation2. Participants in the evaluation process

◎ Two types of participants:

* outsiders: consultants, inspectors, or administrators to

provides teachers with some perceptions with

independent observation and opinion

30

Issues in Program Evaluation3. Quantitative and qualitative evaluation

◎ Quantitative measurement:

* something can be expressed numerically

=> collect information from a large number of

people and analyze statistically

* more fair but some limitations

31

Issues in Program Evaluation3. Quantitative and qualitative evaluation

◎ Qualitative measurement:

* something can not be expressed numerically and rely

on subjective judgment and observation

=> collect information from classroom observation,

interviews, journal and so on

* more holistic and naturalistic but hard to analyze

◎ Combine both Quantitative and Qualitative

32

Issues in Program Evaluation4. The importance of documentation

◎ Relevant documentation:

a) Course statistics:

b) Relevant course documents

c) Course work

d) Written comments

e) Institutional documents

33

Issues in Program Evaluation5. Implementation

◎ to review the process of evaluation

1) Scope 5) Representativeness

2) Audience 6) Timeliness

3) Reliability 7) Ethical considerations

4) Objectivity

◎ to decide how to make use of the information obtained

(see p. 299)

34

Procedures used in conducting evaluations 1. Tests

1) institutionally prepared tests

2) international tests

3) textbook tests

4) student records

35

Procedures used in conducting evaluations1. Tests

◎ Advantage:

1) direct measure of achievement or performance

◎ Disadvantages:

1) hard to make sure the tests are a direct of teaching or are

liked to other factors=> further investigation

2) “reliability and validity” problems

36

Procedures used in conducting evaluations2. Comparison of two approaches to a course

＊ two different versions of the course

=> compare the effects of two or more different teaching

conditions

◎Advantage:

1) control all relevant factors and investigate strictly

◎Disadvantage:

1) teachers’ load (maintain the difference at the same time)

37

Procedures used in conducting evaluations3. Interview

＊ could get many different views of the course

＊ structured interview is more useful

◎Advantage:

1) obtain more deeper (in-depth) information

◎Disadvantages:

1) time-consuming

2) could not be generalized

38

Procedures used in conducting evaluations4. Questionnaire

＊ could get wide range of teachers’ and students’ comment

◎Advantages:

1) easy for administer

2) generalization

◎Disadvantages:

1) questionnaire design (elicit unbiased answers)

2) difficult to interpret the information

=> follow-up investigation

39

Procedures used in conducting evaluations5. Teachers’ written evaluation

＊ use structured feedback form

◎Advantages:

1) good position to write the comment

2) provide information quickly

◎Disadvantage:

1) only teacher’s point of view

40

Procedures used in conducting evaluations6. Diaries and journals

＊ provide a narrative record of things

◎Advantage:

1) provide relatively detailed and open-end information

◎Disadvantages:

1) hard to decide how to use the information

2) need cooperation and a time commitment

41

Procedures used in conducting evaluations7. Teachers’ records

＊ reports of lessons taught, material covered, attendance,

students’ grades, and time allocation

◎Advantage:

1) provide a detailed account of some aspects of the course

◎Disadvantages:

1) some information will not be relevant

2) some information will be subjective (teachers’ view)

42

Procedures used in conducting evaluations8. Student logs

＊ ask students to provide their opinion of the course

◎Advantages:

1) provide students’ opinion of the course

2) give insights that teachers may not be aware of

◎Disadvantages:

1) need cooperation and time commitment

2) no benefit for students to attend this activity

43

Procedures used in conducting evaluations9. Case study

＊ to do how student made use of lesson plans throughout a

course, or trace the progress of a particular learner

◎Advantages:

1) provide more detailed information

2) provide a rich picture of different dimensions of

courses

◎Disadvantages:

1) the subject may not be representative

2) time-consuming

44

Procedures used in conducting evaluations10. Student evaluation

＊ written or oral feedback on the teachers’ approach, the materials used, and their relevance to the students’ need

◎Advantage:

1) easy to obtain information which contains a wide range

of topics and a large number of learners will be involved

◎Disadvantages:

1) subjective and impressionistic

2) difficult to interpret or generalize

45

Procedures used in conducting evaluations11. Audio- or video-recording

＊ provide some examples of different teaching styles and

lesson formats

◎Advantage:

1) provide many teachings and record information

◎Disadvantage:

1) recordings are hard to set up

46

Procedures used in conducting evaluations12. Observation

＊ by other teachers or a supervisor (give a specific task and

provide procedures to the observer )

◎Advantage:

1) observers can focus on many things in class

◎Disadvantage:

1) need preparation and explicit guidance

47

~Thank you~

1 Curriculum Design 9 Approaches to evaluation Instructor: Prof. Mavis Shang Erin 9610002M 97/06/12.

Documents

Transcript of 1 Curriculum Design 9 Approaches to evaluation Instructor: Prof. Mavis Shang Erin 9610002M 97/06/12.