3-6-2005Niovi Antonopoulou, Ealta Conference, Voss, 2nd-5th June 2005 1.

3-6-2005 Niovi Antonopoulou, Ealta Conference, Voss, 2nd-5th June 2005

1


2

First part

a. A very brief preview on the evolution of testing and assessment.

b. A literature review relative to the codes of practice and of ethics.

Second part

a. Definitions of the code of ethics and the code of practice and fairness in testing.

b. A brief description of the structure of the test for the certificate of attainment in Greek.

c. Evaluation/critical review of the examinations for the certificate of attainment in Greek.


3

“… Professional books and journal articles, by and large,

have tended to concentrate heavily on theoretical and statistical

aspects of test validity and reliability and are most often written

by academics or educational measurement specialists who are

primarily concerned with large-scale test construction and

Validation.” (Brindley. 2001: 127)


4

Could it be that all the frameworks are very elaborate and bureaucratic for the language testers or teachers to imbed?

Could it be that people engaged in this field address only experts?

Could it be that a code of ethics will restrict FL test constructors and they would not be able to “do the job”?

Could it be that in the effort to cover all the various cases, we end up with a vast framework, vague and obscure, where comprehension of terms can only be achieved –if ever– with the help of a special dictionary?

Could it be that from all those mentioned in a code of ethics only a selection would compile a general framework for a code of good practice, which could be used universally by all testers?


5

“… we had selected six specific sets of standards,

because these sets were good examples of the kind of

work that has been done in the recent past and

because each set of standards adds something new to

our understanding of what standards can and need to

be.” Alderson et al (1991: 254)


6

Which content and methodology are best and will this agreement last for more than a few years, when a new fashion changes our views about what the most appropriate content and methodology should be?Will any one set of standards be suitable for the range of tests on offer, or will standards which are suitable for one sort of test force all other tests into the same mould (e.g. discrete-point vs. performance-based tests)?How comprehensive should standards be? Should they be confined to testing instruments and test procedures, or should they concern themselves with test use? Should they go even further and provide guidelines for dealing with political realities and the dubious intentions of some people in our societies?How detailed should they be? Where is the line between essential detail and triviality?What sort of language should be used - language that will be understood by testers or language that the general public can understand?Which languages should they be written in, and who will do the translations?How idealistic should they be? Should they describe the minimum in good practice or should they describe the maximum? How prescriptive should and can they be?Should some standards be more dispensable than others? Is the primary, secondary and conditional distinction useful or is it confusing?Should standards be enforceable? If so, who will enforce them? If not, how can we be sure that all parties are putting their best into the effort?Is there a way of 'piloting' the standards, so that we can find out if they are effective?


7

“…I’m not sure it’s possible to create anything that’s universally

usable.”

Mace (1998 cited in Johnstone, C. J. 2003)


8

To what extent is that true?

What exactly do they mean by that?

How consistent with the rules are the procedures they employ?

Would their professionalism really stand up to scrutiny?

What does ethics in language testing mean?

Is it ethical to accept even the positive washback, while we force FL teachers and in a way test designers too, to adjust their teaching to the various types of testing?

Do testers take into consideration, not only the linguistic needs, but also the opinion, of teachers and candidates while constructing a test and to what extend?


9

“Moral may indeed be more often applied to the practice or

actions of individuals but in practice the terms ethics/ethicality

and morals/morality appear to be used interchangeably”

Davies, A: 1997: 238


10

Code of Ethics in Testing

=

Code of Fairness in Testing

Code of Good Practices

Professional Standards


11

“One way of thinking of professional ethics is that is

assembles guidelines for action. The ethical principles of

the American Psychological Association (APA), for

instance, presents itself just this way, saying that the code

“is intended to provide both the general principles and the

decision rules to cover most situations encountered by

psychologists.”

(Bishop Sharon, 2004: 111)


12

The Certificate of Attainment in Greek, (hereafter CAG),

was established in 1998 by the Ministry of National

Education. The Centre for the Greek Language (hereafter

CGL) has been designated as the official organizer of the

relevant large-scale examination locally and abroad and

has conducted examinations since 1999. Today there are

73 authorised examination centres in 40 countries

throughout the world and the total number of nationalities

represented is about 47. The candidates who entered the

examination this year were 2.550.


13

Listening Examination (25%)

Level Parts Durat Means

A 2 25 min cass

B 2 25 min cass

C 2 30 min cass

D 3 40 min cass

Reading Examination (25%)

Level Parts Durat

A 2 30 min

B 2 30 min

C 2 40 min

D 3 45 mi.

Writing Examination (25%)

Level Parts Durat

A 2 35 min

B 2 45 min

C 2 75 min

D 2 110 min

Speaking Examination (25%)

Level Examin Cand Durat

A 2 2 10 min

B 2 2 10 minC 2 2 12-5 m

D 2 2 15-20 m

The examination is structured as follows:


14

CEF A1 A2 B1 B2 C1 C2

CAG ----- A B C D -----


15

It is intended to offer a benchmark of satisfactory ethical behaviour by all language testers.

Ethics is based on a blend of the principles of beneficence, non-maleficence, justice, a respect for autonomy and for civil society.

Ιt endeavours to reflect the ever changing balance of societal and cultural values across the world.

Language testers are independent moral agents and sometimes they may have a personal moral.

The Code of Ethics by ILTA (adopted at the annual meeting of ILTA held in Vancouver, March 2000


16

Principle 1Language testers shall have respect for the humanity and

dignity of each of their test takers. They shall provide them

with the best possible professional consideration and shall

respect all persons’ needs, values and cultures in the

provision of their language testing service.


17

Principle 2Language testers shall hold all information obtained

in their professional capacity about their test takers

in confidence and they shall use professional

judgment in sharing such information.


18

Principle 3

Language testers should adhere to all relevant ethical

principles embodied in national and international

guidelines when undertaking any trial, experiment,

treatment or other research activity.

Principle 5

Language testers shall continue to develop their

professional knowledge, sharing this knowledge with

colleagues and other language professionals.


19

Principle 7

Language testers in their societal roles shall strive to

improve the quality of language testing, assessment

and teaching services, promote the just allocation of

those services and contribute to the education of

society regarding language learning and language

proficiency.

Principle 4

Language testers shall not allow the misuse of their

professional knowledge or skills, in so far as they are

able.


20

Principle 6

Language testers shall share the responsibility of

upholding the integrity of the language testing profession.

Principle 8

Language testers shall be mindful of their obligations to

the society within which they work, while recognizing

that those obligations may on occasion conflict with

their responsibilities to their test takers and to other

stakeholders.


21

Principle 9

Language testers shall regularly consider the potential

effects, both short and long term on all stakeholders of

their projects, reserving the right to withhold their

professional services on the grounds of conscience


22

First stage

First part

Planning

• Description of the participants.• Test takers communicative language needs analysis. • Content and structure of the test. • A Comprehensive Examination Syllabus and specifications.• Criterial levels of performance (rating scales). .

Second part

a.Production of guidelines and specifications

b.Training

• Handbook for test takers, test users and staff. • Handbook for the establishment of examination centre.• Handbook for test construction. • Guidelines and specifications for raters, interviewers,

invigilators, administrators. • Supporting material for teachers and test takers.

• Selection and training of the test constructors.• Selection and training of interviewers, raters and staff.• Training seminars in the examination centers.

Third part

Material for administration

• Application forms • Notification about various modifications• Questionnaires for test takers


23

Second stage

Test development • Choice of the elements to be tested in line with the specifications.

• Selection of the texts and items.• Choice of the tasks.• Grading of items.• Informal trialling of the test. • Analysis of results of the trial.• Moderation and rewriting of items. • Validation on the final version of the test items.• Preparation and recording (training of the actors,

choice of music etc.)


24

Third stage

Administration • Printing of test papers. • Dispatching material to the examination centres. • Examination.• Standardization meeting.• Scoring procedure.• Results.• Analysis and interpretation of the results. (validity,

reliability, discrimination, item analysis etc.).• Final report.• Records.• Gathering information for improving tests.• Research.


25

Test items for the Certificate of Attainment in Greek:

are realistic, valid, reliable and objective;

are directly related to everyday activities;

correspond to the candidates' needs;

aim at the use of the language in contexts that are relevant to the aims and needs of candidates;

do not cause any unnecessary anxiety or stress to candidates;

involve as authentic and as interesting as possible topics, which reflect everyday situations;

do not involve topics which may offend any candidates;

take into account factors like gender, the status, and the personality of candidates and, to a certain extent, their possible knowledge and experience, in that they do not include topics which may involve unknown or specialized language, e.g. terminology.


26

3-6-2005Niovi Antonopoulou, Ealta Conference, Voss, 2nd-5th June 2005 1.

Documents

Transcript of 3-6-2005Niovi Antonopoulou, Ealta Conference, Voss, 2nd-5th June 2005 1.