NOTA MTE3112 (EGLISH VERSION)

Compilation of Notes of MTE3112

By Cg Mohd Ridzuan al-Kindy™ (IPG KDRI)

MTE3112 – ASSESSMENT PRACTICES IN MATHEMATICS TOPIC 1: TESTING AND EVALUATION Definition

Ass

essm

ent /

Tes

t

A systematic way to measure changes in an individual behaviour. (R.M. Gagne,1965)

Required pupils to show explicitly in the form of activities

Measuring instrument: observation and formal test (e.g.: exercise, quiz)

Always happened in the class during the lesson and on-going process.

Aims: o Determine pupils’ behavioural changes o Determine pupils achievement in certain

lesson o Place pupils in suitable group according

to result o Motivate pupils to revise o Train pupils make appropriate

preparation for public examination

Mea

sure

men

t

A process or system used to determine the quantitative value of a certain test in learning. (Mok Soon Sang, 2004)

Expressed in a fixed unit of number (e.g.: %, or Grade)

Covers all aspect of testing to determine the position and achievement of pupils in class: o Ordinal scale measuring instrument to rank pupils’

grade / position Compare the rank of pupils in a class Determine pupils’ performance and

achievement o Interval scale unit of mark with same weightage in

its value Compare the grades between one

class to another Compare the pupils’ achievement

among pupils of a number of classes. Aims:

o Determine level of achievement and pupils’ position in same / among several classes.

o Compare pupils’ achievement and position in certain subject / overall academic achievement

o Allocate pupils for appropriate streaming o Make analysis, interpretation and

conclusion for certain research based on result measurement

o Award certificate in certain school level.

Eval

uatio

n

A process to determine, obtain and provide useful information so as to make objective consideration for follow-up action. (Dr. Shufflebeam, 1971)

In education – a system or process which cover activities to collect information relating to strategy of T&L activity and to make analysis and interpretation of result, with the aim to take appropriate follow-up action.

5 stages of evaluation: o Test preparation o Test implementation o Assessment and measurement o Analysis of result o Interpretation and follow-up action

Aims: o Collect information concerning pupils’

achievement after T&L activities, so as to interpret and take appropriate action

o Determine pupils’ strength and weaknesses in T&L certain skill so as to plan for enrichment and remedial activities.

o Determine the effect of teachers’ presentation in class so as to plan for a more effective T&L strategy

o To determine the achievement of pre-determined learning objective so as to plan for T&L activities that follow.

o Record the evaluation result in the individual’s Performance and Profile Record

o Assess the effectiveness of implementing a certain new programme (e.g.: KBSR)

Relationship among Test, Measurement and Evaluation

Assessment / test refer to the collection of data to

describe or better understand an issue. Measurement is the process of quantifying

assessment data. Evaluation refers to the comparison of assessment

data that measured to a standard for the purpose of judging worth or quality.

Evaluation

Measurement

Assessment / Test



Evaluation in KBSR Roles:

Types of Evaluation in KBSR

o Evaluation to ascertain the effect in:

Assessment Design Principles of Assessment

Planning a Assessment / Test

Improve pupils' learning by modifying the T&L activities based on the result obtain in School-based

Evaluation

Understand clearly the pupils' leraning strength and weaknesses so that appropriate follow-up activities

can be carried

Stimulate pupils to improve leraning through teacher's guidance

Prepare pupils to acomodate themselves in testing situation through contonuous evaluation

To raise standard of evaluation by constructiong high quality test items

Improve teacher ability and efficiency in conducting test and T&L activity

•Conducted in class during lesson•Find out weakness and correct it on the spot

• Informal evaluation

Skill mastering

•Carried after complete several related skills in class

•To understand pupils' progress in certain learning skills

•To determine the goup of pupils which need enrichment or remediation

•Can be formal or informal evaluation•Result recorded to Pupils' Performance Record

Learning process

•Carried out after teacher has taught a few learning topics

•Usually held before the end of a school term

•To differentiate pupils' achievement after a few learning topics

•Carried out in formal way

Acheivement

•Determine the reasons why you are assessing the students

•Design an assessment that will fulfil your needs

The purpose of the assessment and task fluffiness

•Ensure assessment is both valid - in that it tests a relevant skill or ability

•Reliable - in that the same result would be achieved if assessment repeated later.

The validity and reliability of the assessment

•To compare the candidates abilities with a common measure – objective criteria or candidates performance

The referencing of the assessment

•Consider the difficulty level of the items •Consider how well the assessment differentiates between the candidates - to provide maximum information the assessment

The construction quality of assessment items

•Grades awarded are very concise summaries of students’ abilities.

•Should be clear and easily understood

The grading of the assessment

1. Determine aim of testing

2. Determine test content

3. Formulate test blueprint

4. Construct items / questions

5. Check items / question



Principles of Item Construction When constructing items, the teacher should put

himself in pupils’ shoes. The question that can help teacher construct the

items: o How these items are understood by students

which have various levels of ability? o Is there any clue of the answer in the item to help

pupils? o Is there any ambiguity? o Is the item was too easy or not? o Is the item related to the important knowledge

and skills? o Is the item was too long or too short? o Is there any relationship between item and the

objective of the subject? Types of Items Construct Objective Items

o Multiple Choice Question

o Multiple Completion Question

Subjective Items o Structural objective / Short Answer Question

o True / False Question

o Matching Items

o Mechanical Problem (Calculation Question)

o Problem Solving Question

Solo / Bloom Taxonomy Solo Taxonomy

o SOLO taxonomy stands for: Structure of Observed Learning Outcomes

o Developed by Biggs and Collis (1982), and is well described in Biggs and Tang (2007).

o Describes level of increasing complexity in a student's understanding of a subject through five stages.

o It is claimed to be applicable to any subject area.

What is the sum of 403 and 607? A. 100 B. 1000 C. 1010 D. 1100

Which statements are TRUE? I. A cube has 6 faces II. All cuboid must contain 6 rectangular

faces III. Cube and cuboid are three

dimensional objects IV. A tower is a type of cuboid

A. I, II and III B. I, II and IV C. I, III and IV D. II, III and IV

What is product of 3 and 8? Answer: _____________________________

(a) 100 + 23 = 123 True False

(a) 110 – 20 = 80 True False

45 + 86

193 - 80

25 x 5

981 ÷ 9

109

113

131

125

Calculate 25% from RM 264.00 Answer: __________

The price of an exercise book is RM 0.50. Ahmad bought 7 exercise books from the bookstore. How much Ahmad need to pay? Answer: __________



Level of Understanding Phrase of Learning

Pre-structural No understanding demonstrated

Quantitative Phase

The amount of detail in the

students response increase

Uni-structural Concrete, minimalistic

understanding of an area. Focuses on one conceptual

issue in a complex case.

Multi-structural Indicates understanding of

boundaries but not of systems. Understanding of several

components but the understanding of each is discreet.

Disorganised collection of ideas or concepts around an issue.

Has not been able to relate the items in the list.

Relational Indicate orchestration between

facts and theory, action and purpose.

Understanding of several components which are integrated conceptually. Can

apply the concept to familiar problems or work situations

Qualitative Phase

The detail in the response becomes

integrated into a structural pattern

Extended Abstract Student conceptualizes at a

level extending beyond what has been dealt with in the actual teaching.

Can generalize to a new area.

Bloom Taxonomy

Level Description

Kno

wle

dge

o Recall data or information. o Key Words: defines, describes, identifies,

knows, labels, lists, matches, names, outlines, recalls, recognizes, reproduces, selects, states.

o Questions like: What are the health benefits of eating apples?

Com

preh

ensi

on

o Understand the meaning, translation, interpolation, and interpretation of instructions and problems. State a problem in one's own words.

o Key Words: classify, describe, discuss, explain, express, identify, indicate, locate, recognize report, restate, review, select, and translate.

o Questions like: Compare the health benefits of eating apples vs. oranges.

App

licat

ion

o Using new knowledge. Solve problems to new situations by applying acquired knowledge, facts, techniques and rules in a different way.

o Key Words: apply, choose, demonstrate, dramatize, employ, illustrate, interpret, operate, practice, schedule, sketch, solve, use, and write.

o Questions like: Which kinds of apples are best for baking a pie, and why?



Ana

lysi

s o Examine and break information into parts by

identifying motives or causes. o Make inferences and find evidence to

support generalizations Analysis of elements

o Analysis of relationships / organizational principles

o Key Words: analyzes, breaks down, compares, contrasts, diagrams, deconstructs, differentiates, discriminates, distinguishes, identifies, illustrates, infers, outlines, relates, selects, separates.

o Questions like: List four ways of serving foods made with apples and explain which ones have the highest health benefits. Provide references to support your statements.

Synt

hesi

s

o Refers to the ability to put parts together to form a new whole.

o Involve the production of a unique communication (theme or speech), a plan of operations (research proposal), or a set of abstract relations (scheme for classifying information).

o Learning outcomes in this area stress creative behaviours, with major emphasis on the formulation of new patterns or structures.

o Key Words: arrange, assemble, collect, compose, construct, create, design, develop, formulate, manage, organize, plan, prepare, propose, set up, and write.

o Questions like: Convert an "unhealthy" recipe for apple pie to a "healthy" recipe by replacing your choice of ingredients. Explain the health benefits of using the ingredients you chose vs. the original ones.

Eval

uatio

n

o Make judgments about the value of ideas or materials.

o Key Words: appraises, compares, concludes, contrasts, criticizes, critiques, defends, describes, discriminates, evaluates, explains, interprets, justifies, relates, summarizes, supports.

o Questions like: Do you feel that serving apple pie for an after school snack for children is healthy? Why or why not?

Curriculum Specification and Planning of Test (Test Blue Print) Also known as Test Specification Table (Jadual

Spesifikasi Ujian - JSU) Purpose of preparing Test Blueprint

o To guide for teacher to construct questions follow accurately to test blueprint

o To ensure the appropriate weightage for various topics and their skills level reflect accurately the pupils’ overall ability.

Important elements: o Topics / skills Refer to curriculum specification

o Skill level Follow the Bloom Taxonomy level (6 levels) For diagnostic test simply use 3 levels – easy,

moderate and hard o Number of items of according topic and skill level

Example of Test Specification Table for Diagnostic Test

for Year 4 Pupils in Topic of Length



School Based and Classroom Assessment Formative and Summative Assessment Aspects Formative Summative

Concept

A type of formal and informal test to assess the effectiveness of skill mastering and progress of learning in class.

A type of formal test to access pupils’ achievement in certain schooling level

Aim

To ascertain the effectiveness of skill mastering and pupils’ progress To correct weaknesses immediately in class

To determine pupils’ achievement based on a few learning units and arrange their grades level accordingly

Activity

To carry out formal or informal test individually, in group or in class during learning

To use a few period to carry out formal test in class or year (cohort)

Time To carry out after teaching one or several skills

To carry out after teaching a few learning units

Format

Informal – observation / questioning and discussion Formal – paper & pencil test in short time

Formal – paper & pencil / oral test within suitable period

Target Individually, groups or whole class

The whole class or year (cohort)

Coverage Cover certain skill (informal) or a few skill (formal)

Cover a few learning units or certain schooling level

Record

No record (informal) and the use of check list / performance record (formal)

Profile record / certificate

Follow Up

Action

Correct weakness immediately / carry out remediation / enrichment

Replacing pupils in suitable learning group

Place pupils in suitable group / class

Plan and modify lesson / curriculum / panel’s plan

Award certificate based on achievement in examination

Alternative Assessment An ongoing process involving the student and

teacher in making judgments about the students’ progress using non-conventional strategies. (Hancock,1994/1)

Form of alternative assessment: o Portfolios o Scrap book o Rubrics & checklists (criterion-reference) o Anecdotal records – teacher, or person who is

observing, writes down their observations of the student providing a record of student growth, strengths and needs.

o Oral performances o Projects o Performance tasks

Reason / Purposes:

Formal and Informal Evaluation

Formal Informal

Formal assessments have data which support the conclusions made from the test (standardized measures)

Informal assessments are not data driven but rather content and performance driven.

Standardized measures

Non-standardized measures

Pupils will be assessed and graded according to standardized grade

Pupil will be assessed according rubric scores

Pupils achievement written in % or grade (standard score)

Pupils achievement written in score (such 10/15 – in non-standard score)

Students are able to display knowledge in many different ways.

Students who are not skilled test takers will be able to succeed in showing their abilities and therefore build self-esteem.

Alternative Assessment methods do not interrupt class time as do traditional tests, but instead are measurements of day to day growth.

Gives ongoing information on strengths and weaknesses of each student providing the teacher opportunity to work within those.

Not tied to a specific culture or language, but rather, is free from prejudice and unfair assumptions.

Directly connected with the curriculum and therefore provides a direct reflection of what is being taught



Assess overall achievement, to compare a student's performance with others

Assess criterion referenced measures or performance based measures

More to quantitative measurement

More to qualitative measurement

Written test, oral test, examination, coursework, monthly test

Questioning and discussion, observation, self-assessment test, homework, in class exercise

Norm-referenced and Criterion-referenced Evaluation

Aspects Norm-referenced Criterion-referenced

Concept

Relative-achievement evaluation (shows performance of group of candidates in form of normal graph)

Objective-referenced evaluation (determine individual achievement based on minimum criterion fixed earlier

Aim

To compare and distinguish performance between candidates / group

To determine ability to master learning based on the certain fixed criterion

Uses

Summative test to distinguish candidates in performance level of distinction, credit, pass and fail

Formative test to improve T&L based on the test result

Target

Comparison of performance among candidates

Determination of performance individually or in small group

Question Charac-teristic

Arranged from easy to hard, and with discrimination among candidates

Have almost the same level of difficulty based on learning objective

Grading

Passing mark / grading is determined after the test result

Passing mark / grading is determined before the test is carried out

Coverage Wide range of learning skills

Limited learning skills

Example Public examination as UPSR, PMR & SPM

Oral test, MUET, Coursework, written exercise

Norm-referenced Evaluation o Obtained normal graph from candidates’ score

Interpretation of Assessment Item Analysis and Interpretation of Items Difficulty Index

o Used in general to ascertain the degree of difficulty of a certain item or question set whether it is too difficult, average or too easy.

o Formulae for푛 < 30:

o Formulae for푛 ≥ 30:

o Interpretation 푫. 푰. Interpretation Decision

푫. 푰. < ퟎ.ퟑ Too Difficult Modify ퟎ.ퟑ ≤ 푫. 푰.≤ ퟎ.ퟖ Average Accept

푫. 푰. > ퟎ.ퟖ Too Easy Modify

퐷푖푓푓푖푐푢푙푡푦퐼푛푑푒푥,퐷. 퐼

= 푁푢푚푏푒푟표푓푐푎푛푑푖푑푎푡푒푠푎푛푠푤푒푟푐표푟푟푒푐푡푙푦(퐶)

푇표푡푎푙푐푎푛푑푖푑푎푡푒푠푎푡푡푒푚푝푡푡ℎ푒푖푡푒푚/푞푢푒푠푡푖표푛(푇)

퐷푖푓푓푖푐푢푙푡푦퐼푛푑푒푥,퐷. 퐼 = 퐶 + 퐶푇

Whereas: 퐶 - Total number of candidates in the high-

score group who answer correctly the items or questions

퐶 - Total number of candidates in the low-score group who answer correctly the items or questions

푇 - Total number of candidates in both groups with high and low score



Discrimination Index o Function to differentiate candidates between

high-score and low-score group o Determine whether a certain item or question is

suitable and can be used for a test o Formulae:

o Interpretation

푫풔. 푰. Interpretation Decision 푫풔. 푰. > ퟎ.ퟒퟎ Positive

Discrimination High

Very suitable and can be accepted

ퟎ.ퟐퟎ < 푫풔. 푰.≤ ퟎ.ퟒퟎ

Positive Discrimination

Moderate

Modify the item / question

ퟎ ≤ 푫풔. 푰.≤ ퟎ.ퟐퟎ

Positive Discrimination

Low

Item / question needs to be

reconstructed

푫풔. 푰. < ퟎ Negative Discrimination Performance of low-score group

is better than high-score group

Item / question is bad and should be removed

Evaluation of Reports and Reporting Stages of evaluation reporting:

Data collection

o Collect the marks of pupils after the test. o Commonly represent in the form of table

Measurement of mean, mode, median and standard

deviation o Mean – the standard average of the set of data /

arithmetic average of a set of values, or distribution Ungrouped data

Ungrouped data in frequency table

Grouped data

o Mode Ungrouped data


Data collection

Measure-ment of mean, mode, median

and standard deviation

Data represen-tative and analysis

Interpre-tation

based on analyzed

data

퐷푖푠푐푟푖푚푖푛푎푡푖표푛퐼푛푑푒푥(퐷푠. 퐼. ) = 퐶 − 퐶12푇

Whereas: 퐶 - Total number of candidates in the high-

score group who answer correctly the items or questions

퐶 - Total number of candidates in the low-score group who answer correctly the items or questions

푇 - Total number of candidates in both groups with high and low score

푀푒푎푛, 푥̅ =∑푥푛

Whereas: ∑푥 - Sum of all score 푛 - Number of score

푀푒푎푛, 푥̅ =∑푓푥∑푓

Whereas: 푥 - The score 푓 - Frequency

푀푒푎푛, 푥̅ =∑푓푥∑푓

Whereas: 푥 - The class midpoint 푓 - Frequency

The value / score that repeated the most number of time in a set of data

The value / score that has highest frequency on the frequency table



Grouped data

o Median – the centre / mid value / score of a set of

data Ungrouped data – data arranged in ascending

or descending order


Grouped data

o Standard deviation Ungrouped data


Grouped data

Representative of data

o Frequency table

o Histogram

o Frequency polygon

푀표푑푒 = 퐿 +∆

∆ + ∆푐

Can be obtained from the highest bar of histogram

Or formulae:

Whereas: 퐿 - Lower class boundary of modal class ∆ - Frequency of modal class – frequency

of before modal class ∆ - Frequency of modal class – frequency

of after modal class 푐 - Size of median class = upper boundary – lower boundary

푀푒푑푖푎푛,푚 = 푇

푇 =(푛 + 1)

2푡ℎ표푏푠푒푟푣푎푡푖표푛

Whereas:

푛 - Number of score

푀푒푑푖푎푛,푚 = 푇

푇 =(푛 + 1)

2푡ℎ표푏푠푒푟푣푎푡푖표푛

Whereas:

푛 - Number of score

푀푒푑푖푎푛,푚 = 퐿 +푛2 − 퐹푓

푐

Whereas: 퐿 - Lower boundary of median class 푛 - Sum of frequencies 퐹 - Cumulative frequency immediately before the median class 푓 - Frequency of median class 푐 - Size of median class = upper boundary – lower boundary

푆푡푑,휎 = ∑ 푥푛

− (푥̅)

Whereas: ∑푥 - Sum of all score square

푆푡푑,휎 = ∑ 푓푥∑푓

− (푥̅)

Whereas: ∑푓푥 - Sum of all frequency x score square

푆푡푑,휎 = ∑ 푓푥∑푓

− (푥̅)

Whereas: ∑푓푥 - Sum of all frequency x class midpoint



o Cumulative frequency table

o Ogive

o Frequency curve

Interpretation of statistical data – frequency curve o Normal curve

Symmetric and has no skew Pupils’ scores are balance Question is effective and acceptable Question constructed can discriminate the

level of weak, moderate and good pupils The data can represent the level of pupils. 푚푒푎푛 = 푚푒푑푖푎푛 = 푚표푑푒

o Positive-skewed curve

The tail is on right of the curve Many pupils get low score since cannot

answer the question given The question constructed is too difficult The question constructed cannot discriminate

well the level of pupils The data cannot represent the level of pupils

accurately. The question need to be modified 푚표푑푒 ≤ 푚푒푑푖푎푛 ≤ 푚푒푎푛

o Negative-skewed curve

The tail is on the left of the curve Many pupils get high score since can answer

the question easily The question constructed is too easy The question constructed cannot discriminate

well the level of pupils The data cannot represent the level of pupils

accurately. The question need to be modified 푚푒푎푛 ≤ 푚푒푑푖푎푛 ≤ 푚표푑푒

o Bimodal

The curve has two modes. It is a mixture of two normal distributions with

the same standard deviation but different means.

The frequency of average score is lower than lower and high score

The frequency of lower score almost same / same with the frequency of high score.



Recording Progress and Monitoring of Students’ Achievement Progress Record

o Usually a checklist o Used to assess and trace the pupils’ progress in

certain topic of Mathematics lesson o Aims: Identify pupils for enrichment and remediation

o Contains: Skills taught Pupils’ name (alphabetically order) Note / follow up actions

o E.g.: Progress Record

Class : 2 Amanah Topics : Unit 8 – Length Skills : 8.1 Use Vocabulary related to Length

8.2a Compare the length of objects by direct comparison 8.2b Measure the length of objects using uniform non-standard unit 8.3 Measure the length of objects using standard unit

No Name Skills Notes /

follow up actions 8.1 8.2a 8.2b 8.3

1 Adam Aiman bin Mansor / / / / E

2 Afiq Fahmi bin Azli / / / / E

3 Azim Faizudin bin Ali / x x x R

4 Imran Fadli bin Rosman / / / / E

5 Irfan Danial bin A.Karim x x x x R

6 Aina Syafiqa binti Hashim / / x x R

7 Nur Hani binti Mohd Azmi x x x x R

8 Hanan Sufiyah binti Mohd / / / / E

9 Liyana Safiah binti Ahmad x / x x R

10 Madihah binti Zulkifle / / / / E

Notes: E – Enrichment R – Remediation

Performance Record

o Record the pupils result in a certain formative or summative result.

o The result can be used to determine the pupils’ level of performance.

o Record the marks and grade achieved by the pupils.

o Usually prepared and kept by teacher in teaching record book.

o Contains: The test name (monthly / term) Pupils’ name (alphabetically order) Marks Grade Analysis of achievement according to grade

o E.g.:

Assessment Administration Test Administration Three main stages:

•Checking number of test question paper

•Arrangement of the candidates' seating according to examination rules and confortable

•Facilities needed : wall clock, candidates checklist name, additional answer paper, etc.

Preparation

•Ensure the candidates enter the room early, 10-15 minutes before examination start

•Ensure pupils sit according to their seat

•Give instruction clearly•Observe pupils during the test

•Remind the time left so as the aware about it

Implemen-tation

•Collect the pupils answer and ensure they write their particular

•Ensure all pass up the answer sheet

•The answers script need to bundle up and safely handed to examiners

Coordination



Test moderation The process of eliminating or lessening extremes To ensure normality throughout the medium on

which it is being conducted Normally occur in public examination such UPSR,

PMR and SPM. Rarely happened in school since the examiners of

the answer script is the subject teacher that marks all the pupils’ answer of his class. But some teacher still discuss with other teacher about the way giving the marks. (can be considered as informal moderation)

Purposes of test moderation: o To standardize the marks given by the different

examiners that marks the same paper. o To decide the grade range of pass, credit,

average, good and excellent. o To balance up the number of candidates achieve

high, moderate and low scores close to the normal distribution curve.

Marking scheme A set of criteria used in marking the candidates

answer paper that be standardized. The purposes:

o To decide the minimum answer that pupils need to write as to get a mark.

o To guide the examiners on how to marks the candidates’ answer script.

o As the main reference on how to marks. o To avoid the biased feeling among examiners by

prepared a guide for them. o Help to reduce the influence of subjective

elements while examiners marking the script. 3 important criteria in marking scheme for

subjective:

Test Characteristics Validity

o Refer to what level the test is able to measure what it is supposed to measure

o Also can be said as tests for determining whether the test is measuring the concept / skills that the teacher thinks is being measured

o Can be classified into several categories: Face validity

Content / curriculum validity

Criterion-related validity

•covers the informations aspects / related fact of answer

•show the understanding and mastering of the concept / skills in mathematics

Content criterion

•covers the aspect of accurate usage of language / skills of represent ideas logically and maturely

Organization criterion

•covers the technique aspect of accuracy, suitability, perfectness, cinsistency, originality and creativity in candidates' answer

Process criterion

Refer to test as if it perceived as assessing what it is supposed to test, checked by teacher that constructed the question on his specialized.

The format and presentation of test paper also can be considered as face validity

The question constructed need to fulfill the aims and skills level that have been determined.

Test blueprint prepared with specific skills and weightage of the skills to be tested

Ensure the pupils already learn the skills that will be tested

The construction of items is following the test blue print accurately

Used to compare scores on a certain assessment with value from external criterion Predictive validity

Refer to criterion evaluated Relates to the ability to predict the

capability or qualification of an individual in carrying out certain duty / work in the future

E.g.: The SPM trial result can be used (valid) to apply the ESU Forecast Universiti Teknologi Petronas Engineering Programme

Concurrent validity Refer to individual achievement in

a test which highly correlated with previous similar test.

Determined by conducting a similar test so as to correlate with result of a previous test.



Construct validity

Reliability o Refer to the degree to which a measurement

procedure produces similar outcomes when it is repeated.

o When designing an assessment, need to consider whether the results achieved will be consistent.

o Several different ways of measuring reliability: Test-retest reliability

Parallel forms reliability

Internal consistency

Objectivity o Refer to the accuracy of an examiner in marking

any candidates answer. o Examiners need to give same marks of the same

answer of candidates according to marking scheme

o Closed-ended question has highest objectivity rather than open-ended question.

o To improve the marking for open-ended question, the analytical marking scheme and moderation.

Administrability o Refer to the smooth implementation of the testing

process in room / examination room. o The whole process of conducting the test from

beginning stage of preparing the question papers until the ending stage of marking the candidates’ answer script can be carried out smoothly.

Assessibility o Refer to the result of the candidates’

achievement which can easily interpret and assessed by the way providing clear and useful information according to the objective of the test.

o This mean: The result is easily collected and converted to

statistical form Clear picture of interpretation that can guide

the analyser to make follow up action Bank Items Can be described as the databank that keeps all the

examination questions whether pre-existing or constructed by teacher.

The past items / question were arranged in systematic file to ensure it is easy to access the items / question.

Refer to the matching between the assessment and those construct which are presumed to inspire test performance

Should include all previous skills related to the assessment of the constructs.

The correlation between candidate’s attempts at the same test. (a candidate attempt a same test more than one time)

The correlation between candidates attempts at two tests which are supposed to be identical. (Two test paper from a same test blueprint answer by a candidates)

Essentially is a pseudo-measure of reliability. It actually is the correlations between the test items.

Can be considered as an estimate of the correlation between the test that was actually delivered with other possible tests that might have been constructed using those items.



TOPIC 2: MATHEMATICAL DIFFICULTIES AND DIAGNOSTIC TEST Diagnostic Test Standard IQ Test An Intelligence Quotient (IQ) indicates a person's

mental abilities relative to others of approximately the same age.

Everyone has hundreds of specific mental abilities –some can be measured accurately and are reliable predictors of academic and financial success.

Most IQ tests measure verbal, non-verbal, working memory and processing speed abilities.

Intelligence tests also assess the following: o Mathematical skills o Verbal skills o Spatial skills o Pattern recognition o Logical reasoning o Short term memory o General knowledge o Classification skills

Purposes of IQ test:

Range of IQ test score:

IQ Range Classification

Above 145 Genius or near genius

130-145 Very superior

115-130 Superior

85-115 Normal

70-85 Dullness

Below 70 Borderline deficiency Example of IQ test:

School Based Assessment (SBA) SBA is the assessment of students’ progress which

occurs, on an on-going basis during the year, as an important part of the students’ learning.

Purposes: o Teachers give regular feedback to their students

to help them learn better. o Enable develop students’ thought process and

their ability to solve problems. o Enable the improvement of students’ personal

development and communicative ability. With SBA, student assessment include:

o Students’ coursework-the school work they do in the classroom and at home during the year

• It yields specific information regarding areas of cognitive strength and weakness.

Self-knowledge and growth

•IQ tests help identifies situations in which the overall ability is significantly higher than ability in one specific area.

Identification of learning disabilities

•Identification of above-average or gifted intelligence. Knowing areas of significant strength can help parents advocate for advanced classes, specialized assignments and a better appreciation of their child's presentation in the classroom.

Identification of gifted children

•Our abilities correspond with specific locations in the brain, a decline in ability in certain areas, compared with previous levels of functioning, can help doctors locate and better understand the nature of a cognitive problem.

Identification of illness or damage



o Students’ personal development (their behavior within the school, their development of personal and social values and their participation in co-curricular activities of the school).

Form of SBA: o Project-based learning o Oral presentations o Group work o External class activity o Co-curricular activity

Student can be assessed on: o Attendance in class and interest in learning o Assessment (class wise) o Assignments (individual/group) o Behavior, values and honesty o Presentation of speech o Individual and group discussions o Leadership qualities o Discipline o Participation in cultural activities o Performance in sports and games o Practical classes in science subjects

Classroom Assessment / Test A series of tools and practices designed to give

teachers accurate information about the quality of student learning.

Purposes:

Steps in the classroom assessment process:

Diagnostic Assessment and Administration Definition of Diagnostic Test An individually administered test designed to identify

the weakness and problem in the learning process. The test is required to assess the current

mathematical ability of pupils. Characteristic:

Implementation and Administration

Aimed at learning improvement, rather than at assigning grades.

Give better understand your students' learning and so to improve your teaching.

Help student become better monitors of their own learning.

Help break down feelings of anonymity, especially in larger courses.

Point out the need to alter study skills.

Provide concrete evidence that the teacher cares about learning.

Check for student background knowledge

Identify areas of confusion

Enable students to self-assess their learning level

Determine students’ learning styles

Target and build specific skills

Test is to analyse which skill that pupils have problem to understand it.

Not covered on all the skills learn, only specific skills that need to be analyse the problem.

Test is to observe the behavioural change in pupils understanding on certain skills.

Every item in the diagnostic test is specifically deciding the skill that need to be diagnose.

The level of item is easy and moderate, the hard level is not recommended (<65% difficulty).

Conducted before any lesson or during the lesson.

Remediation activity needs to be planned to overcome the pupils’ problem.

Marks of pupils in diagnostic test are record as individual profile (Easily know the behavioural change)

Decide the skills that need to be tested (in more specific and unit skill)

Draft and prepare the blueprint of the diagnostic

Items need to be constructed based on blueprint built.

Decide its validity, reliability, objectivity, administrability and assessibility

Decide how the items will be tested; test paper or in-class exercise

Prepare record table for the pupils’ achievement

Plan the suitable remediation activity



Misconception and Mathematical Difficulties Misconception Misconceptions – incomplete half-truths idea that

actively constructed by the from the children everyday experiences.

Effects: o It affect with learning when students use them to

interpret new experiences. o Students are emotionally and intellectually

attached to their misconceptions, because they have actively constructed them.

o Hence, students give up their misconceptions, which can have such a harmful effect on learning, only with great reluctance.

Newman Error Analysis

Reading o Reading the problem question o Recommend question: “Please read the question to me.” “Read the question to me. If you don’t

know a word tells me.”

Comprehension o Understand what the question ask o Recommend question: “Tell me what the question is asking you

to do.”

Transformation Skills o Carrying out a transformation from the

words of the problem to the selection of an appropriate mathematical strategy

o Recommend question: “Tell me a method you can use to find

and answer to the question.” “Now tell me what method you used to

find the answer.”

Process Skills o Applying the process skills demanded by

the selected strategy o Recommend question: “Show me how you worked out the

answer to the question. Explain to me what you are doing as you do it.”

“Now go over each step of your working, and tell me what you were thinking.”

Encode o Encoding the answer in an acceptable

written form o Recommend question: “Now write down your answer to the

question.” “Tell me, what is the answer to the

question? Point to your answer.”

Carelessness o When the pupils know redo the question for

second time and answer it correctly, the pupils is said careless.

Motivation o The motivation of pupils during answering

the question is something implicit to be measured.



TOPIC 3: SPECIAL NEEDS IN MATHEMATICS EDUCATION Effective Teaching Skills for Special Needs Teaching and Learning Strategies for Special Education Needs Classroom management

Time management and transitions

Presentation of materials

Assessment, grading and testing

Behaviour

Enrichment Activities A learning activity which is interesting, challenging

and provide opportunities for students to develop and support curious properties, self-reliance, critical and creativity thinking skills.

Carried out after the students master basic skills to enriching their mathematical skills.

Usually the task given is more challenging and out of ordinary thinking in the class.

Purposes: o Apply basic mathematical skills to solve problems

(complex and challenging) o Developing trends and talent o Nurturing interests o Strengthen the concepts and skills o Promoting self-reliance

Seat student in area free from distractions

Eliminate all unnecessary materials from student desk to reduce distractions.Keep an extra supply of pencils, pens, books and paper in the classroom.

You may have to allow the student frequent breaks.

Have an agreed upon cue for student to leave the classroom.

Reduce visual distractions in the classroom.

Space short work periods with breaks

Provide additional time to complete assignment.

Allow extra time for homework completion.Inform student with several reminders, several minutes apart, before changing from one activity to the next.

Reduce amount of work from usual assignment.

Provide a specific place for turning in assignments.

Modify expectations based on students’ needs.

Break assignments into segments of shorter tasks.

Provide a model of end product.

Provide written and verbal direction with visuals if possible.Number and sequence steps in a task.

Provide outlines, study guides, copies of overhead notes.Explain learning expectations to the student before beginning a lesson.Make sure you have the students’ attention before beginning a lesson.Limit the number of concepts presented at one time.

Provide incentives for beginning and completing material.

Divide test into small sections.

Grade spelling separately from content.

Allow as much time as needed to complete.

Avoid time test.

Change percentage of work required for passing grade.

Permit retaking the test.

Provide monitored breaks from test.

Avoid confrontations and power struggles.

Provide an appropriate peer role model.

Modify rules that may discriminate against student

Develop a system or code that will let the student know when behaviour is not appropriate.

Ignore attention seeking behaviours that are not disruptive to the classroom.

Arrange a designated safe place that student can go to.

Develop a code of conduct for the classroom and visually display it in an appropriate place where all students can see it, review it frequently.

Develop a behaviour intervention plan that is realistic and easily applied.

Provide immediate reinforces and feedback.



Way to do enrichment activity:

Among students With teachers o Suitable for moderate

or advance student o Suitable for all

o Informal o Formal o Teacher as facilitator o Teacher fully conduct

the activity o Occur interaction

among students o Occur interaction

among teacher and students

Guide to select materials:

o Help pupils to do self-learning (individual / group) o Links with the mathematical skills they have

learned o Can help reinforce math skills o Can broaden knowledge o Interesting, challenging and relevant to the their

ability of interests and talents o Suitable time allocation

Form of enrichment activity: o Math games o Puzzle o Project o Competition o Experiments o Worksheet

Steps to execute the enrichment activity

Example:

Remedial Activities Remediation is a plan to help students with learning

difficulties in acquiring a skill by using alternatives approach, teaching materials and activities.

Sign of students who have difficulties: o Repeating the mistakes o Always make error in calculation o Unable to give an explanation when asked o Have difficulty solving the problem solving

questions o Showed a negative attitude o Low achievement in test

Steps in remediation:

Factors of learning difficulties:

o Learning environment o Students' lives and personal (family background) o Lack of intelligence o Physical factors

Enrichment

Horizontal enrichment (based on ability of

students)

Vertical enrichment (based on students' interest and talent)

Teach the basic mathematics skills

Give instruction on how to do the activity

Give suitable time to student to finish up the activity

Teacher monitor students work, help student which face any problem

Evaluation of students' work

Apply the value, award for students that can execute the activity well

Identify students with difficulty (screening test)

Analyze the problem (diagnostic test)

Planning and execution of remedial activity

Evaluation from last activity

Follow up action



Other Types of Learning Disabilities Dyslexia Meaning

o Learning disability that impairs a person's ability to read, and which can manifest itself as a difficulty with phonological awareness, phonological decoding, orthographic coding, auditory short – term memory and / or rapid naming.

Symptoms o Noticeable difference between the pupil's ability

and their actual achievement o Family history of learning difficulties o Difficulties with spelling o Confusion over left and right o Writing letters or numbers backwards o Difficulties with mathematics o Difficulties with organizing themselves

E.g.:

Dyspraxia Meaning

o Difficulties with movement and coordination. o It is also known as "motor learning disability". o Somebody with dyspraxia finds it hard to carry

out smooth and coordinated movements. Symptoms

o Early childhood Many will have difficulties getting dressed. Problems carrying out playground movements

(jumping, playing hopscotch, catching a ball, kicking a ball, hopping, and skipping)

Problems with classroom movements, such as using scissors, coloring, drawing, playing jig-saw games.

Problems processing thoughts. Difficulties with concentration (hard to focus

on one thing for long period) o Pre-school Finds it hard to keep friends Hesitates in most actions, seems slow Does not hold a pencil with a good grip Such concepts as in, out, in front of are hard

to handle automatically

o Later on childhood Tries to avoid sports and physical education Learns well on a one-on-one basis, but

nowhere near as well in class with other kids around

Reacts to all stimuli equally Mathematics and writing are difficult Spends a long time getting writing done Does not follow instructions

Treatment o Occupational therapy Observe how the child manages with

everyday functions both at home and at school.

Then help the child develop skills specific to activities which may be troublesome.

o Speech and language therapy Carry out an assessment of the child's speech Then help him / her communicate more

effectively. o Perceptual motor training Involves improving the child's language,

visual, movement, and auditory skills. A series of tasks, which gradually becoming

more advanced, are set - the aim is to challenge the child so that he/she improves, but not so much that it becomes frustrating or stressful.

Dyscalculia Meaning

o Specific learning disability or difficulty involving inborn difficulty in learning or comprehending mathematics (math disability).

o It is akin to dyslexia and can include confusion about math symbols.

o Dyscalculia can also occur as the result of some types of brain injury

Symptoms o Confusing signs (+, -, /, *) o Reversing and/or transposing numbers o Reliance on counting strategies rather than

“knowing” o Difficulty with mental arithmetic and

measurements o Disability to grasp and remember math rules /

formulas Treatment

o No therapy has been properly verified and proven to be effective

o Anecdotal evidence also suggests dyscalculia individuals might pursue such systems out of need or interest.

o Software intended to remediate dyscalculia has been developed.



E.g.:

Dysphasia Meaning

o A language disorder, where the person experiences an impairment of power of speech, writing or signs and of comprehension of spoken or written language.

Symptoms o Having difficulty in talking, understanding,

listening and writing on his language o Difficulty in doing numeral calculations (writing)

and word problems in Math o Difficulty in comprehension – reading

comprehension, o Difficulty in the usage of language – abstractions

(verbal), semantic meaning, complex grammatical structure and linguistic function

Treatment o Speech therapy – to improve their

communication. o Talking slowly and repeating things o Using gestures or drawings o Avoiding noisy areas o Need a lot of emotional support.

TOPIC 4: APPLICATIONS OF TECHNOLOGY IN ASSESSMENT ICT in Assessment The potential for the use of new technologies in

education was recognized from the earliest days of interactive computing, and courseware (computer based learning materials) usually includes some self-assessment for formative purposes.

The examinations bodies can use only the simplest forms of ICT based assessment (objective or multiple choice tests) until learners are familiar with newer styles.

Roles for ICT in assessment: o Improve efficiency and effectiveness in

assessment o Systematically manage the data of assessment o Opportunities in association with new styles of

learning that using ICT o Administrate the test with easy and speed way.

Item Construction Item Construction The usage of specific software can help teacher

build a good item. o Word processing software Teacher type the questions / item easily and

set up the margin and paging. E.g.: Microsoft Office Word

ICT in Assessment

Items Construction

Items Analysis



o Mathematical diagram sketch software Teacher can draft / sketch mathematical

diagram easily in this software. E.g.: Geometer’s Sketchpad

o Data and Graphical sketch software Teacher can key in data and produce the

graph easily. E.g.: Microsoft Office Excel

Items Construction using Hot Potatoes The Hot Potatoes is a web-based software suite that

includes six applications, enabling you to create interactive multiple-choice, short-answer, jumbled-sentence, crossword, matching/ordering and gap-fill exercises for the World Wide Web (online).

But it also can be used in offline mode. Hot Potatoes is freeware, and you may use it for any

purpose or project you like. It is not open-source (e.g: Linux, Mozilla Firefox).

Can be downloaded it’s homepage from: http://hotpot.uvic.ca/

Applications in Hot Potatoes:

Jclose – Create fill-in-the-blanks exercise

Jquiz – Create multiple-choices or short-answer quiz

JCross – Create a crossword puzzle

Jmix – Create a jumbled-sentence exercise

Jmatch – Create a matching exercise

The Masher – Build a sequence of exercises into a complete indexed unit

Item Analysis Item analysis is the process for determining the

items that was built achieving a standard or a set criterion.

Rasch Measurement Model (George Rasch, 1960): o Assumptions: The probability of successful

candidates to answer an item is dependent on the ability of the candidate and the difficulty of the item.

o Calibration and item difficulty measure the ability to place candidates on a linear scale.

o Feature allows the comparison is made between the ability of the candidate and the difficulty of the item.

o Items found to be suitable (fit) will be stored in the Item Bank

o Rasch model is a mathematical formula based on the following formula:

푷 = 풇(풃 − 풅) where 푃 = probability

푓 = function 푏 = the ability candidates 푑 = difficulty of item

Unit for the difference (b - d) is in logits (the probability of expression)



Item analysis software o QUEST o ConQUEST o RUMM o BigSteps o WinSteps

Item Analysis using Quest QUEST is item analysis software developed by the

Australian Centre for Educational Research (ACER) from Australia.

Interface of Quest:

Advantages:

o Correcting item – identify items such errors with have no answer, two answers and wrong answers

o Candidate diagnostic information – items that are difficult (not mastered), distractor that was chosen (misconception) and identify students with difficulty

o Improving the quality of testing and assessment – for building a better item and maintaining documentation of the statistical assessment instruments

Useful for the analysis of items: o Objective test o Subjective tests o Rating scales

QUEST analysis gives: o Item estimation o Cases estimation o Fit statistics in table form

Estimation is reported in units of Logits. Fit statistics reported in the Mean Square Residuals. To implement the estimation and output, QUEST

only need: o Control files o Data files

Analyze QUEST output: o Please refer to appendix.

Prepared by: Cg Mohd Ridzuan al-Kindy™

Mohd Ridzuan bin Mohd Taib (Facebook - Cg Mohd Ridzuan al-Kindy)

http://jilmuallim.blogspot.com PISMP Mathematics Semester 6 IPG Kampus Dato’ Razali Ismail.

©Copyright 2010

NOTA MTE3112 (EGLISH VERSION)

Documents

Transcript of NOTA MTE3112 (EGLISH VERSION)