A View from Highland on the use of CEM Data

Peter FinlaysonQuality improvement Officer

February 2013

1. How CEM data might help inform the Transition process

2. Some knowledge of pupil feedback information from SOSCA

3. Pupil Tracking-how SOSCA helps4. Where CEM Data might fit in a school’s

self evaluation5. Where added value can be identified6. Self Evaluation for the Authority

Ideas to take away from today

Why Highland are using CEM data

29 Secondary schools 183 Primary Schools All Primary schools use InCAS in P3,P5 and

P7 All secondary schools use SOSCA in S2 A few use MidYIS in S1 Financial Commitment!!

Highland Background

• Improve outcomes for Young People

• Teachers still developing Assessment Standards

• Qualifications will still be a major part of a pupil’s portfolio of achievement

• Qualifications will be norm referenced• CEM Background

Why are Highland using Standardised Testing(CEM)?

How do we know how well pupils are progressing

Are they achieving their potential?

Broad General Educationand

Senior Phase

Key Questions

About Assessment for Excellence (AfE)

· AfE is a suite of assessments that provides information about pupils' attainment and attitudes, complementing teachers' assessments of pupils' progress through CfE.

· CEM's AfE tests and feedback software enable a personalised approach to assessment in a format that pupils can relate to.

· AfE can ease assessment processes and aims to provide a stable and long-term approach to school evaluation.

· CEM assesses more than one million pupils every year and has a wealth of experience in providing accurate, reliable information. This information can help inform teaching and management decisions and can really make a difference to the outcomes of the child.

· CEM's AfE produces data which is nationally representative and have year-on-year comparisons so teachers can compare students' performance against established norms.

Key Principles

"It comes down to the quality of what happens in the classroom, putting children at the centre. If we know a child can attain better, how can we help? We need to become interventionists.“Head of Education for Fife, Craig Munro

"The big question for many people is, 'How do we translate this data into improving learning?' We (CEM) offer research-based advice on how to help children succeed in the classroom and we give detailed feedback which the teacher can use alongside their detailed professional knowledge of each pupil.“Director of Research and Development for CEM, Christine Merrell

"We don't want assessments that don't support learning. We want to promote improvements in learning, teaching and schools' performance.“Ken Greer, Executive Director of Education for Fife Council

The information provided through CEM assessments can help teachers monitor pupils' progress, set targets, facilitate school improvement and inform the teaching and learning process.

Transition

All primary schools in Highland use InCAS All Primary schools use the Highland SPP

Approach Professional Judgement backed up by InCAS

data 5-14 on its own was flawed!! All Secondary schools receive the AfE data

from Primary as Standardised Scores through Phoenix e1

Information from Primary Schools

Moving Forward with Assessment

The Personal Progress Scale

1 – is consistently achieving his/her potential

2 – is consistently achieving success but has the potential to achieve more

3 – is only sometimes achieving potential

4 – is yet to achieve potential

Moving Forward with Assessment

The Performance Scale

A - is consistently performing beyond the expected standard for his/her stage

B - is consistently performing at the expected standard for his/her stage

C - is sometimes performing at the expected standard for his/her stage

D - is not yet performing at the expected standard for his/her stage

SOSCA Feedback• Assessed by a computer adaptive test

• School and students’ test performances are compared to a nationally representative sample

Assessment Feedback: Pupil scores and records, and school Band Profiles

‘Predictions’ Feedback: To S4 qualifications (currently Standard Grade and will for

National 4 and 5) based on overall curriculum score

Value-Added Feedback: For Standard Grade and will for National 4 and 5 from

curriculum baseline (SOSCA) for curriculum assessment (SOSCA) from P7 InCAS or S1 /S2

MidYIS baselines

AfE S2 Curriculum (SOSCA) Feedback

InCAS and SOSCA Data is used to inform teacher judgements around the SPP

Chances Graphs

SOSCA Predictions

0 0 0

5

28

49

17

0

5

10

15

20

25

30

35

40

45

50

55

7 6 5 4 3 2 1

Per

cen

t

Grade/Level

Individual Chances Graph for student 3- Standard Grade EnglishOverall AfE Score 111, Band A

Predictions: Chances Graphs49% chance of a grade 2 – the most likely single grade. 51% chance of a different grade

Chances Graphs based on Pupil’s AfE Test Score

‘Predictions’…...are based on

Average performance by similar students in past examinations

So the term ‘prediction’ should not be used. Alternatives could include:

‘expected grade’: from the table of data

‘most likely grade’: from the chances graphs

Reliability of ‘predictions’?

• School factors inc. catchment area

• How seriously the pupil took the assessment.

“We have used them for target setting in S3.”

“We do 'working at grades' twice a year and this year we used SOSCA as a guide to what they should be working at.” “I have used them to compare results and predictions in the department.”

“All HOD's have been trained on the use of SOSCA.” “All departments are aware of the information and should know what the information means (!) More time was spent with Guidance, LS, English, Maths and Science going over the data. Guidance staff have all printed the IPRs, they have been placed in the pupil files and refer to them regularly. ”

“Learning Support and English make regular use of the information. ”  

Quotes from Highland Schools

Important role for the Key Adult?

Tracking of attainment-one aspect Individual Pupil record from SOSCA? Course Choice at end of S2? Course Choice at end of S3?

Personal Support

STACS –comparator schools SOSCA Tables Band Profile Graphs Value Added Data

School Self Evaluation

Standardised Scores

Mean Score: 100

Standard Deviation: 15

Sort Data By...

School A

School B

AfE S2 Curriculum based Standardised Feedback 2012

Scores Standardised On AfE Curriculum based Sample 2012

St.

Sco

re

Ban

d

St.

Sco

re

Ban

d

St.

Sco

re

Ban

d

St.

Sco

re

Ban

d

St.

Sco

re

Ban

d

St.

Sco

re

Ban

d

St.

Sco

re

Ban

d

St.

Sco

re

Ban

d

St.

Sco

re

Ban

d

St.

Sco

re

Ban

d

St.

Sco

re

Ban

d

St.

Sco

re

Ban

d

Average97.7 97.0 98.7 97.3 91.3 93.4 94.9 92.2 94.0 98.0 94.0 94.9

St. error 1.3 1.3 1.3 1.3 1.3 1.4 1.3 1.4 1.3 1.3 1.3 1.3

Tex

t C

om

pre

hen

sio

n

Pas

sag

e C

om

pre

hen

sio

n

Ove

rall

Rea

din

g

Nu

mb

er,

Mo

ney

&

Mea

sure

men

t

Info

rmat

ion

H

and

lin

g

Reading Mathematics Science

Sh

ape,

Po

siti

on

&

Mo

vem

ent

Ove

rall

M

ath

emat

ics

Liv

ing

Th

ing

s &

th

e P

roce

sses

of

Lif

e

Ear

th &

Sp

ace

En

erg

y &

Fo

rces

Ove

rall

Sci

ence

Sp

eed

Rea

din

g

Average103.7 101.1 100.7 102.2 103.1 106.2 101.1 103.7 101.5 105.0 103.2 103.8

St. error 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0

School A

34

24 2319

0

10

20

30

40

50

60

70

80

90

100

D C B A

Per

cen

t

Band

Overall Reading 2012: Percentage of Pupils in each Band

44

2318 15

0

10

20

30

40

50

60

70

80

90

100

D C B A

Per

cen

t

Band

Overall Mathematics 2012: Percentage of Pupils in each Band

35

2229

15

0

10

20

30

40

50

60

70

80

90

100

D C B A

Per

cen

t

Band

Overall Science 2012: Percentage of Pupils in each Band

School B

16

30 2925

0

10

20

30

40

50

60

70

80

90

100

D C B A

Per

cen

t

Band

Overall Reading 2012: Percentage of Pupils in each Band

1722 23

38

0

10

20

30

40

50

60

70

80

90

100

D C B A

Per

cen

t

Band

Overall Mathematics 2012: Percentage of Pupils in each Band

1722 24

37

0

10

20

30

40

50

60

70

80

90

100

D C B A

Per

cen

t

Band

Overall Science 2012: Percentage of Pupils in each Band

0.2

1

0.2

1

0.2

1

-0.5

-0.4

-0.3

-0.2

-0.1

0

0.1

0.2

0.3

0.4

0.5

0.28

Average Standardised Residuals MidYIS Y8/S1 to SOSCA Value-Added 2012

Overall Mathematics Standardised for Nationally Representative Sample

Av

era

ge

Sta

nd

ard

ise

d R

es

idu

al

0.2

1

0.2

1

0.2

1

-0.5

-0.4

-0.3

-0.2

-0.1

0

0.1

0.2

0.3

0.4

0.5

0.17

Average Standardised Residuals MidYIS Y8/S1 to SOSCA Value-Added 2012

Overall Reading Standardised for Nationally Representative Sample

Av

era

ge

Sta

nd

ard

ise

d R

es

idu

al

Value Added Data Added Data

Authority View

School A and School B Inspection reports Attainment Data SIMD Data

Highland Example

By the end of S2, pupils were making very good progress andattaining well across the Curriculum..Most pupils were attaining appropriate levels in mathematics,reading and writing.By the end of S4 , the proportion of pupils attaining five or moreAwards at SCQF levels 4 and 5 was well above national averages, and in line with schools with similar characteristics.

School A 2008

Across the curriculum, young people achieve well in S1/S2The majority achieve appropriate national standards by the end of S2In recent years, the success of young people in S4, in national examinations, has generally been in line with the national average. It is stronger than in schools which serve young people with similar needs and backgrounds

School B 2009

Centre nameNo. pupils in known datazones

No. pupils from 15% most deprived datazones

No. pupils from 20% most deprived datazones

% pupils from 15% most deprived datazones

% pupils from 20% most deprived datazones

School B 427 35 81 8.2% 19.0%School A 692 0 0 0.0% 0.0%

5119

332

5119

634

0.5 1.5 2.5 3.5 4.5

30

40

50

60

70

80

90

100

110

120

130

140

150

160

170

103.1

AfE S2 Curriculum based Baseline 2011/2012Overall AfE Scores Standardised for AfE S2 Sample

Selected Schools

Cohort Mean Consortium MeanOutliers

School B

SOSCA Information

School A

Percentages and NCDs based on S4 Roll for passes achieved by end of S4

All Candidates

 

School A   

School B

  5+ Level 4 5+ Level 5   5+ Level 4 5+ Level 5

  % NCD % NCD   % NCD % NCD

2012 92 2 55 2 2012 82 5 30 7

2011 95 1 55 2 2011 72 8 22 9

2010 91 2 55 2 2010 85 4 29 7

2009 93 1 54 1 2009 84 4 32 6

2008 90 2 48 2 2008 85 3 31 6