1 Annual Measurable Achievement Objectives (AMAOs) June 2009 State of Maine Title III...

1

Annual Measurable Achievement Objectives

(AMAOs)

June 2009

State of MaineTitle III Accountability

2

The No Child Left Behind Act (NCLB) waspassed in 2001 and requires the following: Establishment of ELP (English Language

Proficiency) standards; Annual assessment of English language

proficiency; Definition of Annual Measurable Achievement

Objectives (AMAOs) and measurement reporting; and

Accountability standards for meeting AMAOs.

No Child Left Behind andTitle III Accountability

3

Basic Title III FrameworkMaine Title III

Accountability System

StateConsortiaDistricts

Title IIIPerformance Indicators

AMAO IMaking Progress

AMAO IIAttaining

Proficiency

AMAO IIIAYP

Determination

4

State-defined English language proficiency targets Based on state English language proficiency standards

and baseline data

Evaluates effectiveness of English language instruction

AMAO I Making progress in proficiencyAMAO II Attaining proficiencyAMAO III Adequate Yearly Progress (AYP) for ELLs

Title III Performance IndicatorsAnnual Measurable Achievement Objectives

(AMAOs)

5

AMAO I – Making ProgressDefinition, Conditions, Key Variables,

Decision Logic and Example

6

AMAO that evaluates annual increases in the number or percentage of children making progress in learning English (P.L. 107-110, Title III, Part A,

Subpart 2, §3122(a)(3)(A)(i)).

Translation:Are ELLs progressing towards proficiency?

AMAO IEnglish Language Proficiency

Making Progress

Definition

7

Weighting system adjusts for how long students have participated in an English as a Second Language (ESL) program in Maine.

Improvement rate fixed at 9.17 index points per year(equates to an increase of 7.37 scaled score points).

Two years of data are required for calculation. Greater points awarded for greater improvement in

fewer years.


Making Progress

Conditions

8

The Duration is the number of years a student receives ESL services in a Maine school district.

Code Used for AMAO I

Calculation

Length in a Maine ESL Program

Category

0 0 – 2 years Short-term

1 3 – 4 years Typical

2 5 or more years Long-term

3 Data missing or invalid

Unknown

AMAO IKey Variable - Duration

9

The Comp PL is the performance level achieved on the ACCESS for ELLs® (Assessing Comprehension and Communication in English State-to-State for English Language Learners).Code Used for AMAO I

CalculationProficiency Status

6 Reaching

5 Bridging

4 Expanding

3 Developing

2 Beginning

1 Entering

AMAO IKey Variable – Comp PL

10

The CompIndex variable is derived from the Duration and Comp PL variables.

The Duration and Comp PL variables are concatenated (linked side by side). This creates the CONCAT variable.

The CONCAT variable is then matched and assigned the corresponding CompIndex score from the value table.

AMAO IKey Variable – CompIndex

11

Value Table 0 - 2 Years in Maine ESL ProgramDuration Comp PL CONCAT CompIndex

0 0.0 00 00 1.0 01 250 1.5 01.5 500 2.0 02 750 2.5 02.5 1000 3.0 03 1250 3.5 03.5 1500 4.0 04 1750 4.5 04.5 2000 5.0 05 2250 6.0 06 250


12

Value Table 3 - 4 Years in Maine ESL ProgramDuration Comp PL CONCAT CompIndex

1 0.0 10 01 1.0 11 01 1.5 11.5 251 2.0 12 501 2.5 12.5 751 3.0 13 1001 3.5 13.5 1251 4.0 14* 1501 4.5 14.5 1751 5.0 15 2001 6.0 16 225

*Used in CompIndex example.


13

Value Table 5+ Years in Maine ESL ProgramDuration Comp PL CONCAT CompIndex

2 0.0 20 02 1.0 21 02 1.5 21.5 02 2.0 22 252 2.5 22.5 502 3.0 23 752 3.5 23.5 1002 4.0 24 1252 4.5 24.5 1502 5.0 25 1752 6.0 26 200


14

Value Table Unknown (Invalid or Missing Data)Duration Comp PL CONCAT CompIndex

3 0.0 30 03 1.0 31 03 1.5 31.5 03 2.0 32 253 2.5 32.5 503 3.0 33 753 3.5 33.5 1003 4.0 34 1253 4.5 34.5 1503 5.0 35 1753 6.0 36 200


15

A student who received ESL services in a Maine school district for three years (Duration=1) obtained a proficiency level of 4 (four) on the ACCESS for ELLs® (Comp PL=4)

The student has a CONCAT value of 14 (by joining the Duration and the Comp PL) and is assigned the corresponding CompIndex points from the value table.

AMAO IKey Variable – CompIndex - Example

15014Expanding

Calculation

4

3 - 4 years

Calculation

1

CompIndex(From ValueTable)

CONCAT Value

Proficiency

Status

Comp PL

Code Used

for AMAO I

Length in ESL

Program

DurationCode

Used for AMAO I

16

Step 1 – Minimum n-count – If there are less than 20 students, AMAO I is MET.

Step 2 – If Step 1 is NOT MET, do the following: Calculate the prior year average CompIndex for all ELL

students. Subtract it from the current year average CompIndex for all ELL students. If the difference is greater than or equal to 9.17 index points,

AMAO I is MET. Process continues


Making Progress

Decision Logic

17

Step 3 – If Steps 1 and 2 are NOT MET, do the following: Calculate the prior two years CompIndex weighted

average for each student (add year one and year two CompIndex points, then divide by the prior two years n-count).

Subtract this from the current year’s CompIndex. If the difference is greater than or equal to 9.17 index

points, AMAO I is MET.

Process continues


Making ProgressDecision Logic

18

Step 4 – If Steps 1, 2, and 3 are NOT MET, do the following: Subtract the current year average of the CompIndex upper limit

(established by the 95% Confidence Interval) from the prior year average.

If the difference is greater than or equal to 9.17 index points, AMAO I is MET.

• Confidence Intervals adjust for sampling error. A 95% Confidence Interval uses a fixed value of 1.96 to make this adjustment.

• The equation pertaining to AMAO I is shown below: Current Year Average CompIndex + 1.96 (Current Year Standard Deviation CompIndex/Square Root of the Current Year N-Count)

Process continues


Making ProgressDecision Logic

19

Step 5 – If the SAU/consortia did not receive a MET in any of the steps, AMAO I is NOT MET.


Making Progress

Decision Logic

20

CalculateComposite

Index

>9.17 Change Threshold

Cell Failed

Data Validation Procedures

Missing Data Point

95%CI

2-year average > 9.17 Change

Threshold

Yes

Processing Path

No

No

No

Yes

Yes

AMAO I Decision Logic

No

No

Yes

Processing Path

N-count< 20

Cell Passed

Yes

21

According to ABC district’s data, did it meet AMAO I in 2009?

2009 2008 2007

# of ELLs / CompIndex 142 121 104

Sum of CompIndex 19225 16275 14125

Average CompIndex 135.4 134.5 135.8

Standard Deviation CompIndex 57.3 62.6 59.2

AMAO IDetermination Example

22

Step 1 - Minimum n-count – If there are less than 20 students, AMAO I is MET.

ABC district’s calculation:

ABC district has 142 ELLs. This is not less than 20, so AMAO I is NOT MET for Step 1. Proceed to Step 2.

AMAO IDetermination Example – Step 1

23

Step 2 - If Step 1 is NOT MET, do the following: Calculate the prior year average CompIndex. Subtract it from the current year average CompIndex. If the difference is greater than or equal to 9.17 index

points, AMAO I is MET.


135.4 - 134.5 = 0.9

0.9 is not greater than or equal to 9.17, so Step 2 is

NOT MET. Proceed to Step 3.


24

Step 3 - If Steps 1 and 2 are NOT MET, do the following: Calculate the prior two years CompIndex weighted average (add

year one and year two CompIndex points, then divide by the sum of the prior two yearsn-count).

Subtract this from the current year average CompIndex. If the difference is greater than or equal to 9.17 index points,

AMAO I is MET.


(14125+16275)/(104+121)=135.11135.4 – 135.11 = 0.29

0.29 is not greater than or equal to 9.17, so Step 3 is NOT MET.

Proceed to Step 4.


25

Step 4 - If Steps 1, 2, and 3 are NOT MET, do the following: Subtract the current year average of the CompIndex upper limit

(established by the 95% Confidence Interval) from the prior year average.

If the difference is greater than or equal to 9.17 index points, AMAO I is MET.

ABC district’s calculation: Apply Confidence Interval: 135.4+1.96(57.3/√142) = 144.81 144.81-134.5=10.31

10.31 is greater than or equal to 9.17, so Step 4 is MET. AMAO I has been MET in 2009.


26

AMAO IIProficiency Status

Definition, Conditions, Targets,Key Variables, Decision Logic

and Example

27

AMAO that evaluates annual increases in the number or percentage of children attaining English proficiency by the end of each school year, as determined by a valid and reliable assessment of English proficiency consistent with Section 1111(b)(7) of NCLB (P.L. 107-110, Title III, Part A, Subpart 2, §3122(a)(3)(A)(ii)).

Translation:Are ELLs attaining proficiency?

AMAO IIEnglish Language Proficiency Status

Definition

28

Threshold values established for the baseline year (6.67%) and increased by 2.87% each subsequent year.Starting point was determined by using weighted percent

proficient for a three-year period (2005-2007). Districts were rank-ordered by percent proficient, and the district at the 20th percentile enrollment was used for baseline.

Based on projection exceeding 30.0% in 2014.

AMAO IIEnglish Language Proficiency Status

Conditions

29

AMAO II English Language Proficiency Status

Short-Term Targets

30

FEP5_6 is defined as a dichotomously transformed variable reflecting data cells with ACCESS for ELLs® proficiency levels of five or higher.Code Used for

AMAO II Calculation(FEP5_6)

Code Used for AMAO I Calculation

(Comp PL)

Proficiency Status

1 6 Reaching

1 5 Bridging

0 4 Expanding

0 3 Developing

0 2 Beginning

0 1 Entering

AMAO II Key Variable – FEP5_6

31

Step 1 – Minimum n-count – If there are less than 20 students, AMAO II is MET.

Step 2 – If Step 1 is NOT MET, do the following: Calculate the sum of FEP5_6 for the current year. Divide by the # of ELLs with FEP5_6 for the current year. If the quotient is greater than or equal to the annual

threshold (18.2% in 2009), AMAO II is MET.

Process continues


Decision Logic

32

Step 3 – If Steps 1 and 2 are NOT MET, do the following: Calculate the current and prior year sum of FEP5_6 and

divide by the prior two years of the number of FEP5_6s.

If the result is greater than or equal to the annual threshold (18.2% in 2009), AMAO II is MET.

Process continues


Decision Logic

33

Step 4 – If Steps 1, 2, and 3 are NOT MET, do the following: Calculate the sum of FEP5_6 for the current year. Divide by the count of all ELLs. If the upper limit (established by the 95% Confidence Interval) is

greater than or equal to the threshold (18.2% in 2009), AMAO II is MET.

Confidence Intervals adjust for sampling error. A 95% Confidence Interval uses a fixed value of 1.96 to make this adjustment.

The equation pertaining to AMAO II is shown below:Current Year FEP5_6% + 1.96(Square Root(Current Year FEP5_6%*(1-

Current Year FEP5_6%)/Current Year N-Count))Process continues


Decision Logic

34

Step 5 – If the SAU/consortia did not receive a MET in any of the steps, AMAO II is NOT MET.


Decision Logic

35

Calculate %FEP 5_6

> Annual Threshold

Cell Failed

Data Validation Procedures

Missing Data Point

95%CI

2-year average > Annual Threshold

Yes

Processing Path

No

No

No

Yes

Yes

AMAO II Decision Logic

No

No

Yes

Processing Path

N-count< 20

Cell Passed

Yes

2005 Annual Target = 6.7%2006 Annual Target = 9.5%2007 Annual Target = 12.4%2008 Annual Target = 15.3%2009 Annual Target = 18.2%

36

According to ABC district’s data, did it meet AMAO II in 2009?

2009 2008

Count of FEP5_6 91 94

Sum of FEP5_6 10 5

AMAO IIDetermination Example

37

Step 1 - Minimum n-count – If there are less than 20 students, AMAO II is MET.


ABC district has 91 ELLs. This is not less than 20, so AMAO II is NOT MET for Step 1. Proceed to Step 2.

AMAO IIDetermination Example – Step 1

38

Step 2 - If Step 1 is NOT MET, do the following: Divide the sum of FEP5_6 for the current year by the count of

all students.

If the quotient is greater than or equal to the annual threshold (18.2% in 2009), AMAO II is MET.


10/91= 11%

11% is not greater than or equal to 18.2%, so Step 2 is NOT MET. Proceed to Step 3.


39

Step 3 - If Steps 1 and 2 are NOT MET, do the following:Add the current and prior year sum of FEP5_6 and divide

by the total of the current and prior year count of FEP5_6s.

If the result is greater than or equal to the annual threshold (18.2% in 2009), AMAO II is MET.


(10+5)/(91+94)= 8%

8% is not greater than or equal to 18.2%, so Step 3 is NOT MET.

Proceed to Step 4.


40

Step 4 - If Steps 1, 2, and 3 are NOT MET, do the following: Divide the current year sum of FEP5_6 by the count of all

ELLs. If the upper limit (established by the 95% Confidence

Interval) is greater than or equal to the threshold (18.2% in 2009), AMAO II is MET.

ABC district’s calculation: 10/91=11%

Apply Confidence Interval: 11%+1.96(Square Root (11%*(1-11%)/91))= 17.4%

17.4% is not greater than or equal to 18.2%, so Step 4 is NOT MET.

AMAO II has NOT been MET in 2009.


41

AMAO III - AYPDefinitions, Conditions, Targets and Decision Logic

FINAL ACCOUNTABILITYDETERMINATIONS

Decision Logic

42

AMAO that measures adequate yearly progress for limited English proficient children (LEP/ELL) as described in Section 1111(b)(2)(B) of NCLB (P.L. 107-110, Title III, Part A, Subpart 2, §3122(a)(3)(A)(iii)).

Translation:Are ELLs making AYP?

AMAO IIIAYP Status for ELLs

Definitions

43

Adequate Yearly Progress (AYP) – The accountability status of a school, district/SAU, and the state as a whole based on federally mandated performance measures.

AMAO IIIAYP

Definitions

44

Participation Rate – Proportion of students who provide at least one valid answer on a test booklet divided by the number of students who were enrolled at the particular grade level, less those receiving “special consideration” status.

Proficiency Rate – The number of students who scored a 3 achievement level or higher on the appropriate state academic assessment divided by those who were eligible to be tested.

AMAO IIIAYP

Definitions

45

Schools attain AYP if the participation, performance, and OAI targets are met for the given year. They do not attain if any indicator is not met for any subgroup that has satisfied the minimum n-count requirements.

Districts/SAUs attain AYP if the participation, performance, and OAI targets are met for at least one of the three grade clusters (elementary, middle, or high school). They do not attain if any one indicator is not met at the elementary, middle, or high school level.

AMAO IIIAYP Attainment

Conditions

46

AYP data for reading and mathematics used to determine if district made AYP exclusively because of ELL subgroup.

Consortia receive the AYP status of member district/SAUs.

If any member does not make AYP for given year, consortium deemed as missing AMAO III.

AMAO IIIAYP Status for ELLs

Conditions

47

Year Grades 3-8% Proficient

High School% Proficient

Grades 3-8% Proficient

High School% Proficient

Reading Mathematics

2006 50% 50% 40% 20%

2007 50% 50% 40% 20%

2008 50% 57% 40% 31%

2009 58% 64% 43% 43%

2010 66% 71% 55% 54%

2011 75% 78% 66% 66%

2012 83% 86% 77% 77%

2013 92% 93% 89% 89%

2014 100% 100% 100% 100%

AMAO IIIAYP Attainment

Short-Term Targets

49

Final Title III AccountabilityDeterminations

Decision Logic

50

MetAMAO I

AMAO I

MetNot Met

No Yes

Yes

Yes

No

Final Title III Determinations

No

AMAO II AMAO III

MetAMAO II

MetAMAO III

51

Thank you

Nancy MullinsDirector

ESL/Bilingual ProgramMaine Department of Education

23 State House StationAugusta, ME 04333

Phone: (207) 624-6788Fax: (207) 624-6789

[email protected]

1 Annual Measurable Achievement Objectives (AMAOs) June 2009 State of Maine Title III...

Documents

Transcript of 1 Annual Measurable Achievement Objectives (AMAOs) June 2009 State of Maine Title III...