Arkansas’ K-12 Achievement & NSLA Funding

Arkansas Categorical Poverty Funding System (NSLA)

Arkansas K-12 Achievement & NSLA Funding

September 4, 2013AAEA

1

OEP is a research center within the College of Education and Health Professions at the University of Arkansas that specializes in Education Research and Policy.

Officeforeducationpolicy.org

AR Education Reports Policy Briefs Report Cards Newsletters Data Resources

2Office for Educational Policy 2Refer to menu bar at the top left of the OEP homepage. http://www.officeforeducationpolicy.org/

OEP HomepageClick on Arkansas School DataAccessing Data Resources through the OEPArkansas School Data has multiple databases at both school and district levels.Arkansas School Data

33Accessing Report Cards, Education Reports and Policy Briefs through the OEPRefer to menu bar at the top left of the OEP homepage. www.uark.edu/ua/oepOEPublications leads to options such as Report Cards, Education Reports and Policy Briefs. Remember to sign up for our weekly e-mail, OEP Web Links (OWL), to get updated on current education news across the state and nation. Please e-mail [email protected] to sign up.Also, sign up for the OEP Blog at www.officeforedpolicy.com to receive alerts when the latest OEP Blog posts are published.

4OEP Outreach We at the OEP believe that teacher quality is important and that all Arkansas classrooms should be lead by a qualified teacher.

The Arkansas Teacher Corps (ATC) program is a collaborative partnership between the University of Arkansas, school districts, and local community organizations that aims to provide an accelerated path to teaching for the highest-performing and most talented individuals to have a lasting impact on students and communities in Arkansas.

arkansasteachercorps.org

5OutlineOverall Achievement: Are we 5th or 49th? BenchmarkNAEPThe NSLA Funding QuestionHas NSLA funding produced gains for FRL students?How have districts spent NSLA funding?3. Our Recommendations for NSLA Funding 6Overall AR Achievement:

How was Arkansas performance on the Benchmark and End-of-Course Exams in 2012-13? Over time?7Benchmark PerformanceGrowth over time, until slight decrease in 2012-13 in literacy and mathSlight decrease can be attributed to many factors, including ceiling effects and CCSS implementation dipGrade-level trends: lower grades perform at higher levels than upper grades

Benchmark, Grade 3 8, % Proficient/Advanced, Over time88Benchmark Performance, By Region

Literacy Benchmark, Grades 3-8

Math Benchmark, Grades 3-8

Higher-performing regions: Northwest and Northeast9EOC PerformanceIn 2012-13, slight decreases in Algebra & Geometry scoresSteady increases in Grade 11 Literacy and Biology scores over time

10How was Arkansas performance on the NAEP in 2011? Over time?11NAEPNational Assessment of Education Progress Nations Report CardAdministered to random sample of 4th and 8th grade studentsMost recent data from 2011New 2013 NAEP data to be released this fall12NAEP Math, 2011Grade 4Grade 4 in math: Slightly below national average13The 2011 NAEP results for math indicate that Arkansas students are below the national average in math performance, as are most of the students in the region. The bars in Figure 2 display the percentage of students mean scale score for each state on the math assessment for the nation, Arkansas, and the region. The brown trend line shows the percentage of students scoring proficient or advanced in math. Arkansas students are below the national average in percent proficient or advanced in both Grade 4 and Grade 8 Math. Arkansas students ranked 34th out of 51 for Grade 4 and 37th out of 51 for Grade 8 based on the mean scale score. For both grades, this is a drop in ranking. When compared to performance in the region, Arkansas students in Grade 4 and Grade 8 are performing at higher levels than students in Louisiana, Mississippi, Oklahoma, and Tennessee; however, students in Missouri and Texas outperformed Arkansas in both grades. The relative performance of Arkansas students to their regional peers is consistent with previous trends.

13NAEP Math, 2011Grade 8Grade 8 in math: Below national average14The 2011 NAEP results for math indicate that Arkansas students are below the national average in math performance, as are most of the students in the region. The bars in Figure 2 display the percentage of students mean scale score for each state on the math assessment for the nation, Arkansas, and the region. The brown trend line shows the percentage of students scoring proficient or advanced in math. Arkansas students are below the national average in percent proficient or advanced in both Grade 4 and Grade 8 Math. Arkansas students ranked 34th out of 51 for Grade 4 and 37th out of 51 for Grade 8 based on the mean scale score. For both grades, this is a drop in ranking. When compared to performance in the region, Arkansas students in Grade 4 and Grade 8 are performing at higher levels than students in Louisiana, Mississippi, Oklahoma, and Tennessee; however, students in Missouri and Texas outperformed Arkansas in both grades. The relative performance of Arkansas students to their regional peers is consistent with previous trends.

14NAEP Reading, 2011Grade 4Grade 4 in reading: Below national average15Much like the math results, the 2007 NAEP reading results indicate that Arkansas students are below the national average in reading performance, as are most of the students in the region (with the exception of students in Missouri). Figure 3 presents the mean scale score for students in the nation, Arkansas, and the region. The line graph depicts the percentage of students scoring proficient or advanced on the 2007 reading assessment for the nation, Arkansas, and the region. Arkansas students are below the national average in both Grades 4 and Grade 8. Nationally, Grade 4 Arkansas students were ranked 37 out of 51 and Grade 8 students were ranked 42 out 51.

15NAEP Reading, 2011Grade 8Grade 8 in reading: Below national average16A review of Figure 3 reveals that Arkansas students are below the national average in both Grades 4 and Grade 8. Nationally, Grade 4 Arkansas students were ranked 37 out of 51 and Grade 8 students were ranked 42 out of 51. Compared to the region, Arkansas students are performing at higher levels than students in Louisiana and Mississippi and at levels similar to students in Oklahoma, Tennessee, and Texas. However, students in Missouri are performing at higher levels than Arkansas students. Unlike the Math scores, students in the 4th grade are not outperforming students in 8th grade in terms of percent proficient at the national level; the results are the exact same. The fourth grade students in some of the surrounding states are outperforming their eighth grade counterparts, but the opposite trend can also be observed among regional peers. There was a two percent difference in the number of students that achieved proficiency in Arkansas in the 4th grade compared to 8th grade students. Ranked out of 50 states and D.C. (Top Rank = 1)Ranked out of 50 states and D.C. (Top Rank = 1)

16NAEP Performance, 2011AR % ProfUS % ProfDiffSurrounding StatesGrade 4 Math37%39%-2%AR > TN, OKA, LA, MSGrade 4 Reading30%32%-2%AR > TN, OK, LA, MSGrade 8 Math29%34%-5%AR > OK, TN, LA, MSGrade 8 Reading28%32%-4%AR > TN, LA, MS17NAEP Performance, Over timeIn math, in grades 4 and 8, Arkansass students have decreased the gap between Arkansas and the nation on the NAEP. However, Arkansas still performs less well than the nation in math and grades 4 and 8 on the NAEP. (Closer in Grade 4)

Math, Grade 4

Math, Grade 8

18NAEP Performance, Over timeIn literacy, in grades 4 and 8, Arkansass students have decreased the gap between Arkansas and the nation on the NAEP. However, Arkansas still performs less well than the nation in literacy and grades 4 and 8 on the NAEP. (Closer in Grade 4)Reading, Grade 4

Reading, Grade 8

195th or 49th?Two stories are out there today:AR is backwards Thank goodness for Mississippi falling way behind in school qualityAR is rapidly climbing 6th in national rankings on the 2012 Quality Counts report and now 5th in 2013!! AR has better schools than in Connecticut, Florida, and Texas. Lets look at comparable data to do a fair comparison of AR scores to US totals.20NAEP Math, 2011Apples to Apples Comparisons Positive Results for ARIn Grade 4, Arkansas FRL students were slightly ahead of the nations average.In Grade 8, Arkansas FRL students were slightly below the nations average.21NAEP Reading, 2011Apples to Apples Comparisons Positive Results for ARIn Grade 4, Arkansas FRL students were slightly ahead of the nations average.In Grade 8, Arkansas FRL students were on par with the nations average.22NAEP v. Region, 2011Arkansas compares well to surrounding states and to the nation when scores are compared by poverty level.Our state suffers in the overall category because more of our students are in the low income group than in other states.23Math and Reading, Grade 4: Comparison to Region/US by IncomeFor the sake of time, lets look solely at 4th grade math scores.23Careful with these resultsWhen comparing performance of FRL students across states, it is important to keep in mind cost of living.Income level of for a family of four at ~$30,000 (free lunch threshold) looks different in Little Rock than in Los AngelesTherefore, FRL is an imperfect measure when examining poverty levels and comparing data across states. This might generate a positive BIAS for ARE.G. LR FRL = $30K ~= $22K in Seattle; thus comparing a wealthier set of AR kids to WA kids.24

NAEP: Ranking States by AchievementAchievement Measure4th Grade Math4th Grade Reading8th Grade Math8th Grade Reading2011 NAEP ScaledScore238217279259Scaled Score Rank(50 States + DC)36383943Difference Score(Achieved Expected)+2.6+2.6+2.6+0.6Difference Score Rank(50 States + DC)1411122125Above is Arkansas rank when comparing simple NAEP scores and a ranking for when each states demographics are taken into consideration (Difference Score Rank)Although Arkansas scores are lower than other states, the state as a whole does well when our demographics are taken into consideration.25OEP Similar Schools Database26Allows for comparisons to districts with similar or the same SES characteristics, including % FRL, % household bachelor degrees, median income, and district enrollment growth.

Find on our website (Officeforeducationpolicy.org), under Arkansas Schools DataBack to the Question at Hand What do we think we know so far?AR students have been improving:Benchmark and EOC growth over time (until 2012-13)But test scores generally increase with time due to test familiarity...so its important to compare AR to the USSlight NAEP overall growth over timeSlight decrease in AR/US gap in 4th grade math/readingRelates to question: Has NSLA funding for FRL students helped?

27The NSLA QuestionHow does NSLA funding work?How do we know if it works? - If it were working, what changes might we expect to see?So, what did we find about possible effectiveness?Given the uncertainty, could we have expected great gains? (How were funds used?)After all this, what would we suggest?

28NSLA: How does it work?In the 2013 Quality Counts report, Arkansas received a B+ on equity funding, ranking it as one of the top states in the nation in distributing equity funding to districts.

Arkansas should be commended for its focus on students in poverty, as the formula does channel more resources toward students in poverty, particularly those in very poor districts.

29NSLA: How does it work?Math (GPA Measure), Districts By % FRLLiteracy (GPA Measure), Districts By % FRL We know that districts with 70% or more FRL students see a drop in achievement. NSLA funding seeks to allocate more funding to those districts. 30NSLA Funding: How does it work?The tiered system creates two cliffs.Cliffs cause districts with very similar demographics to be treated differently in the funding system. For example, a district with 69% FRL receives less funding per FRL pupil than a district with 70% FRL; however, student bodies with 69% and 70% FRL look relatively similar.

31The Big Q How would we know if NSLA funding worked?We might observe Hypothesis 1: Increased scores for FRL students (relative to non-FRL students) this may be the most important!Hypothesis 2: Districts just above the cliffs performing better relative to those just below the cliffs.Hypothesis 3: Districts with influxes in NSLA funds performing better than in past.

32Hypothesis 1:FRL Students vs Non-FRL StudentsIf NSLA Funding were working, we might expect to see increase in achievement for FRL students relative to non-FRL students. 33Benchmark AchievementMath2005-06 2011-12Percentile Point GrowthNon-FRL students62nd 66th+4 FRL students40th 40th 0Literacy2005-06 2011-12Percentile Point GrowthNon-FRL students63rd 66th +3FRL students39th 43rd +4Math, 2005-06 to 2011-12Literacy, 2005-06 to 2011-12In math, the gap between FRL and non-FRL students has widened over time.In literacy, FRL students have slightly closed the gap; but FRL students still perform less well. 34NAEP Growth, 2003 to 201135Math and Reading Score Gains, 2003 to 2011Over the past decade, Arkansas scores have grown by leaps and bounds, but that statistic is padded by lower baseline scores.The greatest gains come in math and for higher-income students.Hypothesis 1:FRL Students vs Non-FRL StudentsAchievement gap between FRL and non-FRL students continues to exist.BenchmarkGap is widening in math performanceGap is slightly shrinking in literacy NAEPNon-FRL produced higher gains than FRL students over time36Hypothesis 2:Cliff DistrictsCliffs cause districts with very similar demographics to be treated differently in the funding system. For example, a district with 69% FRL receives less funding per FRL pupil than a district with 70% FRL; however, student bodies with 69% and 70% FRL look relatively similar. The cliffs allow us to compare the performance of relatively similar districts (e.g. 69% to 70%) that receive different amounts of funding. Thus, if NSLA were working, we would see greater performance for districts above the cliffs37Hypothesis 2:Cliff DistrictsBenchmark Math GPA, 2007-08 to 2012-13Benchmark Literacy GPA, 2007-08 to 2012-13On the math and literacy benchmark exams, the districts just above and below the cliff (thus, districts who are socio-economically equal) perform nearly identically.Achievement Comparisons at the 70% Cliff*3838On the math and literacy benchmark exams, districts just below the 90% cliff outperformed the districts above the cliff. Hypothesis 2:Cliff DistrictsBenchmark Math GPA, 2007-08 to 2012-13Benchmark Literacy GPA, 2007-08 to 2012-13Achievement Comparisons at the 90% Cliff*3939

Hypothesis 3:Increased Funding

When a district moves up a tier by having a higher % of FRL students, FRL students may perform at higher levels after the district has received more funding. Thus, if NSLA were working, we would see greater performance for districts after the new fundsSince 2004-05, some districts have moved into a higher tier of poverty funding. The achievement of these districts was compared and at both the 70% and 90% cliffs, no district showed an increase in achievement as a result of a financial windfall.40So, what do we know about NSLA?It is important to note that we do not have the counterfactual to examine how districts would perform without poverty funding. Nevertheless, we do know that:Most agree that additional resources should be provided to schools with higher concentrations of poverty (to help students overcome additional challenges associated poverty).No research indicates exact $$ amount needed to create equal opportunities for poor students.From data presented thus far, no justification for funding cliffs (theoretical or empirical).

So, how do districts use NSLA funding?

41So, how do districts use NSLA funding?Expenditure Categories Year Coded as Exp. Percent of NSLA Funding in 2011-12Literacy, Math, and Science Specialists and Coaches200316.51%Other activities approved by the ADE-11.56%High Qualified Classroom Teachers20039.42%Transfer to ALE Categorical Fund-8.63%School Improvement Plan-8.62%Counselors, Social Workers, Nurses20038.30%Teachers Aides20038.17%Curriculum Specialist20034.69%Pre-Kindergarten20033.27%Before and After School Academic Programs20032.76%Supplementing Salaries of Classroom Teachers-2.77%Tutors20032.35%Transfer to ELL Categorical Fund2.28%Professional Development in Literacy, Math, and Science20032.02%Summer Programs20031.28%Early Intervention20031.22%Transfer to Special Educations Programs-0.93%Transfer to Professional Development Categorical Fund-0.87%District Required Free Meal Program20110.70%Parent Education20030.52%ACT Fees for 11th Graders and Operating/Supporting a Post-Secondary Preparatory Program20110.10%Scholastic Audit-0.37%Districted Reduced-Lunch Meal Program20110.05%Remediation activities for college20110.05%Teach For America professional development20110.03%Implementing Arkansas Advanced Initiative for Math and Science20110.01%Hiring Career and College Coaches20110.00%Materials, supplies, and equipment including technology2003-Expenses related to a longer school day2011-Expenses related to a longer school year2011-Shaded box denotes a coded use originally set in 2003.42How do districts use NSLA funding?The majority of districts distribute funding among 8 or more expenditure codes.

Districts seldom focus the money in one or two specific areas; therefore, it seems if many districts use the funding to plug gaps in budgets.

It is unclear as to whether all districts are specifically pinpointing the funding towards students in poverty (or schools serving these students).

For example, a district may spend a large portion of funding on Highly Qualified teachers or Specialists these teachers may or may not work specifically with the low-income students.

Furthermore, districts do not use all the funding many have balances at the end of the year. 43How do districts use NSLA funding?Given the uncertainty, could we have expected great gains? Funds have spent across the board by most districtsNo clear evidence that funding has been focused for students in povertyMoney is allocated to district offices and not even to schools with high levels of poverty44What do we recommend?Two main discussions this year:Distribution of fundsSmooth sliding scale to replace the current tiered systemDistribute more funding for districts with higher concentrations of FRL studentsWeighting the funding to differentiate between poverty levels by factoring in the difference between free and reduced lunch studentsLeftover balances by districtsUse of funding: more or less prescriptive?45Example of a Smooth Distribution

Smooth sliding scale Weighted to account for differences in free and reducedWeights are 75% for Reduced-Lunch Students and 100% for Free-Lunch Students.46Problem: Several affluent districts would lose $$Smoother but prescriptive?Should the use of NSLA funding be more prescriptive?Long debate over extent of mandating the spending matrix

Arguments for prescriptive use:Current lack of focus of fundsPinpoint only to students in povertyUse prescriptive manner as a way to figure out what worksArguments against prescriptive use:Flexibility is necessary: State-wide policies may not fit for all.What do you prescribe? Research isnt conclusive on what works best4747Concluding ThoughtsArkansas scores on the Benchmark and EOC have improved in the past 10 years, but much of the NAEP increases occurred before 2003 (slides 18 and 19)On the NAEP, Arkansas students have only slightly increased scores in 4th grade and 8th gradeFRL and non-FRL students have produced gains; but non-FRL students have experienced greater gainsIt is difficult to determine the effectiveness of NSLA funding over the past 10 years.The gap between FRL and non-FRL students has not shrunk.Policymakers and districts need to continue to strategically think about how NSLA funding can be pinpointed so that students in poverty can achieve at higher levels. 48Comments? Questions?Thank you for your time and input!

4949Chart13

Chart23

Bench Over Time Percent Proficient and Advanced on the Arkansas Benchmark Exams in Math and Literacy, Select Academic Years2012-132011-122010-112007-082005-06Math75%78%77%68%55%Literacy79%81%75%64%59%

Bench Over Time by RegionPercent Proficient and Advanced, Math Benchmark Exam, Select Academic Years2011-122005-06Region 5 (SE)70%42%Region 4 (SW)73%50%Region 3 (CN)76%52%Region 2 (NE)77%54%Region 1 (NW)82%62%Arkansas78%55%

Percent Proficient and Advanced, Literacy Benchmark Exam, Select Academic Years2011-122005-06Region 5 (SE)75%48%Region 4 (SW)79%53%Region 3 (CN)80%56%Region 2 (NE)81%59%Region 1 (NW)85%65%Arkansas81%59%

EOC Over TimePercent Proficient and Advanced on End of Course Exams, Select Academic Years2011-122010-112007-08Biology42%41%29%Grade 11 Literacy68%60%51%Geometry75%73%60%Algebra85%78%66%

EOC by RegionPercent Proficient and Advanced on End of Course Exams, By Region 2011-2012AlgebraGeometryGrade 11 LiteracyBiologyArkansas Overall81%75%68%42%Region 1 (NW)85%81%75%51%Region 2 (NE)78%76%65%39%Region 3 (CN)77%72%69%40%Region 4 (SW)73%67%61%34%Region 5 (SE)73%69%54%30%

ITBS Over TimeNational Percentile Rank on the Iowa Test of Basic Skills, Select Academic Years2011-122010-11Science NPR6156Language NPR5246Reading NPR5248Math NPR5749Overall District NPR5448

ITBS By RegionNational Percentile Ranks on the Iowa Test of Basic Skills (ITBS), 2011-2012Science NPRLanguage NPRReading NPRMathematics NPROverall District NPRRegion 1 (NW)6855566158Region 2 (NE)6051515553Region 3 (CN)5950515553Region 4 (SW)5649485350Region 5 (SE)4945434946Arkansas6152525754

Sheet12004-052005-062006-072007-082008-092009-102010-112011-12Math P/A45%55%62%68%73%75%77%78%Gain10%7%6%5%2%2%1%Literacy P/A52%59%59%64%68%73%75%81%Gain7%0%5%4%5%2%6%

LitChart

Chart14

Bench Over Time Percent Proficient and Advanced on the Arkansas Benchmark Exams in Math and Literacy, Select Academic Years2011-122010-112007-082005-06Math78%77%68%55%Literacy81%75%64%59%


Percent Proficient and Advanced, Literacy Benchmark Exam, Select Academic Years2012-132011-122005-06Region 5 (SE)73%75%48%Region 4 (SW)77%79%53%Region 3 (CN)78%80%56%Region 2 (NE)78%81%59%Region 1 (NW)83%85%65%Arkansas79%81%59%


EOC by RegionPercent Proficient and Advanced on End of Course Exams, By Region 2011-2012AlgebraGeometryGrade 11 LiteracyBiologyRegion 1 (NW)85%81%75%51%Region 2 (NE)78%76%65%39%Region 3 (CN)77%72%69%40%Region 4 (SW)73%67%61%34%Region 5 (SE)73%69%54%30%Arkansas Overall81%75%68%42%




Chart16









Chart18




EOC Over TimePercent Proficient and Advanced on End of Course Exams, Select Academic Years2012-132011-122010-112007-08Biology44%42%41%29%Literacy70%68%65%51%Geometry72%75%72%60%Algebra77%81%78%66%





Arkansas’ K-12 Achievement & NSLA Funding

Documents

Transcript of Arkansas’ K-12 Achievement & NSLA Funding