The Effective Use of Some School-level Indicators of Student Learning Growth: NWEAs Learning...
-
Upload
kaleigh-skipwith -
Category
Documents
-
view
215 -
download
0
Transcript of The Effective Use of Some School-level Indicators of Student Learning Growth: NWEAs Learning...
The Effective Use of Some School-level Indicators of Student Learning Growth:
NWEA’s Learning Productivity Measurement (LPM) System
Y. M. Thum, Ph.D.
Sr. Research FellowNorthwest Evaluation Association
Portland, Oregon
12th Annual Maryland Assessment ConferenceRiggs Alumni Center, University of Maryland, College Park, MD 20742
October 18-19th 2012
Y M Thum - Modeling Growth 2
Some Important Cross-cutting ConcernsoWhat should the focus be, i.e. Goals?
– Attention from Educational Quality to Learning Progress– De-emphasizing Achievement, to favor Learning Growth, and
BACK.– Characterizing: Effectiveness, Improvement, or Productivity?
oGiven the above, what data are available in support ?– Do the available data support the measurement of the
quantities of interest?
oModel needs to match the purpose of the information.
– Monitoring for Accounting vs. Accountability: Is a “trait” or “state” assumption being implied – how much reliability/stability should one anticipated?
– Analysis is Description vs. Causal (value-added) : Is the leap from “how your students performed” to “how you have helped your students performed” too often made?
– Realize that indicator/ranking/metric does not, by itself, produce change – unlike incentive systems (e.g. Tax code)
Model
Introduction
10/9/2012
Goal
Data
Now you have a shot!
Y M Thum - Modeling Growth 3
Outline of the Presentation
Specific Goals of the PresentationoComments on some persistent issues for measuring growth
– Interval scale and impact on rankings, etc.– Dimensionality, construct “shift”– How growth is defined, implicitly, by the choice of growth models– Residualized gains, growth, vs. value tables– What result, or pattern of results, is hypothesized?
oAn example of How Growth is being defined and measured– NWEA’s Measures of Academic Progress, K-12– Achievement and Growth Norms, conditional on a student’s instruction
time
oAn example of School Productivity Measurement– Analyses focus on the historical data for a school and address specified
questions/hypotheses about growth and productivity patterns based on the data.
– Examples are offered regarding its use for district and school monitoring and data dialogues activities.
10/9/2012
Y M Thum - Modeling Growth 4
How to Classify Growth ModelsDifference over TIME seems key to “Growth” – 3 species
• Residualized gains model– Anytime the outcome sits alone on the LHS– Time is implicit, sometimes absent when variation in time is zero (e.g., a cross-sectional
post-on-pre regression)– “Growth” is defined as a residual against an estimated baseline or base-plane– Colorado Growth Model, Rob Meyer, etc.
• Value Tables– Thum (2002) pointed out that this is an “index scoring” approach– Nominating “worth” scores, disregards scale– Properties not generally clear
• (Conventional) Growth Curve Model– Outcomes are all on the LHS, all treated as stochastic, heteroscadesticity treated easily– Regression of observed gains is a simple (and crude) example– Sanders & Horn (1997), Bryk et al. (2003), Thum (2002~2006)– “Growth” is intra-individual, repeated measures, variation btw students, schools, etc.
The above growth models are core, amendable to the introduction of covariates – i.e., models for value-added analyses.10/9/2012
Y M Thum - Modeling Growth 5
Vertical Scales, Growth Models, Valued-addedSome sprinkling …
• Briggs & Weeks (2009)– Constructed 8 vertical scales based on real item response data, varying IRT
model, calibration and estimation approaches– Applied Sanders’ “layered model” for school effects to scores– Finds that
• while the ordering of school effects estimates is insensitive to the underlying vertical scales
• Only the precision of such value-added estimates seem to be sensitive to the combinations of choices made in the creation of the scale.
• Shafer et al. (2012) compared many “growth models”– No real differences, see also Tekwe (2004).
• Lockwood et al. (2007, JEM)– Teacher value-added estimates are more sensitive to the choice of a
student achievement measure than to differences in value-added model for the same achievement measure.
10/9/2012
Y M Thum - Modeling Growth 6
Uni-dimensionality and Measurement• Multi-dimensionality
– No-one disagrees: Nothing is likely to be 1-dimensional in reality– A wonderfulous quest, and we have seen how it may be more
challenging in MEASUREMENT then in ANALYSIS– However, uni-dimensionality IS part of the core proposition in
purposive scaling • Shirt-sizes example
– Construct shift (Martineau, 2006)• Are we mistaking changes in content for “shifts” in construct? • Which is also understandable but that is not what “measuring” aims to
do, I believe.
• Not an argument against our multi-dimensional world; just being focused on the problem at hand.– Separate scaling of variables FROM their analyses
10/9/2012
Y M Thum - Modeling Growth 7
A Curious Problem with Value TablesTransition matrix, value tables
• Committee nominates/imposes values to express the “worth” of difference types of progress observed
• Ignores scale, arbitrary, even absolute, disregards any potential for measurement/classification “errors”
• Can be problematic! Value tables DEFINE Growth rather than “EVALUATE” it. See e.g. from Shafer et al. (2012) next.
• Thus NOT “model-free,” but may still be amendable to “serious” analysis, e.g., Thum’s (2002b) work on the California Academic Performance Index (API).
10/9/2012
Y M Thum - Modeling Growth 8
TUp Value Points for Year-One & Year-Two Cell Combinations Y AdvH 4 4 4 4 4 4 3 2 1
e AdvM 4 4 4 4 4 3 2 1 0
a AdvL 4 4 4 4 4 3 1 0 0
r ProfH 4 4 4 3 2 1 0 0 0
ProfM 4 4 3 2 1 0 0 0 0
T ProfL 4 4 3 1 0 0 0 0 0
w BasH 4 3 1 0 0 0 0 0 0
o BasM 3 1 0 0 0 0 0 0 0
BasL 1 0 0 0 0 0 0 0 0
BasL BasM BasH ProfL ProfM ProfH AdvL AdvM AdvH
Year One (Basic, Proficient or Advanced; High Middle or Low) Linear Model: 1.27 -0.30*Y1 + 0.55*Y2 -0.03*Y1*Y2, adj-R2=0.798
Value Tables DEFINE, not EVALUATE
TUpDn Value Points for Year-One & Year-Two Cell Combinations Y AdvH 8 8 6 6 4 4 2 2 0
e AdvM 7 6 5 4 3 2 1 0 -2
a AdvL 6 5 4 3 2 1 0 -2 -2
r ProfH 5 4 3 2 1 0 -2 -2 4
ProfM 4 3 2 1 0 -2 -2 -4 -4
T ProfL 3 2 1 0 -2 -2 -4 -4 -6
w BasH 2 1 0 -2 -2 -4 -4 -6 -6
o BasM 1 0 -2 -2 -4 -4 -6 -6 -8
BasL 0 -2 -2 -4 -4 -6 -6 -8 -8
BasL BasM BasH ProfL ProfM ProfH AdvL AdvM AdvH
Year One (Basic, Proficient or Advanced; High Middle or Low) Linear Model: -0.37 -1.03*Y1+1.06*Y2, adj-R2=0.946
The behavior of TProg is hailed as the one with enough reliability and stability for teacher HIGH-STAKES assessment applications, but it represents not as much “growth” as Y2 status (and nothing from Y1 as believed).
10/9/2012
TProg Value Points for Year-One & Year-Two Cell Combinations Y AdvH 25 24 23 22 21 20 19 18 17
e AdvM 23 22 21 20 19 18 17 16 15
a AdvL 21 20 19 18 17 16 15 14 13
r ProfH 19 18 17 16 15 14 13 12 11
ProfM 17 16 15 14 13 12 11 10 9
T ProfL 15 14 13 12 11 10 9 8 7
w BasH 13 12 11 10 9 8 7 6 5
o BasM 11 10 9 8 7 6 5 4 3
BasL 9 8 7 6 5 4 3 2 1
BasL BasM BasH ProfL ProfM ProfH AdvL AdvM AdvH
Year One (Basic, Proficient or Advanced; High Middle or Low) Linear Model 1: 9.0 – 1.0*Y1 +2.0*Y2, corr(TProg, Y2)=0.9, adj-R2=1.0 !!!!! Linear Model 2: 17.0-1.0*Y1, adj-R2=0.19 Linear Model 3: 5.0+2.0*Y2, adj-R2=0.798
Y M Thum - Modeling Growth 9
NWEA’s Growth Norms:
How Growth is Defined and Measured
10/9/2012
Y M Thum - Modeling Growth 10
• A stable and reliable vertical scale across the relevant (narrow) grade-range
• Students are measured uniformly well across the range – variance of SEMs relatively low over theta
• Precision of scores is provided, will be used• Scores are criterion referenced, with performance norms,
to provide clear interpretation
NWEA’s MAP possesses all the above qualities. Fall and Spring testing additionally offer a huge design advantage over once-a-year assessment programs
Test and Scale Requirements
10/9/2012
Y M Thum - Modeling Growth 11
Measure Status, Infer Growth• “Growth” is observed, an inference about a
change on a scale for status over time; not an “independent” metric (Thum & Hauser)
• Methodological Approach– Multilevel polynomial growth model– Latent variable regression of (latent linear) Gain
coefficient on (latent) Initial Status coefficient– Make inference of an Observed Gain wrt the
appropriate Conditional Gain Distribution• Result
– Condition Growth Index (CGI)– Conditional Growth Percentile (CGP)
10/9/2012
Y M Thum - Modeling Growth 12
RIT
Instructional Days
150 200 275 330 380
150
200
Spring Grade 2
Fall Grade 3
Winter Grade 3
Spring Grade 3
Fall Grade 4
Achievement over Time
CenteredInstructional Days (Weeks)
10/9/2012
Y M Thum - Modeling Growth 13
RIT
Instructional Days
150 200 275 330 380
150
200
Predicted score distribution after275 instructional days
Mean: 198.91S.D.: 12.6
Spring Grade 2
Fall Grade 3
Winter Grade 3
Spring Grade 3
Fall Grade 4
Predicted mean score after275 instructional daysMean: 198.91s.e.: 0.25
Conditional Achievement Norms
10/9/2012
Y M Thum - Modeling Growth 14
Trajectory of Status over Time
1. Achievement norms are specific to the instructional day!2. Growth norms are conditional on
• how much instruction students have had between assessments • how well they performed at baseline
Academic peers – same test, amount of instruction, baseline performance!
10/9/2012
Y M Thum - Modeling Growth 15
Observed and Inferred Growth
10/9/2012
Y M Thum - Modeling Growth 16
RIT
Instructional Days
150 200 275 330 380
150
200
Starting RIT
Conditional Growth NormsAchievement
Norms
Growth Norms
10/9/2012
Y M Thum - Modeling Growth 17
Additional Facts/Features• Short-term gain is measured on a within-grade scale• Instructional time within a grade is the “time” dimension• Scores weighted for precision, using their standard errors of
measurement• Post-stratification weighting to reflect nationally representative
student body• In this procedure, we measure achievement status, and not
growth. Theoretically, growth is observed and to assess its true magnitude requires an inference, a result that is based solely on an acceptable descriptive model of change in achievement status over time.
• The CGI is – not absolute, not relative, but a normative measure of change– facilitates aggregation over students, subjects, grades, etc.
10/9/2012
Y M Thum - Modeling Growth 18
NWEA’s Learning Productivity Measurement (LPM) System:
Assessing a School’s Academic Health
10/9/2012
Y M Thum - Modeling Growth 19
Learning Productivity Measurement• The LPM System provides a set of perspectives into the
learning growth of the students in your school based on their individual MAP performance histories . See Thum (2003, 2006).
• Describes and compares the trends of both– achievement status, as well as – achievement growth.
• Presents results as evidence summaries to facilitate data-dialogue/client discussions about – how student learning in your school has changed over time– was there discernable change, if so
• where, i.e., which grade-level? • when, i.e., is the change recent?
10/9/2012
Y M Thum - Modeling Growth 20
Multilevel Multi-Cohort Growth Model
Overview: LPM System Flow-chart
Representative Growth Curve for
Age-Cohort, c
Students inan Age-Cohort
School j Data Block
Estimated Growth Rates
1ˆc jContrasts of Age-
cohort Growth Rates
1ˆas c jh
Predicted Grade-YearMeans
Predicted Grade-Year Means
ˆ gyjyContrasts of Predicted
Grade-Year Means
ˆbs gyjh y
Growth Reports
T
• Tests Age-Cohort growth patterns• Tests Grade- Cohort growth patterns• Growth pattern norms• Compare schools• Compare to similar schools• Compare districts• Predicts status & gains for following year• Validation of predictions
10/9/2012
Y M Thum - Modeling Growth 21
Assumptions re Inputs:Validity & Quality of Outcome Measures
o We assume that we have an outcome of student learning which the user believes to be a valid/useful measure of the intended construct.
o The outcome measure possesses the necessary psychometric (scale) properties supporting its use.
o To the degree that either, or both, the construct validity of the measure, and its scale-type (interval), are approximate in practice, we submit that the validity of the interpretation using this outcome needs to be tempered accordingly.
o Faced with this complex of nearly unsolvable issues, I rest some of my choices on a “satisficing principle” (Simon, 1956).
10/9/2012
Y M Thum - Modeling Growth 22
School A Mathematics Data
2004 2005 2006 2007 2008 2009Fall Spring Fall Spring Fall Spring Fall Spring Fall Spring Fall Spring
Grade 4 4 5 5 6 6 7 7 8 8 9 9
5
CohortN
RIT105 103 122 123 119 123 110 111 141 143 119 125
214.21 219.19 216.56 223.24 219.14 222.71 219.79 225.14 214.4 222.59 221.19 231.21% 54 57 64 65 54 67 N
Gain 101 119 117 106 138 116 4.77 6.83 3.86 5 8.03 10.79
Grwth Idx -1.41 0.69 -2.23 -1.07 1.85 4.74CGI -0.23 0.11 -0.36 -0.17 0.3 0.77% 43 53 41 46 57 71
4
Cohort 5 5 6 6 7 7 8 8 9 9 10 10N
RIT114 109 122 123 103 106 118 126 123 122 138 142
206.03 216.68 207.36 218.12 209.38 217.12 205.94 213.61 206.46 220.31 207.81 221.52% 57 60 64 56 59 60 N
Gain 104 118 99 117 114 134 10.12 10.71 8.17 7.71 13.5 14.39
Grwth Idx 3 3.65 1.2 0.56 6.43 7.34CGI 0.52 0.63 0.21 0.1 1.11 1.26% 63 65 55 52 76 77
3
Cohort 6 6 7 7 8 8 9 9 10 10 11 11N
RIT109 109 110 107 118 117 117 117 125 128 124 122
196.81 205.28 197.8 208.17 195.47 203.36 195.53 204.58 195 205.9 193.96 204.4% 62 64 56 56 56 54 N
Gain 103 103 115 113 121 118 8.01 10.16 7.68 9.01 11.02 10.08
Grwth Idx -1.42 0.81 -1.83 -0.52 1.47 0.49CGI -0.24 0.14 -0.31 -0.09 0.25 0.08% 42 54 41 46 55 51
2
Cohort 7 7 8 8 9 9 10 10 11 11 12 12N
RIT100 102 129 126 115 116 116 118 117 120 122 124183 196.69 180.05 195.3 183.19 196.48 181.98 195.7 176.99 194.93 179.49 199.84
% 61 54 61 57 43 50 N
Gain 98 120 111 113 111 115 13.67 14.94 13.21 13.69 17.93 21.05
Grwth Idx 2.85 3.89 2.41 2.78 6.59 9.91CGI 0.48 0.65 0.4 0.47 1.11 1.67% 62 66 59 61 74 85
10/9/2012
• Overlap in data IS natural in learning• Information “ballast” imparts
stability, standard for evidence• Highly specific questions being
asked, and addressed• School-by-school analysis, very much
an “Age-Period-Cohort” analysis.
Y M Thum - Modeling Growth 23
Your School’s Performance:A Holistic Take
2
3
4
5Grade
RIT
F S F S F S F S2006 2007 2008 2009
School Year
Age-cohort
Students in the same age-cohort share a relatively homogenous school experience in terms of their peers and teacher group.
Grade-level
Groups of students attending the same grade-level are exposed to the same teachers and materials.
10/9/2012
Y M Thum - Modeling Growth 24
A Pictorial Guide to How LPM System Works
10/9/2012
Y M Thum - Modeling Growth 25
1 2 3 4 5Year
1
2
3
6
4
5
Grade
Score
800
700
600
500
1 2 3 4 5Year
8
7
6
5
4
3
2
1
Grade
Cohorts of Longitudinal Student Data
Cohort 5
10/9/2012
Y M Thum - Modeling Growth 26
1 2 3 4 5Year
1
2
3
6
4
5
Grade
1 2 3 4 5Year
1
2
3
6
4
5
Grade
Score
800
700
600
500
Cohort Regressions
Individual Growth Curves of Students in Cohort 5
10/9/2012
Y M Thum - Modeling Growth 27
1 2 3 4 5Year
1
2
3
6
4
5
Grade
Score800
700
600
500
Observed Grade-Year Means
10/9/2012
Y M Thum - Modeling Growth 28
Overall Strategy:Obtain a good fitting (measurement) model for each school (surface for math), then construct and evaluate relevant valued-added hypotheses for
the school.
Year
Grade
Outcome
Note: Surface need not be a “flat.”
10/9/2012
Y M Thum - Modeling Growth 29
Comparing Age-cohort Growth Rates
2
3
4
5Grade
F S F S F S F S 2006 2007 2008 2009
School Year
Age-Cohort BGrowth Trend
Age-Cohort AGrowth Trend
Age-Cohort CGrowth Trend
An Sample Question about the Pattern of Age-cohort Growth : Fall Achievement growth rate for Age-cohort A is stronger than
the average of the Fall achievement growth rates of Age-cohorts B and C.
Note: Age-cohort A is the “most recent” age-cohort in this analysis.
10/9/2012
Y M Thum - Modeling Growth 30
What Being More Productiveover Time Means
F S F S F S F S 2006 2007 2008 2009
School Year
Evidence: • More Recent Age-Cohort Slopes are
StrongerConclusion: • School is increasingly more productive
Evidence: • More Recent Age-Cohort Slopes are
WeakerConclusion: • School is becoming less productive
2
3
4
5
Grade
F S F S F S F S 2006 2007 2008 2009
School Year
10/9/2012
Y M Thum - Modeling Growth 31
Patterns of Fall Achievement Trends
Your SchoolOther SchoolsYour DistrictSchools like yours
Legend
Reports in Progress:e.g., “Band-Aid” Graph
If Observe: Slopes Decrease over TimeConclude: Decreasing Productivity
If Observe: Slopes Increase over TimeConclude: Increasing Productivity
?Pattern
of Growth Rates
RIT
Year
Stronger
RIT
Year
Weaker(a) (b)
10/9/2012
Y M Thum - Modeling Growth 32
1 2 3 4 5Year
1
2
3
6
4
5
Grade
Score800
700
600
500
Is Grade 1 Predicted Average Increasing?Is Grade 1 Predicted Average Increasing?
Is Grade 4 Predicted Average Increasing?Is Grade 4 Predicted Average Increasing?
Predicted Grade-Year Means
Based on a model for the information contained in the data-block …
Based on a model for the information contained in the data-block …
10/9/2012
Y M Thum - Modeling Growth 33
Fall Achievement Fall-Spring Gains C22 GG1 GG2 C41 C42 GG3 GG4 GG5Evidence
Supports
Does Not Support
Pattern/Trend Questions ConsideredOutcome Who # Concern/Inquiry
Fall Achievement
Student Cohort C22 More recent cohorts is growing faster.
Per Grade Level GG1 Most recent grade is higher than prior years.
Per Grade Level GG2 Avg. has been increasing with time.
Fall-Spring Gain
Student Cohort C41 Gains are accelerating on average for cohorts.
Student Cohort C42 More recent cohort accelerates the most.
Per Grade level GG3 Most recent students higher than prior years.
Per Grade Level GG4 Avg. has been increasing with time.
Per Grade Level GG5 Gains are positive on average (over time).
10/9/2012
Y M Thum - Modeling Growth 34
Corresponding Hypothesis TestsOutcome Who # Hypothesis (“R” for Growth Rate, “M” for “Mean”)
Fall Achievement
Student Cohort C22 [… < RC-2< RC-1 < RC ] , “c” means “most recent cohort”
Per Grade Level GG1 My > AVG[My-1,My-2, …] , “y” means “current year”
Per Grade Level GG2 […, < My-2 < My-1 < My]
Fall-Spring Gain
Student Cohort C41 AVG[…, Rc-2, RC-1, RC] > 0
Student Cohort C42 [… < RC-2< RC-1 < RC ]
Per Grade level GG3 My > AVG[My-1,My-2, …]
Per Grade Level GG4 […, < My-2 < My-1 < My]
Per Grade Level GG5 AVG[ …, My-2, My-1, My] > 0
Fall Achievement Fall-Spring Gains C22 GG1 GG2 C41 C42 GG3 GG4 GG5Evidence
Supports
Does Not Support
10/9/2012
Y M Thum - Modeling Growth 35
Sample Results– Peterson ISD
The School Challenge IndexA new measure of how schools in a state compare in terms of the challenges and opportunities they operate under as reflected by an array of factors they do not control. This school measure generally taps the collective economic circumstance of its students, but it also offers a broader view of the “economic strain” they experience as seen through a relevant set of socio-demographic, organizational, and educational policy programming factors.
Age-cohort C22 patterns appear to be stable.
Age-cohort C42 patterns appear to be stable.
Age-cohort C41 patterns appear to be stable.
Age-cohort Pattern ofFall Ach. Slopes
Age-cohort Pattern ofF-S Gain Slopes
Age-cohort Avg. ofF-S Gain Slopes
10/9/2012
District and School names are completely fictitious.Any resemblance to actual districts or schools, healthy
or otherwise, is purely coincidental.
Y M Thum - Modeling Growth 36
More Recent Fall Ach Slopes are Stronger Mathematics
Percentage of NWEA schools in your State with Stronger, Same, or Weaker Growth Rates of Fall Scores for their more recent Age-cohorts: 2007, 2008, and 2009
%
MathematicsMathematics
10/9/2012
Age-cohort Pattern ofFall Ach. Slopes
Y M Thum - Modeling Growth 3710/9/2012
Y M Thum - Modeling Growth 38
Summing Up• We noted, and asked questions about, several persistent
issues with growth modeling and accountability. I hope to hear your objections, in order to better assess if I have been adequately provocative.
• In presenting the NWEA norming procedure, we have presented a definition of growth that is “novel;” of someone who runs around with a rule he is happy with.
• In presenting the LPM System, we have returned the focus of accountability to an internal dialogue about how a school has been improving, in effectiveness and in productivity – which is where I think data, rendered well, will be most helpful to the improvement of schools and learning.
• Thank you!
10/9/2012
Y M Thum - Modeling Growth 39
References1. Betebenner, D. W. (2008). Toward a normative understanding of student growth. In Ryan, K. E. and Shepard, L.
A., editors, The Future of Test-Based Educational Accountability, 155–170. Taylor & Francis, New York. 2. Briggs, D. C., & Weeks, J. P. (2009). The Sensitivity of Value-Added Modeling to the Creation of a Vertical Score
Scale. Education Finance and Policy, 2009, 4, 384-414.3. Martineau, J. A. (2006). Distorting value added: The use of longitudinal, vertically scaled student achievement
data for growth-based value-added accountability. Journal of Educational and Behavioral Statistics, 31(1), 35-62.
4. Schafer, W. D., Lissitz, R. W., Zhu, X., Zhang, Y., Hou, X., & Li, Y. (2012). Using Student Growth Models for Evaluating Teachers and Schools. (Technical report). College Park, MD: MARCES.
5. Simon, H. A. (1957). Models of man: Social and rational. New York: Wiley.
6. Thum, Y. M.(2002a). Measuring student and school progress with the California API. CSE Technical Report No. 578. Los Angeles, CA: University of California, National Center for Research on Evaluation, Standards, and Student Testing, UCLA.
7. Thum, Y. M. (2002b). No Child Left Behind: Methodological Challenges & Recommendations for Measuring Adequate Yearly Progress. CSE Technical Report 590. Los Angeles: Center for Research on Evaluation, Standards, and Student Testing, UCLA.
8. Thum, Y. M. (2003). Measuring Progress towards a Goal: Estimating Teacher Productivity using a Multivariate Multilevel Model for Value-Added Analysis. Sociological Methods & Research, 32 (2), 153-207.
9. Thum, Y. M. (2006). Designing Gross Productivity Indicators: A proposal for Connecting Accountability Goals, Data, and Analysis. In R. Lissitz (Ed.), Longitudinal and Value-Added Models of Student Performance. (pp. 436 - 479). Maple Grove, MN: JAM Press.
10. Thum, Y. M., & Bowe, B. (2009). An Effect-Size Indicator for Measuring the Productivity of a Teacher based on the Learning Growth of her Students with Applications. Unpublished report, National Heritage Academies.
10/9/2012