The Illinois State Board of Education (ISBE) recently released student growth percentiles (SGPs) to Illinois school districts in preparation of the new school accountability system and the launch of the new Illinois school report card. While incorporating student growth into the school accountability system is a step in the right direction, it is important to recognize that ISBE SGP results are reported within an accountability context not a school improvement context. As you draw inferences from your recently released SGP results, avoid a common pitfall when interpreting ISBE SGP growth results.
The current ISBE SGP model and its reporting of SGPs does a great job facilitating state-wide school accountability, but it is not designed to drive school improvement efforts at the local level. The ISBE SGP model can be broadly described as a conditional proficiency model. That is, it describes a student’s performance relative to students with similar testing histories. Specifically, what percentage of students across the state did an individual student perform equal to or better when compared to students with a similar performance baseline.
Why then is it inappropriate to compare growth results by proficiency levels? Because the way in which the 2018 PARCC SGP results are reported to schools is alongside 2018 PARCC proficiency results. This makes perfect sense for accountability, but not for school improvement. Since 2018 PARCC scale scores are formulaically tied to both proficiency and growth, a correlation is induced between proficiency rates and growth rates. Below is real data from a school district showing the relationship between growth and proficiency using 2018 PARCC results.
One can clearly see the strong relationship; the higher your proficiency, the higher your growth. However, this should not be interpreted as lower proficiency students showed lower growth and higher proficiency students showed higher growth. Instead, this pattern is a statistical artifact of comparing 2018 proficiency rates to 2018 growth rates. This relationship will always exist. The inability to examine growth by proficiency as reported by ISBE renders SGP results to have limited utility for school improvement as growth should always be interpreted within the context of proficiency.
Given this undesirable induced correlation between growth and proficiency, the astute analyst may ask:
Can’t this correlation be removed by looking at last year’s proficiency versus this year’s growth?
Normally yes, but in the case of the ISBE SGP model and its current reporting, no. ISBE has implemented the SGP model in a way that allows two different students with the same PARCC scale score last year and the same PARCC scale score this year to receive two different SGP growth scores, effectively eliminating a transparent baseline for which to evaluate growth. This is a result of a student’s baseline being established from up to two prior years of PARCC data.
Given there is no single baseline proficiency level, can’t we replace the baseline with a projected score from the SGP model?
Normally yes, but in the case of the ISBE SGP model and its current reporting, no. The SGP model adopted by ISBE does not produce a projected score. One could conceivably engineer a projected score using the conditional percentiles, and recalculate growth against the projection, but this is not reported by ISBE and is not consistent with the ISBE accountability system.
There will always be competing interests between student growth for accountability and student growth for continuous quality improvement. Growth models within state accountability contexts will always prioritize alignment to broader state policies and universal benchmarks, causing state growth models to be less relevant at the local level to support the school improvement process. In general, the adage is external benchmarks for accountability (national/state norms) and internal benchmarks for school improvement (local norms).
Schools should adopt local growth norms to judge the impact of school improvement efforts. Local norms provide a far more rigorous and scientific framework to quantify and measure improvements at the local level. This is especially true of subgroups and other special sub populations for which strong correlations exist among subgroups and proficiency rates.
A basic tenet of the scientific method is to standardize as many confounding variables as possible to isolate the effects of the variables of interest. The use of longitudinal local norms removes confounding and other contextual factors that cloud the ability to transparently measure growth. Local longitudinal growth norms provide a powerful framework to drive the school improvement process.
For more information on how to best utilize student growth at the local level to drive the school improvement process, please join us on October 25th, 2018 for an IL-Empower professional development session. ECRA Group is an approved IL-Empower professional learning partner. For more information, please contact melinawright@ecragroup.com or register at https://ecragroup.com/event-directory.
