School Rankings and the Diversity Penalty
posted by Sarah Waldeck
Those in legal education are familiar with the deleterious effects of the U.S. News rankings, but have not paid much attention to similar popular rankings of elementary, middle, and high schools. Because perceptions of public school quality often dictate where parents of school-aged children choose to live, these rankings are tremendously important.
My colleague Tim Glynn and I have recently examined rankings by private entities of schools in Illinois, New Jersey, and Ohio, and sampled school report cards from 18 states. Our analysis, available here, demonstrates that school rankings are neither accurate nor neutral measures of quality. Instead, rankings penalize socioeconomic and racial diversity and are biased toward wealthier and Whiter schools.
Most rankings use a student body’s overall performance on standardized proficiency tests to gauge school quality. This ignores the achievement gap—the well-documented phenomenon that, on average, wealthier students outperform poorer students on these tests and Asian and White students outperform Black and Hispanic students. The achievement gap is not inevitable, and educators are working hard to close it. But while the gap persists, wealthy and White schools will almost always have higher aggregate proficiency scores and thus outrank schools with a diverse mix of students. And that’s true even if a particular school serves each subgroup of its student population better than the higher ranked schools do.
This diversity penalty exists across popular school ranking systems in all areas of the country. Consider the website SchoolDigger and its rankings of New Jersey and Illinois high schools. Millburn High School—located in an affluent northern New Jersey town and often described in the media as one of the best high schools in the state—ranked 22 for tested year 2010. (The top spots were held by magnet schools that pre-select their students based on academic achievement.) The high school in neighboring South Orange-Maplewood—a far more socioeconomically and racially diverse community—ranked 179. But isolating performance at these two schools by demographic subgroup creates a very different impression of relative school quality. For example, when the two schools are re-ranked based just on the test scores of White students, they are in a virtual dead heat. The high school in Montclair, another nearby diverse community, performs comparably. Similarly, in Illinois, New Trier Township High School—which draws students from several affluent Chicago suburbs—ranked fifth for tested year 2010. Nearby Evanston High School—located in a far more diverse community—ranked 126. But when the two schools are re-ranked in ways that account for the achievement gap, they are essentially tied. Oak Park & River Forest High School, another diverse Chicago suburban school, is competitive as well. This pattern repeats itself in different years and different states and for elementary schools as well as high schools.
Parents should care about more than just the performance of their child’s demographic peers. But rankings that rely on aggregated scores are a misleading indicator for all demographic subgroups, including low-income students and historically-disadvantaged minorities. The problem is not that disadvantaged subgroups drag down aggregated test scores. Rather, by lumping all students together without regard for socioeconomic and racial differences, rankings reveal little about how a school actually serves its student population.
Because of the achievement gap, diverse schools in which both disadvantaged and advantaged students outperform their demographic peers will often still have lower aggregated proficiency scores—and hence lower rankings—than schools with mostly wealthy and White students. The rankings therefore penalize diversity and reward wealth and White racial homogeneity. Parents who rely on rankings will conclude that wealthy and White schools are better, even when the statistics show their children would do just as well or better in a diverse school.
Many parents see the value of diversity and would happily opt for schools that are both diverse and academically strong. And integrated learning environments benefit all students. But popular school-ranking systems suggest, contrary to reality, that academic strength and diversity seldom co-exist. When parents choose school districts based on rank, those with means will select away from diverse schools and the neighborhoods in which they are located. This distortion of local housing markets contributes to school and neighborhood segregation and may help explain why highly diverse communities are so rare.
School report cards contain data about demographic subgroup performance, and some private ranking systems also make this information available. But because the disaggregated data is usually buried beneath the headlines, many parents do not focus on it. Moreover, disaggregated data does not provide what many parents want—a bottom-line assessment of overall school quality.
Given their popularity, rankings are not going to disappear anytime soon. The question, then, is how to dampen their damaging effects. More on that in a later post.