School Rankings and the Diversity Penalty
posted by Sarah Waldeck
Those in legal education are familiar with the deleterious effects of the U.S. News rankings, but have not paid much attention to similar popular rankings of elementary, middle, and high schools. Because perceptions of public school quality often dictate where parents of school-aged children choose to live, these rankings are tremendously important.
My colleague Tim Glynn and I have recently examined rankings by private entities of schools in Illinois, New Jersey, and Ohio, and sampled school report cards from 18 states. Our analysis, available here, demonstrates that school rankings are neither accurate nor neutral measures of quality. Instead, rankings penalize socioeconomic and racial diversity and are biased toward wealthier and Whiter schools.
Most rankings use a student body’s overall performance on standardized proficiency tests to gauge school quality. This ignores the achievement gap—the well-documented phenomenon that, on average, wealthier students outperform poorer students on these tests and Asian and White students outperform Black and Hispanic students. The achievement gap is not inevitable, and educators are working hard to close it. But while the gap persists, wealthy and White schools will almost always have higher aggregate proficiency scores and thus outrank schools with a diverse mix of students. And that’s true even if a particular school serves each subgroup of its student population better than the higher ranked schools do.
This diversity penalty exists across popular school ranking systems in all areas of the country. Consider the website SchoolDigger and its rankings of New Jersey and Illinois high schools. Millburn High School—located in an affluent northern New Jersey town and often described in the media as one of the best high schools in the state—ranked 22 for tested year 2010. (The top spots were held by magnet schools that pre-select their students based on academic achievement.) The high school in neighboring South Orange-Maplewood—a far more socioeconomically and racially diverse community—ranked 179. But isolating performance at these two schools by demographic subgroup creates a very different impression of relative school quality. For example, when the two schools are re-ranked based just on the test scores of White students, they are in a virtual dead heat. The high school in Montclair, another nearby diverse community, performs comparably. Similarly, in Illinois, New Trier Township High School—which draws students from several affluent Chicago suburbs—ranked fifth for tested year 2010. Nearby Evanston High School—located in a far more diverse community—ranked 126. But when the two schools are re-ranked in ways that account for the achievement gap, they are essentially tied. Oak Park & River Forest High School, another diverse Chicago suburban school, is competitive as well. This pattern repeats itself in different years and different states and for elementary schools as well as high schools.
Parents should care about more than just the performance of their child’s demographic peers. But rankings that rely on aggregated scores are a misleading indicator for all demographic subgroups, including low-income students and historically-disadvantaged minorities. The problem is not that disadvantaged subgroups drag down aggregated test scores. Rather, by lumping all students together without regard for socioeconomic and racial differences, rankings reveal little about how a school actually serves its student population.
Because of the achievement gap, diverse schools in which both disadvantaged and advantaged students outperform their demographic peers will often still have lower aggregated proficiency scores—and hence lower rankings—than schools with mostly wealthy and White students. The rankings therefore penalize diversity and reward wealth and White racial homogeneity. Parents who rely on rankings will conclude that wealthy and White schools are better, even when the statistics show their children would do just as well or better in a diverse school.
Many parents see the value of diversity and would happily opt for schools that are both diverse and academically strong. And integrated learning environments benefit all students. But popular school-ranking systems suggest, contrary to reality, that academic strength and diversity seldom co-exist. When parents choose school districts based on rank, those with means will select away from diverse schools and the neighborhoods in which they are located. This distortion of local housing markets contributes to school and neighborhood segregation and may help explain why highly diverse communities are so rare.
School report cards contain data about demographic subgroup performance, and some private ranking systems also make this information available. But because the disaggregated data is usually buried beneath the headlines, many parents do not focus on it. Moreover, disaggregated data does not provide what many parents want—a bottom-line assessment of overall school quality.
Given their popularity, rankings are not going to disappear anytime soon. The question, then, is how to dampen their damaging effects. More on that in a later post.
July 30, 2012 at 8:03 am
Posted in: Civil Rights, Education, Property Law
Print This Post
Responses (6)
Ken Arromdee - July 30, 2012 at 10:23 am
Ah, Asians and whites are racial homogeneity.
What you mean is “racial heterogeneity of the wrong kind”, not “racial homogeneity”.
Jonny - July 30, 2012 at 5:36 pm
It seems like this post ignores the fact that going to school with students who score better on standardized tests may also be more intelligent (assuming the proxy of test scores means anything and I believe that it does) and thus there may be some benefit to having your child exposed to other really bright students. In other words, the quality of education delivered is not the only factor integral to the education of students (as you acknowledge by touting the benefits of diversity) – so too is the student body an important dynamic when it comes to the holistic benefits of a school.
Sarah Waldeck - July 30, 2012 at 7:28 pm
Jonny,
The post doesn’t address the point you raise, but the underlying paper to which I link does. There is a substantial literature on peer effects–the idea that students benefit from being around high achievers. You should check out the paper if you are interested in this phenomenon, but the short answer is that peer effects aren’t as straightforward as your comment suggests. Rather, children and teens tend to be most influenced by their socioeconomic and racial peers. Therefore, parents can’t simply assume that the school with the highest aggregate test scores will have the most positive peer effects. Instead, parents should look closely at the achievement of the portions of the student body that they consider most relevant to their child. Parents who lump all students together learn less about a school and its probable effects on their child than parents who disaggregate.
Londoner - July 30, 2012 at 10:51 pm
I disagree with this post (I haven’t had time to read the paper yet) on a number of grounds, but I’ll stick to this one for now. Your theory seems to presuppose that the “achievement gap” exists independently of the quality of the school, rather than as a consequence of it.
I would suggest that it’s far more likely that the race gap exists because the schools in areas with many African-Americans and Latin@s are bad than that the schools get bad rankings because of their racial composition. Rather than adjusting the scores for racial composition, the better methodology is to adjust for non-school factors (poverty, crime, native language, etc.) that happen to affect AAs & L@s more (in the case of poverty and crime, largely due to the legacy of discrimination). I think they’ll show that schools in high-AA/L@ areas are “bad” schools (often due to bad funding), and this explains the racial gap, rather than the racial gap explaining the bad performance of the school.
Boiled down, my complaint is that you take the achievement gap as a given, rather than treating it as the problem to be addressed.
Sarah Waldeck - July 31, 2012 at 1:47 pm
Londoner,
My co-author and I agree that the achievement gap is not a given– that’s one of the reasons the post notes that the gap is not inevitable and that educators are working to close it. It is also true that socioeconomics, school quality and a variety of other factors feed the gap. As you anticipate, the underlying paper discusses this literature. However, while achievement gaps persist, we shouldn’t ignore the effect they have on comparisons of relative school quality. This is especially true because aggregated data has the unfortunate effect of obscuring which schools are doing a good job of closing the gap.
Kelly - August 7, 2012 at 12:21 pm
Great article. Taking a closer look at specifically what data shows can be very beneficial. As an educator I can see the benefits of both, but as in anything parents and educators have to be advocates for themselves and their families.
Leave a Reply