Does Blind Review See Race?*
posted by Kaimipono D. Wenger
In a comment to my earlier post suggesting that law review editors should seek out work from underrepresented demographic groups, my co-blogger Dave Hoffman asked an excellent question: Would blind review remedy these concerns? It seems to me that the answer here is complicated. Blind review would probably be an improvement on balance, but could still suffer from — err, blind spots. Here are a few reasons why.
The paradigmatic case for the merits of blind review comes from a well-known study of musician hiring, published about a decade ago by Claudia Goldin and Cecilia Rouse in the American Economic Review. Goldin and Rouse gathered data on symphony auditions, and found that blind auditions — that is, ones which concealed the gender of the auditioning musician — resulted in a significantly higher proportion of women musicians auditioning successfully. As Rouse commented,
“This country’s top symphony orchestras have long been alleged to discriminate against women, and others, in hiring. Our research suggests both that there has been differential treatment of women and that blind auditions go a long way towards resolving the problem.”
The Goldin-Rouse study shows that blind review can be a useful tool in combating bias. Would a similar review system work in the law review context?
Well, maybe.
It seems quite likely that blind review would reduce the effects of some problematic existing biases. For instance, blind review could greatly reduce or even eliminate the effects of letterhead bias and CV bias, as discussed in my earlier post. That would likely have very positive results for women and people of color.
There still might be some concerns. Let’s go over them.
An initial concern is that blind review might not always be blind. For instance, there might be structural loopholes which some authors could exploit, such as symposia or reply articles. In addition, authors might be able to use the expedite process to lever their way up the stack — that is, an expedite from a nonblind journal might result in priority review at an otherwise blind journal. Some editors might even deliberately undermine the blind process, such as by searching SSRN. (And in a related vein, blind review could also be undermined by internal signals in the article. If the article is based on prior experience as a Supreme Court clerk, for instance, then it is limited in its blindness.)
So, a system of blind review could potentially suffer from gaps in the blindfold. But let’s set those aside for a moment, and assume perfect blind review. Some biases could still exist (although on balance, they would probably have much smaller distorting effects than letterhead bias, CV bias, or implied association).
For instance, law review editors who are disproportionately white and male might not realize the importance of some issues, due to blind spots in their own background. For instance, an all-male editorial board might not appreciate the scope of an important article about the Family Medical Leave Act. I don’t want to overstate the case; but, it would be surprising if there were no bias reflected in the background experience of gender- and race-skewed editorial boards. On a similar note, it may be the case that there are stylistic differences — for instance, that men and women are typically socialized to frame arguments slightly differently — and that male editors would prefer a male-sounding framing, even if both are equally valid.
Thus, even if quality articles are distributed evenly across demographic groups, it seems possible that some bias could still creep in even under a system of blind review. This could happen either because of gaps in the blind review process; or because if the process replicates white male experience, it might not adequately account for valid perspectives of people outside that framework.
One final concern, which is in many ways the most complicated: What if blind review does result in a selection which more or less accurately reflects the contents of the submission pool, but the pool itself is biased in problematic ways against women and people of color?
Some commenters on the earlier post suggested that gaps in manufacture might be due to inherent differences in ability between groups. But before simply jumping to that explanation, should we see whether the pre-selection manufacture of law review articles is also biased in structural ways?
For instance, to use a somewhat stylized example, let’s say that due to implicit biases, law deans tend to channel women professors disproportionately into time-intensive committee work, thus reducing the relative number of women articles. (There are many other ways that structural disadvantage could creep into manufacture: Women could be more often assigned to labor-intensive courses like legal writing; women and people of color might have more difficulty finding co-authors; they might be invited less often to participate in panels; they might have reduced access to research stipends or RAs; and so forth.) If the pool itself is unbalanced, it could be the result of many factors, some of which are deeply problematic.
So if there are race and gender disparities in the pool itself, how should law reviews respond? Observers could reasonably come to different conclusions here, I think.
One approach would be to say, the problem is not at the selection stage but at the manufacture stage. It should be addressed at that stage. Deans should not disproportionately burden women faculty; panel organizers should not under-invite people of color; and so forth.
What if that response is not happening?
It is complicated to suggest that law reviews correct for a market distortion in manufacturing with a counter-distortion in selection. An argument can be made, certainly, that this kind of recalibration is far outside the scope of the law reviews. But one could also make the case that law reviews should reflect the experience of various groups, not only the experience of the dominant group. And that consideration would weigh in favor of law reviews seeking out work from underrepresented groups. The proper balance of these concerns is certainly open to debate. But I don’t think it would be crazy for a law review editor to say, even _if_ it’s the case that there are fewer X-quality articles from women, our commitment to fostering legal discourse — including the experience of marginalized groups — is such that we are willing to make that trade-off in some cases. And that kind of decision could only be made by relying on author demographic information.
(And as a side note, as many people mentioned on the earlier thread, article quality is a somewhat amorphous concept — there is quite possibly more wiggle room built in here than critics are allowing.)
For all those reasons, I think that blind review would probably not be a perfect solution to the problem of bias in law review article selection. That said, I think it would almost certainly be a significant step forward in reducing the overall level of bias, because current distortions like letterhead and CV bias likely have such a significant skewing effect.
–
*FN: Yes, my title here is a take on Osagie Obasogie‘s excellent article _Do Blind People See Race?_. Which you should go read, if you haven’t yet done so.
(Image: Wikicommons, The Blind Flower Girl of Pompeii)
February 18, 2013 at 7:11 pm
Tags: blind review, gender, law reviews, Race, unconscious bias
Posted in: Feminism and Gender, Law School (Law Reviews), Law School (Scholarship), Race
Print This Post
Responses (5)
Brett Bellmore - February 19, 2013 at 7:08 am
I don’t know what I can say beyond this:
If the only acceptable selection process is one which is guaranteed to produce the numbers you want, shouldn’t you just stop beating around the bush, and admit you want a quota?
Kaimipono D. Wenger - February 19, 2013 at 9:45 am
Brett, I appreciate your attempt to increase comment diversity by adding straw people to the conversation. I’m sure this is done with helpful intent. However, you should be aware that straw people are not in fact an underrepresented group in blog comments.
AYY - February 20, 2013 at 1:55 am
In fairness to Brett, you did argue in favor of affirmative action.
BTW, there are female Deans and females on law review editorial boards, and there’s no reason to think that a woman could evaluate an article on the Family Leave Act better than a man, and you didn’t cite much evidence to support your hypotheses.
The proper analogy wouldn’t be to music performance, it would be to musical composition. You can blind test all you want and you can still tell the difference between Beethoven and Lady Gaga.
A.J. Sutter - February 20, 2013 at 2:36 am
@AYY: Actually, apropos of music, I read a book of aesthetic philosophy a number of years ago that made the opposite point. (Unfortunately, the book is either buried in storage or lost, and in either case on another continent from where I now sit.) You might not be able to tell the difference easily between composers that are roughly contemporary — e.g., Haendel and some Baroques, Haydn or Mozart or even early Beethoven and some Classicals, between various Romantics, etc. The author there argued that it’s legitimate to know the name of the composer because it gives the listener some context.
E.g., if you know a piece is by Alkan you might think it’s quite characteristic, whereas if you know it was Liszt you might think he was having an odd day that day (or copying Alkan); a piece that might be an OK minor work for Haydn might sound quite juvenile for Beethoven, what’s typical Milhaud or Webern might be very prophetic for Berlioz or Mahler, etc. On a more contemporary note, you might also check some of the YouTube viewer threads at the links in Frank’s recent post about Eurovision: against the claim that Cascada is copying Loreen’s song, some fans claim that Cascada is being true to a style they’ve had since 2004.
While I think this impairs your analogy, I think it’s at best neutral about Kaimi’s point, too. The philosopher was saying it matters that Alkan is Alkan and that Cascada is Cascada, not necessarily that, say, Alkan was Jewish or that Cascada is a German band with an English lead singer.
Brett Bellmore - February 21, 2013 at 8:30 pm
“In fairness to Brett, you did argue in favor of affirmative action.”
In fairness, I took the argument to be something along the lines of, “Merit selection might have concealed bias, and the pool being selected from might be biased, and this, and that…” And really, doesn’t this come down to the only proof of the process being fair that you’ll accept is the numbers turning out “right”?
And if your only criteria for the process being fair is the numbers coming out right, and you feel entitled to tweak the process until it’s ‘fair’, how is this to be distinguished from a quota?
Leave a Reply