Author: Jeffrey Selbin

0

Greiner and Pattanayak: The Sequel

In a draft essay, Service Delivery, Resource Allocation and Access to Justice: Greiner and Pattanayak and the Research Imperative, Tony Alfieri, Jeanne Charn, Steve Wizner, and I reflect on Jim Greiner and Cassandra Pattanayak’s provocative article reporting the results of a randomized controlled trial evaluating legal assistance to low-income clients at the Harvard Legal Aid Bureau. (The Greiner and Pattanayak article was the subject of a Concurring Opinions symposium last March.) Studying the outcomes of appeals from initial denials of unemployment insurance benefit claims, Greiner and Pattanayak asked, what difference does legal representation make? Their answer is that “an offer of HLAB representation had no statistically significant effect on the probability that a claimant would prevail, but that the offer did delay the adjudicatory process.” That is, not only was an offer of legal assistance immaterial to the case outcome, it may have harmed clients’ interests.

The Greiner and Pattanayak findings challenge our intuition, experience and deeply-held professional belief that lawyer representation of indigent clients in civil matters is fundamental to the pursuit of justice. Our first reaction is that the study must have fatal conceptual or methodological flaws – the researchers studied the wrong thing in the wrong way. Even when we learn that the study is credible and well designed, we doubt that this kind of research is a worthwhile use of our time or money relative to serving needy clients. Finally, and perhaps most importantly, we worry that the published results will only serve as fodder for the decades-long political assault on legal services for the poor.

If replicated across venues, however, studies like Greiner and Pattanayak’s can tell us a great deal about individual representation, program design and systemic access to justice questions. In fact, we cannot make genuine progress in any of these areas – much less marshal the case for more robust legal aid investments and the right to counsel in some civil cases – without better evidence of when, where and for whom representation makes a difference. Fortunately, developments in law schools, the professions and a growing demand for evidence-driven policymaking provide support, infrastructure and incentive for such research. For these reasons, we urge legal services lawyers and clinical law professors to collaborate in an expansive, empirical research agenda.

 

1

Good to Evaluate Our Pet Notions

I’m delighted that Jim and Cassandra have conducted this study. We have accomplished many things in clinics, but we have not even scratched the surface of their potential to serve as sites of inquiry – what some of us have called “clinic lab offices” – to ask basic service delivery questions. Their literature review is a cautionary tale about what we know (or think we know) about the impact of representation. Their original research is pathbreaking, if controversial.

There is plenty to praise here, which I have done privately to the authors, but like Kevin I will focus on some of the challenges. In addition to the methodological limits of this study, the paper raises a lot of important questions that can only be answered through additional research, especially with respect to causal mechanisms. I’ve shared specific questions and suggestions in recent months, but my general view is that an organizational sociology/ethnography dimension – to generate more data about the administrative agency, the service provider and this particular subset of clients – would inform this study (and others) to help account for otherwise not obvious variables.

For example, w/r/t to the administrative agency, what incentives exist for ALJ to grant/deny benefits, especially to represented vs. unrepresented claimants? Do ALJs get punished/rewarded for false positives/negatives? My experience with agency adjudication is that there are often hidden nudges that operate much more powerfully than surface considerations (Social Security funds district offices based on the handling of some kinds of claims but not others; VA rewards certain workers for productivity, but not accuracy, etc.). How has this venue changed over time in response to focused representation by HLAB? Maybe this is a story of triumph – HLAB representation has been so effective that unrepresented claimants now have a level playing field, or at least a user-friendly setting.

In empirical work, the best data (in statistically significant terms) is often generated by the most narrowly drawn question (by eliminating/controlling for other causal mechanisms). That is, we sacrifice breadth for depth, which is a version of the limits that have frustrated critics of this study. RCTs are the research gold standard, but socially complex relationships can rarely be reduced to quantitative assessments, and are often better understood through the application of mixed methods, including a thick qualitative description of what’s motivating various actors within the system.

In 1967, referring to the HLS Community Legal Assistance Office established a year earlier and funded as a demonstration project by the federal Office of Economic Opportunity, Frank Michelman observed: “[W]e have undertaken to construct and demonstrate what we have been pleased to call a ‘model’ of a law-school-affiliated legal-services program. Stripped of pretension and reduced to practicality, what this means to me is that we are committed to a continuing effort to generate alternative methods, to put into operation whatever recommends itself to our objective appraisal, and to evaluate remorselessly our fondest pet notions.”

It’s taken more than 40 years, but Jim and Cassandra’s paper, with all its strengths and limits, is a testament to the potential and pitfalls of evaluating remorselessly our fondest pet notions. Hopefully the interest it has generated – and their forthcoming work, which I eagerly await – will spur others to undertake rigorous study of legal representation on behalf of the poor. The absence of such data impairs our ability both to allocate scarce resources effectively and to make the case persuasively to public and private funders for greater investments in the field.