What Difference Representation – A Response
I am the Executive Director of Greater Boston Legal Services, the primary provider of civil legal services to poor people in the greater Boston area. My program and I have a great stake in assuring that our limited resources are used where they can be most effective. Indeed we are participating with Professor Greiner in a study of the impact of our staff attorneys’ representation in defense of eviction cases. My comments refer to the draft dated February 12, 2011.
It is important with any study; however, to know what it concludes and what it does not. For instance, and most importantly, the study concedes on page 43 that it could draw no conclusions about the effect on outcome for claimants actually receiving representation, as opposed to just an offer of representation. Thus, this study should be recognized for what it is: a limited analysis of the somewhat abstract concept of “offering” assistance. Indeed, the study wisely cautions against drawing any conclusions from the study about the usefulness of free legal assistance or even about the usefulness of offers of representation in unemployment cases in general (page 47).
I feel some changes are necessary to avoid much confusion about (and misuse of) this study’s conclusions (or lack thereof) as to the effect of representation itself, as opposed to just the offer. For instance, given that this study’s principal conclusions are about an offer of representation and not actual representation, a more accurate title to this study would be, “What Difference an Offer of Representation?” And the very first sentence of the Introduction on page 5 currently reads, “Particularly with respect to low-income clients in civil cases, how much of a difference does legal representation make?” It is only a footnote that explains the study looks at offers as well as effect, and much later in the study (page 32) that no conclusions were reached at all as to the effect of representation. Similarly, the conclusion (“Where Do We Go From Here?”) states, “the present study primarily concerned representation effects on legal outcomes affecting the potential client’s pecuniary interests.”
I am concerned also that the results reported in the study with respect to offers of representation by HLAB are misleading at best and of little utility at worst. This is because nearly half of the control group were represented by counsel and, more significantly, probably that many and perhaps more in the control group got an offer of free representation from my program or another providing free legal services in unemployment cases. To make an analogy to the medical world, suppose there was a Pfizer drug trial where 50% of Pfizer’s control group were offered the exact same medication from Merck. Wouldn’t that cast serious doubt on the outcome of the study? There is no mention of this 49% in either the abstract or introduction which unfortunately are all many readers will read.
The article purports to study the impact of an “offer” of representation rather than the impact of actual representation. One implication of studying offers is that the control group i.e. the group that did not get an offer, may, and probably will, contain some people who will end up being represented because they will have found other means to obtain representation. The article refers to them as “Go-Getters” (text accompanying footnotes 133 to 137).
The problem in the study with the control group is that during the study period in the same geographical area several other organizations including my own were also making offers of free representation to those in the control group. This was built into program design. “If the randomization was not to offer, the student attorney so informed the claimant by telephone and provided her with names and telephone numbers of other legal services provides in the area who might take her case.” (text accompanying footnotes 103 to 104) In the study 49% of those in the control group ended up being represented. (text accompanying footnotes 134-136).
The article characterizes those who in the control group who obtained representation as “Go-Getters” and with no empirical evidence the authors say:
Our instincts are that the process of finding an attorney once turned down by HLAB required effort, energy articulateness, and persistence from a claimant. (text accompanying footnotes 136-137).
This could be plausible for those in the control who found a private attorney. But in Boston, there were other organizations, primarily my own, also offering free representation. And everyone in the control group was given our phone number and phone numbers of other free providers.
Participants got into the study by making a phone call to HLAB. (text accompanying footnotes 101-102) HLAB is simply one of several organizations listed in the information provided by the Massachusetts Division of Unemployment Assistance to all unemployment claimants participating in hearings concerning their eligibility for unemployment insurance benefits. So this was a group with the ability and desire to make a phone call in order to get representation. All that was needed “to find an attorney once turned down by HLAB” was to make a second phone call to another organization listed by the Division or provided by the HLAB student. This hardly seems to require a greatly different amount of “effort, energy and articulateness and persistence.”
Our staff reports that in at least some cases, the HLAB student attorneys, concerned that persons in the control group who especially needed assistance would end up without representation, actually sent GBLS the names and phone numbers of the persons randomized out. Our staff then called and in each such instance both offered and provided them representation. So, in at least some cases, those in the control group got second offers of free representation with no extra effort on their part. For those making phone calls in the summer, which were apparently a very small part of the group studied, our program had a special line set up to represent people in UI cases, so all that was needed was one phone call.
Most callers seeking representation, would be screened by our sister organization LARC (the Legal Advocacy and Resource Center). In a slightly different context the article discusses the difficulties in getting through to LARC (text accompanying footnotes 241-245) To be sure there are times when it is hard to get through to LARC. At other times, there is no significant delay. LARC annually handles 12,000 phone calls a year, and last year 1000 of them concerned employment matters. So it is equally plausible to say that that for those in the control group, it was not significantly harder to get through to LARC than for them to have made their initial phone call to HLAB. This is particularly so since LARC is open after working hours while HLAB is only open during working hours and during the summers and school vacation has even more limited hours. Moreover, as described above, individuals are provided with the telephone numbers of a range of free providers by the Division of Unemployment Assistance. At the least, there is no evidence that folks we represent who got through the LARC line are different from those in the study group.
In continuing to try to minimize the 49% figure, the study states, “The second hypothesis we do not credit is the possibility that that the control group had too many claimants who ended up finding representation so that there is an insufficient contrast… [between the two groups].” (text accompanying footnotes 149-150).
But the study does not set out what difference there might have been the between represented and unrepresented. If the represented in the control group got better outcomes than the group as a whole, then removing them from the group would result in the control group showing worse outcomes. And if, as I suggest, the represented group were not “go getters” but simply people who did get an offer after either doing nothing or making one more phone call, then an offer of representation from HLAB or another free service does make a difference.
Nor does the article study the final “outcome” of any of the cases measured. In the treatment group, HLAB’s victory in cases ensured that the claimant obtained unemployment benefits. If the claimant lost HLAB would continue to represent these claimants and might end with a favorable outcome. Moreover, even if claimants “win” a hearing, if there are other “issues” needing to be adjudicated, such as immigration status, whether or not the claimant is able and available for work, etc. further representation is often needed to secure benefits. In contrast, for the control group, we have no data on what happened after the initial win or loss at the hearing for the represented or unrepresented group. If the study purports ultimately to measure what difference representation makes, the final outcome for the claimant is important to know.
To measure truly the impact of an offer of free representation, I suggest the study needs to remove from the control group those who also had an offer of free representation or do some statistical taking into account of those in the control group who also had an offer of free representation. (This is different from removing everyone in the control group who got representation which the article with some reason rejects (text accompanying footnotes 137-138)). For those who got a private attorney (I suspect few, if any) the go-getter hypothesis is at least plausible.) Without removing the impact of those in the control group who had an offer for free representation, the conclusion of the article are not warranted.
The possibility that the offer of representation made a difference is strengthened by the fact that the win rate in the control group was higher than the state average. When claimants in the control group appealed they won 65% of the time compared to the state average of 47%. When claimants in the control defended they won 83% of the time compared to the state average of 75% (text accompanying footnote 150) The article gives three hypotheses to explain this(text accompanying footnotes 150-155) but never suggests what to most lawyers would be the obvious conclusion, 49% of the were represented by counsel. I do not have the data, but it seems unlikely that in the state as whole close to 75% of the claimants in the system are represented (49% of the control group and virtually all of the studied group).
Why is this so important to me? I believe that the title and abstract, as well as the control group issue have the possibility of making the study and the article very misleading. If this were simply a case of a purely academic study then I would be more than happy to leave it to the academics and use my time to provide legal services. Further studies might clarify things and get us all in agreement. But further studies are way in the distance. Even our eviction defense study with Profess Greiner is months away from being finished. Meanwhile, despite all the careful caveats in the article, it is being cited by others for very broad propositions. In the Turner case now before the U.S. Supreme Court this article is cited for the proposition that lawyers are unlikely to make a significant difference in civil contempt hearings.
If the article is misleading, or leads to a misreading, it could do real harm to people who do need assistance of counsel for some basic human needs.