Success: California Law Review Drops Foolish Rules
posted by Dave Hoffman
Several weeks ago, I blogged about rules promulgated by the California Law Review that appeared to discourage empirical submissions. Michael Heise, at the ELS blog, then added to the chorus.
For fun, I checked the website of the CLR today, and found that the offending rules (#4, no images or graphics; and #5, only five charts/graphs/tables allowed) were gone. Way to go, anonymous California Law Review student editors!
Next target: the Yale Law Journal’s incomprehensible policy on replication, which, as Katie Porter pointed out, seems to require authors to send YLJ a copy of STATA/SPSS with their submission. Query: has anyone actually complied with this rule? If so, did you pay for an extra program license?
April 6, 2008 at 3:42 pm
Posted in: Law Rev (California), Law School (Law Reviews)
Print This Post
Responses (8)
anon - April 6, 2008 at 4:46 pm
There is nothing “incomprehensible” in the YLJ’s policy on replication. They want you to send them your dta and do files, not the STATA package itself. This is a completely standard request in econ/poli-sci/etc journals. What *is* incomprehensible is the number of law school “empiricists” who don’t understand the difference. Hey, at least Katie Porter doens’t think YLJ wants her motherboard too! Along with her office chair.
tim zinnecker - April 6, 2008 at 5:09 pm
If only the “top tier” law reviews would abolish the unwritten rule which states: “We shall not extend an offer of publication to the author of any manuscript which includes one or more citations to either (i) the federal income tax code or (ii) the Uniform Commercial Code until after we have extended an offer of publication to every other author (living, dead, or imaginary) who has, in the past twelve months (or longer, if necessary) submitted a manuscript, essay, book review, or e-mail message which addresses — directly or indirectly, obviously or unintentionally, seriously or sarcastically, or in the text or any footnote — any issue pertaining to constitutional law, civil procedure, evidence, criminal law, criminal procedure, antitrust, real property, personal property, intellectual property, admiralty, labor law, insurance law, torts, administrative law, ADR, business associations, environmental law, family law, immigration law, or any other non-tax or non-UCC subject taught at any law school (accredited or otherwise) in the United States (or elsewhere).”
Rafael Pardo - April 6, 2008 at 5:15 pm
Here is the Yale Law Journal’s submission guideline (copied verbatim) regarding data:
4. Data: For empirical work, we require access to all materials needed for replication (including computer programs and data sets). Please provide us with: (a) a web link, (b) an electronic copy,
or (c) a CD-ROM of the materials.
The guideline clearly distinguishes between (1) computer programs and (2) data sets. Using STATA as an example, the software itself would fall within the first category and .dta files would fall within the second category. If all the Yale Journal is requesting are .dta and .do files from authors (as anon suggests), their guideline goes well beyond that by requesting a computer program. And I do not think one can interpret the request for a “computer program” as a request for a do-file. Do-files are text files that Stata can read and execute. Without the software (i.e., the computer program), do-files are worthless. As currently written, I think one must conclude that the guideline requests statistical software from authors. I would agree with Professor Hoffman’s assessment that such a request is absurd.
AE - April 6, 2008 at 5:49 pm
With this success with Cal. L. Rev., maybe you can go after the Westlaw people next. Doesn’t it seem likely that part of the concern with publishing graphics and the like is that when people go pull up the articles on Westlaw or Lexis, they won’t be able to see these graphics/tables/etc.? If a lot of the value of an article is in this information, and the primary type of access doesn’t make this information available (or at least as accessible?), that’s a substantial problem.
david hoffman - April 6, 2008 at 7:18 pm
Anon: If the YLJ only wanted the .dta and do files, it could have found a more clear way of saying so. I read the rule as Professor Pardo does: a request for the underlying computer program.
AE: I agree with you 100%. WL/Lexis need to do better. I’ll post on this soon. (On the exceedingly remote chance that they care.)
Andrew - April 6, 2008 at 10:06 pm
That’s not how I read their policy–I read “programs” as including .do files (and their equivalent), not requiring an upload of Stata or another commercial software package.
Professor Pardo’s interpretation of the word “program” would exclude everything written in an interpreted programming language (or scripting language). Everything written in Python and Perl is equally “useless” without an interpreter. (I disagree that they are, in fact, truly useless–someone can read the code and see their methodology, even if one does not execute it. For that matter, compiled .java files are useless without a Java Runtime Environment.) In executing .do files, Stata is more like a Python or Perl interpreter than like a standalone application. If I sent them a Python .py file, it would be absurd for me to send them the Python interpreter…I’d just say, “Run this with Python.” (I might elaborate a bit more, but that’s effectively what I’d say.) But I think of the .py file as the “program.” As Professor Pardo says, the the guideline clearly distinguishes between programs and data sets. But .do files are manifestly not data sets, so they must be “programs,” right? Just like Python and Perl scripts are “programs,” even though they, like .do, are technically “scripts” that require an interpreter. Does you guys think that if I write something in a .bat file, that they expect me to send them a copy of Windows as the “underlying computer program”?
As Professors Pardo and Hoffman agree, such a requirement would be manifestly unreasonable. But they set aside the principle of charity, and choose the unreasonable interpretation of Yale’s instruction, because they “could have found a more clear way” to request so. I wonder: what would such an instruction look like? “including data sets and programs, except for commercially available programs?”
david hoffman - April 7, 2008 at 10:54 am
I’m pretty sure that they could say simply:
“4. Data: For empirical work, we require access to all materials needed for replication (including computer scripts and data sets). Please provide us with: (a) a web link, (b) an electronic copy,
or (c) a CD-ROM of the materials.”
This would clarify the requirement, while making it possible to editors to replicate any findings if they’ve a licensed copy of STATA/SPSS/other around in a nearby stat lab.
Paul Gowder - April 7, 2008 at 5:39 pm
Yet another advantage for R users.
Leave a Reply