Wikitruth Through Wikiorder
posted by Dave Hoffman
Almost four years ago, I blogged at Prawfs about a weird dispute on Wikipedia about the Kelo case. I wrote that “[t]here is a whole ADR and conflict resolution system being set up behind the scenes, in the absence of (a) money; (b) the Bar; or (c) personal contact. And we don’t have to go to Shasta County for months on end to see it.”
Wiki’s DR process continued to fascinate me, and I eventually teamed up with Temple’s Salil Mehra, a comparative IP scholar, to write about the system. We’ve finished just finished a draft, which starts with the following snippet:
Charles Darwin and Abraham Lincoln were both born on February 12, 1809. When some individuals hear about this coincidence, it seems remarkable. To others, it is mundane. To Wikipedia editors working on the encyclopedia’s articles about Darwin and Lincoln, the factoid was the subject of a contentious dispute resolution process that encompassed two polls, outside editor comments, a request for mediation, and a formal arbitration proceeding that generated over 30,000 words in evidentiary submissions and thousands of volunteer man-hours.The problem motivating the fracas was whether or not the shared birthday merited inclusion in the Wikipedia’s biography of Darwin. Because Wikipedia’s editing process is open, editors who disagree might endlessly recycle their views, leading to unstable articles, entrenched disagreement and a loss of initiative, altogether destroying the site’s utility. In response, Wikipedia has developed a volunteer-run, highly articulated, dispute resolution system. That system starts with the informal, guided, exchange of views, muddles through mediation, and terminates in an Arbitration Committee, which hears evidence presented by the parties before issuing findings of fact and conclusions of policy and law. Such decisions, organized by volunteer arbitration clerks and disseminated by volunteer reporters, have created a virtual Wiki-common law.
As the result of the binding arbitration in the Darwin Birthday Dispute, two editors were banned from the site for a month for their lack of cooperation with others, and one was further prohibited from editing either Darwin’s or Lincoln’s article. A third individual was formally thanked by the arbitrators for his work as a counselor to one of the banned parties. The Arbitrators, per their usual rule, did not resolve the content of the dispute: non-banned parties were free to continue testing whether the Emancipator and the Scientist’s shared birthday was worthy of note.
There are at least two separate levels of strangeness about this story.
• Why do people spend time editing Wikipedia articles and why they would care enough about this particular fact to disagree?
• Why does Wikipedia have a dispute resolution system that doesn’t resolve disputes?
Interested in reading more? Download our draft, which just went up on SSRN. Or, if you are a law review editor, check your inbox. We’re in there!
March 6, 2009 at 6:32 pm
Posted in: Wiki
Print This Post
Responses (19)
eric - March 6, 2009 at 6:23 pm
Extremely interesting. I can’t wait to read the full piece. Just one question: why wouldn’t you want to spend several months in beautiful Shasta County?
Mike Guttentag - March 6, 2009 at 9:00 pm
I can offer one answer to the riddle about why people spend so much time trying to resolve disputes in a quasi-legal manner. My claim is that humans instinctively turn to legal systems to organize social behavior. I’ve just posted a paper, “Is There a Law Instinct?” at http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1346649 . This paper formalizes and evaluates the hypothesis that reliance on legal systems to organize social activity is an integral part of human nature, just as language and morality now appear to be directly shaped by innate predispositions.
What do you think? Am I allowed to mention that this paper is also now available for publication on your blog?
anon - March 6, 2009 at 10:04 pm
You might be interested to know that one of the current members of Wikipedia’s Arbitration Committee is actually a law student with a strong interest in IP law and scholarship.
A.J. Sutter - March 6, 2009 at 11:45 pm
Regarding Mike Guttentag’s comment and referenced article: I’m agnostic about the existence of a law instinct, but would bet a hefty chunk of change that there is a universalization instinct among academics in Anglo-Saxon countries, which makes them leap from the categories of Anglo-Saxon norms and scholarship directly into biological generalizations (especially ones dealing with evolution or “neuro”- anything). Thaler & Sunstein’s Nudge is an example, along with Mike G.’s paper.
Mike, you don’t have a single cite to a foreign-language source, much less to any primary sources concerning anthropological evidence. Moreover, the standard of evidence is the weak one encouraged by the student editors of law reviews, viz., if you can cite a source supporting your position, however specious that support may be, you have borne your burden of evidence. Your discussions of ancient law are based exclusively on written texts that have chanced to survive, which says little about how the law operated in life; cf. the gap between the Old Testament (and, to some extent, even the Talmud Bavli) and Jewish religious law in practice. To assert that less than four pages of text, citing only to law professors and one 45-year-old anthropological article, is sufficient for a review of pertinent anthropological evidence (@49), is not at all persuasive, to put it mildly. The unquestioned reification of the concept of “instinct” and ex-post evolutionary arguments are additional weaknesses. (You define instinct as “a behavior … produced by a complex interaction between genes and the environment” (@9n34) — one might as well talk about a blogging instinct or a television-watching instinct: what behavior, perhaps other than autonomic functions like breathing, is not so produced?)
I’m not by any means singling you out personally for the complaints in this comment; you have far too much company among your peers. Since this post began with Darwin, his example of allowing a long brewing time for his ideas might be among the better ones to emulate (notwithstanding its Anglishness). Another example is the footnotes to Marcel Mauss’s The Gift [Essai sur le don]. I suggest you retract the paper for now and reconsider it after spending a few decades deepening your scholarship, especially outside the confines of your current intellectual tradition, if your commitment to this topic comes from something more sincere than careerist opportunism. Perhaps that could include some primary anthropological research of your own.
A.J. Sutter - March 7, 2009 at 10:27 am
Back on point, about the Wikipedia paper: it’s quite an interesting process that you’ve documented, and the idea of the conflict resolution process being constitutive of the community (at least, of a portion of the community, since I guess not everyone goes through the LSSP) is an interesting one. Two minor points: (i) mightn’t “cultivate” be a better metaphor than “weeding in”?, and (ii) I didn’t think the game theory analysis was necessary for the paper, and maybe you should leave it out in future versions — less may be more in this case.
dave hoffman - March 7, 2009 at 11:52 am
Thanks all for the comments, keep ‘em coming!
To anon: I didn’t know that there is a law student arbitrator, although I knew there were as many as two practicing lawyers. I sent the paper to the arbitration committee, and I hope to continue to receive feedback from them. (I’d also like to get feedback from current mediators, but that group seems quite diffuse.) What’s terrific about Wikipedians, apart from their generosity in sharing their insights, is that they are generally quite self-conscious in how they go about their work.
A.J.: Sorry you didn’t enjoy the game theory. That particular section responds, as you may know, to a particular argument in the literature about how to model social production. Not for everyone, but it makes a contribution, I believe.
Mike Guttentag - March 7, 2009 at 12:27 pm
To A.J. Sutter: Thanks for taking a look at the paper, and realize this is not the place to go into a full-fledged discussion. Still, I would make two observations. First, I do consider this paper a beginning rather than an end to the exploration of the law instinct hypothesis. Second, I did not intend to put too much weight on the anthropological and historical discussion, in large part because I do not think such analysis will ever be able to prove or disprove my claims.
The more promising avenue of research is, I believe, experimental work, both in developmental psychology and, especially, economic experiments. Experimental work is crucial to the effort to uncover universal patterns that cannot be explained as a product of culture. There is much exciting experimental work going on directly relevant to my claims, which the paper attempts to present fairly.
Seth Finkelstein - March 7, 2009 at 10:17 pm
“Why do people spend time editing Wikipedia articles and why they would care enough about this particular fact to disagree?”
Because Wikipedia is a cult that preys on a certain mindset, e.g. people who want to be considered professors. See the “Essjay” fraud for a blatant example:
http://www.guardian.co.uk/technology/2007/mar/08/media.comment
“Why does Wikipedia have a dispute resolution system that doesn’t resolve disputes?”
Because it makes the supposed resolvers feel very important.
Seth Finkelstein - March 7, 2009 at 10:22 pm
Hmm, let me try to link that:
http://www.guardian.co.uk/technology/2007/mar/08/media.comment
In fact, see my _Guardian_ columns page, many of which discuss Wikipedia from an uncommon perspective.
Seth Finkelstein - March 7, 2009 at 10:27 pm
Also, go tell the folks at Wikipedia Review about your paper – they’ll love it.
Especially the part about “The data shows that Wiki-dispute resolution ignores the content of user disputes, instead focusing on user conduct”.
A.J. Sutter - March 7, 2009 at 11:05 pm
Mike, thanks for taking my comments in the constructive spirit in which they were given. About experimental work, though:
(1) Unless your experimental “subjects” come from a wide variety of cultures and social strata, they don’t reflect much other than about the group from which they come. E.g., university students in developed countries — and probably anywhere — are not a sufficient basis from which to draw universal conclusions; nor are managers. Stephen Marglin mentions in The Dismal Science that economics students are particularly more self-interested than other experimental participants; if L&E is anywhere near to being as influential on current legal education as I infer from reading this blog, I don’t imagine law students would be a whole lot less tainted.
(2) An even bigger problem with experiments is that the categories themselves that are used for framing the hypotheses are very culturally determined. The categories of “instinct” and even “law” are examples. The catgories of neoclassical economics and its incrementally tweaked version, behavioral economics, are not at all universal. (Contrary to what many people in Anglo-Saxon countries think, behavioral econ is not universally regarded as revolutionary.) This is where becoming acquainted with more heterodox versions of economic, social and psychological thought is important. And one doesn’t need to go to West Africa or Inner Mongolia before finding heterodoxy: you can start with France — the economy of conventions, antiutilitarianism (MAUSS group), décroissantistes, anthropolgical work of Pierre Clastres, etc. etc. To say nothing of Ivan Illich.
dave hoffman - March 8, 2009 at 12:32 pm
I’ve read your column about the Essjay incident, but I wonder whether you are being too quick to assume motive in others. The paper makes a different set of arguments about motive and about the reason that the dispute resolution system is structured the way it is. We’d love your comments, if you’ve a chance to read it.
I looked at Wikipedia Review, and couldn’t find a way to easily post about the article. You are welcome to — we’re certainly in the market for more feedback.
anonymous coward - March 18, 2009 at 8:31 am
Is your flowchart intentionally hilarious, or is that really how you perceive the structure? Of course it’s funny precisely because it has a lot of truth in it…
I think you overemphasize the role of the Wikiquette noticeboard (given a section in the paper) and Editor review (mentioned in your flowchart). My impression was that many more conflicts are channeled through the Administrators’ noticeboard and its Incidents subpage on their way to ArbCom, possibly via an RfC. Did your analysis include counting which venues of dispute resolution were tried before arbitration was accepted? I would also be interested to see if the number and type of pre-ArbCom DR attempts is related to the ultimate remedies, which would naturally lead into studying how successful those venues are.
THOR - March 18, 2009 at 4:09 pm
For what it’s worth to your paper, Tthe actual article “February 12, 1809″ was nominated for deletion in August 2006. See: http://en.wikipedia.org/wiki/Wikipedia:Articles_for_deletion/February_12,_1809
dave hoffman - March 18, 2009 at 4:41 pm
The flowchart isn’t mine. I pulled it from a wikipedia page describing the DR system. I find it sort of hilarious. Regarding AC’s other comments, I think that studying the pre-ARB-Com DR process is a valuable extension of our work, which we’re currently looking into.
Jon Awbrey - March 21, 2009 at 11:02 pm
Just a data point for your next study:
Mignight Judgment
Based on several years observation, I think you will find that the above example is much more typical of how Wikipediot disputes get “resolved” — more by lynchings in the dead of night than anything resembling even so much as the pseudo-legalisms of that kangaroo court known as ArbCom.
All the best in your “further research” — it is indeed much needed.
Barry Kort - March 22, 2009 at 1:24 am
The “Law Instinct” was treated in Fyodor Dostoevsky’s novels (e.g. Crime and Punishment and The Brothers Karamazov).
It’s not so much an instinct as a lamentable meme that got lodged in human culture about 4000 years ago.
It’s a lamentable meme because, while on the face of it, it looks like a good idea, it’s a mathematically flawed idea.
Deep thinkers have long sensed the flaw in the Crime and Punishment Regulatory Model, but only in the past century has the level of scholarship emerged from the domain of theology to the domain of mathematical systems theory to expose the flaw.
Alas, while it’s easy to appreciate that such technical analyses exist both in ancient theological texts and in modern systems theoretic papers, the analysis seems too daunting for the average layperson to digest.
Jon Awbrey - March 22, 2009 at 10:23 am
There is an ongoing discussion of this topic at The Wikipedia Review, on this thread.
P.S. That was a typo for “Midnight Judgment” — no allusion to old Russian fighters was intended.
ttt - April 11, 2009 at 9:28 am
Saturday, 9:27 am EST, April 11.
Sir, you paper, “Wikitruth through Wikiorder”, is no longer available to download on:
http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1354424
When ever I attempt to access this page, I get an error message “Service Unavailable” In fact, the entire site, http://papers.ssrn.com, I get a message, “Service Unavailable”
Leave a Reply