Predicting Supreme Court Outcomes Using AI ?
Is it possible to predict the outcomes of legal cases – such as Supreme Court decisions – using Artificial Intelligence (AI)? I recently had the opportunity to consider this point at a talk that I gave entitled “Machine Learning Within Law” at Stanford.
At that talk, I discussed a very interesting new paper entitled “Predicting the Behavior of the Supreme Court of the United States” by Prof. Dan Katz (Mich. State Law), Data Scientist Michael Bommarito, and Prof. Josh Blackman (South Texas Law).
Katz, Bommarito, and Blackman used machine-learning AI techniques to build a computer model capable of predicting the outcomes of arbitrary Supreme Court cases with an accuracy of about 70% – a strong result. This post will discuss their approach and why it was an improvement over prior research in this area.
Quantitative Legal Prediction
The general idea behind such approaches is to use computer-based analysis of existing data (e.g. data on past Supreme Court cases) in order to predict the outcome of future legal events (e.g. pending cases). The approach to using data to inform legal predictions (as opposed to pure lawyerly analysis) has been largely championed by Prof. Katz – something that he has dubbed “Quantitative Legal Prediction” in recent work.
Legal prediction is an important function that attorneys perform for clients. Attorneys predict all sorts of things, ranging from the likely outcome of pending cases, risk of liability, and estimates about damages, to the importance of various laws and facts to legal decision-makers. Attorneys use a mix of legal training, problem-solving, analysis, experience, analogical reasoning, common sense, intuition and other higher order cognitive skills to engage in sophisticated, informed assessments of likely outcomes.
By contrast, the quantitative approach takes a different tack: using analysis of data employing advanced algorithms to produce data-driven predictions of legal outcomes (instead of, or in addition to traditional legal analysis). These data-driven predictions can provide additional information to support attorney analysis.
Predictive Analytics: Finding Useful Patterns in Data
Outside of law, predictive analytics has widely applied to produce automated, predictions in multiple contexts. Real world examples of predictive analytics include: the automated product recommendations made by Amazon.com, movie recommendations made by Netflix, and the search terms automatically suggested by Google.
Scanning Data for Patterns that Are Predictive of Future Outcomes
In general, predictive analytics approaches use advanced computer algorithms to scan large amounts of data to detect patterns. These patterns can be often used to make intelligent, useful predictions about never-before-seen future data. Many of these approaches employ “Machine Learning” techniques to engage in prediction. (I have written about some of the ways that machine-learning based analytical approaches are starting to be used within law and the legal system here).