By Andrew Cohen
Professor Justin McCrary admits that his varied scholarly interests often tempt him to “act like a Labrador, where every Frisbee looks fun to chase.” Legal analytics—which harness unwieldy data, improve research efficiency, and help predict outcomes—allows him to maintain a frenetic research pace.
McCrary and other Berkeley Law scholars are using new search technologies and various efficiency models to improve the study, analysis, and predictive quality of their work. The director of UC Berkeley’s Social Science Data Library, McCrary uses these advancements across myriad legal areas.
He recently helped build a Web portal for the California Attorney General’s Office—with extensive information on arrests, violence against police, and deaths in police custody. “This is a more helpful release and understanding of data than we’ve seen from any other attorney general in the U.S.,” he said. “It stands out as a beacon for best practices.”
Previously, McCrary worked with the Equal Employment Opportunity Commission to develop data systems that better reveal factors influencing private sector and governmental diversity. He also teamed with Berkeley Law Professor Robert Bartlett to analyze the organization of U.S. financial markets—and the impact of various features in regulating high-frequency trading.
“What’s exciting about seeing data analytics become an emerging field is that people from many different disciplinary perspectives are using similar tools,” McCrary said. “This helps Berkeley provide substantial technical assistance that lets state agencies and other organizations create more efficiency and transparency.”
McCrary and Professor Kevin Quinn are teaching a new class, Litigation and Statistics, which reveals how big data is changing legal practice—particularly litigation strategy. Quinn also explores analytics in the E-Discovery course he teaches with Professor Anne Joseph O’Connell, and follows developments in computer science, computational linguistics, and related fields that help to summarize vast amounts of information.
“Much of what legal academics pursue is rooted in large bodies of text, be it court opinions, agency regulations, or statutes,” he said. “Automated text analysis streamlines the process. It doesn’t eliminate the need for human judgment, but it can make more efficient use of one’s human resources by helping to focus attention on subsets of documents worth intensive review.”
Quinn has used such methods to align U.S. Senate floor speeches into topically coherent categories. He and fellow Berkeley Law Professor Mark Gergen have also analyzed cases from the New York Court of Appeals, a key court in tort and contract jurisprudence, during the first half of the 20th century. Their research unearthed an underlying structure of how judges decided these cases—which types of cases produced consistent voting and which ones were more unpredictable.
“Scholars too often look at only the high-profile cases rather than the full body of law,” Quinn said. “Sometimes, the high-profile cases are outliers that aren’t representative of more general patterns. These statistical methods for textual data have the potential to open up new areas of research.”
Seeing the patterns
A search method called Item Response Theory (IRT) has been used to depict patterned voting in the U.S. Supreme Court. Applying this model to the New York Court of Appeals, Quinn and Gergen found patterned judicial voting based on more than just political ideology. In some periods, the court’s right wing regularly favored defendants in workers’ compensation and constitutional rights cases, yet favored the plaintiff in personal injury cases that split the court.
Once the IRT model identified patterns of consensus and dissension among judges, Gergen searched for explanations. “The goal is to investigate the underlying dimensions of disagreement among judges across a wide range of cases,” he said.
Examining the degree of behavioral consistency within and between judges, Gergen and Quinn found that the nature of these disagreements changed over time. Examples of the bench divides: moralistic versus pragmatic, liberal versus conservative, and stability versus flexibility advocates.
The professors are now conducting a similar study of the California Supreme Court for the 20th century. Targeted keyword searches—using a term or multiple terms likely associated with an issue such as capital murder—helps them streamline the process. “While cases do eventually have to be read to confirm the correlation between subject matter and patterned voting, the effort is minimized,” Gergen said. “It’s also relatively easy to check against false positives and false negatives.”
Helping public law catch up
Professor Eric Biber and former Berkeley Law professor Eric Talley will soon begin analyzing appellate briefs and opinions in National Environmental Policy Act (NEPA) cases. They hope to develop predictions about how courts will decide them, which could help government officials determine if a development project might be successfully challenged by NEPA—and guide those interested in challenging NEPA compliance.
“These cases are very fact-specific and sometimes unpredictable in terms of how courts resolve them,” Biber said. “Because they all refer to one main statute, we think it’ll be easier to unpack what’s going on.”
After examining the judicial opinions and related briefs, he and Talley will try to identify patterns that correlate with outcomes to build the infrastructure for their analysis.
“We want to get a sense of how to do this predictive work in public law,” Biber said. “Much has been in done in private law, and public law may be tougher because it involves a wider range of statutes. But it’s vital for non-government entities and small organizations to have access to these same methods.”