Lawyers, Bots, and Query

The use of AI in legal research is becoming more common.

AI.

Have ever two letters looked so sinister together?

“A” is the alpha. The firstborn. Numero uno. Firsts are often a little stiff, burdened as they are with being so visible, so vulnerable—if this whole thing fails, won’t it be their fault? Yet with great responsibility comes great likeability.

As for “I,” who among us can’t relate to its skyscraper-like appearance, signifying that attention must be paid? Put these revered runes together, however, and they spell D-O-O-M, according to some. Others see the digraph not as dastardly but as our salvation.

Of course, artificial intelligence, or AI, is neither good nor bad. It is a tool, an apparatus, a thing to be used. No doubt many of those uses are outstanding. Yet the downsides can be significant, too.

Send an email to your Representatives to show your support for libraries!

For example, according to the Brennan Center, a recent survey asked Americans about their concerns regarding AI, and 85 percent said they were “very concerned” or “somewhat concerned” about the spread of deepfakes: artificially generated pictures or videos powered by a type of machine learning called “deep” learning.

Moreover, AI makes it easy to copy—and distort—the work of artists and photographers. And programs like ChatGPT enable students to produce competent but fraudulent term papers in minutes. (One study by the plagiarism detection company Turnitin found that as many as 11 percent of student papers contained some AI-produced content.)

AI software is also making inroads into the practice of law. There have been headline-making missteps, such as when Jae S. Lee, a New York attorney, was sanctioned for submitting a court filing that included a fake citation—one supplied to him by ChatGPT. Other cases have involved similar errors. It even happened to Michael Cohen, Donald Trump’s fixer-turned-foe. 

This tendency of chatbots to fictionalize content is called “hallucinating,” and it can cause real problems. In his 2023 Year-End Report on the Federal Judiciary, Chief Justice John Roberts highlighted such failures, saying they reinforce the view that “human adjudications, for all of their flaws, are fairer than whatever the machine spits out.”

Take action today to support libraries!

None of this means, however, that AI is useless in the field of law. It does some things very well. Back in 2017, the American Bar Association described some of those things.

• "review of documents for discoverable or otherwise relevant information, generally referred to as technology-assisted review (TAR).
• legal research through automated searches of a universe of case law and statutes.
• contract and legal document analysis.
• proofreading, error correction and document organization."

I agree with document review, contract analysis, and proofreading, as those tasks are formulaic, meaning they require less originality, inspiration, and high-level problem-solving than other duties. Using AI software for these minutiae can free up attorneys to concentrate on weightier activities.

What I question is the inclusion of “legal research,” which, if my twenty-plus years as a law librarian are a representative sample, rarely follows a pattern. Some aspects are rote—retrieving a document by its citation, for instance. The hard part is when an attorney knows what she wants to write and what argument will best help her client, but there isn’t a citation to an obvious supporting document.

Or maybe the attorney knows the kind of content needed, but that document doesn’t use the expected language. This happens all the time in legal practice. For example, suppose a researcher needs the North Carolina statute giving buyers the right to demand repairs to defects in their new car within the first year or two of ownership. Most states have such laws, which are colloquially called “lemon laws.” North Carolina’s is N.C.G.S. Chapter 20, Article 15A. Yet if the researcher types “lemon law” into a legal database, they won’t find this statute. Why? The words “lemon law” don’t appear in it.

Sign the pledge to vote for libraries!

Legal research, then, is rarely as cut-and-dried as contract drafting or document review. It requires flair, ingenuity, resourcefulness, stick-to-it-iveness, and not a little luck. Of course, that hasn’t stopped the major providers of legal research databases—Westlaw, LexisNexis, and Bloomberg Law—from developing generative AI add-ons to their systems. Paxton, Vincent, and Copilot are some low-cost competitors.

I recently asked some attorneys I work with to test the Westlaw and Lexis AI modules. They took some questions they had already answered and fed them into the modules, comparing the AI responses to their own previous research.

One attorney found the AI responses to be “almost worthless.” The question was about restrictions on abortion prior to twelve weeks, which, having already researched the issue, he knew to be prohibited under state law. He posed the question in several forms, and each time, the AI assistant gave him the wrong answer. “If this had been a request from a legislator,” he told me, “and I hadn’t known the answer in advance, I would have passed along incorrect information.”

Another attorney found the AI feature useful, though he stressed that it would be important to “check the information, verify it, and do independent research.” To me, this doesn’t generally sound like a time saver—if I’m going to verify AI results with independent research, then I may as well rely only on that research—but there may be projects where it could help.

Westlaw and LexisNexis have been around for decades. When they first came along, many attorneys saw them as harbingers of the death of books, not to mention the erosion of legal research skills. Now, of course, one can scarcely be a legal researcher without them.

AI is the next evolution, whether everyone likes it or not. The technology isn’t perfect, but it will get better. Those who resist it will eventually be left behind.

Visit www.everylibrary.org to learn more about our work on behalf of libraries.

#librarymarketers: Enjoy this story? Want to use it for your library newsletter, blog, or social media? This article is published under Creative Commons License Attribution-NonCommercial 4.0 International and is free to edit and use with attribution. Please cite EveryLibrary on medium.com/everylibrary.

This work by EveryLibrary is licensed under CC BY-NC 4.0