Alaska's AI Court Chatbot Faces Troubled Launch

Table of Contents

The Challenges of Implementing AI in Legal Systems

Dealing with the belongings of a loved one after their passing is an emotionally challenging task. However, in Alaska, the state courts have faced an unexpected challenge: an artificial intelligence chatbot that has proven to be unreliable and misleading. This situation highlights the difficulties that government agencies across the United States face when trying to implement powerful AI systems in real-world scenarios where accuracy and reliability are crucial.

Alaska’s court system has been working on a project called the Alaska Virtual Assistant (AVA), which was intended to help residents navigate the complex process of probate. Probate involves transferring property from a deceased person to their heirs and involves numerous forms and procedures. AVA was meant to be a quick solution to increase access to justice, but the project has taken over a year to develop.

“Aubrie Souza, a consultant with the National Center for State Courts (NCSC), said that AVA was supposed to be a three-month project,” she explained. “We are now at well over a year and three months, but that’s all because of the due diligence that was required to get it right.”

The development of AVA has revealed the challenges that government agencies encounter when using AI for real-world applications. Stacey Marz, the administrative director of the Alaska Court System and one of the AVA project leaders, emphasized the importance of accuracy in such projects.

“With a project like this, we need to be 100% accurate, and that’s really difficult with this technology,” Marz said. “I joke with my staff on other technology projects that we can’t expect these systems to be perfect, otherwise we’d never be able to roll them out. Once we get the minimum viable product, let’s get that out there, and then we’ll enhance that as we learn.”

However, Marz believes that this chatbot should be held to a higher standard. “If people are going to take the information they get from their prompt and they’re going to act on it and it’s not accurate or not complete, they really could suffer harm. It could be incredibly damaging to that person, family or estate.”

Many local government agencies are experimenting with AI tools for various purposes, from helping residents apply for a driver’s license to speeding up municipal employees’ ability to process housing benefits. However, a recent Deloitte report found that less than 6% of local government practitioners were prioritizing AI as a tool to deliver services.

The AVA experience demonstrates the barriers that government agencies face in leveraging AI for increased efficiency or better service. These include concerns about reliability and trustworthiness in high-stakes contexts, along with questions about the role of human oversight given fast-changing AI systems.

Marz envisioned AVA as a low-cost version of Alaska’s family law helpline, which provides free guidance about legal matters ranging from divorce to domestic violence protective orders. “Our goal was to basically try to replicate the services with the chatbot that we would provide with a human facilitator,” Marz told NBC News.

Tom Martin, a lawyer and law professor who launched a law-focused AI company called LawDroid, developed the chatbot. He highlighted the critical decisions and considerations involved in the design process, such as choosing and shaping an AI system’s personality.

“Different models have almost different types of personalities,” Martin said. “Some of them are very good at rule-following, while others are not as good at following rules and kind of want to prove that they’re the smartest guy in the room.”

For a legal application, you don’t want that, Martin added. “You want it to be rule-following but smart and able to explain itself in plain language.”

Even traits that would otherwise be welcomed become more problematic when applied to topics as consequential as probate. Working with Martin, NCSC’s Souza noted that early versions of AVA were too empathetic and annoyed users who might have been actively grieving and simply wanted answers about the probate process.

Beyond the system’s superficial tone and pleasantries, Martin and Souza had to contend with the serious issue of hallucinations, or instances in which AI systems confidently share false or exaggerated information.

“We had trouble with hallucinations, regardless of the model, where the chatbot was not supposed to actually use anything outside of its knowledge base,” Souza told NBC News.

Martin has worked extensively to ensure the chatbot only references the relevant areas of the Alaska Court System’s probate documents rather than conducting wider web searches.

Despite its many fits and starts, AVA is now scheduled to be launched in late January, if all goes according to plan. For her part, Marz remains optimistic about AVA’s potential to help Alaskans access the probate system but is more clear-eyed about AI’s current limits.

“We did shift our goals on this project a little bit,” Marz said. “We wanted to replicate what our human facilitators at the self-help center are able to share with people. But we’re not confident that the bots can work in that fashion, because of the issues with some inaccuracies and some incompleteness.”