Insurance Chatbot Saves Agents 2.4 Seconds Per Search: A Cost-Benefit Analysis

A research collaboration between Dakota State University and regional insurer Safety Insurance has produced experimental data on AI's practical value in insurance workflows. Their chatbot, called Axlerod, saves independent insurance agents an average of 2.42 seconds per information retrieval task—a seemingly small gain that reveals significant questions about AI implementation economics.

The Axlerod System Architecture

Axlerod is built as a lightweight wrapper around Google Gemini 2.5 Pro, connected to Safety Insurance's internal data sources through a middleware layer called LiteLLM. This proxy translates API requests from OpenAI's standard format to Google's API while handling Vertex AI authentication. The system uses a microframework called Smoltalk for creating agentic applications.

The chatbot handles four specific information retrieval tasks: finding client policy numbers, determining AutoPay eligibility, identifying covered vehicles, and determining billing plans. According to the pre-print paper "Introducing Axlerod: An LLM-based Chatbot for Assisting Independent Insurance Agents," the system achieved a 93.18% success rate during testing.

Time Savings Breakdown

Without the chatbot, agents took 7.55 seconds on average for search-oriented tasks. With Axlerod, that time dropped to 5.13 seconds. For more complex tasks requiring navigation through multiple screens or searching by customer name, the time savings were "notably faster," though specific figures weren't provided in the paper.

The researchers argue that business-facing chatbots make more sense than consumer-facing ones because insurance agents possess domain expertise sufficient to interpret subtle policy language and identify AI hallucinations. This human-in-the-loop approach addresses a key concern in high-stakes environments where accuracy matters.

The Economics of Seconds

The financial calculation reveals the challenge of quantifying micro-efficiency gains. Using an average insurance agent salary of $80,000 annually (approximately $0.01 per second based on 250 eight-hour work days), the math works as follows:

• Per search savings: 2.42 seconds × $0.01 = $0.0242
• Daily savings at 20 searches: 48.4 seconds = $0.484
• Annual savings: 12,100 seconds = $121

However, this assumes the saved time converts directly to productive output rather than becoming "another 48 seconds of drinking coffee each day," as the study authors note. The employer cannot deduct $121 from an agent's annual pay; instead, it represents potential productivity gains that may or may not materialize.

Cost Structure and ROI

Each chatbot inquiry costs an average of $0.0075. Khandaker Mamun Ahmed, co-author and assistant professor at Dakota State University, provided an alternative calculation using 80 daily searches (10 per hour over an eight-hour day):

• Daily cost: 80 × $0.0075 = $0.60
• Daily savings: 80 × $0.0242 = $1.936
• Net daily benefit: $1.336
• Daily ROI: 222%

The break-even point occurs at just 0.31 searches per day—meaning an agent needs to make only one search every three days for the system to pay for itself.

Real-World Implementation Challenges

Scott Johnson, an independent insurance broker at Marindependent, offered practical perspective. He estimates making 10 to 20 policy queries daily, but notes that existing software like EZLynx already automates fetching carrier data, eliminating about 80% of routine inquiries.

"When agents do have to reach out for information, questions tend to be complicated ones that chatbots wouldn't be able to answer," Johnson explained. He also observed that many chatbots struggle with industry-specific terminology and common requests like policy information or declaration sheets.

The Broader Context

The US property and casualty insurance market includes over 4,100 carriers with combined revenues exceeding $1 trillion. Agents routinely handle 50–200 customer interactions daily, with information lookups potentially numbering much higher when friction is reduced.

Ahmed acknowledges that "robust field testing is still essential." Expanded evaluation with practicing agents will be critical to validate real-world utility, assess agent trust, and measure efficiency across both human-in-the-loop and more automated workflows.

Key Takeaways for Implementation

1. Usage volume matters: The ROI calculation is highly sensitive to how frequently agents use the system. Low usage scenarios may not justify implementation costs.

2. Domain expertise requirement: The researchers emphasize that agent-facing deployments work best when humans remain in the loop to validate outputs in high-stakes environments.

3. Integration complexity: Axlerod required custom middleware to connect with Safety Insurance's internal systems, suggesting implementation isn't plug-and-play.

4. Existing automation: Many agencies already use tools that automate routine tasks, potentially reducing the addressable use cases for AI chatbots.

5. Success rate considerations: While 93.18% success sounds high, the remaining 6.82% could create friction in workflows, especially if agents must verify outputs or handle failures manually.

The study demonstrates that even marginal time savings can yield positive ROI with sufficient volume, but it also highlights the importance of realistic usage projections and integration planning when evaluating AI tools for professional services.

For more details, see the pre-print paper "Introducing Axlerod: An LLM-based Chatbot for Assisting Independent Insurance Agents."