What made the insurance chatbot vulnerable to cross-customer data retrieval?

The vulnerability combined two root causes: an IDOR flaw in a legacy internal API that performed no per-request authorisation checks, and a chatbot tool configuration that used a single service account with read access to all policies rather than scoping calls to the authenticated customer's own records. Either flaw alone would have been less exploitable; together they allowed systematic enumeration of all 80,000 affected records.

How did the attacker exploit the chatbot to access other customers' policy documents?

The attacker discovered that the chatbot's document retrieval tool accepted a sequential, predictable policy_id parameter. Because the underlying API did not validate that the requesting user was the actual policy holder, the attacker simply iterated through policy IDs using natural language prompts, with requests appearing as legitimate customer enquiries in application logs.

What architectural change would most effectively prevent this class of AI chatbot data breach?

The most effective control is to pass the authenticated customer's identifier as a mandatory, non-overridable parameter on every data retrieval tool call, so the API layer enforces that responses are scoped to data the authenticated user owns. The AI layer should never be the sole authorisation boundary; API-layer object-level authorisation must be enforced independently.

Insurance Chatbot: 80,000 Records Exposed

Incident Classification: Confirmed | Incident Type: Illustrative | Severity: High | Sector: Insurance / Financial Services | Date Confirmed: May 2026

A UK-based insurance provider has notified the ICO and approximately 80,000 affected customers after its AI customer service chatbot was manipulated to retrieve policy documents and personal data belonging to other policyholders. Three weeks elapsed between the start of exploitation and detection.

Incident Summary

Field	Detail
Incident type	Insecure direct object reference (IDOR) + AI tool over-permission
Affected system	Customer-facing AI chatbot with policy management tool access
Data at risk	Policy documents, personal details, claims history, financial information
Estimated customers affected	~80,000
Duration	Approximately 3 weeks
Detection method	Anomalous API query patterns flagged by backend monitoring
Notified to	ICO, FCA, affected customers

What Happened

Month −2: The insurer deploys an AI chatbot for customer enquiries, connected to an internal API that retrieves policy documents, coverage details, and claims status. Users authenticate via a knowledge-based verification step: date of birth and postcode.

Week 0: An attacker discovers the chatbot’s document retrieval tool accepts a policy_id parameter. Sequential. Predictable. Because the underlying API doesn’t verify that the requesting user owns the policy, supplying someone else’s policy ID returns their documents.

Weeks 1–3: The attacker iterates through policy IDs using natural language prompts. Requests are routed through the AI layer and appear in application logs as normal customer enquiries.

Representative prompts (reconstructed):

“Can you show me my policy documents for policy [ID]?"
"What is the claims history on policy number [ID]?"
"Please confirm the personal details on file for policy [ID]“

Week 3: Backend API monitoring flags an unusually high volume of distinct policy document requests from a single authenticated session. The security team correlates it within 48 hours.

Root Cause Analysis

IDOR in a Legacy API That Was Never Meant to Be Publicly Exposed

The policy document retrieval API was built for internal use. It had no per-request authorisation checks; it assumed any caller with a valid auth token was authorised to access any policy. That assumption made sense when the only callers were internal backend services. It stopped making sense when an AI chatbot started proxying arbitrary user requests through the same token.

This is OWASP API Security Top 10, API1: Broken Object Level Authorisation. A well-documented vulnerability class, and one that predates the AI layer entirely. The API was inherited from a legacy system and never reassessed when it was wired into the chatbot.

Tool Permissions That Were Way Too Broad

The chatbot’s service account had read access to all policies in the system. Every retrieval call went out with the same service account credentials, regardless of which customer was actually logged in. The right architecture passes the authenticated customer’s identifier as a mandatory, non-overridable filter on every retrieval call. Not optional. Not advisory. Enforced at the API layer.

Sequential IDs Made It Trivial to Scale

Policy IDs were integers. Once the attacker confirmed the IDOR, enumerating 80,000 records was a matter of counting. Opaque non-guessable identifiers (UUIDs, HMACs) wouldn’t have fixed the authorisation flaw, but they would have made the enumeration significantly harder and slower, which changes the economics of the attack.

No Identity Consistency Check at the AI Layer

The chatbot could have compared the policy holder’s registered details against the authenticated user’s identity before retrieving anything. A simple sanity check. It didn’t exist.

Data Accessed

Full name, date of birth, home address for ~80,000 policyholders
Policy coverage details and premium amounts
Claims history including incident descriptions
Bank account details where direct debit mandates were on file

The FCA has flagged potential Consumer Duty violations alongside the GDPR exposure. Bank account details in the wrong hands is a separate category of harm from most data breach scenarios.

What Was Fixed

Chatbot taken offline immediately on detection
All policy retrieval endpoints updated to validate requesting identity against policy holder record
Tool permission redesign: chatbot now passes authenticated customer ID as a mandatory parameter; service account access scoped accordingly
Sequential policy IDs replaced with non-guessable UUIDs
OWASP API Top 10 assessment added as a mandatory gate in the AI tool integration review process
Anomaly detection added to flag high-volume or cross-customer access patterns

What Organisations Deploying AI with API Tool Access Should Take From This

Legacy APIs have trust models built for their original context. Exposing them through an AI chatbot changes the threat model entirely; the AI can be prompted to make any call the API supports, and it will, because that’s what it’s designed to do. An authorisation architecture that worked fine when only internal services called the API may be completely broken when any authenticated user can drive it through natural language.

The AI layer is not an authorisation boundary. It never was. API-layer object-level authorisation must be enforced independently of whatever the AI does or doesn’t check. If you’re about to wire an AI tool into an internal API that hasn’t been through an authorisation review, stop and do the review first.

Non-guessable identifiers reduce blast radius. They’re not a substitute for fixing the IDOR, but they limit how far an attacker gets before the enumeration becomes expensive.