The assistant on your website speaks for you

More companies put an assistant on their website or in their app. It answers at night, in every language, and it never runs out of patience. It also speaks in the company’s name, it sits on the open internet where anyone can type anything into it, and behind it there is usually a database, a set of keys and a connection to other systems.

That makes it a security matter, not only a marketing one. The cases of the last two years show three ways it goes wrong: the assistant promises what the company never offered, it is talked into saying what it should not, and the data behind it leaks.

It speaks for you

The case cited most often is Air Canada. After a bereavement, a passenger, Jake Moffatt, asked the airline’s website chatbot about reduced fares. It suggested he could buy a ticket and apply for the bereavement fare afterwards. The airline’s policy did not allow that, and it refused. Before the Civil Resolution Tribunal of British Columbia in February 2024, Air Canada argued that the chatbot was a separate legal entity responsible for its own actions. The tribunal member, Christopher Rivers, disagreed: the chatbot was “still just a part of Air Canada’s website”. The airline was ordered to pay 812 Canadian dollars.

The amount was small. The principle is not: what the assistant says on your website, you said. In April 2025 the makers of the code editor Cursor learned the same lesson in public. Their support bot, signing as Sam, told users that a subscription could not be used on several machines at once. There was no such rule. A cofounder, Michael Truell, wrote that the answer had come from an AI support bot and was wrong, and that AI answers in email support are now clearly labelled.

Anyone can type anything

In December 2023, a user told the assistant on the website of a Chevrolet dealer in Watsonville, California, built on ChatGPT, to agree with everything a customer says and to end each answer by calling it a legally binding offer. He then asked for a 2024 Tahoe for one dollar, and the bot agreed. No car changed hands, and the dealer switched the chatbot off. In January 2024, a customer of the parcel service DPD got its chatbot to swear, to write a haiku calling itself useless and to call DPD the worst delivery firm in the world. DPD blamed an error after a system update and disabled the AI element.

Those were embarrassments. The next one was a security hole. In August 2025, researchers at Cybernews showed that Lenovo’s support assistant, Lena, could be made to produce harmful HTML with a single prompt of about 400 characters. The code sent session cookies to the attacker, and with a support agent’s cookie an attacker could log into the support system as that agent. Lenovo fixed it. An analyst at Everest Group drew the lesson: treat chatbots as full applications, not as AI pilots.

What sits behind the chat window

Some of the worst cases needed no clever prompt at all. They came from the systems behind the assistant. At the end of June 2025, the researchers Ian Carroll and Sam Curry looked at McHire, the hiring platform McDonald’s uses with Olivia, an assistant made by Paradox.ai. An administrator login accepted 123456 as both username and password. From there, changing a number in a request opened the records of what they counted as more than 64 million applicants: names, email addresses, phone numbers, home addresses and the tokens to read their chats. The login stopped working within two hours of their report.

In August 2025, attackers stole the access tokens of Drift, a chat tool companies run on their websites, and used them to export data from the Salesforce systems it was connected to. Google’s threat intelligence group saw them search the stolen data for passwords and cloud keys. Cloudflare, which used the Drift chatbot on its own site, found 104 of its API tokens in support cases where customers had pasted them, and rotated every one.

And in March 2026, Security Magazine reported that the researcher Jeremiah Fowler had found three unprotected databases behind the AI customer service assistant of Sears Home Services, with 3.7 million records: chat logs, call recordings and transcripts, names, addresses and phone numbers. Some recordings ran for up to four hours, because the line had stayed open after the call.

What a safe assistant needs

None of this is a reason to switch the assistant off. It is a reason to build it like any other application that faces the internet, with a few rules of its own.

Scoped data. The assistant sees public content and, after sign in, only the records of the person it is talking to. The check of who may see what runs in your systems, not in the model. OWASP says it plainly: authorisation must not be delegated to the language model.

No secrets in reach. The instructions behind an assistant should be treated as public, because they can be extracted. Keys, passwords and connection details do not belong in them, and the tokens of the tool itself are kept narrow and rotated.

Firm limits. Prices, discounts, refunds and commitments come from your systems, not from the model. The assistant may explain a policy. It may not change one. Its output is treated as untrusted too, so it can never place code in a page.

Guardrails outside the model. A separate check on what goes in and what comes out, because a system prompt asking for good manners is a wish, not a control.

Logging with care. Every conversation and every action is recorded, so a mistake can be found and explained. Those logs hold personal data too, so they need access control, encryption and a deletion date.

A way to a person. When the assistant is unsure, when the customer is upset, when money or a promise is involved, it hands over to someone who can decide. AI can take the routine questions at any hour. Judgement, goodwill and the responsibility for a promise are the parts a company should keep human.

Testing and labelling. Try to break it before launch and after every update, with the tricks above: role play, hidden instructions, requests for discounts. And say that it is a machine. Switzerland’s data protection commissioner says people have a right to know when they deal with one, and the EU AI Act’s transparency rules, which require the same, have applied since 2 August 2026.

How we build it

We build assistants to the same rules. Twin, the company brain we build for clients, answers a customer with the passage from the manual as its source, handles a return by the company’s own policy, and hands the conversation to a person when it is not sure. Access is by role, and every step is logged.

Studio, where a company runs its website and its marketing, has protection against injected instructions built in, and its assistant drafts while a person decides. For data that must not leave the company, Eigna, built by our partner Eigna Labs, runs as a private model in the company’s own cloud or on its own servers, trained to refuse hacking and attacks.

A company with an assistant online can start with five questions. What can it see? What can it promise? Which secrets could it reach? Who reads its logs? And when did someone last try to break it?