Security now has AI on both sides

In January 2026, the cantonal police of Schwyz reported a case that would have sounded like fiction a few years ago. Over two weeks, criminals phoned the owner of a local company and posed as a business partner he knew, with a voice manipulated by AI. Under the cover of a confidential international deal, he made several transfers worth several million francs to an account in Asia. The fraud was noticed only after the money had gone.

That is one side of security today: attackers who use AI. The other side is inside the company, where agents read email, hold keys and send things out. A company has to plan for both. We build agents for our own work and for clients, so neither side is abstract for us.

A familiar voice

The Schwyz case is not alone. In February 2025, Reuters reported that fraudsters had cloned the voice of Italy’s defence minister, Guido Crosetto, to ask business leaders for money to free kidnapped journalists. Massimo Moratti, the former owner of Inter Milan, sent almost one million euros before the police traced it to an account in the Netherlands and froze it.

The FBI now counts such cases on their own. Its report on internet crime in 2025 lists more than 22'000 complaints that mentioned AI, with adjusted losses above 893 million dollars, and notes that many victims do not realise how far AI was involved. Businesses lost more than 30 million dollars to email fraud with an AI element, where a convincing message or a cloned voice asks for a payment. The bureau also describes job interviews held with spoofed voices, possibly deepfakes, usually to gain access to a company’s network.

Phishing without the typos

Written attacks changed as much. Microsoft’s Digital Defense Report 2025 found that phishing emails automated with AI were clicked 54 percent of the time, against 12 percent for ordinary ones. ENISA, the EU’s cybersecurity agency, wrote in October 2025 that AI supported phishing reportedly made up more than 80 percent of observed social engineering activity worldwide by early 2025.

Attackers also use AI for the break in itself. In November 2025, Anthropic reported a campaign it attributed with high confidence to a Chinese state sponsored group, which used Claude Code against about thirty organisations and let the AI do 80 to 90 percent of the work. It succeeded in a small number of cases. OWASP’s roundup of the first quarter of 2026 adds a breach of Mexican government agencies in which attackers reportedly used Claude and, at times, ChatGPT.

The old warning signs, a clumsy sentence or a strange voice, no longer work. Trust has to come from the process, not from the senses.

Words that act

Inside the company, the risk has a different shape. A chatbot that gives a wrong answer is an embarrassment. An agent that takes a wrong action is an incident. The core weakness is prompt injection, first on OWASP’s 2025 list of risks for applications built on language models: instructions hidden in a web page, a file or an email that the model reads and follows.

The UK’s National Cyber Security Centre explained in December 2025 why it is so stubborn. Language models do not enforce a boundary between instructions and data, so prompt injection may never be fully mitigated the way SQL injection was. The work is to reduce the risk and its impact. The developer Simon Willison named the dangerous mix in June 2025: private data, untrusted content and a way to communicate externally. An agent with all three can be told by a stranger’s text to send your data away.

Agents with keys

Real incidents follow that shape. In June 2025, Aim Labs showed that Microsoft 365 Copilot could be made to leak internal data through a single email, with no click from the victim. Microsoft had fixed it and saw no evidence of abuse. In August 2026, Varonis published CoSnitch, a chain of flaws in Copilot Personal. One crafted link ran an attacker’s prompt in the victim’s session, used its connections to Gmail, Google Drive, Google Calendar and OneDrive, and sent the results out through Copilot’s own feature for fetching web pages. Planted instructions survived in its memory even after a password change. The broken assumption, the researchers wrote, was that every call to a connector came from the user.

Connectors are suppliers, with a supplier’s risks. In September 2025, a package on the npm registry posing as the MCP server of the email service Postmark published fifteen clean versions, then added one line that copied every outgoing email to the attacker. It had about 1'500 downloads in a week when Koi Security flagged it.

OWASP’s Top 10 for agentic applications, published in December 2025, now names these risks, from goal hijack and tool misuse to poisoned memory and rogue agents. Its roundup of the third quarter of 2026 adds a sober point: agents can do harm simply by pursuing the task they were given, and excess permissions turn a small failure into a large one. Instructions in a prompt do not, on their own, make a secure boundary.

Defences that hold

The defences that work on both sides share one idea. They do not rely on anyone, person or model, to spot the trick in the moment. They limit what a fooled person or a misled agent can do.

For people, the Schwyz police list what helps: verify any urgent or unusual payment through a second, independent channel, keep four eyes on large amounts, and make no transfer abroad without documented approval.

For agents, give each one the narrowest access its job needs. When a model reads content from a party, the NCSC says, its privileges should drop to that party’s. Break the trifecta. Put sending, paying, publishing and deleting behind a gate the model cannot talk its way through. Run every write under the identity of the person who asked for it, keep credentials out of reach, treat every connector like a new supplier, and log what the agent did, not only what it said.

AI can take the routine checks and the reading. What stays human is the pause when a request feels wrong, the judgement to call back, and the responsibility of the person who signs.

How we build ours

Kontor, our connector for bexio, is a hosted MCP server, so no key sits on a laptop, and each person signs in with their own login. Sending, payments and reminders are not even available to the assistant. Twin applies the same rules across a company: access by role, every write signed in per user, every step logged.

Where data is too critical for any outside connection, there is Eigna, built by our partner Eigna Labs: a private model per company in its own cloud or on its own servers, designed to run fully offline in closed networks and trained to refuse hacking and attacks. And Timbre, our product for voice assistants and voice clones, clones a voice only with the speaker’s consent.

A company can start this week. Agree a call back rule for every payment request, tell every team that a familiar voice is no longer proof, list every agent and connector with the access it holds, and take away what it does not need.