broccolli.xyz
Blog

Artificial Intelligence | Process Automation

AI Agent Security: Limits, Data and Human Oversight

· 7 min

AI agents are the first category of software that takes action on your company's behalf without anyone pressing a button. They do not suggest. They act — sending the email, updating the CRM record, issuing the credit note.

That changes what security means. When an assistant gets it wrong, you get a bad sentence on screen. When an agent gets it wrong, a customer receives an invoice with your company's name on it.

Eurostat reports that 20.0% of EU enterprises with 10 or more employees used AI technologies in 2025. Portugal sits lower, at 11.5% according to the national statistics institute, up 2.9 percentage points in a year. Adoption is moving faster than the internal rules meant to govern it.

This article covers the three decisions that determine whether AI agent security holds up in a smaller company: what limits you set, what data you let the agent touch, and how you supervise it.

With AI agents, the risk moves: it is no longer what AI says, it is what AI does

The OWASP GenAI Security Project's quarterly exploit round-up, covering 1 January to 11 April 2026, documents eight incidents involving AI systems. The breakdown is worth reading closely.

Three of the eight involved excessive agency — the agent held more permissions than the job required. One involved indirect prompt injection. Only one of the eight carried a formal CVE (CVE-2025-59528, remote code execution in Flowise, with 12,000 to 15,000 instances exposed on the public internet).

That last figure is the instructive one. Most AI agent security failures do not come from a catalogued code flaw. They come from misconfiguration, design mistakes and badly considered data flows. A patch does not fix those. An architectural decision does, or fails to.

In the OWASP Top 10 for LLM applications, excessive agency climbed from sixth place in 2025 to third in 2026. It is the fastest-rising category, and it is precisely the one that applies to anyone building agents.

Limits: the rule of two out of three

One design heuristic is worth more than any checklist of best practices, and it comes down to three dangerous properties.

An agent becomes exploitable when it holds all three at once:

  1. Access to private data — the CRM, the mailbox, the customer database, internal files.
  2. Exposure to untrusted content — incoming emails, supplier PDFs, web pages, customer tickets.
  3. The ability to communicate externally — sending mail, calling an outside API, publishing something.

With all three, an attacker who can get text into the input channel can instruct the agent to read private data and send it out. With only two of the three, that same attack cannot close the loop.

Meta formalised this as the "agents rule of two": an autonomous agent should hold at most two of these properties. If the use case genuinely needs all three, the agent stops being autonomous and must ask a human before acting.

In practice this gives any decision-maker one question to put to the technical team: does this agent read something that came from outside, hold access to our data, and have the ability to send messages? If the answer is yes three times, either the design is wrong or it is missing an approval step.

The flaw vendors are not going to patch

It helps to understand why prompt injection is not fixed by an update.

A language model receives the system instructions, the user's request and any retrieved external text as a single stream of tokens. There is no reliable way to mark some of those tokens as commands and others as data. It is the same class of problem SQL injection was before parameterised queries — except that here the equivalent of the parameterised query does not yet exist.

Prompt injection remains first in the OWASP Top 10, and it maps to six of the ten categories in the list dedicated to agentic applications. The instructions no longer arrive only as text either: they can sit inside an image, an audio file or a document the agent processes.

None of this argues against using AI agents. It argues against designing them on the assumption that the model will always obey whoever is nominally in charge. Assume the opposite and cap the damage.

Data: with AI agents, where the workflow runs is not where the model runs

This is the most expensive confusion we see in smaller companies.

Running the automation platform on your own server, in European hosting, is a sound decision — and it is one reason n8n is a common choice for workflow automation in firms handling sensitive data. But it solves orchestration, not inference.

If your workflow runs on a server in Lisbon and then calls a model API hosted in the United States, the prompt — customer name, invoice value, conversation history — travels to the United States. For GDPR purposes, the boundary that matters is where inference happens, not where the workflow engine sits.

Three concrete decisions follow:

  • Minimise what enters the prompt. An agent classifying complaints does not need the customer's tax number or bank details. Filtering fields before calling the model is the cheapest and most effective control available.
  • Know where your inference provider is and make sure the processing agreement covers it. A European provider or a self-hosted model changes the answer, with different cost and performance.
  • Log what was sent. Without a record of prompt and response, you cannot audit an incident or answer a subject access request.

There is also the AI nobody authorised. On the figures OWASP cites, only 37% of organisations have a policy in place to detect unapproved AI use. In most companies, someone is already pasting customer data into a tool management has never heard of.

Oversight: approve AI agent actions that cannot be undone

Supervising everything is the same as supervising nothing — within three days the team is approving on autopilot. Oversight only works when it is selective.

The test is reversibility. Reading an email, classifying a ticket and drafting a reply are reversible: if the agent is wrong, you correct it at no cost. Sending a customer communication, deleting a record, issuing a credit note or making a payment are not.

The n8n documentation on human-in-the-loop approval for tool calls implements exactly this distinction: human review is enabled per tool, not for the whole agent. When the agent decides to use a tool flagged for review, the workflow pauses and sends an approval request through Slack, Telegram, Gmail or the platform's own chat, showing which action is about to run and with what parameters. If it is denied, the action is cancelled and the agent is told it was refused — which matters, because the agent does not then proceed as if the task were done.

The pattern that works: the agent handles the whole path, and only the final irreversible action passes through a person. It is the same logic already applied to approval workflow automation that shortens decision cycles, except that now it is software asking for the sign-off.

A worked example, with the arithmetic shown

There is no public study measuring this, so what follows is an estimate with its assumptions stated.

A company receives 40 emails a day to its support address. Triage, a standard reply, and in roughly 3 cases a day, a credit note to issue. Manual triage takes about 4 minutes per email: 160 minutes a day, close to 2.7 hours.

Build the agent badly — CRM access, reading inbound email, permission to issue credit notes and reply to customers — and you have all three properties of the rule of two at once. A crafted email can ask the agent to issue a credit note or forward another customer's data.

Build the same agent with limits: it classifies and drafts a reply for all 40 emails, while issuing a credit note is a tool with mandatory human review. Only the 3 cases a day reach a person, at 2 minutes each: 6 minutes.

The arithmetic is 160 minutes against 6. The gain does not come from the agent being fast — it comes from concentrating oversight on the 7.5% of cases where a mistake costs money. The same principle of putting human attention where it actually decides something runs through the back-office tasks that still need human oversight.

What the law already requires, as of August 2026

The EU Artificial Intelligence Act does not apply only to those who build models. It applies to those who deploy them, and two obligations already in force catch any smaller company.

Article 4, on AI literacy, has applied since 2 February 2025 to both providers and deployers, with SMEs named explicitly. It requires you to ensure that the people operating the system know enough to use it in an informed way. In plain terms: if your team runs an agent without understanding what it can and cannot do, you are non-compliant.

Article 50, on transparency, has applied since 2 August 2026. If a customer is interacting with an AI system, they must be told, clearly and no later than the first interaction. An agent answering customer emails without disclosing that fact does not meet the bar.

Both apply to AI agents you merely deploy, not only to ones you build. One calendar note, because there is genuine confusion here: the Digital Omnibus package, in force since 27 July 2026, deferred the Annex III high-risk obligations from 2 August 2026 to 2 December 2027. That is what moved. Article 50 transparency and Article 4 literacy did not.

The decision in front of you

It is not whether you will use AI agents. It is how much authority you give them.

Pick one process — just one — and answer three questions before anyone writes a line of configuration: what data does this agent genuinely need to see, which of the actions it can take are irreversible, and what happens if the text it reads was written by someone acting in bad faith.

If you cannot answer all three, the problem is not information security. It is process design, and that is where it gets solved.

Which process would you trust an agent to run end to end, except for the last click?