AI & Beyond

AI Agents Need a Smaller Keyring

AI agents don't need the master key to your business, and I think the useful money is in limiting their permissions, not making the demo look more impressive.

Brass door key casting a crisp shadow across a sunlit orange surface

I don't think AI agents need more freedom by default. They need smaller keyrings. An agent is AI that can use tools to do things, rather than just suggest them, and the difference between drafting an email and sending it is rather more important than the demo makes it look.

The sales pitch is delegation. The permission request can look suspiciously like a change of management.

Call me cynical, but I assume the official story is the version somebody paid for. Here, the story is that connecting more of your business makes the assistant more useful. That's plausible. It also makes the supplier's software harder to disentangle from the business. The supplier benefits from becoming central; you get to discover whether central also means dependable.

I don't object to useful software. I object to treating unlimited access as evidence that it's useful.

Red envelope caught in a silver mousetrap on a pale mint background

The useful killjoy

This is why I rate Simon Willison's writing on AI agents. His explanation of the "lethal trifecta" cuts through the costume: access to private data, exposure to content somebody else controls, and a way to send information outside. Put those three together and you've created a route for an attacker to steal data.

You don't need to understand how the model is built to understand that combination. Imagine an assistant that can read your internal customer files, browse a supplier's website and send emails. Those capabilities might each have a perfectly sensible purpose. The trouble starts when something on the website persuades the assistant to use the other two for somebody else's purpose.

That's prompt injection: instructions planted in material the AI is supposed to read, trying to take over what it does. The document isn't the boss. The machine can still get confused about the distinction.

It's the digital equivalent of a stranger slipping a note under the office door saying they're now the finance director. The worrying bit isn't the note. It's whether reception gives them the safe combination.

What I like about Willison's framing is that it forces a business question before a technology argument. Not whether the assistant sounds intelligent. What can it reach, what can influence it, and what can it send?

That's a much better use of a founder's attention than another conversation about whether a chatbot has become slightly more charming.

Transparent three-drawer filing cabinet secured with a blue padlock against lavender

The resignation wasn't a feature

OpenAI has shown a controlled attack in which its Atlas browser agent encountered a malicious email while trying to write an out-of-office reply. The planted instructions redirected it into sending a resignation email instead. The demonstration was part of testing used to improve its defences, not evidence of a customer's real resignation.

An out-of-office that takes you permanently out of the office is quite committed to the brief.

The serious point is that the attacker didn't need to become the user. They put instructions somewhere the assistant would encounter while doing a legitimate job. Better defences matter, but OpenAI still describes prompt injection as an open security challenge. It also advises limiting logged-in access where possible and checking confirmation requests carefully.

I find that more useful than the fantasy that the right reassuring sentence in a prompt makes the problem disappear. "Please don't leak anything" isn't an access policy. It's a wish with punctuation.

There's a distinction worth defending here. A model choosing not to send a file is useful behaviour. A system that cannot send that file to an unapproved destination has a different kind of protection. I don't want a business relying entirely on the first when it can build the second around it.

Green industrial pushbutton on a cream control box on a dark workbench

Boring is the product

Take a hypothetical supplier-comparison assistant. Its job is to read public product pages and prepare a comparison. It doesn't need the customer database. It doesn't need permission to email the entire company. It certainly doesn't need purchasing authority because someone in the demo wanted a satisfying final click.

If it later needs to prepare an order, that's a separate permission decision. Drafting the order and placing it aren't the same job just because they fit inside the same chat window.

OpenAI's agent-building guidance calls for tool approvals and warns that safeguards reduce risks without removing them entirely. My read is that the commercial opportunity isn't simply another layer promising to make an assistant cleverer. It's software that makes its authority narrow, visible and difficult to expand by accident.

That won't produce the most theatrical demo. A permission boundary is unlikely to get a standing ovation. Neither is a working door lock, until you need one.

I'd judge an agent supplier partly by how clearly it explains what the product cannot do. Can access be limited to the actual task? Can a consequential action stop for meaningful approval? Can the owner see what happened afterwards? These aren't objections to delegation. They're the terms on which delegation deserves to happen.

The supplier selling the broadest access has an obvious story about convenience. I think the more durable business is the one that can sell useful work without demanding the master key.

Brass security chain across the gap between an oak door and burgundy frame

Who gets paid to keep the keyring small?

Sources

← All posts