The agent delusion: why knowledge work is placing faith in the wrong layer of the AI system stack
AI agents are not the primary unit of intelligence. The organisational context they’re provided with is. Confusing one with the other isn’t just sub-optimal – it’s a dangerous bet.
Meet Barry.
Barry... Botman.
The company CEO has just created Barry to be their new ‘virtual Chief of Staff’.
Barry prepares board updates. He summarises leadership meetings. He tracks strategic priorities, drafts internal briefings, chases unresolved decisions, coordinates with specialist agents, works twenty-four hours a day. And he has never once asked whether the current operating model is, perhaps, a tiny bit deranged.
For a while, for all of the above reasons as far as the CEO is concerned, Barry is thought to be bloody marvellous.
Until, that is, Barry starts getting... a bit weird.
Old priorities resurface in new briefings. Leadership decisions blur together. Board updates sound plausible, but wrong. An action from three months ago appears as if it were agreed yesterday. Then there was that time when a cancelled “we don’t talk about that anymore” initiative wandered into a meeting agenda pretending it still had budget.
The temptation, of course, is to place the blame on... well, the AI. The agent. Obviously.
“The AI is rubbish.” “The model needs fixing.”
“Barry, you bloody imbecile.”
Except, it would be a mistake to blame Barry himself.
Why?
Because Barry isn’t real.
There is no ‘virtual Chief of Staff’. There never was.
Anthropomorphised perceptions of AI agents as actual ‘workers’ with an attached sense of personhood and sentient intelligence are, undeniably, a convenient abstraction. They let non-technical executives quickly grasp the functional value of agentic AI applied to business.
But they also act as a hugely misleading – and increasingly dangerous – wrong turn for most knowledge work industries trying to grapple with the value that AI actually offers to business operations.
This is an article about that wrong turn, and what course correction looks like. From “out-of-the-box AI is human-equivalent intelligence, therefore representing displacement of human labour” to “AI is system intelligence that has a huge role to play in assisting and enhancing pre-existing human labour (without threatening it)”.
And this isn’t a thought experiment.
The New Zealand government is currently cutting nearly 9,000 public-sector roles under a restructuring programme that names accelerated AI adoption as part of its official rationale. The political debate has focused on austerity, ideology, and employment. Almost nobody is asking the operational question underneath: what exactly will these AI systems be executing against?
So first, back to Barry. If you’re not real... what are you (not)?
What Barry actually is. Not.
Strip the name away.
At the simplest level of explanatory abstraction, Barry is essentially:
A large language model (e.g. Opus 4.7, GPT 5.5, choose your poison) pointed at a slice of organisational context – documents, instructions, workflows, tools, constraints, and source knowledge – which then triggers system execution against the contents of that organisational context (’go here, follow these steps, use this tool, run this script, produce x, followed by y, in order to synthesise z’).
That is the workflow. That is what is running.
The foundation capability layer is the raw capability of the model: generate, classify, extract, transform, reason, synthesise. Powerful, but generic. It knows nothing of your organisation.
The agentic execution layer is the operational machinery around that capability: skills, workflows, agents, orchestration, tools, permissions, routing, observability, escalation.
The context layer is the thing between them. The system of understanding – comprised of the substance, structure, and operational governance that turns organisational knowledge into something machine-operable.
Models provide capability. Context provides meaning. Agentic systems provide execution.
Now, there’s no doubting that the sense of there really ‘being’ an agent on the other side of the screen – that acts, produces, decides, routes, judges – is felt as real.
But the source reality creating that effect is: an LLM parsed through whatever organisational context it was given as inputs, producing outputs that mirror the input’s nature.
Add a cheerful name (or ‘Barry’), plus just enough projected personality flourishes to make it feel less weird, and there it is. You’ve ‘created’ your agent.
This is the first perceptual breakthrough leadership teams need to make if they're going to grasp what they're actually building – and what they're making decisions over.
Meaningful intelligence lives in the organisational context layer – the webpages, marketing collateral, policies, strategy docs, meeting notes, dashboards, and all the weird-and-wonderful strategic detritus that accumulates anywhere humans gather to do "human worker stuff". Not in the AI model itself.
The distinction matters enormously. Not just because it deepens leadership understanding of what’s inside the agentic AI black box – but because it’s the difference between two very different internal conversations.
The first: “I know at a systems level what agentic AI is, and I get that it would be dangerous to treat this as an immediate trigger for human labour replacement within our business.”
The second: “I genuinely believe these are intelligent virtual workers. This is magic! We’ve entered the age of AGI, folks, how soon could we reduce our workforce by 35%?”
Perceiving agents as actual virtual colleagues leads to the latter response.
If you think Barry is a virtual colleague, you train him, coach him, manage his performance – and start quietly thinking of him as a cheaper replacement for human labour.
This is the agent delusion.
If you understand Barry as a reasoning processor layer pointed at organisational context, you’re more likely to see AI as augmenting human labour, not replacing it. You don’t inspect ‘the agent’ when things go south (because there is no embodied agent to inspect) – you inspect the human-made context that’s feeding the effect of agentic intelligence.
This is the agent reality.
And no prizes for guessing which lens the New Zealand cuts are being made through.
The context layer: where intelligence actually lives
Right. So if it’s not the model, and it’s not the orchestration – what is the context layer, and why does it carry the load?
Three properties. Substance. Structure. Governance. Get all three right and agentic AI systems start behaving like the intelligence leadership thought they were hiring (albeit, grounded in an understanding of what’s actually at play).
Get any of them wrong, and the agent behaves like… Barry.
Substance is the meaning itself. The actual operational truth of what the organisation knows, intends, claims, sells, promises, and decides. Substance is what you mean. It lives in webpages, marketing collateral, policies, strategy decks, customer-facing artefacts, knowledge bases, training materials, sales enablement, internal documentation – every place an organisation stores its understanding of itself.
To take a concrete operational example: if you’re building a customer support knowledge system, the first substantive work is editorial substance. Which knowledge base articles are canonical and which are duplicative or redundant? What’s cruft that should never go near an agent at all? What’s written to style guide and what isn’t? Which definitions actually describe current product reality and which describe a feature that was deprecated eighteen months ago?
This is the human, judgement-led discipline of deciding what the organisation actually knows and means in this area, and getting it into a form a system can reliably point at.
Structure is how that meaning gets encoded so a system can actually work with it. Taxonomies, content models, metadata, definitions, entity relationships, knowledge graphs. The discipline of making meaning machine-operable. This is the territory the semantic engineering field has owned for decades, and it matters more now than ever.
Worth flagging: some enterprise discourse is now using “context layer” to refer specifically to this structural encoding work – data definitions, metric semantics, lineage graphs, ontologies. Real, necessary, correct. But it’s actually incomplete: being just one of three properties of a context layer (substance + structure + governance), not the whole thing.
Structure determines whether an agent retrieving a policy can also retrieve the related policy, the superseding policy, the exception to the policy, and the canonical definition of the term the policy turns on. Without structure, Barry retrieves documents. With structure, Barry retrieves meaning.
Governance is what keeps substance and structure true over time. Ownership. Lifecycle workflows. Standards. Update cadences. The roles, responsibilities, and routines that catch drift before it propagates. Organisational meaning is not static; it changes constantly as products evolve, strategies pivot, positioning sharpens, policies update, people move. The agent doesn’t know the document is six months out of date. It just executes against whatever is provided to it. The governance layer is the difference between a context layer that keeps working and one that steadily turns into a liability.
(Made considerably worse, of course, if you built the liability on top of cutting the human workforce that’s now required to fix it. Oops.)
Substance. Structure. Governance.
A healthy context layer has all three. A broken one is broken in at least one of them – and usually broken across all three at once, because organisations that haven’t invested in substance also haven’t built the structure, and don’t have the governance routines to maintain either.
This is what I increasingly refer to as content system design work. The discipline of building and maintaining a context layer (composed of not only structural semantic relationship mapping, but editorial source truth and an operational content governance model) deliberate enough, stable enough, and current enough to point a probabilistic system at without inheriting compound failure.
Different business contexts get different content systems to be designed for corresponding AI use cases – marketing, product, customer experience, internal operations – but the underlying anatomy is the same.
Substance, structure, governance. Maintained as a live operational asset, not a one-off implementation project.
That is what sits underneath the agent. Not out of the box intelligence. And not merely a knowledge base with a nicer name. A designed operational environment for meaning to dynamically move between human and machine consumers.
The danger of the anthropomorphised AI system
A large language model is a probabilistic, pattern-spotting system. It generates plausible continuations against whatever context it has been given.
Point a probabilistic system at ambiguous organisational context and expect deterministic operational reliability from the output, and you are making a category error at the foundation of your AI strategy.
The better the agent feels like a colleague, the harder it becomes to inspect the system underneath; the more convincing the ‘virtual colleague’ framing, the easier it is to forget the system is only ever acting through the context it can access.
That isn’t just a technical concern. It’s commercial. It’s strategic. And in knowledge industries currently betting their cost base on AI workforce replacement, it’s existential.
Most organisations don’t have enough stable operational meaning to support deterministic execution from a probabilistic system.
The model is a growing commodity. The agent is the visible behaviour. The context layer underneath both is where competitive advantage actually lives – and where it can also disappear in ways no one is currently set up to notice.
The agent is not underperforming in the way the leadership team thinks it is.
The agent is reflecting the context layer it was pointed at.
When an AI agent recommends a decision based on an old document, the first question is not “why did the agent do that?”
The first question is: why was that document included as a plausible source of truth within our content system the AI is feeding from?
When the agent blurs together two versions of the same strategy, the first question is not “does the model need better instructions?”
The first question is: why did the organisation have two live versions of the strategy?
So the next time someone presents an agent architecture, ask the missing-layer question.
Not only: how does the system coordinate and orchestrate execution?
But: what context layer is it executing against?
What does the organisation’s informing content actually mean, intend, expect, and need?
Where is that meaning defined? How is it encoded? Who maintains it? How is it validated when the organisation changes?
Between the raw capability of AI models and the sophisticated execution of agentic systems sits the layer now revealing itself as the most difficult – and most strategically decisive – part of the stack:
organisational context.
Which is expressed through the organisation’s content.
And in most cases...
Ooph, what a mess.
I'm currently working on a set of designed content system templates that sit underneath agentic AI in knowledge-intensive organisations – the context layer the agents are actually executing against.
Watch this space. And if that’s something you’d like to chat more about, I’d be happy to hear from you.