AI can do the work

The claim

There is a lot of discussion about when AI will start replacing knowledge workers. Recent warnings go further, suggesting AI could soon conduct its own research, improve itself and eventually operate beyond meaningful human control.

I'm not convinced we're there yet.

Not because AI isn't impressive. It clearly is. The problem is that being able to produce something and being able to reliably determine whether that thing is correct are two very different capabilities.

I see this constantly when using AI myself.

I can ask it a question in an area I understand well and get a confident, well-structured answer. Sometimes it's excellent. Other times there's a subtle mistake, an incorrect assumption or a constraint it has missed.

I know it's wrong because I understand the subject.

That raises an uncomfortable question: how impressive does AI look when the person using it doesn't know enough to recognise the mistake?

Capability isn't the same as autonomy

Most discussions about AI replacing jobs concentrate on whether AI can perform the task.

• Can it write code? Yes.
• Can it produce a design? Yes.
• Can it analyse data? Yes.
• Can it write a business case? Yes.

But that isn't the threshold for replacing someone.

The more important question is whether it can do the work reliably, repeatedly and without someone knowledgeable checking it.

At the moment, humans still provide that layer.

An experienced person doesn't simply produce output. They challenge assumptions, recognise unusual situations, remember why previous decisions were made and spot when something technically correct is actually the wrong thing to do.

That knowledge is difficult to see because much of its value comes from preventing things from happening.

Then there is the economics

Even if AI can perform the work, replacement only makes sense when the economics work.

The comparison isn't:

£80,000 employee vs £20 AI subscription.

It is closer to the cost of the human team versus AI, plus compute, integration, context retrieval, governance, verification, rework, human supervision and risk.

That distinction becomes increasingly important as the work becomes more complex.

Software is a good example. AI can create a small application remarkably quickly. Maintaining a mature enterprise platform is a different problem.

An experienced developer can retain a mental model of a system. They know where something lives, what depends on it and why a strange-looking decision was made three years ago.

AI has to reconstruct enough of that context to make the change safely. It doesn't necessarily read an entire codebase every time. It can search, index and retrieve relevant sections. But those mechanisms have an operating cost too.

As the software grows, so does the problem.

Cheap code could create expensive software

When generating code becomes cheap, the incentive is to generate more of it.

But more code means more software to understand, test, secure and maintain. That creates a potentially awkward cycle:

Cheaper generation → more code → more complexity → more context → more AI processing → greater maintenance cost.

Technical debt makes this worse. Ask an AI to implement a feature and it will generally optimise towards completing that task. Unless technical debt, architectural direction and maintainability are part of the brief, they may not receive the same attention.

The implementation works. The tests pass. The ticket closes.

Repeat that hundreds of times and every individual decision can be reasonable while the overall system gradually gets worse.

Experienced engineers recognise this because they don't just understand individual pieces of code. They understand the direction the system is travelling.

Who marks the AI's homework?

This may be the bigger problem. Imagine an autonomous AI development process:

AI interprets requirement → AI writes code → AI writes tests → AI runs tests → AI reviews result → AI declares it correct.

Everything can pass while the original interpretation was wrong. The AI has successfully proved that its solution satisfies its own understanding of the problem.

That's not independent verification.

We don't accept that structure from humans either. Software teams have code reviews, QA, security reviews, architecture reviews, product acceptance and user testing for a reason.

Using another AI to check the first one improves things, but it doesn't necessarily remove the problem. Two systems can share the same assumptions and blind spots.

If the final human doesn't understand the work sufficiently to challenge either system, they aren't really providing oversight. They're approving the result.

So will AI replace jobs?

Almost certainly some. It will also change jobs, remove parts of jobs and allow smaller teams to produce things that previously required much larger ones.

But "AI can perform 70% of this person's tasks" does not automatically mean "we can remove 70% of these people."

The real threshold is much harder: can AI deliver a reliably correct outcome for less money, with less supervision and acceptable risk?

For some work, we've probably already crossed that threshold. For complex knowledge work, I don't think we have.

And perhaps the biggest mistake in the current debate is measuring AI by what it can produce rather than by what happens when nobody knowledgeable is standing behind it checking the work.

Because producing something that looks right is relatively easy. Knowing when you're wrong is much harder.