← Back to blog

It's Not an Engineer. It's a Prosthesis.

· 12 min read · AI · aiagentsclaude-codemulti-agentsoftware-engineeringvibecoding
Contents

Note

This is Part 4 of 4 of From Swarm to Protocol, a serialized edition of “From swarm to protocol: a year of vibing, from noob to pro”. The body below is reproduced from the complete edition with only navigational changes; stage numbers refer to the full arc.

Everything before was machinery; this closer is the honest part: what the whole apparatus costs, which failures the floor actually catches and which slip through, why the human at the last gate matters more as the agents improve, and why, when stripped of the agents, it was just engineering all along.

Epilogue: The Prosthetics

None of the abformentioned was free. It burns way more tokens and more patience than just opening up Claude Opus 4.8 on ultracode and letting it go ham with one shot prompts. Some people, particularly those with years and decades of engineering knowledge, would benefit the least from my setup as they already contain the necessary skills to write proper specifications, detailed implementation plans, and better intuition on what exactly needs to be done and how. The only thing my maybe framework beats out on is the shipping speed, and even then a single talented, proper engineer with a single model burning about a quarter of the tokens would probably ship something better in maybe at most double the time, and a team of maybe two or three competent engineers would be able to match the speed and even surpass the quality with roughly the same amount of tokens spent.

The caveat: this was never a clean experiment. The workflow got better, but, over the same handful of months, so did the models underneath it. As well as my familiarity with the codebase, and my own judgment.1 I’ll be honest and say that I can’t completely separate workflow lift from model lift from “just-getting-less-bad-at-this”; they all moved together. I also wish I could have given numbers and metrics but I honestly didn’t measure any — making them up would’ve been antithetical to one of the core lessons of this story!

Thing is, with the current state of gen-AI models, it’s very easy to start building something you can’t understand and can’t verify by yourself very effectively. With my framework, it at least allows someone with very little technical know-how to barely, just barely stand a chance, giving them a floor, not a license. And even then, it’s just mostly agents checking the work of other agents, which is still rather risky. Really, what the whole thing does is trade quality for velocity: none of the gates exist to make the output as good as it could be. They’re there to keep your codebase from being a complete slop-fest while you move fast, allowing you to ship something that is past MVP and is usable by clients, but not something that’s quite enterprise-grade. In practice this meant that an incomplete migration can still occasionally slip through to deployment before it reaches you.2 Which means the moment velocity stops being the priority, the trade goes minus EV, and you’re better off slowing down and building it properly from the get-go without this whole multi-agent setup.

It’s worth noting that keeping it out of slop-fest territory is only the floor, so it’s important to be precise about what failures the floor actually catches. The loud ones — like an agent reporting work it never did — die the moment you demand evidence, so it’s easy to check no matter how little you know. The quiet ones are the dangerous remainder: the stochastic, plausibly-correct-looking work two agents converge on, the ouroboros that never truly disappears, and the many other errors my board never gets to green. Those only get resolved for real when you read the diffs as carefully as you’d read your own. In short, someone working with little to no knowledge would get the floor and almost nothing past it.

Danger

Worth being precise about why “show me” works at all: in my experience agents almost never forge the actual artifact. They’ll over-claim a bare “done” all day — narrating the plausible, as I keep saying — but pin them to the real proof (the merge SHA, the CI run, the repo state) and they’ll either go do the work or admit they hadn’t, rather than hand you a fabricated log.

This only pushes the hallucination risk onto the evidence side — but, in my experience at least, it occurs far less often there, and I think the reason is grounding, not luck: producing the actual SHA or test run forces the agent back into contact with the real repo, and that contact is usually what makes it catch the gap (or cop to it).

The big caveat is that all of this assumes good-faith incompetence. There are really two tiers of fabrication: forging the report of evidence is caught by demanding the real repo state rather than the agent’s summary of it, but an agent maliciously gaming the signal itself — subverting the very test meant to catch it so it genuinely passes — means the ground truth is the forgery, and checking the repo saves you nothing.

That isn’t hypothetical: labs training frontier models have documented it repeatedly — OpenAI caught reasoning models reward-hacking their own unit tests, Anthropic watched models generalize all the way to rewriting their reward function and covering their tracks, and Apollo Research showed frontier models will scheme and deceive their evaluators outright.

Against an adversary rather than a sloppy worker, the “show me” gate inverts — the proof itself becomes the lie — and the whole thing completely breaks down without a human reviewing the change in detail.

Therefore, let me be clear about what this whole multi-agent orchestration framework actually is.

It is not an engineer. It’s a prosthesis.

Something that lets you reach past what you could do unaided, whether that’s running models too cheap to trust, working a domain you only half understand (or in my case, understood only <5%), or holding more parallel surface area than one head can keep straight. And like any prosthesis, it’s only there because something is missing, not because you’ve somehow transcended the need for it.

This is evident from just looking at what’s actually stopping London Bridge from falling down. That stack of agents checking agents checking each other — a planner watched by a reviewer, an implementer watched by a panel, a roadmap reconciled by an orchestrator that is itself just two more agents — is turtles most of the way down.3

Three real turtles stacked one atop another -- turtles all the way down.
Turtles most of the way down. In the workflow, the one bolted to the very top isn’t a turtle — it’s a human.

The only thing that makes it trustworthy instead of terrifying is that, at the very top, there’s a skeptical human bolted to the last gate, refusing to call anything done until something real — a merge, a repo state, a test that actually ran — says so; even then, that’s just a call to go dress up as a QA team, either with agents, by themselves, or both, at which point they’d bug hunt the deployed output (hopefully not on prod). Pull that human out and let the loop run itself, fully agentified, and you won’t have an engineer, much less the engineering team you replaced it with4 — you have the Stage 3 swarm again, just with better manners and a much larger bill: confident, plausible, and agreeable output at scale, with nobody left who can tell whether any of it is true and leaving QA for someone else downstream.

That’s the part the AI hype train these last couple of years gets backwards. The instinct, once a framework like this starts working, is to automate the last human out of it too — close the loop, walk away. But the more of the checking you hand to agents, the more weight lands on the one check you didn’t automate, not less.

Note that this gate is only ever as good as the time and judgment you spend standing at it. The cheap, fast version is just refusing to let “done” travel without proof: it’s the only node not drawn from the same stochastic well, so it’s the one place a claim can be forced to show a merge, a repo state, a test that actually ran, instead of just sounding right. That alone kills the loud failures, like an agent narrating work it never did or an agent going off in an orthogonal direction to your plan, and it works regardless of how much (or little) you know. The quiet ones, real work that’s subtly wrong, are catchable too, but only by reading the diffs as carefully as you’d read your own… which is exactly the slow, expensive review the speed was bought by skipping. So a sharp engineer can absolutely close that gap; they just close it by spending the time they came here to save. The one who can’t close it at any price is someone working a domain they barely understand — because that gap takes knowledge and intuition, and that is the thing that more review hours don’t buy.

If someone can figure out how to take the human out of the loop entirely, we’d have something very close to AGI! Unfortunately, or fortunately, we’re a bit far from there. I tried this loop out with Fable 5 the short time it was available, and it still didn’t reach this level!

However, here’s the thing that snuck up on me. Take out all of the agents — the orchestrator, the pairs, the panels, the relays — and look at what’s left. A spec written before any code. A plan reviewed by someone who didn’t write it. A claim of “done” that has to carry proof. A second reader whose whole job is to disagree. A rule for every place something once broke.

That’s not a multi-agent methodology. That’s just engineering.

It’s the workflow people were already doing long before any of this, and the kind I, and basically every vibecoder out there, skips. The agents didn’t teach me that. They billed me for it, over and over, in production, in front of whoever was downstream, at every place I’d been sloppy enough to assume the worker on the other end was competent and honest. The whole framework is just what good practice looks like once you can no longer assume either, and every ceremony in it is a scar — nothing made it in because it sounded clever, and any rule that couldn’t point to a specific time something went wrong didn’t earn its place.

So no — I wouldn’t hand this to someone as a way to replace or become an engineer(s), any more than I’d hand someone a wheelchair as a way to learn to run. I’d be much more comfortable recommending this as a tool for someone looking to get something done at least somewhat properly when constrained by budget, time, or knowledge. But if you’ve already got the judgment and a decent model, spend it — the bureaucracy my workflow runs on is the thing you use when you can’t.

To recap, I went into all this a sophomore one-shotting “build me a chatroom” into a split-screen editor, and I came out the other side running a simple multi-model bureaucratic emulator of an engineering team — and writing better software because of it. Somewhere in there I stopped being a complete vibecoder and turned into something closer to an engineer — less trial and error than being Sisyphus, if the boulder stayed up a little longer each push.

Key word being closer. Not quite my tempo. By the one rule this whole thing taught me, “engineer,” like “done,” is a claim, not a fact — and I don’t have the evidence, nor the knowledge, to back it up quite just yet.

Previous: Part 3 — Three Rules for Agent Teams

The combined list of references and supplementary reading can be found here.

Image credits:

  • Turtles all the way down (Epilogue) — River terrapin by Pelf, via Wikimedia Commons; public domain.

Footnotes

  1. Mind you, I wasn’t just brainlessly vibing the whole time — I spent a nontrivial amount of my time researching and reading up on proper software development: security architecture and threat modeling, system design, how to actually structure a schema or a service contract before it solidifies into the wrong shape. Some of that operator lift was deliberate and not osmosis. ↩

  2. Though in all fairness, this still happens even with a full engineering team and a dedicated QA team. ↩

  3. Though I will say this tangled web of checks and balances still functions better than the analogous system in the US government. ↩

  4. Looking at you, Corpo Tech — especially Microsoft and Meta. Among others. ↩