“Too many cooks and nobody’s the boss — so they kind of pick randomly.” — Lou, on why more agents produced worse output
Session context: 2026-07-23_Mastermind — after Lou’s account of an editor agent that wouldn’t behave, the group converged on the count itself as the problem. Mazie supplied the proverb in chat; Scott supplied the structural fix.
Core Idea
The default reflex when an AI workflow underperforms is to add another agent — a reviewer, a fact-checker, a second opinion. It feels like rigour. Lou cited research finding that assigning more agents to a task degraded performance rather than improving it, and both he and Jay described hitting it in practice. Jay’s version is the sharper one: extra agents can help on mechanical dimensions like punctuation or source checking, but “if it gets [the important part] wrong, it’ll double down on the wrong part, and then you just get something that’s useless.” Adding voices does not add judgment. It adds ways for the wrong judgment to get reinforced.
Lou’s diagnosis of the mechanism is the useful part. The failure isn’t quantity as such, it’s unowned conflict. Two writer agents with different editorial guidelines don’t negotiate; the orchestrator picks one, more or less arbitrarily, and the result is non-deterministic quality with no visible cause. Nobody’s the boss.
Three rules came out of it.
Fewer, better agents, each with an independent task. Don’t create near-duplicate agents that do similar work differently. If two agents overlap, one of them is a bug.
One canonical library. Keep every skill in a single repository so a single pass can compare them all, surface contradictions and blind spots, and — the payoff Lou emphasized — so that one fix propagates to every folder that inherits the skill. “If you detect an error and you fix it, then every folder that uses that particular skill automatically gets the fix.” Distributed copies make an audit impossible and a correction endless.
The auditor never reports to the audited. Scott’s line in chat is the structural constraint that makes any of this trustworthy: “the auditor agent can only report to the orchestrating agent, not any agent it’s auditing.” An auditor with a channel back to its subject is a negotiation. The finding has to travel up to whoever can act on it, never sideways to whoever caused it.
There is a corollary on cost that came in from the same direction. Lou stopped using the maximum-parallelism mode almost entirely — “it fans out so many agents… not all of them need to be independent context, and it burns tokens like a mother” — and Scott’s chat verdict was blunter: “Ultra is tokenmaxxing.” Fan-out is fast and expensive and not measurably more reliable. Speed is not the same as correctness, and you’re paying for both as if they were.
Practical Application
Inventory your agents this week. List every one, and next to each write the single sentence that says what only it does. Any two agents whose sentences overlap are candidates for merger — pick the better one and delete the other, don’t try to differentiate them. Then check one thing: does any auditing or reviewing agent you have report its findings back into the thing it reviews? If so, re-route it to the orchestrator. Finally, confirm every remaining agent lives in one canonical location, so the next fix only has to be made once.
Related Insights
- Insight - Verify With Cheap Agents, Adjudicate With an Expensive One — the case where more agents does help, and why: independent verifiers plus a single adjudicator, not a committee.
- Insight - Skill Composition — Many Small Skills With Shared Modes Beat One Big Monolith — the tension worth holding: many small skills beats a monolith, while many overlapping agents does not.
- Insight - Skill Slop — Unconstrained Skills Get Invoked in Unintended Ways — what a proliferating, unpruned library does to invocation reliability.
- Insight - Your Own Rules Are the Bug — Use One Agent to Debug Another — the diagnostic that surfaces conflicting instructions across agents.
- Insight - The Tool Tax — When 40% of Your Time Goes to Sharpening the Axe — every extra agent is maintenance surface, and maintenance is where the time goes.
Evolution Across Sessions
Sharpens Insight - Verify With Cheap Agents, Adjudicate With an Expensive One (2026-07-16), which established a multi-agent verification architecture that does work, by naming the conditions under which multi-agent architectures fail instead: overlapping responsibilities, unowned conflict between instruction sets, and no clear adjudicator. The distinction is ownership, not headcount — Jay’s two verifiers work because exactly one agent decides. New to this session: the canonical-library argument for auditability and single-point fixes, Scott’s auditor reporting-line rule, and the observation that maximum-fan-out modes buy speed rather than reliability while costing significantly more.