Topic
Why AI agents that “refuse” instructions are usually obeying an older instruction from you — and the ten-minute diagnostic that finds it.
Target Reader
A knowledge entrepreneur or consultant six to eighteen months into building their own AI setup. They have custom instructions, rule files, maybe a few named agents. They are past “how do I prompt this” and into “why won’t this thing do what I say.” Technical enough to have written rules; not technical enough to think of those rules as a codebase.
The Fear / Frustration / Want / Aspiration
The specific frustration of losing half a day to a machine that won’t listen — and the creeping doubt underneath it: maybe I’m not good enough at this. The reader has been rephrasing prompts with increasing precision and getting the same wrong output, and has started to wonder whether the whole approach is beyond them.
Before State
Treats bad AI output as a prompting problem. When an agent misbehaves, escalates by rewriting the request — more explicit, more emphatic, ALL CAPS if it comes to that. Has never opened their own rule files while debugging, because it doesn’t occur to them that their instructions are the suspect.
After State
Recognises the layer error. Understands that a rule-level constraint silently beats any prompt-level correction, that the newest models are more literal about instruction-following (so this failure mode is growing, not shrinking), and that the fix is to read the instruction set as source code — ideally with a second agent doing the reading.
Narrative Arc
Lou spends half a day trying to convince his editor agent to extract the structure of an article instead of copying its content; the agent keeps inserting Charlie Munger quotes because Munger was in the source. The turn: he stops debugging the output and points a second agent at the first agent’s instruction files. In one pass it reports back — of course it’s doing that, you told it to — quoting a rule Lou wrote months earlier and forgot. The resolution: your agents don’t disobey, they obey things you’ve stopped remembering, and past a certain number of agents you need a standing audit function rather than a frustrated afternoon.
Core Argument
An AI agent that ignores your instruction twice is not failing to understand you — it is following a constraint you wrote at a layer your prompt cannot reach, and the fix is to read the code, not argue with the runtime.
Key Evidence / Examples
- Lou’s editor agent inserting citations from Charlie Munger specifically when asked to model an article’s structure rather than its content — a perfect illustration of literal compliance producing apparent disobedience.
- The half-day cost, versus the one-pass diagnosis once a second agent read the rule files. The asymmetry is the whole argument.
- Scott Delinger’s structural rule from the same session: “the auditor agent can only report to the orchestrating agent, not any agent it’s auditing.” An auditor with a channel to its subject is a negotiation.
- The two-writer-agents-with-conflicting-editorial-guidelines case: the orchestrator picks one more or less arbitrarily, producing non-deterministic quality with no visible cause.
- The canonical-library payoff — Insight - Fewer, Better Agents — And the Auditor Never Reports to the Audited — one repository means one audit pass and one fix that propagates everywhere.
Proposed Structure (5–7 beats)
- Open cold on the half-day: the request, the wrong output, the escalating rephrases, the growing suspicion that you’re the problem.
- The reveal — a second agent reads the first agent’s rules and finds it immediately. Name the layer error explicitly: prompts cannot override rules.
- Why this is getting worse, not better: better instruction-following means better compliance with your mistakes. The discipline that made you good at this is the thing generating the bug.
- The diagnostic, verbatim and copyable — the exact prompt to hand a second agent.
- Scott’s reporting-line rule, and why an audit that talks to its subject isn’t an audit.
- The scaling problem: at five agents you can read everything, at fifty you can’t, and self-improving agents guarantee you can’t. Audit as a standing function.
- Close on the reframe worth keeping: your instruction set is a codebase, and nobody debugs a codebase by arguing with it.
Related Insights
- Insight - Your Own Rules Are the Bug — Use One Agent to Debug Another
- Insight - Your Rubric Can Certify Garbage — When the Evaluator Drifts With the Work
- Insight - Fewer, Better Agents — And the Auditor Never Reports to the Audited
- Insight - Skill Slop — Unconstrained Skills Get Invoked in Unintended Ways
- Insight - The Universal Audit Phrase — Errors, Omissions, Oversights, Duplications, Contradictions
Editorial Notes
Lead with the failure and Lou’s own admission of the wasted half-day — the confession earns the reader’s trust before any advice lands. Do not frame this as “here’s a mistake beginners make”; the reader who has this problem is good at this, and the article should say so. Avoid the temptation to make it about a specific model’s literalism — that dates fast and isn’t the point. The through-line is a layer distinction, not a model quirk. Pairs naturally with the rubric-drift piece; consider them a two-part sequence on things you built that quietly stopped serving you, but each must stand alone.
Next Step
- Approved for drafting
- Needs revision
- Deprioritised