PowerUp Coaching — Living Knowledge Base

The Two-Agent Rule: How to Stop Hallucinations Without Blowing Your Token Budget

6 min read

Topic

A concrete, cheap, automatic architecture for factual verification: two low-cost sub-agents verify independently, one high-reasoning model adjudicates the result — and the gate belongs on anything being written to memory, not just on what you read.

Target Reader

A knowledge entrepreneur, coach, or consultant who has moved past casual AI use into building actual systems — skills, agents, a knowledge base with memory. They ship AI-assisted work to clients or an audience. They know hallucination is real, they’ve added “don’t fabricate, cite your sources” to their prompts, and they believe that’s mostly handled. It isn’t, and they don’t know it yet.

The Fear / Frustration / Want / Aspiration

“I’ve told the AI not to make things up. I’ve told it to cite sources. But I’ve also seen it confidently produce garbage, and I don’t have a real system for catching it — I have a habit of glancing at things and hoping. And every fix I read about sounds expensive: run everything through the big model, verify every claim by hand. I can’t afford that in tokens or in time. I want something structural that doesn’t cost me my rate limits.”

Before State

Verification is a prompt constraint and a personal habit. The reader trusts instructions like “only use verified information” to hold, and backstops them with occasional spot-checks driven by whether something feels off. They assume robust verification means spending their most expensive model on everything, so they don’t do it. Nothing gates what enters their knowledge base — a fabricated claim written to memory becomes permanent ground truth, cited back to them with confidence in every future session.

After State

The reader has one line in their global config that makes verification structural instead of aspirational, and they understand the tiering that makes it affordable: cheap models verify, the expensive model adjudicates. They know why the instinct to do this the other way around is backwards. Critically, they’ve moved the gate to the point where a claim earns permanence — the memory write — because that’s where a hallucination stops being a bad answer and starts being a bad foundation.

Narrative Arc

Open on Scott’s project: explicit instructions not to fabricate, and Gemini invented 50–70% of the results anyway, including the URLs. The instruction was correct; it simply didn’t hold — which means prompt-level constraints are not a verification system, they’re a wish. The turn: Jay’s architecture makes verification structural rather than instructional, and the surprise is the economics — the verifiers don’t need to be smart, they need to be independent, so they run cheap, and the expensive model does only the narrow adjudication. Resolution: the gate matters most where you weren’t looking — on the memory write, because that’s where a lie stops dying with the conversation and starts compounding.

Core Argument

Asking a model not to fabricate is not verification — verification is structural, and the cheapest structure is two independent low-cost agents checking against the source with one high-reasoning model adjudicating the result, applied at the moment a claim earns permanence.

Key Evidence / Examples

  • Jay Drobez’s standing rule, verbatim: “In every skill, I have a line saying: do two independent verifications with two different agents, then present it to the main adjudicator. If there’s a disagreement, do a deep dive. If there’s no disagreement, then I take that risk of two agents being wrong.”
  • The measured cost finding, from two weeks of running it during aggressive rate limiting: “I thought I was just going to blast through my limits. It really does not burn as many tokens.” Sonnet verifiers, Fable/Opus adjudicator.
  • Scott Delinger’s confabulation case: explicit anti-fabrication instructions, 50–70% invented results, fabricated Google search URLs, and the tell that caught it — “I’m going to say that these three nodes were decommissioned so that the number of CPUs remains the same.”
  • Lou’s extension, the article’s real payload: “If you put a hallucination in memory, then you’re really cooked.”
  • Jay’s honest accounting of residual risk: “I take that risk of two agents being wrong.” The architecture prices its error rate rather than claiming zero.
  • Jay’s inference-time variant: a small instruction file alongside the source files, asking for a source-accuracy estimate on a 1–100 scale with exposed reasoning — challenging the model as it infers rather than after.
  • Related insight: Insight - Your Growing Trust in AI Is the New Attack Surface — the psychological reason readers stopped verifying in the first place.

Proposed Structure (6 beats)

  1. The instruction that didn’t hold. Scott’s story — told carefully, because the reader has written the same prompt and believes it works.
  2. Why prompt-level constraints fail. The model is optimizing for your satisfaction, not for truth. You cannot instruct that away, and the outputs that pass your gut check most easily are the ones most shaped to pass it.
  3. The architecture. Two independent verifiers, one adjudicator, deep dive on disagreement. Short — it’s genuinely simple.
  4. The economics nobody expects. Verifiers don’t need intelligence, they need independence. Cheap tier. The expensive model does one narrow comparison. Jay’s measured result.
  5. Move the gate to the write. The core reframe: a bad answer in a conversation dies with the conversation; a bad answer in memory becomes ground truth forever.
  6. The line to paste, and the honest caveat. Two agents can both be wrong. Name the residual risk — that’s what makes it usable.

Editorial Notes

Tone: Practitioner-to-practitioner. Jay’s pattern is the star — attribute it clearly and let his own words carry the technical beats. This is not a fear piece; the reader already believes hallucination exists. The news is that it’s cheap to gate, and that they’re gating the wrong end of the pipe.

Angle to avoid: The generic “AI hallucinates, be careful” article. That’s been written a thousand times and this vault already has Brief - A Field Guide to Trusting AI Output Without Getting Burned covering trust calibration. This piece is strictly architectural — one pattern, its cost profile, its placement.

Competing brief: Brief - A Field Guide to Trusting AI Output Without Getting Burned (2026-04-08, source: 80-20 Rule) covers when to trust. This covers what to build so trust isn’t the question. Complementary, not overlapping — but they should not be drafted back-to-back, and this one should reference that one rather than re-arguing calibration.

Verify before publishing: Jay’s token-cost claim is his own measurement over ~2 weeks, not a benchmark. Present it as a practitioner report (“Jay measured…”), never as a general finding. Do not generalize the Sonnet/Fable pairing into a recommendation for model choices we haven’t tested.

The strongest line in the source material is Lou’s “inference is not truth.” It may belong in this piece or in the trust piece — don’t spend it in both.

Next Step

  • Approved for drafting
  • Needs revision
  • Deprioritised

Graph View