2025-08-14_Mastermind

2025-08-14 AI Mastermind for Leaders

Table of Contents

• Insight - RAG Is Raw Material, Not Answers — Design for the Right Retrieval Architecture

Session Overview

Lou opened August 14 with a note that it is “almost the last quarter,” signaling the shift in focus toward the end-of-year business run for entrepreneurs and coaches. He used the first portion of the session to share ongoing learnings from his legal AI project — specifically the deeper architectural questions that emerged from client testing. The session became an extended, practical exploration of what RAG can and cannot do, why architectural mismatch is a more common problem than bad prompting, and how knowledge graphs might solve the relational retrieval challenge that naive RAG cannot.

The centerpiece moment came from a live client session earlier that day: the client’s very first question caused the system to fail. Not because the prompting was wrong, but because the client wanted comprehensive, relational analysis across all 2,200 documents simultaneously — exactly the task RAG was not designed for. Lou’s transparent, in-the-moment problem-solving framing made this a high-quality learning experience: he demonstrated how to diagnose the actual architecture problem rather than trying to prompt-engineer around a structural limitation.

The session included rich discussion from Bally Binning on the training implications for end users of RAG systems, and from Alex F on the parallels between language document indexing and engineering graph models. The group collectively converged on the knowledge graph direction, with InferNotus and Neo4J emerging as concrete options. Don Back’s question about whether Lou was considering turning the legal tool into a general product led to an important tangent on business model decisions: equity vs. development fees, directing vs. executing, and the danger of resentment when obligations replace enthusiasm.

High-Signal Moments

• Lou’s file cabinet metaphor for RAG: your assistant has to know which file folder to pull, which page to look at — and they don’t always know, and neither do you when prompting
• The “naive RAG is top 5 chunks” clarification — the most common misconception about RAG capabilities, addressed directly
• Contextual RAG explained: add a document summary to every chunk so all pieces from one document retrieve together
• “There are about 15 different RAG models now” — the acknowledgment that “RAG” is not one thing but a family of architectures
• Don Back’s product question and Lou’s answer: “Direct it. Don’t do it” — a spontaneous piece of coaching that applies well beyond AI product development
• “I’m already noticing things I can’t get around to that I want to, because I’m obligated to do this, and I’m resentful of it” — a raw, honest moment about the energetic cost of taking on obligations misaligned with how you want to work
• Lou’s response: equity + handoff to a partner who can execute, not taking on a full-time development role
• Bally Binning’s coaching knowledge base use case: RAG for decision-making frameworks and coaching tools — a concrete PowerUp application

Open Questions

1. Is a knowledge graph the right architecture for comprehensive cross-document relational retrieval, or does it just push the problem one level deeper?
2. How do you handle pronoun resolution and co-reference at scale in RAG — without moving to knowledge graphs?
3. For coaching knowledge bases specifically: what is the right chunk size and architecture for frameworks vs. transcripts vs. journal entries?
4. At what scale of document library does the effort of building a knowledge graph become worth it vs. relying on a long-context model?
5. How do you know, before building, whether your use case is a RAG problem or a knowledge graph problem?

Suggested Follow-Through

• Before building any RAG system, write down the actual query pattern you expect users to run — and check whether it is a “find the most relevant piece” question or a “analyze across everything” question
• Test InferNotus (notus.ai) with a subset of the legal documents as a proof-of-concept for knowledge graph retrieval before building a custom implementation
• Explore Neo4J as the open-source alternative for knowledge graph construction
• Review the “Contextual RAG” pattern for any use case where complete process steps or frameworks need to be retrieved intact
• Revisit the “Direct it, don’t do it” principle in your own business: where are you executing tasks that should be delegated, and what is the resentment cost?

Additional Resources

Links & Tools Shared in Chat

• InfraNodus — knowledge graph visualization tool for text analysis — https://infranodus.com/ (shared by Donald Kihenja; noted it came to mind during the knowledge graph discussion)

Tools Mentioned in Chat

• Instantly.ai — cold email outreach automation tool (mentioned by Donald Kihenja in the context of outbound lead sequences)
• Vapi — voice AI for phone calls and call automation (mentioned by Donald Kihenja)
• n8n — open-source workflow automation (mentioned by Donald Kihenja as part of an outbound automation stack)

Books Mentioned

• Pre-Suasion by Robert Cialdini — mentioned by Donald Kihenja in the context of timing-based persuasion; “priming them — like it” (Bally Binning); the principle: prime the prospect with a relevant gift or value before making an ask
• The Road Less Stupid by Keith Cunningham — mentioned by Bally Binning; referenced in context of avoiding costly thinking errors in business decisions

Ideas from Chat

• Don Back’s question: “Create a JSON summary file?” — as a lightweight solution for giving an AI system persistent structured context about a project; worth exploring as a minimal memory architecture
• Donald Kihenja: “Knowledge graph sounds like a 3D AI database” — an accessible metaphor for non-technical clients
• The reciprocity-sequenced outbound lead flow: send value-loaded content → prime with relevant gift → book call — Donald Kihenja’s framing of a Cialdini-informed lead generation sequence
• Don Back: “Interesting, load up with the reciprocity driver and book the call” — the tactical summary of the above
• Chain of Density summarization technique was referenced (Don Back: “Chain of Density was talked about in Client Engine 5 coaching”) as a known framework in the group’s context

Derived Artifacts

• transcript-miner (Transcript Miner — transcript-to-thought-leadership pipeline)
• wisdom-doctrine (Wisdom Doctrine — excavating expertise below conscious access)
• SKILL (Meeting Nuggets — session recap skill)