You are hiring a senior practitioner.
Their portfolio is impressive. Three case studies. Two thought-leadership pieces. A framework that looks well-developed.
References say they’re sharp. You’re 60% sure.
Eighteen months ago, you’d have been 90% sure with that data. Now you’re at 60% and you can’t quite say why.
Here’s why.
The artifacts in their portfolio could have been produced by judgment they actually built. Or by AI output they edited. From the outside, you cannot tell which.
Neither can their references. Most references don’t see the work being made. Only the work being delivered.
The hiring problem in this market is not finding people with strong portfolios. AI-augmented portfolios are everywhere. The problem is telling whose portfolio reflects real judgment.
Here is how to actually do that.
OPERATOR FILE #24 (Hiring)
Expert hiring managers test for what’s underneath the portfolio.
Average hiring managers evaluate the portfolio.
Commodity hiring managers trust the references.
Here is the mechanism, then the practice.
When someone uses AI extensively in their work over a year or two, two things diverge.
Their output.
And their underlying capability.
The output stays high. The capability quietly atrophies in places — particularly in the judgment-formation pathway. The cognitive activity that builds discrimination from doing, failing, and recovering.
Call this the phantom expert.
They produce real outputs. They are not faking. The problem is that the outputs are downstream of borrowed scaffolding rather than earned discrimination.
The difference does not surface until a high-stakes situation requires the discrimination directly.
A hiring process that evaluates portfolios is evaluating outputs.
You will hire phantom experts at scale if your process doesn’t probe for what’s underneath.
Three protocols change that.
Protocol #2 — The Portfolio Reverse-Engineer
Pick one piece from their portfolio. Tell them:
“Walk me through how you’d produce this from scratch today, without using AI. What would your week look like?”
Watch their face for the first three seconds.
A practitioner who actually produced the work this way will describe a process. They will mention specific decisions. Specific moments where they considered alternatives. Specific failures along the way that shaped the final form.
The process will have texture.
A practitioner who AI-mediated the work will produce a recipe. They will describe the steps in clean, generic terms — “I’d start with research, then draft an outline, then iterate” — without the texture of actual judgment-formation.
This is not about catching AI use.
AI use is fine.
This is about telling whether the candidate has the capability that would have produced this output if AI hadn’t existed.
If they can’t produce that capability under your gaze, they probably don’t have it.
What These Protocols Won’t Catch
They are designed to detect phantom expertise in existing domains where the candidate’s portfolio claims judgment.
They will not reliably detect a candidate who’s expanding into new domains and hasn’t yet built judgment there. For new-domain capability, the test is different — see the acquisition discipline in Article 3.
For senior roles, the existing-domain test is usually the load-bearing one.
You’re hiring for established judgment. Not potential.
The Operational Close
Before your next senior hire, write down the three signals that would tell you their expertise is real, not phantom.
If you can’t write them down, you don’t have them yet.
That is the work.
The cost of a bad senior hire in this market is far higher than it was three years ago.
The floor on portfolio quality has come up.
The floor on judgment hasn’t.
Hire the difference. Not the artifacts.