Lesson 4 · Failure mode #3, continued

Build the feedback loop before you theorize

When an agent hits a hard bug, its instinct (and yours) is to stare at the code and guess. That is the exact failure this skill prevents. The whole discipline reduces to one move: get a command that goes red on this specific bug before forming a single hypothesis. Everything after that is mechanical.

The one idea

Phase 1 is the skill If you have a tight pass/fail signal that goes red on this bug, you will find the cause: bisection, hypothesis-testing, and instrumentation all just consume that signal. If you don't have one, no amount of staring at code will save you. Spend disproportionate effort here. Be aggressive, be creative, refuse to give up.

Matt's rule is blunt: no red-capable command, no hypothesis. If you catch the agent (or yourself) reading code to build a theory before that command exists, stop. Jumping to a hypothesis is the failure the skill exists to block.

Ten ways to build a loop, roughly in order

You do not need all of these. You need the first one that reaches the bug:

A failing test at whatever seam reaches the bug.
A curl / HTTP script against a running dev server.
A CLI invocation with a fixture input, diffed against known-good output.
A headless browser script that drives the UI and asserts on DOM / console / network.
Replay a captured trace: save a real payload to disk, replay it through the code path.
A throwaway harness: minimal subset of the system, one function call.
A property / fuzz loop for "sometimes wrong output" bugs.
A bisection harness for "it broke between two known states" (git bisect run).
A differential loop: same input through old vs new, diff the outputs.
A human-in-the-loop bash script, last resort, but still structured.

Then tighten the loop like a product: faster (cache setup, narrow scope), sharper (assert the exact symptom, not "didn't crash"), more deterministic (pin time, seed RNG, freeze network). A 2-second deterministic loop is a debugging superpower; a 30-second flaky one is barely better than nothing. For flaky bugs, the goal is not a clean repro but a higher reproduction rate: loop it 100x, parallelize, add stress until it is debuggable.

The rest is mechanical: phases 2 to 6

2. Reproduce + minimise. Run the loop, watch it go red on the user's exact symptom (not a nearby lookalike). Then cut inputs, callers, config one at a time until every remaining piece is load-bearing.
3. Hypothesise. Generate 3 to 5 ranked, falsifiable hypotheses before testing any. Each states a prediction: "if X is the cause, changing Y makes the bug vanish." Show the ranked list to the user, they often re-rank it instantly ("we just deployed #3").
4. Instrument. One variable at a time. Prefer a debugger breakpoint over ten logs. Tag every debug log with a unique prefix like [DEBUG-a4f2] so cleanup is one grep. For perf bugs, measure first, never "log everything and grep".
5. Fix + regression test. Write the failing regression test before the fix, but only if a correct seam exists (one that exercises the real bug pattern). No correct seam is itself a finding: the architecture is preventing lockdown. Flag it.
6. Cleanup + post-mortem. Original repro gone, regression test passing, all [DEBUG-...] removed, correct hypothesis written into the commit message. Then ask: what would have prevented this? If the answer is architectural, hand off to improve-codebase-architecture (Lesson 5).

Notice the single thread: every phase either feeds or consumes the feedback loop. Same lesson as TDD, aimed at bugs instead of features.

Check yourself

According to the skill, what is "the skill" in diagnosing a hard bug?

When may you move on to forming hypotheses?

Why generate 3 to 5 ranked hypotheses instead of chasing the first one?

Do this now (your one win)

Next real bug in any repo (a script throwing, a VAPI relay misbehaving, a blog-pipeline step producing wrong output):

Say "use the diagnosing-bugs skill. Phase 1 only: build me one red-capable command before any theory."
Do not let the agent explain a cause until it has pasted a command and its red output that asserts your exact symptom.
Make it tag any debug logs with a prefix so cleanup is one grep.
At the end, ask it the post-mortem question: what would have prevented this? Note whether the answer is architectural.

Report back: what loop did it build (test? curl? harness?), and did it try to theorize before the loop went red? Catching that jump is the whole lesson.

Primary source · read this
diagnosing-bugs/SKILL.md. The Phase 1 section is the part to internalize.

Your teacher is here Got a live bug? Paste the symptom and say "Phase 1 me", and I will refuse to theorize until we have a red command. Best way to feel the discipline.