Lesson 3 · Failure mode #3: code doesn't work

Give the agent a feedback loop it can't fool

You and the agent are aligned (Lesson 1) and speaking the same language (Lesson 2). The code still comes out wrong. That is a feedback problem: the agent is flying blind. Test-driven development is the loop that lets it see. But most agent-written tests are worthless, and this lesson is really about telling the good ones from the fakes.

Why feedback, and why TDD specifically

Always take small, deliberate steps. The rate of feedback is your speed limit. David Thomas & Andrew Hunt, The Pragmatic Programmer

Without feedback on how its code actually runs, an agent guesses and hopes. You want the usual tranche of loops: static types, browser access, and automated tests. TDD's red-green-refactor is the tightest of these because the agent writes a failing test first, then makes it pass, so every step is checked against something concrete.

RED:      write a failing test for one behavior
GREEN:    write the minimal code to make it pass
REFACTOR: improve the design, tests stay green

One rule to hold the agent to immediately: never refactor while red. Get to green first, then clean up.

The anti-pattern that makes agents produce crap tests

Left alone, agents "do TDD" wrong. They write all the tests, then all the code. Matt calls this horizontal slicing, and it produces bad tests every time:

WRONG (horizontal):
  RED:   test1, test2, test3, test4, test5
  GREEN: impl1, impl2, impl3, impl4, impl5

RIGHT (vertical, "tracer bullets"):
  test1 -> impl1
  test2 -> impl2
  test3 -> impl3

Why horizontal fails: tests written in bulk test imagined behavior, not real behavior. They end up asserting the shape of things (data structures, signatures) instead of what the user can do. They pass when behavior breaks and fail when it is fine. Insist on one test, then one implementation, then repeat. Each test reacts to what the last cycle taught.

Good test vs bad test: the one distinction that matters

A test should verify behavior through the public interface, not implementation details. The acid test: if you rename an internal function without changing behavior, does the test break? If yes, it was testing implementation, and it is lying to you.

Good: describes what, uses the public API, survives refactors

test("user can checkout with valid cart", async () => {
  const cart = createCart();
  cart.add(product);
  const result = await checkout(cart, paymentMethod);
  expect(result.status).toBe("confirmed");
});

Bad: coupled to how it works, breaks on harmless refactors

test("checkout calls paymentService.process", async () => {
  const mockPayment = jest.mock(paymentService);
  await checkout(cart, payment);
  expect(mockPayment.process).toHaveBeenCalledWith(cart.total);
});

Red flags: mocking internal collaborators, testing private methods, asserting on call counts or order, verifying through the database instead of the interface. (Mock only at real system boundaries: external APIs, time, randomness. Never mock your own code.)

The subtle killer: tautological tests

This is the highest-value idea in the lesson. A tautological test computes its expected value the same way the code does, so it passes by construction and can never disagree with the code. Break the code wrong, and the assertion breaks the same wrong way. Zero confidence, looks like coverage.

// BAD: expected value is recomputed the way the code computes it
test("calculateTotal sums line items", () => {
  const items = [{ price: 10 }, { price: 5 }];
  const expected = items.reduce((sum, i) => sum + i.price, 0);
  expect(calculateTotal(items)).toBe(expected);   // asserts code == code
});

// GOOD: expected value is an independent, known literal
test("calculateTotal sums line items", () => {
  expect(calculateTotal([{ price: 10 }, { price: 5 }])).toBe(15);
});

The rule The expected value must come from an independent source of truth: a known-good literal, a worked example, the spec. Never a value recomputed by the same logic under test. This is the single most common way agent-written tests turn out fake.

Check yourself

Which test is tautological (passes by construction, gives no confidence)?

Why is "write all tests, then all code" (horizontal slicing) discouraged?

You rename an internal helper without changing behavior, and a test fails. What does that tell you?

Do this now (your one win)

Next time the agent fixes a bug or adds logic in a real repo (a script in scripts/content/, the Plaud sync, an automation), run it test-first:

Say "use the tdd skill: one failing test, then minimal code, then repeat. No horizontal slicing."
For each test the agent writes, apply two checks yourself: (a) would this survive renaming an internal function? (b) is the expected value an independent literal, or recomputed the way the code computes it?
Kill any test that fails either check. Watch how many of the agent's first-draft tests are tautological.

Report back: did the agent try to slice horizontally or write a tautological test, and how did you catch it? That is a strong learning record, it shows you can now police the feedback loop, not just run it.

Primary source · read this
tdd/SKILL.md and its tests.md (good vs bad, tautological examples). Both short.

Your teacher is here Want to try it on a real function right now? Paste one and say "grill my tests" or "drive a red-green-refactor loop on this with me". I will play the tdd skill and you catch the fakes.