Skip to module content
🧠AI Academy
0/91
Sign in
Module 07 Β· ~10 min

The Debugging Discipline

A five-trace drill: hypothesis first, answer overleaf.

Reading progress
0/5 Β· 0%

The big idea

πŸ’‘Key idea
Before you paste any error into an AI, do four things. First, read what the error literally says β€” the file, the line number, the error type, the message. Don't skim it; read it. Second, write a one-sentence hypothesis. In writing, even if you're not sure. Third, now consult the AI β€” but you're checking a hypothesis, not asking an oracle to think for you. Fourth, close every session by asking: 'Is this the real fix, or a workaround?' The gap between what you thought was wrong and what was actually wrong? That gap is your entire curriculum.
Quick check
1 question Β· instant feedback
0/1
  1. The API returns 400 with 'messages.1.content: Input should be a valid list.' Where is the bug?

Deep dive

2/2 open

Here's what each trace teaches you.

**Trace 1 β€” TypeError: Cannot read properties of undefined.** Real cause: the code trusted the response shape without checking `res.ok` first. Real fix: throw on `!res.ok` with a structured error, then validate the success shape with Zod. The band-aid to reject: `return (body?.elements ?? [])` β€” this makes the crash disappear and installs silent error-swallowing in its place.

**Trace 2 β€” 400: messages.1.content: Input should be a valid list.** Real cause: an assistant tool-use turn was replayed as a bare string. Real fix: store and replay `response.content` verbatim; type the history structure so this can't happen silently again.

**Trace 3 β€” relation does not exist (works locally, fails in staging).** Real cause: the migration was never applied to staging. Real fix: migration lives in the repo; CI checks that migration history matches the target project.

**Trace 4 β€” 429 contention across accounts.** Covered in depth in the example above. Retrying harder cannot fix contention between requests.

**Trace 5 β€” eval suite catches numeric mismatches after `.slice(0, 50)` is added.** Real cause: the 'performance cap' silently changed semantics β€” the model summarised a sample while the report quoted totals. Real fix: compute totals in code over all rows; pass computed aggregates to the model. If context length forces truncation, truncate detail rows β€” never numbers.

Every AI-suggested fix comes in two flavours: the one that makes the error disappear, and the one that actually fixes the broken invariant. They are not the same thing.

Get in the habit of asking the AI directly: 'Is this the real fix, or a workaround?' Then read its answer critically β€” the AI will sometimes tell you it's a workaround if you ask.

Here's the harder truth: the highest-severity bugs in AI systems don't throw exceptions at all. Your error-reading habit (traces 1–4) catches what crashes. Your eval suite (trace 5) catches what silently lies. You genuinely need both β€” one without the other leaves a category of bugs invisible.

Quick check
1 question Β· instant feedback
0/1
  1. Same code works locally, fails on staging with 'relation does not exist.' First move?

In the field

πŸ”¬Worked example
Trace 4 Β· Rate-limit contention Β· [mcp] LinkedIn API 429 Too Many Requests β†’ retry β†’ 429 β†’ retry β†’ 429 β†’ giving up. Started failing this week; code untouched for a month; also new: n8n weekly-report workflow went live for three more accounts. The retry logic is working as designed. The 429s are the API correctly refusing because aggregate demand changed β€” several workflows hammering the same per-app rate limit simultaneously (same trigger time, same token). Retrying harder inside one request cannot fix contention between requests. Real fix: stagger schedules, serialise MCP calls through a queue with a global rate cap in the MCP server itself (the right place β€” it owns the API relationship), cache/share responses where accounts overlap, and only then request a higher quota tier. Consultant-grade observation: this failure was predictable at onboarding β€” 'what happens to shared rate limits when we add accounts?' belongs in the risk register.
🚫When not to reach for it
The habit is universal, but scale honestly β€” a 20-error log kept honestly is worth more than a 200-error log copy-pasted from AI. And the discipline is orthogonal to speed: skilled practitioners often reach the AI in 10 seconds, but with a written hypothesis first.
Quick check
1 question Β· instant feedback
0/1
  1. 429s appeared after you added three more client accounts. Real fix?

Pitfalls & takeaways

Failure modes

  • Paste-and-pray. Error into the model, plausible band-aid returned, workaround merged as fix, real cause never surfaced.
  • Fixing the crash, not the missing error boundary. `return (body?.elements ?? [])` makes the crash disappear and installs T4 (error swallowing) as a 'fix.'
  • Ignoring what the provider literally said. 4xx errors from a provider are specifications being enforced β€” the message names the exact path and expectation.
  • Reading code when environments differ. Works locally, fails in staging β†’ stop reading code, diff the environments (migrations, env vars, secrets, versions, permissions).
  • Retrying harder to fix contention. Some errors are messages about capacity, not defects.
  • The highest-severity bugs don't throw. Silent semantic changes (e.g. slice(0,50) truncating rows before summarisation) require the eval suite to catch.

Durable takeaways

  • Write the hypothesis before consulting the oracle.
  • Provider 4xx errors literally tell you where and what.
  • Works-locally, fails-in-staging β†’ diff environments, not code.
  • The scariest bugs never throw β€” evals catch what errors don't.
Quick check
1 question Β· instant feedback
0/1
  1. 38/40 golden cases pass; 2 numeric-mismatch failures after an AI refactor added `.slice(0, 50)` for perf. Right move?

Do the work

πŸ‹οΈProve you learned it

Start the log today, seeded with your five hypotheses from the trace drill and the gaps you scored. Then fill it with the next twenty real errors from your own work, hypothesis-first, every time. When the gap column is mostly empty, you've stopped needing the oracle and started using an assistant.

0 chars
πŸ“¦Artifact to produce
debugging-log.md β€” one line per error: date Β· error Β· my hypothesis Β· actual cause Β· gap.
Quick check
1 question Β· instant feedback
0/1
  1. The habit being installed by this drill is…