Debugging: What Actually Works When I'm Stuck

5 min read
DebuggingJavaBackendEngineering

I used to treat debugging as a heroic guessing game. Something breaks in production, my heart sinks, and my first instinct is to stare at the code that I think is responsible and start changing things until it stops failing. It almost never worked on the first try.

A handful of painful production incidents taught me that debugging is not intuition — it's a process. Reproduce, isolate, verify. Repeat. These are the habits that actually save me.

The bug is never where I think it is

The first lesson is the most humbling: every time I was certain a bug lived in a specific place, I was wrong. Not sometimes — basically always. The worst incident I remember started with a client reporting "orders are not going through." My mind went straight to the payment service. After hours of staring at it, the actual culprit turned out to be a timezone mismatch in the order-creation timestamp that made a database constraint silently reject inserts.

That experience rewired me. The right reaction to a bug is not "where is it?" but "what do I actually know?". If I'm confident about the cause without any evidence, that confidence is the problem.

Reproduce it first, always

You cannot debug what you cannot see. If I can't make the bug happen on demand, I'm debugging blind — and blind debugging produces random fixes.

So before anything else I ask: how do I make this happen again? Sometimes that means writing a test that triggers the exact input. Sometimes it means checking whether the input in production differs from what I assumed it was. In the timezone incident above, the "aha" came from logging the actual Instant being written, not from reading the code.

Rule: no reproduction, no fix. If I can't reproduce it, my job is to gather more data, not to guess harder.

Read the stack trace like a detective

Stack traces used to intimidate me — then I learned to read them in the right order.

Caused by: org.postgresql.util.PSQLException: ERROR: duplicate key value violates unique constraint "orders_pk"
  at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(...)
  ...
  at com.myapp.service.OrderService.createOrder(OrderService.java:84)

Three things I now look for:

  1. The top line — what kind of exception is this? The type alone (PSQLException, NullPointerException, ConstraintViolationException) already narrows the search.
  2. Caused by: chains — the real cause is usually at the end of the chain, not the start. The first exception is often just the symptom.
  3. My frames near the bottom — framework frames (org.postgresql, org.springframework) are noise. The frames from my package are where the bug actually lives. Follow the stack down to my own code.

Change one thing at a time

The fastest way to never find a bug is to change three things simultaneously. If the fix works, I have no idea which one did it. If it doesn't, I'm more confused than before.

So I change one variable at a time and re-test after each. When the bug is "it used to work," I bisect instead:

# Find the commit that introduced the bug — binary search across history
git bisect start
git bisect bad            # current state is broken
git bisect good v1.2.0    # this tag was fine
# git checks out the middle commit; test, then mark bad/good
# repeat until only one commit remains — that's your culprit

git bisect has found more bugs for me than any amount of staring. Logging is not an apology.

Logging is a debugging tool, not an apology

Early in my career I treated log statements as clutter to be removed before "shipping clean code." I was wrong. Logging is how a production system talks to you, and you can't attach a debugger to a server you don't control.

I now add logs at the boundaries that matter: what came in (the request), what goes out (the response), and every database call in between. When something breaks, the question becomes "which log line is missing?" — and that missing line is usually the bug.

log.info("creating order userId={} amount={}", userId, amount);
// ... if this line is present but the next one isn't,
// the bug lives between here and there.

This has a beautiful side effect: logs written for debugging double as observability for whatever incident comes next.

The rubber duck on my desk

There is a technique so old it sounds silly, and it has unblocked me more than any tool: explain the problem out loud to someone — or something — that can't answer. A rubber duck. A teammate. A comment box.

The act of articulating what I expect to happen against what actually happens almost always exposes the flaw in my mental model. Half the time I finish the sentence and realize the bug. The other half, I've at least made the question precise enough to search for.

The usual suspects

After enough bugs, I've noticed the same faces showing up again and again:

  • Null where an object was expected — an API returned something unexpected, or a field was never set.
  • Timezone and date math — Instant, LocalDateTime, and UTC vs server-local. The bug doesn't show locally because my laptop is in a different timezone than the server.
  • Configuration drift — a database URL, feature flag, or secret that differs between my local machine and production.
  • Off-by-one — pagination, loop boundaries, and "the last item is missing" complaints.

When I'm stuck, I check this list before I check anything clever.

Closing thought

The best debugging advice I ever got is also the simplest: good code is not code that works — it's code that's easy to debug when it doesn't. Log the boundaries, name things clearly, keep functions small enough to reason about. The bug will still come; it just won't take a week to find.

← Back to blog