How to Debug AI-Generated Code and Verify That a Fix Actually Works

AI assistants write code that looks right. It's formatted well, named sensibly, and often explained with confidence. Then it crashes on the first real input — or worse, it runs without errors and quietly returns the wrong answer. This guide walks through a repeatable way to debug that code and, more importantly, to check that a fix actually works. Everything here works in your own terminal; no special tools required.

Why AI-generated code fails in subtle ways

Language models predict plausible code, not code that has been run. Common failure patterns include assuming a dictionary key or object property already exists, assuming input is always well-formed, and mixing idioms from different libraries or language versions. When you paste the error back and ask for a fix, you often get a patch that makes the error disappear — which is not the same as making the code correct.

Step 1: Capture a minimal, reproducible failure

Never patch code you haven't seen fail. Strip the problem down to one file with fixed sample input and no network, database, or randomness. If you can run one command and see the same error every time, you can also run one command to prove it's gone.

Step 2: The bug — an aggregator that crashes on first contact

Suppose you asked an assistant for a function that counts unique active users per day from event logs. You got this:

def aggregate_active_users(logs):
    activity_by_day = {}
    for record in logs:
        day = record["timestamp"].split("T")[0]
        activity_by_day[day].add(record["user_id"])
    return {day: len(users) for day, users in activity_by_day.items()}


sample_logs = [
    {"timestamp": "2026-10-24T14:32:00Z", "user_id": "u_101"},
    {"timestamp": "2026-10-24T15:10:00Z", "user_id": "u_102"},
    {"timestamp": "2026-10-25T09:05:00Z", "user_id": "u_101"},
]

print(aggregate_active_users(sample_logs))

Running it produces a real traceback and a non-zero exit code:

$ python3 aggregator.py
Traceback (most recent call last):
  File "aggregator.py", line 15, in <module>
    print(aggregate_active_users(sample_logs))
          ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
  File "aggregator.py", line 5, in aggregate_active_users
    activity_by_day[day].add(record["user_id"])
    ~~~~~~~~~~~~~~~^^^^^
KeyError: '2026-10-24'
$ echo $?
1

Read tracebacks from the bottom up. The last line names the exception — KeyError: '2026-10-24' — and the frame above points at line 5. The code calls .add() on activity_by_day[day], but nothing ever put a set there. The function assumes every day already has an entry. That's the root cause, not "something wrong with the timestamps."

Step 3: Beware the false fix (when exit code 0 lies)

A common patch — and one assistants frequently suggest — is to just assign the value instead:

        # "Fix": create the entry if it's missing
        activity_by_day[day] = {record["user_id"]}

Run it again. No traceback, exit code 0:

$ python3 aggregator.py
{'2026-10-24': 1, '2026-10-25': 1}
$ echo $?
0

Looks fixed. It isn't. Two different users (u_101 and u_102) were active on October 24, so the correct count is 2. The patch replaces the set on every record, so only the last user of each day survives. A one-line assertion with the answer you worked out by hand exposes it immediately:

# test_false_fix.py
from aggregator import aggregate_active_users, sample_logs

result = aggregate_active_users(sample_logs)
assert result == {"2026-10-24": 2, "2026-10-25": 1}, result

$ python3 test_false_fix.py
{'2026-10-24': 1, '2026-10-25': 1}
Traceback (most recent call last):
  File "test_false_fix.py", line 4, in <module>
    assert result == {"2026-10-24": 2, "2026-10-25": 1}, result
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: {'2026-10-24': 1, '2026-10-25': 1}

Step 4: The verified fix and unified diff

The real fix is to make sure a set exists before adding to it. collections.defaultdict(set) does exactly that: missing keys get an empty set automatically.

--- aggregator.py (original)+++ aggregator.py (fixed)@@ -1,5 +1,8 @@+from collections import defaultdict++ def aggregate_active_users(logs):-    activity_by_day = {}+    activity_by_day = defaultdict(set)     for record in logs:         day = record["timestamp"].split("T")[0]         activity_by_day[day].add(record["user_id"])

$ python3 aggregator.py
{'2026-10-24': 2, '2026-10-25': 1}
$ echo $?
0

Clean exit and — this time — the counts match what we expected: two users on the 24th, one on the 25th.

Step 5: Edge cases to test before you trust it

One passing input is still a narrow test. Here is the fixed function against three inputs AI-generated code commonly mishandles: empty input, duplicate events from the same user, and a timestamp in a different format.

>>> aggregate_active_users([])
{}
>>> aggregate_active_users([
...     {"timestamp": "2026-10-24T08:00:00Z", "user_id": "u_101"},
...     {"timestamp": "2026-10-24T09:00:00Z", "user_id": "u_101"},
... ])
{'2026-10-24': 1}
>>> aggregate_active_users([{"timestamp": "2026-10-24 08:00:00", "user_id": "u_101"}])
{'2026-10-24 08:00:00': 1}

Empty input and duplicates behave correctly. The third result is a silent bug: a timestamp written with a space instead of T becomes its own "day" called '2026-10-24 08:00:00'. String splitting was never really parsing dates. Parsing the timestamp properly fixes both cases — valid ISO formats are normalized, and genuinely malformed values raise an error instead of corrupting the counts:

from collections import defaultdict
from datetime import datetime


def aggregate_active_users(logs):
    activity_by_day = defaultdict(set)
    for record in logs:
        ts = datetime.fromisoformat(record["timestamp"].replace("Z", "+00:00"))
        activity_by_day[ts.date().isoformat()].add(record["user_id"])
    return {day: len(users) for day, users in activity_by_day.items()}

Now lock the behavior in with assertions you can rerun after every future change:

# test_aggregate.py
from aggregator import aggregate_active_users

sample_logs = [
    {"timestamp": "2026-10-24T14:32:00Z", "user_id": "u_101"},
    {"timestamp": "2026-10-24T15:10:00Z", "user_id": "u_102"},
    {"timestamp": "2026-10-25T09:05:00Z", "user_id": "u_101"},
]

assert aggregate_active_users(sample_logs) == {"2026-10-24": 2, "2026-10-25": 1}
assert aggregate_active_users([]) == {}
assert aggregate_active_users([
    {"timestamp": "2026-10-24T08:00:00Z", "user_id": "u_101"},
    {"timestamp": "2026-10-24T09:00:00Z", "user_id": "u_101"},
]) == {"2026-10-24": 1}

try:
    aggregate_active_users([{"timestamp": "24/10/2026 08:00", "user_id": "u_101"}])
except ValueError as exc:
    print("rejected malformed timestamp:", exc)
else:
    raise AssertionError("malformed timestamp was accepted")

print("all assertions passed")

$ python3 test_aggregate.py
rejected malformed timestamp: Invalid isoformat string: '24/10/2026 08:00'
all assertions passed
$ echo $?
0

Even this doesn't prove the function is correct for every possible input — it proves it's correct for the cases you thought to test. That's why choosing good test cases matters more than seeing a clean run.

A short checklist

• Reproduce the failure with one command and fixed input.
• Read the traceback bottom-up and name the root cause before changing code.
• Write the expected output down before running the fix.
• Treat exit code 0 as "didn't crash," not "correct."
• Test empty, duplicate, and malformed inputs.
• Review the diff so you know exactly what changed.

Speeding up the loop

You can do all of this by hand. If you'd rather not shuttle code between a chat window and a terminal, tools that execute code can help. For example, APReason's AI Code Debugger runs supported Python, JavaScript, and TypeScript in a sandbox, checks that the original fails and the fix runs cleanly, and shows the diff. That's execution-based evidence, not a guarantee — you still need assertions like the ones above to confirm the output is right.

More guides are on the APReason blog, or head back to the homepage.