AI-maintained trading bot logged 18,000 tracebacks while its scheduler stayed green

A postmortem published on dev.to describes a failure mode worth knowing about for anyone running unattended automation: a scheduled task that reported success on every run for more than three weeks while the program inside it was crashing on nearly every cycle. The author runs a one-person company where Claude Code writes and maintains the code, including trading bots on Windows Task Scheduler that poll a broker API every five minutes, around the clock, with nobody watching in real time.

The failure the scheduler never saw

According to the post, one bot's scheduled task returned exit code 0 on every five-minute run for over three weeks, and the scheduler's own logs showed nothing but success. Inside the task, the Python process was failing on almost every cycle because of an unhandled authentication error returned by the broker's API. An outer process wrapper caught those crashes and then reported that the wrapper had run and exited cleanly — accurate as a description of the wrapper, useless as a description of the bot. Over the three-week window the bot made more than 8,000 attempts and accumulated over 18,000 tracebacks, because several code paths kept retrying and failing. Not a single real trade was recorded during that entire period. The author only found the problem by opening the raw log file directly; nothing in the monitoring setup prompted the check.

A quieter failure: confident conclusions from stale data

The post describes a second, milder incident with a different shape. A bot equipped with a circuit breaker — a mechanism that force-closes all positions once cumulative paper losses cross a threshold — triggered for real. The close was confirmed directly against the broker's own API, but it was never written to the trade history file that the reporting system reads. The author's daily status report, a script that collects each bot's logs and has an AI model summarize them, therefore kept describing a position as open for days after it had actually closed, and suggested the bot might have crashed. The bot had behaved exactly as designed; the summary was misreading stale data as current state.

Why the standard monitoring categories stayed green

The author walks through why the tools they already knew of would not have caught either problem. LLM observability products, in their experience, trace individual model API calls while a developer is actively building — suited to questions like why a single prompt cost too much — rather than watching a background job nobody is looking at. Dead-man's-switch services, which alert when a job stops checking in, would also have shown green for all three weeks, because the wrapper process never stopped running or pinging.

The common thread, per the post, is that both failures were invisible to anything that checks only whether a process exited 0 or whether some plain string landed in a log. Neither case involved dishonest code: the wrapper genuinely did not crash, and the position genuinely had been open at some point. The gap was between the shell of the job appearing healthy and the job actually accomplishing what it exists to do.

The author's takeaway

The post frames the lesson as a distinction between an agent that performs well while you are watching it and one whose failures you will actually notice once you stop watching — related to earlier incidents the author describes, in which an agent with write access broke things, but with a different failure shape. Uptime monitoring answers whether something is alive, they argue, while the more useful question for a background agent is whether it is still doing its structured job, which is a stricter claim than returning an HTTP 200. As a next step, the author is building a small monitoring layer aimed at solo developers and small teams running unattended AI agents: schedule-aware, and treating "the process exited 0" and "the agent did its job" as two different claims, flagging the gap between them instead of waiting for the process to die outright. They are also inviting readers who run unattended scrapers, bots or pipelines to share their own "everything said green and it wasn't" stories, to test whether the pattern generalizes beyond these two data points.

Why it matters

As AI agents take on writing and operating background automation, health checks inherited from traditional operations measure the wrong thing. An exit code says a wrapper terminated; a heartbeat says a process is alive; neither says the work happened. For jobs whose output is structured records rather than a simple response, the check that matters is at the outcome level: whether the expected trades, rows or files actually appeared, and whether the numbers move the way the schedule implies they should. This postmortem is a compact demonstration that "the job ran" and "the job did its job" can diverge for weeks without a single red flag — and that in an AI-operated stack, periodically reading raw logs by hand remains the check of last resort.