Agent frameworks found executing tool calls cut off mid-argument by token limits

Half-written tool calls get executed anyway

When a large language model hits its output token limit while writing a tool call, the arguments it emits are cut off mid-stream. A post on dev.to, whose author identifies as an AI agent that ran the experiments and filed the resulting bug reports itself, documents that some agent frameworks do not discard that fragment. They execute it anyway, with partial arguments or none, and record the run as a success. Two open-source projects are named, Langroid and Heym, and both have merged fixes.

Two routes to the same bug

According to the post, Langroid (issue #1291) hit the problem through a lenient JSON parser. A model asked to refund 1000 was cut off right after the digits 10, leaving invalid JSON behind. A library called json_repair closed the fragment into a valid object, so the tool ran with an amount of 10.0 and reported a $10 refund. Streaming and non-streaming paths behaved identically.

The same setup reportedly showed a variant closer to production. Early in a conversation, with a 512-token output budget, an email tool call went out whole. Later, after the history grew and the framework lowered that turn's budget to 102 tokens, the model stopped in the middle of the email body — finish_reason was length — and the message was still sent, ending on the fragment "If i". The framework had a fallback that treats an unparseable tool call as plain text, but the repair step meant the fallback never ran.

Heym (issue #902) reached the same outcome differently. Its tool loop caught JSONDecodeError, substituted an empty arguments object, and ran the tool regardless. For tools with required parameters this fails loudly but misleadingly, with a missing-arguments error that implies the model sent nothing. For tools whose parameters are all optional it is worse: the post describes a truncated close_tickets call that ran with defaults — an empty customer ID and a zero-day cutoff — closing tickets for every customer, logging success with empty arguments, and prompting the agent to reply "Done". As the post points out, when filters are optional a truncated call can act more broadly, not less.

The signal that goes unread

Each provider reports a limit stop under a different name, the post explains: finish_reason length in Chat Completions, stop_reason max_tokens at Anthropic, finishReason MAX_TOKENS at Gemini, and status incomplete with a max_output_tokens reason in the OpenAI Responses API. Frameworks supporting several providers often normalize these into one field, and the tool loop then never reads it.

There is a second trap. On Chat Completions, truncated arguments are at least invalid JSON, so a parse failure gives a signal. On the Anthropic, Gemini and Responses transports, arguments can arrive as an already valid object, with only the stop reason revealing the cut. A fix keyed off JSONDecodeError alone misses those cases.

The fix, and the traps inside it

The rule the post proposes is simple: if a turn ended because of the output limit, run none of its tool calls. Answer each call with an error telling the model to resend it shorter, and cap retries so an overrunning model cannot loop. Langroid's maintainer wrote and merged their own fix in #1292, while the post's author wrote Heym's fix in #903, which was also merged.

Even the refusal path proved to be its own code path. Heym's automated code review flagged two problems in the first fix, later resolved in #911. First, storing a refused call's raw arguments as a single string bypassed key-based redaction, letting secrets inside complete calls reach traces and persisted records; the follow-up parses arguments when they parse and otherwise records only their length. Second, after the initial fix a malformed call to a human-approval tool simply errored, allowing a valid sibling call in the same batch to run without approval; the follow-up defers every sibling in that batch instead.

Testing without a model

No API key is needed, according to the post. Run a small localhost server speaking the provider's protocol, script one complete and one truncated tool-call response, and assert three things: the complete call reaches the tool with the right arguments, the truncated one never reaches the tool handler, and the model receives an error result. Making the fake server honor max_tokens also reproduces the late-conversation case where budgets shrink as history grows.

Why it matters

Truncated output is a routine condition, not an edge case, and it becomes more likely as conversation history eats into per-turn budgets. The failure is quiet: tools run with fewer or broader arguments than intended and report success, which production traces will not flag on their own. Anyone building or running an agent should check whether the code path that executes tools ever reads the stop reason, treat tool arguments as an action rather than text to repair, and ask what the default call of an all-optional tool actually does.