GPT-6 Luna wrote no Godot 3 code in 150 answers, but other bugs remain

A 150-answer test finds no Godot 3 code

The familiar complaint that ChatGPT hands out Godot 3 code no longer holds for at least one current model, according to a test published on dev.to by Ziva.sh, the company behind Ziva, an AI agent for Godot. The team asked GPT-6 Luna ten beginner Godot questions under several conditions, with "Godot 4" named in the prompt, with no version mentioned at all, and with web search switched on, repeating each prompt three times per setup. Across the 150 resulting answers, captured on 10 November 2026 and checked against Godot 4.7.2, the model produced zero Godot 3 API errors. Every answer used current Godot 4 forms such as @export, @onready, CharacterBody2D with move_and_slide(), instantiate(), create_tween() and randi_range(). A further 30 runs routed through the Ziva agent on the same model were also free of Godot 3 code.

The background: Godot 4.0 shipped on 1 March 2023 and renamed or removed many Godot 3 classes, methods and keywords, including KinematicBody2D, export var and yield. Forum threads from October 2024, September 2025 and September 2026 all describe AI assistants blending the two syntaxes, in one case even after the user explicitly requested Godot 4. Only one of those posts names a specific model, GPT-4o, and as Ziva.sh notes, the new result says nothing about whatever models produced the older complaints.

How the testing worked

The ten prompts came from real forum threads: a 2D platformer player, a third-person controller, enemy spawning on a timer, saving a high score, a Start button, fading out a coin, tile placement from code, a health bar, random numbers and a timed trap. None named an API that changed between versions. The model was reached through OpenAI's Codex endpoint with a ChatGPT account login rather than the ChatGPT app, which the authors say can behave differently. An AI reader and a scanning script checked every code block against a checklist of Godot 3 forms built from the official upgrade guide and Godot 4.7.2's own rename map, every script was passed through Godot's --check-only mode, and runtime errors were confirmed by running the code headless.

Two details stand out. Without a version in the prompt and without search, 18 of 30 answers stated they targeted Godot 4 and the rest named no version; none targeted Godot 3. With search enabled, the model on its own restricted every query to Godot's stable documentation site, and although outdated Godot 3 docs surfaced in the results of 23 of 60 search-enabled answers, none of those answers cited them or emitted Godot 3 code.

Ziva.sh is careful about the limits of a zero: if the answers were independent, 0 in 150 would put the true error rate below roughly 2% at 95% confidence, and because answers to the same prompt correlate, the real uncertainty is larger. It is also worth noting that the tester sells an AI coding agent for Godot, a connection the post discloses.

What broke instead

The failure mode has shifted rather than disappeared. Seven of the 150 plain-chat answers contained code that does not work in Godot 4.7.2, four that fail to parse and three that parse cleanly and break at runtime, plus two more with incorrect setup instructions. Three patterns dominated:

1. Setting monitoring to false on an Area2D from inside its own body_entered handler passes the parser but is rejected at runtime, because Godot blocks that property change during signal emission. The fix is set_deferred, and the buggy line appeared in five places across the test.

2. A health bar script clamped an exported variable inside a setter that also wrote to a ProgressBar child, which fails when the Inspector applies the saved value before that child exists. An is_node_ready() guard plus an update in _ready() resolves it.

3. Several tile-placement answers used an untyped loop variable over an array of Vector2i constants, which Godot cannot infer and rejects at parse time; annotating the loop variable fixes it.

Notably, a --check-only pass catches the third class of bug but not the first two, which only surface when the game actually runs.

Why it matters

For AI-assisted game development, the actionable picture has changed. Version confusion appears to be largely a problem of older models, and naming your engine version remains cheap insurance. The residual risk is subtler: code that clears a syntax check and fails at runtime, particularly signal-time property changes and node-ordering issues. The workflow the test supports is simple: use a current model, state your Godot version, and run the generated code in Godot rather than trusting that it parses.