deniz.in

Markets

Weather

Loading weather

· via dev.to (home feed)

Codex model defaults shifted three times in 15 days, ending with GPT-5.5's retirement

Between September 29 and October 14, Codex changed how its default model is chosen and retired GPT-5.5 for ChatGPT sign-ins. A dev.to walkthrough argues unpinned model defaults quietly break agent evaluations.

Codex model defaults shifted three times in 15 days, ending with GPT-5.5's retirement

Between September 29 and October 14, OpenAI's Codex coding agent absorbed three model-related changes, none of which would appear in a user's repository diff. That sequence — a new default model, a change to where defaults come from, and the retirement of GPT-5.5 — is laid out in a dev.to analysis that argues a model you never chose behaves like a dependency you never pinned.

Three changes in fifteen days

According to the post, which draws on OpenAI's release notes and documentation, a pull request merged on September 29 gave GPT-6.1 Sol the highest priority in the model catalog bundled with Codex CLI 0.159.1. That made it the model a session receives when no model is requested. On October 7, version 0.161.0 listed GPT-6.1 Sol as the default in both the bundled catalog and the Amazon Bedrock catalog.

The October 8 release, CLI 0.162.0, is what the post calls the quiet change: fresh terminal threads now take their model and reasoning-summary defaults from the Codex app server they connect to — a local or remote daemon — unless the launch explicitly overrides them. The same release also stopped marking entries as "(default)" in the TUI model picker.

On October 14 at noon PDT, GPT-5.5 left Codex for everyone signed in with ChatGPT. API-key sessions were unaffected. Plus, Pro, Business, Enterprise and Edu subscribers were pointed to GPT-6.1 Sol, while Free and Go users were directed to GPT-6.1 Luna in the desktop app, in both cases only when available.

Open about it, but easy to miss

The post concedes none of this was concealed: every change sits in a release note or changelog, and the retirement came with an exact timestamp and a named successor. OpenAI also claims GPT-6.1 Sol nearly matches GPT-6 Astra at roughly a fifth of Astra's token price — figures reported by Unite.AI and The Batch, but originating from OpenAI's own numbers. The author's point is narrower: even a welcome upgrade moves your benchmark results, and your evaluations should be able to say so.

What this does to evaluations

Most agent evaluation is not a formal harness, the post notes — it is remarks like "Codex is worse at migrations than Claude Code" or "this task took 40 minutes last week and 12 today." Each is a claim about a model, and if the model was whatever the default happened to be, the claim has no fixed subject. A regression might trace to a prompt change, or to a catalog file nobody on the team chose.

Reasoning effort compounds the problem: the October 8 release treats it like the model, with fresh threads inheriting the server's default. So does sign-in method, since the retirement split ChatGPT and API-key sessions — one gpt-5.5 setting could mean two different outcomes depending on how the run authenticated. The post's minimum record for any agent result is four fields: the model, the reasoning effort, the account type, and the commit it ran against.

The pattern is not unique to Codex. Claude Code's model documentation describes the opus, sonnet and haiku aliases as pointing to recommended versions that update over time, while full model names pin a specific release. Moving targets suit a chat product, the post argues, but not anything you measure, compare or promise to a teammate.

Pinning, and its price

For daily use, the post recommends a Codex profile: a file in ~/.codex/ setting model and model_reasoning_effort, launched with codex --profile. Explicit profile launches keep the resolved configuration, so the pin holds even for fresh TUI threads. For scripted comparisons, codex exec accepts -m for the model and -c for arbitrary config keys, letting a single script carry both the pin and the logged result.

Pinning still breaks sometimes, as October 14 demonstrated: anyone pinned to gpt-5.5 with a ChatGPT sign-in had to update workspace defaults, saved model settings, managed configurations, custom agents, scheduled tasks and scripts. That is the useful kind of failure, the post argues — it lands in a diff, gets rerun, and leaves comparable rows on either side of the change.

Why it matters

Model identity is an experimental variable, not an implementation detail. When a vendor can retarget a default with a catalog-file edit and retire a pinned model on a schedule, any before-and-after comparison made against a moving default becomes uninterpretable. The practical fix is to treat models like library versions: choose one per task, pin it, and record the model, effort, account type and commit alongside every result — so that when the numbers move, you know whether your code did.

  • #openai
  • #codex
  • #ai-agents
  • #llm-evals
  • #developer-tools

Related posts