Polars 2.0 release candidate switches LazyFrame queries to streaming by default

First release candidate lands

The Polars team has shipped the first release candidate of Polars 2.0, with the final version expected within a matter of weeks. In the announcement on the Polars blog, which reached the front page of Hacker News, the maintainers describe the release as a cleanup milestone rather than a feature showcase: the major version bump exists mainly to retire old design decisions and move to more sensible defaults, and the team expects the upgrade to be an uneventful one for most users.

Streaming becomes the default engine

The change with the widest impact is that calling collect() on a LazyFrame now routes queries to the streaming engine by default, with engine="auto" resolving to streaming. According to the announcement, typical users should see large reductions in memory usage along with major performance gains, and the team expects the streaming engine to be around five times faster overall.

The reason this required a major version bump is behavioural: the streaming engine does not guarantee row order by default for certain operations, including joins, group_by and unpivot. Queries that depend on observable ordering can opt back in through the maintain_order setting, while teams that want the previous behaviour can set an engine affinity back to the in-memory engine process-wide, or pass engine="in-memory" to an individual collect call.

Stricter typing and concatenation

The other theme of 2.0 is strictness. The Polars philosophy, as the post explains, is to surface errors as early as possible rather than letting a pipeline run for twenty minutes before failing, and to make implicit handling of data mismatches opt-in because those mismatches can conceal bugs. The team also ties this to the rise of AI-assisted development: agents can call collect_schema() to validate a query's structure and catch schema-level problems without materialising data, giving them faster feedback while they iterate.

Concretely, is_in expressions no longer silently cast mismatched types to a common supertype. The announcement highlights a case where integer user IDs checked against a list of IDs that had become floats would previously coerce the integers to Float64; because integers beyond 2^53 exceed what float64 can represent exactly, a different ID could round down to the flagged value and produce a false match. Polars 2.0 raises InvalidOperationError instead and requires an explicit cast.

Horizontal concatenation is also stricter: frames of different lengths now raise a ShapeError rather than padding the shorter frame with nulls, and padding remains available explicitly via how="horizontal_extend". A set of ambiguous casts has been removed as well. Casting integers to enums or categorical types, and casting strings to dates or datetimes, now fail, replaced by dedicated methods such as .cat.to(), .cat.physical() and .str.to_date(), with the string parsers letting you specify an explicit format.

Errors that point to the fix

To soften migration, Polars 2.0 adds two typed exceptions, AttributeRemovedError and ArgumentRemovedError, whose messages direct users to the replacement API. Calling the removed melt method, for instance, now tells you to use unpivot with index and on instead of id_vars and value_vars, and the message for the removed join_nulls argument notes that it was renamed to nulls_equal. The team points out that most of the removed functionality has been deprecated for a long time, invites feedback from anyone who relied on something that got cut, and has published a full migration guide covering the remaining changes.

Why it matters

Polars has become a widely used dataframe library in the Python data ecosystem, and version 2.0 changes behaviour that existing code may silently depend on: row order after joins and group-bys, implicit type coercion, and null-padding in concatenation. Pipelines that relied on any of these will either error out or, worse, keep running with different results, so testing against the release candidate via pip install polars==2.0rc1 before the final lands is cheap insurance. Beyond the breaking changes, streaming by default brings larger-than-memory workloads within reach of ordinary queries, and the roadmap sketched in the post, including out-of-core streaming support, a new IO plugin design, a faster S3 reader, expanded SQL coverage, a cost-based planner, join reordering and fully async pipelines, indicates where the project is heading next.