deniz.in

Markets

Weather

Loading weather

· via dev.to (home feed)

PostgreSQL 19 enables data checksums online, catching silent storage corruption

PostgreSQL 19 can activate data checksums on a live cluster with pg_enable_data_checksums(), turning silent storage corruption into detectable errors without a shutdown.

PostgreSQL 19 enables data checksums online, catching silent storage corruption

What is new

PostgreSQL 19 adds the ability to turn on data checksums while the cluster keeps serving traffic. According to a hands-on post by Franck Pachot on dev.to, a new function, pg_enable_data_checksums(), activates checksums in the background and moves the cluster to a fully verified state without a shutdown, potentially with throttling to reduce the impact on the running workload.

Why checksums matter

Without checksums, PostgreSQL performs only basic sanity checks on each page header when reading from disk. There is no cryptographic verification of the page contents themselves, so corruption introduced below the database, in the storage or I/O layer, can pass unnoticed and quietly produce wrong query results.

To show the risk, Pachot initialized a cluster with initdb's --no-data-checksums option, using a PostgreSQL 19 beta 3 container image, created a table holding the text 'Hello World!', and flushed the data out of shared buffers. He then edited the underlying file directly with sed. When PostgreSQL read the table again, it returned the tampered value with no warning at all.

As the article notes, checksums have been the default for newly initialized clusters since PostgreSQL 18, verified on every read from disk and recalculated on every write. Clusters upgraded from earlier versions, however, usually still run without them, and the traditional remedy, the pg_checksums utility, requires shutting the cluster down for a scan that can take considerable time on large installations.

How online activation works

The new function requires superuser access and must be run on the primary; the resulting state changes replicate to standbys through WAL. Progress is visible in the pg_stat_progress_data_checksums view, which reports the current phase along with counts of databases, relations and blocks completed. The data_checksums setting moves from off through an intermediate inprogress-on state to on once every block has been processed. A companion function, pg_disable_data_checksums(), reverses the setting under the same restrictions.

What changes once checksums are on

Repeating the file-tampering experiment on a protected cluster produced a hard invalid-page error naming the affected relation and block, instead of corrupted output. That error is the feature working as intended: it converts an invisible problem into a visible one, letting the operator fail over to a standby or restore from a backup before bad data spreads further.

Backups gain protection too. In the demo, pg_basebackup verified checksums while copying and aborted after reporting a mismatch. For backup tools that do not verify, Pachot recommends validating the restore with pg_checksums --check.

Caveats to plan around

The post lists several operational sharp edges:

  • The enable operation consumes two background-worker slots, so max_worker_processes needs headroom.
  • It waits for open transactions and temporary tables in every database, so a single long session can delay it indefinitely.
  • Pages get checksummed during the run, but reads are only verified after the final transition to on.
  • A crash or restart while in the inprogress-on state means starting the whole process over from scratch.
  • Standbys may need forced restart points that block WAL replay, creating replication lag that can in turn stall the primary; reducing max_wal_size beforehand can mitigate this.

Why it matters

Silent data corruption is one of the worst failure modes a database can have, because backups, replicas and applications all faithfully propagate whatever the storage layer delivered. Checksums turn that scenario into an explicit error. Until now, enabling them on an existing cluster meant a maintenance window proportional to the size of the data, which is likely why many long-running installations never did it. Online activation removes that barrier for most setups, though as the caveats show, it is not a zero-impact operation and deserves the same scheduling and monitoring care as any other maintenance task.

  • #postgresql
  • #databases
  • #data-integrity
  • #open-source
  • #reliability

Related posts