· via Hacker News – Front Page (native)
Onehouse Lakegres adds vector search directly on Hudi and Iceberg tables via Postgres
Onehouse's Lakegres now runs similarity search on existing Hudi and Iceberg tables through a Postgres endpoint, storing the vector index in the table's own metadata instead of a separate vector database.

What Onehouse announced
Onehouse has added vector search to Lakegres, its query layer for open lakehouse tables. The feature lets teams run similarity searches over data already stored in Hudi or Iceberg tables, using an endpoint that speaks the Postgres wire protocol. Any Postgres client or agent framework can therefore issue vector queries in SQL alongside the analytical scans, indexed point lookups and text search that the same endpoint already serves.
The notable design choice is where the index lives. Rather than syncing vectors into a dedicated vector database, Lakegres stores the vector index in the table's own metadata, in the same object-storage bucket as the data files. According to Onehouse, every commit that inserts, updates or deletes rows also updates the index, so it stays consistent with the table without a separate sync job or a second copy of the vectors.
The release follows a companion announcement from Onehouse covering unstructured data processing, which parses documents, images and video and writes their embeddings into open tables. With the new index, agents can query that unstructured content alongside the business records already in the lakehouse.
Why in-memory indexes break at lake scale
Most production vector search today relies on HNSW, a graph index that performs well only when the entire graph fits in memory. Onehouse points to AWS's sizing formula for OpenSearch, 1.1 × (4 × dimensions + 8 × m) bytes of RAM per vector. On Onehouse's benchmark table of 247 million vectors at 1,024 dimensions, with m set to 16, that works out to roughly 1.15 TB of memory for a single table, and around 46 TB at 10 billion vectors.
Lakes are routinely far larger than application databases. Onehouse cites LinkedIn reporting an exabyte across its Hadoop clusters in 2021, Uber's 350-petabyte lake ingesting 6 trillion rows a day, Pinterest's 500-petabyte Iceberg lake and Netflix describing its Iceberg warehouse as exabyte scale. Embedding even a small fraction of such datasets yields billions of vectors. This is why Onehouse, like VAST and turbopuffer, turned to storage-based clustering rather than memory-resident graphs; graph traversal is also a poor match for object storage, since each hop is a small random read.
Clustering and quantization instead
At build time, k-means grouping assigns each vector to a cluster anchored by a centroid, and vectors are encoded relative to that centroid using RaBitQ quantization at 1 to 8 bits per dimension, with 4 bits as the default. The stored codes support L2, cosine and dot-product distance calculations. Clusters are divided into posting segments, with roughly 1,000 to 4,000 vector entries packed into posting blocks of about 512 KB each, which avoids creating billions of individual metadata records.
Because the index is derived metadata, it can be added to an existing Hudi or Iceberg table that has a vector column without rewriting the underlying data files, rebuilt with different settings, or dropped when it is no longer needed. Existing analytical queries keep working, and Onehouse notes there is no need to move off Parquet to formats like Lance or Vortex just to get vector search, although Lakegres can already query those file types as well.
How other vendors place their indexes
Onehouse contrasts its approach with the rest of the market. Snowflake's Cortex Search runs as a service synced from a table, holds the index inside Snowflake and does not support VECTOR columns on Iceberg tables. Databricks' AI Search syncs a Delta table into a serving endpoint, with the index stored in Databricks-managed storage outside the table. AWS keeps vectors in a dedicated S3 Vectors bucket type separate from the Iceberg-based S3 Tables. Google BigQuery limits its IVF and TreeAH vector indexes to standard tables rather than Iceberg, and VAST Data keeps its vector store in its own Element Store outside Hudi or Iceberg. Onehouse's argument is that these vendors advocate open table formats while keeping search indexes in closed formats, forcing customers to rebuild every index when they change engines.
Why it matters
Vector search has become a standard capability in application databases, but the data AI agents most need to search, such as support transcripts, logs, documents, product catalogs and years of event history, sits in lakes that Onehouse estimates are 100 to 1,000 times larger than a typical Postgres instance. Copying that data into a database just to search it means paying to store it twice and operating a pipeline to keep the copy in sync. Storing an open, portable index alongside the table itself removes both costs and avoids the lock-in of vendor-specific index formats.
The design is also being developed in the open: Onehouse says Iceberg support builds on Hudi-backed indexing work in Apache XTable (Incubating), that the design was shaped with engineers from Uber and Walmart for planet-scale tables, and that the Hudi implementation is being upstreamed with XTable in release-candidate testing ahead of OSS releases other engines can integrate. Training-data curation, which uses repeated large-scale similarity searches to select examples and remove near-duplicates, is another workload where cost per search matters as much as latency.
- #vector-search
- #data-lakehouse
- #apache-hudi
- #apache-iceberg
- #postgres