Developer traces 4-second RAG queries to pgvector HNSW lock contention and unnormalized embeddings

A RAG memory layer that hit a wall

A developer integrating RAGFlow, an open-source engine that combines retrieval with agent capabilities, has published a firsthand account of a production incident in which vector search became the system's bottleneck. Writing on dev.to, Samuel Oakley reports that the agent memory pipeline stalled at roughly 50 concurrent document insertions per second, query latency climbed from 80 milliseconds to more than four seconds, and recall dropped by about 30 percent. The datastore underneath was PostgreSQL with the pgvector extension.

The diagnosis pointed at two independent causes: contention on the HNSW graph index during heavy writes, and embeddings stored without normalization.

HNSW locks choked concurrent writes

According to the post, pgvector's HNSW index does not handle high-concurrency writes well. During bulk ingestion, INSERT statements began blocking one another, creating a convoy effect in which queued writes stacked up and latency compounded.

Rather than reaching for connection tuning, the team restructured indexing. An IVFFlat index, built with a list count of 100, absorbs inserts without the same blocking behavior; a separately tuned HNSW index (m = 16, ef_construction = 64) serves read queries. The HNSW index is rebuilt asynchronously with a concurrent reindex during low-traffic windows, so maintenance never stalls the write path. The trade-off is that both indexes live on the same table, adding storage and write amplification in exchange for decoupling the read and write paths.

Unnormalized embeddings quietly hurt recall

The second problem was subtler because the system kept working. The pipeline stored raw embeddings from OpenAI's API as-is. Cosine similarity in pgvector assumes unit vectors, and the varying magnitudes meant semantically similar chunks ranked worse than they should have — results kept coming back, just poorer ones.

The remedy was L2 normalization at write time, dividing each embedding vector by its norm before insertion. After the change, recall@10 on the author's test set rose from 0.68 to 0.91.

Hybrid search for exact matches

With latency and recall stabilized, the team tackled a remaining weakness: pure vector search misses exact keyword matches. They implemented hybrid retrieval that runs pgvector similarity alongside PostgreSQL's native full-text search, then fuses the two ranked lists with reciprocal rank fusion using the conventional constant of 60. In A/B testing, the hybrid setup improved answer relevance by 23 percent, per the post.

The measured outcome

After all changes, the post reports:

• Insert throughput: from 50 per second to 180 per second
• Query p95 latency: from 4.2 seconds to 120 milliseconds
• Recall@10: from 0.68 to 0.91

Why it matters

When a RAG system answers badly or slowly, the instinct is often to blame the model. This incident is a useful counterexample: the retrieval layer — index choice, its behavior under concurrency, and basic embedding hygiene — dominated both latency and answer quality. For teams building agent memory on pgvector, the post offers a concrete playbook: decouple write and read indexes, normalize embeddings before they reach the database, and pair vector search with keyword search rather than relying on either alone. Oakley's closing advice is to stress-test the memory layer early, because index strategy and embedding hygiene mattered more to the outcome than model selection.