Docker's default networking, not CPU, caused ecommerce checkout timeouts at 900 req/s

How checkout broke under load

A case study published on dev.to details how a containerized ecommerce marketplace watched its checkout p95 latency jump from 280 milliseconds to more than 2.1 seconds whenever traffic spiked toward 900 requests per second. The workload was a PHP monolith that had been containerized roughly a year earlier and ran on a single host with Docker Compose: application containers plus Redis, a worker queue and a search sidecar, all behind a managed load balancer. The site served around 40,000 daily active users and had been stable for six months before flash sales started producing tickets about the site freezing.

According to the dev.to write-up, the team's first response was to add more application containers on the assumption of CPU or memory pressure. That made the situation worse. Postgres query times held between 8 and 14 milliseconds throughout the incident, ruling out the database as well. The actual bottleneck was in the network layer between containers, a place where, as the authors note, default tooling offers almost no visibility.

Three compounding network faults

The audit found three issues, none of them fatal on its own:

• Default bridge overhead: every inter-container hop (app to Redis, app to Postgres, app to search) went through userland proxying on Docker's default bridge, adding about 1.8 milliseconds per hop. With three to four internal calls per request at 900 req/s, the cost compounded quickly.
• Conntrack table exhaustion: nf_conntrack_max was still at the kernel default of 65,536. Short-lived Redis and Postgres connections churned through entries during spikes until the table filled and the kernel silently dropped packets, unnoticed because syslog wasn't being shipped anywhere useful.
• DNS resolution overhead: Docker's embedded DNS at 127.0.0.11 resolved service names on every new connection instead of the app caching results, and under connection churn the lookups began queuing behind one another.

The authors emphasize that none of this shows up with ten test users in staging, but all of it shows up with 900 concurrent real ones.

Why they didn't reach for Kubernetes

The write-up explicitly rules out migrating to Kubernetes, arguing it would not have fixed anything: Kubernetes has its own versions of the same problems, including CNI plugin choice, kube-proxy mode and CoreDNS caching. Swapping orchestrators without addressing the root cause just relocates it. The team also declined to rewrite the application to make fewer internal calls, on the grounds that the call pattern was normal and the network layer should handle it efficiently rather than the other way around.

The fix, in four layers

First, kernel tuning: nf_conntrack_max was raised to 262,144, the established TCP timeout set to 600 seconds, tcp_tw_reuse enabled and somaxconn raised to 4,096, applied through a sysctl configuration file. Conntrack table utilization was also added to monitoring, since a full table fails silently.

Second, internal traffic was moved off the default bridge onto a user-defined bridge network with inter-container communication enabled, a 9,000-byte MTU and its own subnet. Application containers, Redis and the search sidecar joined this network so east-west traffic avoided Docker's default NAT path; the public-facing load balancer stayed on a separate network, leaving the external attack surface unchanged.

Third, a dnsmasq sidecar provided local DNS caching, forwarding upstream to Docker's resolver with a 1,000-entry cache and a 10-second local TTL. That TTL was chosen to absorb connection churn during spikes while still picking up replaced containers quickly — the authors specifically verified that a restarted container resolves correctly within one TTL window.

Fourth, PgBouncer was introduced in front of Postgres in transaction pooling mode, configured for 2,000 maximum client connections and a default pool size of 50. This step was deliberately sequenced last: fewer short-lived connections means fewer conntrack entries and fewer DNS lookups, so pooling pays off most once the network layer itself is sound. The entire fix, the case study stresses, required no application code changes.

Why it matters

This incident is a useful template because the failure mode is invisible to the metrics most teams watch. CPU, memory and database latency all looked healthy while the checkout path degraded, and the instinctive remedy — scaling out containers — actively deepened the problem by generating more short-lived connections to churn through conntrack and DNS. The broader lesson from the dev.to piece: when a containerized app slows down only under load, check conntrack utilization, the bridge driver your traffic actually uses, and DNS resolution behavior before adding capacity. It also pushes back on treating Kubernetes as a cure-all, since equivalent pitfalls exist there under different names. Finally, it exposes a monitoring gap worth closing proactively: kernel-level packet drops from a full conntrack table produce no obvious error, only silence.