My Go API benchmark once showed an almost 7x throughput improvement after I moved a read‑heavy endpoint from PostgreSQL to a local in‑memory cache. The result looked obvious: PostgreSQL was the bottleneck, the cache removed it, and the API became much faster. Later I realized that this interpretation was too simple. ApacheBench had measured an entire request path consisting of HTTP handling, connection management, pgxpool waiting, SQL execution, decoding, serialization, and response writing. I decided to decompose that path and understand what the 7x result actually meant.
The number was correct, but my explanation was too simple
The original setup was deliberately small. A Go service exposed an endpoint that returned a user by ID. Without caching, every request reached PostgreSQL. With caching enabled, frequently requested objects were returned directly from process memory.
The performance difference was large. ApacheBench showed enough separation between the two implementations that the result could reasonably be summarized as an approximately sevenfold throughput improvement.
At first I treated that as evidence that PostgreSQL itself was the bottleneck.
That sentence now bothers me.
A PostgreSQL‑backed HTTP request is not a SQL query. It is a chain of operations, and the query is only one of them.
The database version of the endpoint effectively followed this path:
HTTP request ↓ router ↓ pgxpool acquisition ↓ PostgreSQL protocol round trip ↓ index lookup and SQL execution ↓ row transfer ↓ Scan into Go values ↓ JSON serialization ↓ HTTP response
The cache hit was much shorter:
HTTP request ↓ router ↓ cache synchronization ↓ map lookup ↓ HTTP response
This was the first thing I had underestimated.
I had not replaced PostgreSQL with a map.
I had removed an entire section of the request pipeline.
In my implementation, the cache could even contain the already serialized representation. In that case a cache hit avoided not only database access, but also scanning values into a struct and running JSON serialization again.
Calling the complete difference PostgreSQL overhead therefore hides too much information.
A more useful simplified model is:
Tdb_request = Thttp + Tpool + Troundtrip + Tsql + Tdecode + Tserialize + Twrite
For a cache hit:
Tcache_request = Thttp + Tlock + Tlookup + Twrite
The benchmark compared these two sums.
It never isolated one SQL operation and one memory lookup.
That sounds obvious once written down. It was much less obvious while looking at a single requests‑per‑second number.
ApacheBench is part of the experiment
The second problem was my mental model of the load generator.
I had implicitly treated ApacheBench as a stopwatch sitting outside the system.
It is not.
A benchmark client creates a particular workload, and the workload affects the result.
Take these two commands:
ab -n 100000 -c 64 http://127.0.0.1:8080/users/42
and:
ab -k -n 100000 -c 64 http://127.0.0.1:8080/users/42
The second enables persistent HTTP connections.
From the perspective of the application code, nothing changed. The handler is identical. PostgreSQL is identical. The cache is identical.
The transport behavior is not.
Without connection reuse, the benchmark creates much more connection‑management work. Even on localhost, creating and closing sockets is not free. The kernel maintains TCP state, the server accepts connections, file descriptors are used, and additional system calls appear around requests.
This matters particularly when one endpoint becomes extremely cheap.
Imagine that the PostgreSQL path requires several layers of work while the cached path is already close to the minimum cost of serving an HTTP response.
Once the cache path approaches the limit of the HTTP stack, making the cache itself twice as fast does not make the API twice as fast.
Amdahl's law appears in a very practical form.
If only a small fraction of the remaining request time belongs to the cache lookup, optimizing that lookup has very little effect on the complete request.
This also means that connection handling can alter the ratio between the database and cache versions without changing either implementation.
The speedup is not a constant property of the architecture.
It is a property of the architecture under a particular workload.
pgxpool can quietly become the real queue
There is another layer between Go and PostgreSQL that is easy to ignore when looking only at SQL execution time.
The connection pool.
Suppose the benchmark runs with a concurrency of 64 while pgxpool is limited to eight connections.
Sixty‑four requests can be active at the HTTP layer.
Only eight can use PostgreSQL at the same time.
The remaining requests have to wait.
From the outside, this waiting appears as API latency.
PostgreSQL may still execute each query extremely quickly.
This creates a confusing but perfectly valid pair of observations:
SQL execution: fast HTTP request: slow
The missing time can exist before PostgreSQL even sees the query.
pgxpool exposes enough information to detect this:
stat := pool.Stat() log.Printf( "total=%d acquired=%d idle=%d acquire_count=%d acquire_duration=%s", stat.TotalConns(), stat.AcquiredConns(), stat.IdleConns(), stat.AcquireCount(), stat.AcquireDuration(), )
The field I care about most here is AcquireDuration.
If it grows sharply as concurrency increases, requests are spending noticeable time waiting for connections.
At that point a benchmark is partly measuring queueing inside the application.
The obvious response is to increase MaxConns.
That helps only until the bottleneck moves again.
With a very small pool, requests wait inside Go.
With a larger pool, PostgreSQL receives more parallel work.
Increase it far enough and more concurrency can start producing contention rather than useful throughput. CPU scheduling, shared structures, memory pressure and database backend activity become increasingly important.
The queue did not disappear.
It moved.
That idea changed how I interpret connection‑pool tuning.
There is no universally correct pool size that makes PostgreSQL infinitely parallel.
A pool is a concurrency control mechanism.
Too small and the application queues excessively.
Too large and the database may receive more simultaneous work than it can process efficiently.
A benchmark that ignores this can easily attribute pool behavior to PostgreSQL performance.
I should have measured the empty HTTP path first
A nearly empty handler is one of the simplest controls I can add:
func baseline(w http.ResponseWriter, r *http.Request) { w.WriteHeader(http.StatusNoContent) }
It does almost nothing.
That is exactly why it is useful.
It establishes a rough lower bound for the cost of serving requests through the current HTTP stack.
Suppose a cached endpoint approaches the throughput of this empty handler.
At that point, continuing to optimize the map implementation is unlikely to produce a proportional API improvement.
The request has another bottleneck.
It may be socket processing.
It may be response writing.
It may be the Go scheduler.
It may even be the benchmark generator.
Without a baseline, I can keep optimizing a component that has already stopped mattering.
The same principle applies to PostgreSQL.
Instead of measuring only the full HTTP endpoint, I want to think in three layers:
PostgreSQL query only Go → PostgreSQL → Go HTTP → Go → PostgreSQL → Go → HTTP
The first isolates database execution as much as possible.
The second adds the driver, pool, protocol transfer, decoding and runtime.
The third adds the HTTP stack.
If the SQL query is fast but the second layer is much slower, SQL optimization may not be where the largest gain exists.
If the second layer is fast while the complete endpoint is much slower, the database may not be the interesting part at all.
This decomposition is far more useful than knowing that one complete endpoint produced some impressive RPS value.
Closed‑loop load changes when the server becomes slow
There was another property of the benchmark that I initially ignored.
With a fixed concurrency, ApacheBench behaves essentially like a closed‑loop load generator.
A certain number of requests are active.
When one finishes, another can take its place.
This creates an approximate relationship:
throughput ≈ concurrency / response_time
If requests become slower, the achieved request rate naturally decreases because existing requests occupy concurrency slots for longer.
That is perfectly valid for one type of benchmark.
It does not behave exactly like every production workload.
A real public API can receive requests at an externally determined rate.
If 20,000 requests per second arrive and the database suddenly becomes slower, clients do not necessarily reduce their arrival rate to protect the server.
Requests continue to arrive.
Queues grow.
Tail latency grows.
Timeouts begin.
Retries can potentially create even more load.
This difference matters particularly close to saturation.
A closed‑loop benchmark partly reduces offered load as the server slows down.
A constant‑rate workload does not necessarily do that.
That is why maximum requests per second should not be the only result I care about.
I also want to know what happens while approaching the saturation point.
p99 tells a different story from average throughput
RPS is attractive because it creates a ranking.
Implementation A produces more requests per second than implementation B.
A wins.
Real systems are less cooperative.
Two implementations can reach similar throughput while having completely different latency distributions.
Consider two endpoints.
Both can sustain approximately the same request rate.
The first has a narrow distribution: p50, p95 and p99 remain relatively close.
The second has a reasonable median but a rapidly growing p99.
They do not provide the same user experience.
The second system may already be building queues even though its headline throughput still looks healthy.
The PostgreSQL path contains several places where variability can enter:
pool acquisition database scheduling query execution protocol transfer row decoding Go scheduling serialization response writing
Most requests may pass through all of them quickly.
A few do not.
Those few define the tail.
The cache path removes several of these stages entirely.
That can make its biggest practical advantage predictability rather than maximum throughput.
For a latency‑sensitive API, reducing p99 from a problematic region can be more valuable than increasing maximum RPS by another impressive percentage.
This is why I now consider a performance comparison incomplete if it contains only average latency and throughput.
At minimum I want one coherent set of measurements:
throughput together with p50, p95 and p99
CPU usage for the Go process and PostgreSQL
pgxpool acquisition time
active connection count
multiple runs of the same scenario
identical connection behavior between compared implementations
a clear description of concurrency or offered request rate
The exact numbers matter.
The shape of the system under increasing load matters more.
The 7x result was not fake
After all this decomposition, it would be easy to conclude that the original benchmark was useless.
I do not think it was.
The result described something real.
Under that workload, on that system, with that concurrency, connection behavior, pool configuration and request distribution, the cached implementation completed dramatically more requests.
That observation still exists.
The mistake was treating it as a universal constant.
An API does not have one performance number.
Its behavior depends on the workload.
The same cache can look completely different under these two request distributions:
one hot ID requested repeatedly
and:
millions of uniformly distributed IDs
The first case can produce an almost perfect hit rate.
The second can destroy it.
The same PostgreSQL query behaves differently when the database runs on localhost and when every query requires a network round trip.
The same local cache behaves differently when it contains ten thousand entries and when it contains several million objects that increase GC pressure.
The same benchmark behaves differently with a warm process and immediately after deployment.
A read‑only test tells me almost nothing about invalidation cost.
A 200-byte JSON response tells me very little about a multi‑megabyte response.
This means benchmark configuration is not implementation detail.
It is part of the result.
The speedup is a function, not a number
The simplest representation of the original experiment is:
speedup = database_path_time / cache_path_time
That looks like one constant.
A more realistic view is:
speedup = f( concurrency, pool_size, HTTP_connection_reuse, cache_hit_rate, request_distribution, payload_size, database_topology, database_state, cache_size, machine_load )
Now the 7x result makes much more sense.
It is one point in a multidimensional space.
Move to another point and the ratio changes.
This is not a weakness of benchmarking.
It is exactly what benchmarking is supposed to reveal.
The dangerous part begins when one point is presented as if the rest of the space did not exist.
For example, suppose caching removes almost all database work and produces a huge improvement at low concurrency.
As concurrency increases, the cached endpoint may eventually hit the HTTP or CPU ceiling.
Its throughput curve flattens.
The database path may flatten earlier because pgxpool or PostgreSQL saturates first.
The ratio between them therefore changes with concurrency.
There is no reason to expect a constant multiplier.
The architecture contains multiple saturation points.
Caching changes which one is reached first.
What I would benchmark now
If I were evaluating the same optimization again, I would begin with the workload rather than the command.
I would define what I actually want to know.
For example:
How much does the cache reduce request latency at a fixed offered load?
At what concurrency does pgxpool acquisition become significant?
Where does PostgreSQL throughput stop scaling?
How close is the cached endpoint to the empty HTTP baseline?
What happens to p99 near saturation?
How does the result change when the hit rate falls?
What happens when the key distribution becomes Zipf‑like instead of one hot object?
How does a remote PostgreSQL instance change the result?
Those are different experiments.
Trying to answer all of them with one ApacheBench invocation produces a precise number with an ambiguous meaning.
This was probably the most useful thing I learned from revisiting the original test.
A benchmark command is not an experiment.
It is one component of an experiment.
The question comes first.
Conclusion
My original benchmark showed an almost sevenfold difference between a PostgreSQL‑backed Go API path and a local cache.
I still believe the optimization was useful.
What changed is my understanding of what the number represented.
ApacheBench was not measuring PostgreSQL.
It was not measuring the cache either.
It was measuring the complete behavior of the system under one particular workload.
The 7x result included HTTP handling, transport behavior, pool acquisition, PostgreSQL interaction, decoding, serialization, response writing and the behavior of the load generator itself.
Caching removed several of those costs simultaneously.
Then the bottleneck moved somewhere else.
That is the part I find more interesting now.
Performance work is often described as making a component faster.
In practice, it is usually a process of moving the queue.
Remove database latency and HTTP starts to matter.
Fix HTTP overhead and CPU becomes visible.
Increase the pool and PostgreSQL becomes the queue.
Optimize PostgreSQL and serialization may appear.
At some point the load generator itself can become suspicious.
I used to look at a result like 7x and ask whether it was large enough to justify the optimization.
Now I ask different questions.
Which stages disappeared from the request?
Where did the next queue form?
What happens to p99 as load increases?
How close am I to the baseline of the surrounding system?
And which assumption in the benchmark is responsible for the number I am looking at?
The cache did not become less useful when I stopped treating 7x as a universal property.
The result became more useful once I understood what was inside it.
