I have become suspicious of optimizations that look spectacular in microbenchmarks.
The pattern is easy to recognize.
You benchmark two implementations. One is two, three, maybe five times faster. The result looks convincing enough to justify changing the production code.
Then you put the faster implementation back into the actual application.
Almost nothing happens.
I wanted to reproduce this effect with something much smaller than a database, so I tested three ways of building a simple in‑memory cache in Go:
a regular map protected by sync.RWMutex
a map split into 64 independently locked shards
sync.Map
The isolated benchmark produced a very clear winner.
sync.Map reached about 29.4 million reads per second.
The simple RWMutex implementation managed about 10.6 million.
That is roughly a 2.8x difference.
Then I put exactly the same cache behind a small HTTP endpoint.
The advantage almost disappeared.
That result turned out to be much more interesting than the microbenchmark itself.
Three deliberately boring caches
I did not try to build a production‑ready cache.
There was no TTL, LRU, eviction, refresh logic, database fallback or distributed invalidation.
I wanted to isolate one question:
How much does the synchronization strategy matter when many goroutines read from the same in‑memory data structure?
The first implementation used a normal Go map and one RWMutex.
type Cache struct { mu sync.RWMutex m map[uint64]uint64 } func (c *Cache) Get(k uint64) (uint64, bool) { c.mu.RLock() v, ok := c.m[k] c.mu.RUnlock() return v, ok } func (c *Cache) Set(k, v uint64) { c.mu.Lock() c.m[k] = v c.mu.Unlock() }
This is probably the first implementation I would write for a small service.
It is easy to understand.
It is easy to debug.
And until contention becomes significant, it may be completely sufficient.
The second implementation used 64 shards.
type shard struct { mu sync.RWMutex m map[uint64]uint64 } type ShardedCache struct { shards []shard mask uint64 } func (c *ShardedCache) Get(k uint64) (uint64, bool) { s := c.shardFor(k) s.mu.RLock() v, ok := s.m[k] s.mu.RUnlock() return v, ok }
Instead of every key competing for one lock, different keys can land in different shards.
The third implementation used sync.Map.
type SyncCache struct { m sync.Map } func (c *SyncCache) Get(k uint64) (uint64, bool) { v, ok := c.m.Load(k) if !ok { return 0, false } return v.(uint64), true } func (c *SyncCache) Set(k, v uint64) { c.m.Store(k, v) }
All caches were populated before the benchmark.
The key space contained 65,536 entries.
The workload was intentionally read‑heavy because I first wanted to see how much the synchronization strategy alone could change lookup throughput.
The microbenchmark
For the first test I removed almost everything except cache access.
No HTTP.
No JSON.
No TCP.
No database.
No request parsing.
Multiple goroutines repeatedly selected keys and called Get.
The benchmark was run several times, and I compared the median results from the same environment.
The numbers were approximately:
RWMutex map: 10.55 million operations per second
64-shard map: 21.04 million operations per second
sync.Map: 29.37 million operations per second
The difference was large enough that it did not need creative interpretation.
The sharded cache was roughly twice as fast as the single‑lock version.
sync.Map was roughly 2.8x faster than RWMutex.
If this were the only benchmark I had run, the conclusion would have been tempting:
Replace the mutex‑protected map with sync.Map and get a huge performance improvement.
But that conclusion contains a hidden assumption.
It assumes that cache lookup is a large part of the work performed by the actual application.
The benchmark never tested that assumption.
What the benchmark actually measured
The isolated test measured something close to:
operation = cache lookup
That is useful.
It tells me something about the cache implementation.
But an HTTP request looks more like this:
request = read HTTP request + route request + parse parameters + convert key + cache lookup + build response + encode response + write response + network stack + scheduler overhead
The cache can become dramatically faster while the rest of this path remains unchanged.
That is exactly what happened.
Putting the cache behind HTTP
For the second experiment I created a small net/http server.
Each request performed roughly the following operations:
read a key from the request
convert it to uint64
call the cache
build a tiny response
return the response over HTTP
I intentionally kept PostgreSQL out of this experiment.
Adding a database would have made the result more realistic for many APIs, but it would also have introduced another large variable.
I wanted to see whether HTTP processing alone was enough to hide a large cache‑level improvement.
The answer was yes.
The load test used:
persistent HTTP connections
128 concurrent clients
40,000 requests per run
the same 65,536-key working set
The tests were performed using Go 1.23.2 in an environment with five available CPUs.
The server and load generator were kept separate so the client benchmark would not execute inside the same Go process as the server.
That distinction matters more than it initially seems.
If the load generator and server live inside one process, both compete for the same scheduler, CPU budget and garbage collector. At that point I may accidentally benchmark my test harness almost as much as the server.
And then the 2.8x advantage disappeared
At the HTTP level, all three implementations ended up in approximately the same range:
15,000 to 16,000 requests per second.
There was run‑to‑run variation.
The ordering between implementations was not stable enough for me to make a serious claim that one cache was consistently faster at the HTTP level.
And that is the important result.
The microbenchmark said:
RWMutex: ~10.6M ops/s sync.Map: ~29.4M ops/s
A huge difference.
The HTTP benchmark said, effectively:
all implementations: ~15K–16K req/s
A small difference.
Changing the cache implementation did not turn a 15K req/s server into a 30K, 40K or 45K req/s server.
The 2.8x component improvement was almost invisible at the application boundary.
This is not a contradiction
The result looks strange only if cache throughput and API throughput are treated as the same thing.
They are not.
Imagine that one request takes 10 microseconds.
Suppose only one microsecond is spent inside the cache.
The remaining nine microseconds are spent somewhere else.
Now make the cache lookup three times faster.
The cache portion falls from approximately 1 microsecond to 0.33 microseconds.
The total request time changes from:
10 us
to roughly:
9.33 us
The cache improved by 3x.
The request improved by only about 7 percent.
And if the cache occupied an even smaller fraction of the request, the difference would be smaller again.
This is basically Amdahl's law appearing in a tiny Go HTTP server.
If fraction P of execution time can be accelerated by factor S, the theoretical total speedup is:
speedup = 1 / ((1 - P) + P / S)
A large S does not help much when P is tiny.
That is the part that microbenchmarks tend to hide.
The microbenchmark was still correct
I do not think the first benchmark was useless.
This distinction matters.
sync.Map really was substantially faster under that isolated read‑heavy workload.
The sharded map really did reduce the cost associated with one global lock.
The benchmark answered a valid question:
Which implementation can perform more concurrent cache lookups under these conditions?
The mistake would be changing the question after seeing the result.
It did not answer:
Which implementation will make my HTTP API 2.8x faster?
That would require an end‑to‑end benchmark.
One of the easiest performance mistakes is getting a correct answer to the wrong question.
Why sharding helped so much in isolation
The RWMutex implementation has one obvious synchronization point.
Every reader needs the same RLock.
RWMutex allows multiple readers to proceed concurrently, so this does not mean all reads execute serially.
But they still interact with the same synchronization structure.
As concurrency increases, that shared point becomes more important.
Sharding changes the geometry of the problem.
Instead of:
all keys | one lock
I get something closer to:
keys 0..N | hash | 64 separate locks
Operations touching different shards no longer compete through one shared lock.
That explains why the sharded version roughly doubled lookup throughput in my isolated test.
But it also adds work.
Every lookup needs to determine its shard.
There is more code.
There are more maps.
The implementation is harder to maintain.
In the microbenchmark, reducing contention was worth the extra complexity.
In the HTTP test, the benefit mostly vanished into the rest of the request path.
That changes the engineering decision.
Faster code is not always a faster system
This sounds obvious until a benchmark produces a large number.
29.4 million ops/s looks much better than 10.6 million ops/s.
It feels like a meaningful optimization.
And at the data‑structure level, it is.
But production software rarely exists at the data‑structure level.
A real handler may also perform:
authentication
authorization
tracing
logging
JSON encoding
decompression
validation
database access
RPC calls
metrics
allocations
filesystem operations
If the cache is responsible for 2 percent of request cost, spending two days making it three times faster is unlikely to change the service.
If it is responsible for 60 percent, the same optimization may be extremely valuable.
Without measuring that fraction, I do not know which situation I am in.
The easiest thing to benchmark is often the easiest thing to over‑optimize
There is another reason these mistakes happen.
Small functions are pleasant to benchmark.
A cache lookup can be wrapped in a loop.
A serialization function can be wrapped in a loop.
A hash function can be wrapped in a loop.
Then Go gives me clean numbers.
ns/op allocs/op B/op
Very satisfying.
The harder problems are usually less clean.
How much time does the request spend waiting for a database connection?
How often does the scheduler become relevant?
What happens to p99 latency during bursts?
Does GC become expensive only when traffic reaches a certain level?
Is the application limited by CPU, sockets, downstream services or memory bandwidth?
Those questions require more effort.
So it is easy to optimize the part that gives the nicest benchmark output instead of the part limiting the application.
I have started treating that as a warning sign.
If an optimization begins with a microbenchmark, I now want an end‑to‑end measurement before calling it a success.
There is also no universal winner between these three caches
I would not use this experiment to argue that sync.Map is always best.
That is another conclusion the benchmark does not support.
The workload here was intentionally favorable to concurrent reads.
Change the workload and the result can change.
For example:
frequent writes change synchronization behavior
continuously adding new keys changes the workload
deleting keys changes it again
a single hot key is different from uniformly distributed keys
64 entries are different from 6 million entries
one goroutine is different from hundreds
large values change cache behavior
pointer‑heavy values may change GC costs
Even the sharding strategy itself matters.
The number of shards matters.
The hash function matters.
The key distribution matters.
The relationship between readers and writers matters.
So I would not ask which Go concurrent map is fastest.
I would ask which implementation is appropriate for this access pattern and whether the map is important enough to optimize at all.
The second question is usually more valuable.
How I would test this in a real service
If I encountered a mutex‑protected cache in production, I would not replace it merely because another implementation wins a synthetic benchmark.
First I would look for evidence that the current cache is actually a problem.
I would check:
CPU profiles
mutex contention
time spent inside cache operations
cache hit rate
read/write ratio
key distribution
allocation rate
GC CPU
request throughput
p95 and p99 latency
the point where the service starts saturating
If the mutex is clearly visible in the profile, then optimizing it becomes interesting.
If it barely appears, replacing the cache may only improve a benchmark.
The application does not care which code looks faster in isolation.
It cares about the critical path.
A useful hierarchy for performance tests
After this experiment, I find it useful to think about benchmarks in three levels.
Level 1: component
This is the microbenchmark.
It answers questions like:
how fast is Get
how expensive is hashing
how much contention does this lock create
how many allocations does this function perform
This is where I observed the 2.8x difference.
Level 2: subsystem
Now the component sits inside part of the real execution path.
For example:
routing + validation + cache + serialization
This tells me whether the component remains important once neighboring work returns.
Level 3: end to end
Now I measure the system at the boundary that users or other services actually see.
For an API this might include:
client → network → HTTP server → application → cache/database → serialization → network → client
At this level, many impressive micro‑optimizations become difficult to see.
That does not invalidate them.
It tells me their contribution to the whole system is small.
The number I care about changed
Before this experiment, I could easily look at:
10.55M ops/s
versus:
29.37M ops/s
and think I had found something important.
Now the number I want immediately afterward is:
How much of the real request does this operation represent?
Without that information, the speedup is incomplete.
The cache benchmark told me sync.Map could process almost 2.8 times as many reads as my RWMutex version under the tested conditions.
The HTTP benchmark told me something more useful:
My HTTP server did not care very much.
That distinction is exactly why I ran the second test.
And, increasingly, it is the distinction I want to see whenever someone shows me a large microbenchmark speedup.
