After experimenting with connection pool behavior, I wanted to test another assumption that looks harmless on dashboards.

If two workloads have the same average request rate, are they really equivalent from the database point of view?

At first, it seems reasonable.

If one service receives 1,000 requests per second on average and another also receives 1,000 requests per second, both appear to place roughly the same load on PostgreSQL.

But averages remove timing.

And timing is exactly where queues appear.

So I built a controlled queueing simulation of a Go service talking to PostgreSQL through a connection pool.

This was not a benchmark of PostgreSQL itself. I deliberately kept the model simple because I wanted to isolate one variable: the arrival pattern.

The total amount of traffic stayed the same.

Only the timing changed.

One workload was smooth.

The other arrived in bursts.

The average RPS was identical.

The model behaved very differently.

What I wanted to isolate

A real PostgreSQL system has too many moving parts for this particular question.

Query plans change.

Caches warm up.

Checkpoints happen.

Autovacuum runs.

Storage latency varies.

CPU scheduling changes.

Background processes compete for resources.

If I ran the same experiment directly against a production-like database, a change in p99 would not automatically prove that traffic burstiness caused it.

So I started with a narrower model.

The simulated application received approximately 1,000 requests per second on average.

Every request required one database connection.

Most operations represented short indexed queries.

A smaller group represented slower operations.

The query-time distribution stayed unchanged between runs.

The connection pool size stayed unchanged.

The test duration stayed unchanged.

The total number of requests stayed unchanged.

Only their arrival times changed.

I tracked:

incoming RPS
completed RPS
connection acquisition time
active connections
request p50
request p95
request p99
queue depth over time

The exact latency values are not intended as PostgreSQL performance numbers.

The useful part of the simulation is the behavior of the queue when the same amount of work arrives in a different temporal pattern.

The smooth workload was boring

That was actually a good sign.

With traffic distributed relatively evenly, the connection pool rarely experienced sustained pressure.

The number of active connections moved around, but the queue remained small.

Temporary waiting appeared and disappeared quickly.

Completed throughput closely followed incoming traffic.

Latency remained stable.

The important part was not the exact number of milliseconds.

It was the shape.

Work arrived.

The system processed it.

Nothing accumulated for long.

In queueing terms, the model remained stable.

Then I changed only the arrival pattern

For the second run, I kept the same total number of requests but grouped them into short spikes.

Instead of approximately 1,000 requests arriving every second, the traffic could look more like this over several consecutive intervals:

1,700

1,600

400

300

1,100

900

Over a longer window, the average remained close to 1,000 RPS.

The amount of work was essentially unchanged.

The timing was not.

During each spike, requests arrived faster than the model could immediately process them.

A queue formed.

When traffic dropped, the system started draining it.

Sometimes the queue disappeared before the next spike.

Sometimes the next burst arrived while part of the previous queue still existed.

That was enough to change tail latency significantly.

Average RPS hid almost everything

Across the complete run, both workloads had nearly the same average incoming rate.

A dashboard showing one-minute averages could make them look almost identical.

That was the point of the experiment.

Average throughput told me how much work arrived during the window.

It did not tell me when that work arrived.

The smooth workload used available service capacity gradually.

The bursty workload repeatedly exceeded short-term capacity and created waiting.

The total work could be the same while the user-visible behavior was very different.

p50 barely moved

Median latency was not the first metric to react.

Most requests still experienced little or no queueing.

During quiet periods, a request could enter immediately.

At the beginning of a burst, some requests also got a connection before the pool filled.

So p50 remained relatively stable.

That created a combination I find dangerous in real monitoring:

average RPS looks normal

median latency looks normal

most individual operations still look fast

and a small percentage of requests experience much worse latency

The problem appeared much more clearly in p95 and p99.

p99 reacted to queue position

Under smooth traffic, requests saw relatively similar conditions.

Under bursty traffic, arrival timing became important.

Imagine two requests with identical simulated query times.

The first arrives while a connection is available.

The second arrives a few milliseconds later, after the pool has become saturated.

The first begins immediately.

The second waits behind other work.

The database operation itself has not changed.

End-to-end latency has.

That is why tail latency became much more sensitive to burstiness than median latency.

The model did not need slower queries to produce slower requests.

Queueing was enough.

Same amount of work, different queue

This became the simplest description of the result.

The workload performed roughly the same total amount of work.

The difference was how that work was presented to the system.

With smooth arrivals, capacity was consumed gradually.

With bursts, periods of spare capacity could not compensate for later spikes.

If the system has unused capacity during one quiet second, it cannot store that capacity and use it during the next overloaded second.

That unused capacity disappears.

Requests arriving later still have to wait.

Capacity is not a bank account

This is where average utilization can become misleading.

Suppose a service can sustainably process around 1,100 requests per second.

Its average traffic is 800 RPS.

It is tempting to describe that as roughly 300 RPS of spare capacity.

That can be useful for rough planning.

But it says nothing about short spikes.

If traffic temporarily reaches 1,500 RPS, the unused capacity from the previous quiet period does not help.

The system has instantaneous capacity.

Not stored capacity.

That distinction is especially important for tail latency.

A service can be comfortably below capacity on average and still create repeated queues.

The connection pool made the mechanism visible

In the simulation, the connection pool acted as the admission boundary.

Before all slots were occupied, requests could proceed immediately.

Once the pool became saturated, new requests waited.

This produced two clearly different components of latency:

connection acquisition time

and execution time after acquisition

That separation matters.

Execution time answers one question:

How long did the work take after it started?

Acquisition time answers another:

How much work was already ahead of this request?

A request can therefore become slow even when the simulated database operation remains fast.

A larger pool would not automatically solve the experiment

One obvious response to bursty traffic is to increase the pool size.

That might reduce application-side waiting.

But this simulation deliberately does not model PostgreSQL contention, so it cannot prove whether a larger pool would improve or worsen a real database workload.

What it can show is simpler.

Increasing the pool changes where the concurrency limit exists.

If the database has enough unused capacity, allowing more concurrent work may help.

If the database is already close to its efficient concurrency limit, a larger pool may only move the queue deeper into the system.

That second part needs a real PostgreSQL benchmark to verify.

I would not infer it from this simulation alone.

Monitoring resolution matters

The model also made aggregation effects easy to see.

Suppose traffic over several short intervals is:

500 RPS

600 RPS

1,800 RPS

1,900 RPS

500 RPS

700 RPS

A wide enough time window can compress this into a completely ordinary average.

The spikes disappear visually.

The queue does not.

The same principle applies to many production metrics.

CPU can briefly saturate and still show a moderate one-minute average.

Active connections can repeatedly hit the limit while a longer aggregation window looks stable.

p99 can spike for short periods without dramatically moving a long-term average.

For burst-sensitive workloads, I would want short-interval metrics placed on the same timeline.

Especially:

incoming RPS
completed RPS
active connections
connection acquisition latency
queue depth
request p95 and p99

The relationship between those metrics matters more than any single number.

Bursts do not have to come from users

Another useful implication is that uneven database traffic can be created internally.

Examples include:

scheduled jobs

batch workers

cache expiration

message consumers restarting

periodic synchronization

retry waves

multiple workers waking up on the same timer

A system can therefore have stable external traffic and still generate highly uneven database demand.

Imagine 100 workers performing the same periodic task every minute.

Their average database traffic can be tiny.

But if all 100 wake up at the same second, instantaneous concurrency can be large.

The average does not describe that behavior well.

Smoothing work can be an optimization

This is probably the most practical conclusion I would take from the model.

Performance optimization does not always mean doing less work or making each operation faster.

Sometimes it means changing when the same work happens.

Imagine a batch job that needs to perform 10,000 database operations.

One approach is to launch as much concurrency as possible and finish quickly.

Another is to use a bounded worker pool and spread the operations over a longer interval.

The amount of useful work is identical.

The second version may produce much less interference with latency-sensitive requests.

The batch takes longer.

The overall service can behave better.

That is load shaping rather than load reduction.

Backpressure matters more under bursty traffic

Bursts are especially useful for exposing systems without effective backpressure.

When the arrival rate exceeds service capacity, something has to happen.

Requests can queue.

They can be rejected.

Producers can slow down.

Workers can limit concurrency.

Or the system can continue accepting unlimited work until another resource becomes the bottleneck.

The first four options are controlled behaviors.

The last one usually is not.

A connection pool is one form of backpressure.

A bounded queue is another.

A rate limiter is another.

The correct mechanism depends on the system, but the goal is similar:

prevent short demand spikes from turning into unbounded concurrency.

Retries could make the effect much worse

I intentionally excluded retries from this simulation.

They would make it harder to isolate the original arrival pattern.

But they are an obvious next step.

Suppose a burst increases queueing enough that some requests time out.

Clients retry.

Now additional traffic appears specifically because the system is already under pressure.

Latency causes retries.

Retries cause more load.

More load causes more latency.

If many clients use the same timeout and retry interval, the retries themselves can become synchronized.

That can turn a short external spike into a longer internally generated wave.

Adding exponential backoff and jitter is one way to reduce that synchronization.

Again, the total amount of useful work has not necessarily changed.

The timing has.

Average RPS is not a workload description

This was the main thing the experiment changed for me.

I used to think about RPS as one of the primary descriptions of workload size.

Now I think average RPS is only one property.

Two services can both average 1,000 requests per second and have completely different operational characteristics.

One might remain between 900 and 1,100 almost continuously.

Another might alternate between 200 and 1,800.

Their averages are similar.

Their queueing behavior is not.

Their required headroom is not.

Their p99 behavior is not.

Their timeout sensitivity is not.

Their connection pool behavior is not.

That is a lot of information for one average to hide.

What I would test against real PostgreSQL

The simulation answers only the queueing question.

The next step would be a real PostgreSQL benchmark.

I would keep:

the query mix

the connection pool

the total request count

and the average request rate

as stable as possible.

Then I would run at least two arrival patterns:

smooth

and bursty

On the Go side, I would collect:

incoming requests
completed requests
connection acquisition duration
active connections
idle connections
pool waiters
request p50
request p95
request p99

On PostgreSQL, I would collect:

active sessions
query latency
transactions per second
CPU utilization
I/O latency
lock waits
relevant query statistics

That would answer a second question that this model cannot:

Does burstiness only create application-side queueing, or does additional database concurrency also change query execution itself?

That distinction matters.

If acquisition latency rises while query execution remains stable, most of the problem is before execution.

If query duration also rises during bursts, PostgreSQL itself is experiencing additional contention.

What I would try before adding more capacity

If production measurements showed that burstiness was the real cause, I would first ask whether the spikes could be reduced.

Possible approaches include:

adding jitter to scheduled jobs

limiting background worker concurrency

breaking large batches into smaller units

using queues

avoiding synchronized cache expiration

rate limiting non-interactive work

using exponential backoff for retries

separating latency-sensitive and background workloads

These changes do not necessarily reduce total work.

They reduce simultaneous work.

That distinction is important.

The next experiment

There is an obvious continuation.

Keep the same bursty arrival pattern.

Keep the same total work.

Change only the admission policy.

I would compare:

one shared connection pool

a smaller bounded pool

separate pools for interactive and background traffic

The interesting metric would not be maximum throughput.

It would be whether interactive p99 changes while total completed work remains approximately the same.

That would test whether connection pools can act not only as reusable connection storage, but also as a simple traffic scheduling mechanism.

What I took away from this

This simulation did not prove that bursty traffic makes PostgreSQL itself slower.

It was not designed to.

What it did show is that two workloads with the same average request rate can create very different queueing behavior before database execution even begins.

The total amount of work stayed approximately the same.

The average RPS stayed approximately the same.

The timing changed.

That alone was enough to change tail latency.

So average RPS tells me how much work arrived during a window.

It does not tell me how that work arrived.

And when a database-backed service operates close to a concurrency limit, that missing information can be more important than the average itself.

I Sent the Same PostgreSQL Traffic in Smooth and Bursty Patterns. Average Load Lied