Reducing serverless cold starts in production

Cold starts get blamed for every slow request in a serverless stack, and about half the time the accusation is wrong. The other half, they are real, reproducible, and worth a serious chunk of engineering time. The problem is that most people attack them in the wrong order: they add a warmer and call it a day.

We spent a quarter measuring and then removing cold-start latency from our production functions. This is what we learned, in the order we learned it.

What a cold start actually is

A cold start is the time it takes to bring a fresh execution environment to the point where your handler can run. It has three parts that compound: the platform provisioning a sandbox or microVM, the runtime booting, and your code initialising — module loading, dependency resolution, connection setup.

None of these are your function code running slowly. They are all overhead before your code runs, which is why profiling the handler itself will never find them. The single most important fact about cold starts is that the fix usually lives in code that never touches a request handler.

Measure before you optimise

The first thing we did was instrument the cold-start path directly, not infer it from total latency. We logged the time between container start and handler invocation from inside the runtime — before our middleware, before anything we control. That gave us a clean number: platform boot plus runtime init.

The second thing we measured was the p99 of requests that hit a cold container versus the p50 of warm ones, per function. The gap between those two numbers is your actual cold-start tax, and it varied wildly across our fleet. A trivial health function paid almost nothing. A handler that loaded a 40 MB SDK graph paid seconds.

You cannot fix a number you are not collecting. If your telemetry does not distinguish cold from warm, you will chase phantom latency for months. We shipped that instrumentation first and optimised second.

The cheap wins: image and runtime

The cheapest wins are also the most boring, and we did them first.

• Shrink the deployment. A smaller package is less to fetch, less to unpack, and less to parse. We cut our Node handler from 42 MB to 6 MB by removing unused SDKs and letting the bundler tree-shake properly.
• Avoid interpreted-fatness at boot. Heavy native modules and large JSON configs loaded at module scope add hundreds of milliseconds before your handler exists.
• Pick the runtime for the path. A compiled runtime with snapshots beats an interpreter on cold start almost every time, at the cost of less forgiving builds.

None of these are exotic. They are the difference between a function that loads quickly and one that drags its entire dependency tree through the interpreter on every cold container.

Fix initialisation, not just the sandbox

The biggest single win came from where we put our side effects. Every function in the fleet followed the same broken pattern at first: create the database client, open the HTTP agent, load config — all at module scope. That is work the runtime cannot reuse across invocations, because there is only one invocation per container.

We moved everything that could be deferred into lazy singletons: first call creates, subsequent calls reuse. Connection pools became lazy, config loads became cached, and the expensive import of the database driver was pushed behind an async boundary so the handler could respond to a ping before the pool existed.

This matters more than any platform feature because it is the part you control. Provisioned concurrency removes the sandbox boot, but if your module-scope init takes 800 ms, a warm container still takes 800 ms before it serves.

Platform levers: provisioned concurrency and snapshots

Once the code was clean, we spent the platform budget deliberately. Two levers matter: pre-warmed capacity and snapshots.

Provisioned concurrency, or its equivalent under whatever name your provider gives it, keeps a floor of warm containers so a spike of traffic does not pay cold-start tax for the first N requests. We do not provision for the whole fleet — that defeats the cost model of serverless. We provision for the functions whose p99 latency is a product requirement, and let everything else scale from zero.

Snapshots are the more interesting trick. Some runtimes can snapshot the fully-initialised container after module scope has run and restore it instead of booting fresh. For Java-style workloads this collapsed our cold start from seconds to hundreds of milliseconds. The constraint is that anything stateful you initialise at snapshot time is frozen into the image — sockets, mutexes, timers — so you have to keep init lazy even after enabling it.

Warmers are a trap

Every team we know tries warmers: a scheduled ping to keep containers alive. They seem free. They are not.

A warmer costs you money for containers that are not serving real traffic, it gives you false confidence because it warms the pinging path rather than the paths users actually hit, and it teaches your team to rely on a mechanism the platform explicitly does not guarantee. Providers recycle warm containers on their own schedule, so the warmer is fighting the very system that owns the lifecycle.

We treat warmers as a smell. If a function needs a warmer to meet its latency target, it needs provisioned concurrency, or it needs to be moved off the cold-start-sensitive path entirely.

Architecture-level fixes

The last layer is the one most teams skip because it is not glamorous: stop sending cold-start-sensitive requests to cold functions at all.

For reads, an edge cache or a hot data service absorbs the traffic before it ever reaches a function. For writes, a queue decouples the user-facing acknowledgement from the work. And for the specific requests that are allowed to be slow — report generation, exports, webhook fan-out — we route them to functions we explicitly do not keep warm, and tell the user the request is going to take a moment.

Once we split the fleet into "latency-critical, keep warm" and "async, let it freeze", the cold-start problem stopped being an operational firefight and became a small monthly budget decision.

The order to do it in

If we started over tomorrow, we would do exactly this sequence: instrument the cold path, shrink and clean the deployment, make initialisation lazy, then spend platform budget only on the functions where latency is a product requirement, and never on warmers.

Cold starts are a finite, measurable budget. Measured first, they turn from a mystery into a to-do list. Everything else is just doing the to-do list.