Real-Time Latency Is a Product Decision, Not a Metric

Engineers treat latency as a number to drive toward zero. That is the wrong frame. Latency is a product decision about what has to feel instant, what can feel fast, and what can quietly lag without anyone caring. Chasing the lowest possible latency everywhere is how you spend a fortune on infrastructure to make a typing indicator 40 milliseconds quicker while your app still feels slow, because the thing users actually notice was never the bottleneck. Set a latency budget per interaction, and spend your engineering where perception lives.

Latency is perceived, not measured

The core mistake is optimizing the metric instead of the experience. Users do not feel milliseconds. They feel whether an action responded before they started to doubt it worked.

Human perception has rough thresholds. Under about 100 milliseconds feels instant, like direct manipulation. Up to a second feels responsive, the system is clearly working. Past a couple of seconds, attention wanders and the interaction feels broken. These bands, not the raw number, are what you are designing against. Getting a cursor from 80 to 50 milliseconds buys you nothing, because both are already in the instant band. Getting a chat send from 1.5 seconds to 400 milliseconds is the difference between an app that feels sluggish and one that feels alive.

So the question is never "how low can latency go." It is "which band does this interaction need to live in," and that is a product call.

Set a latency budget per interaction

Different interactions have wildly different requirements, and a single global latency target is always wrong for most of them. So budget per interaction type.

Live cursors and typing indicators need to feel instant, under the perception threshold, because they are direct feedback. Chat messages need to feel responsive, fast enough that the sender does not wonder if it sent. Presence updates, the online dot, can lag several seconds and nobody notices or cares. A dashboard metric can be seconds stale and still be useful.

Once you write those budgets down, your architecture gets clear. You spend effort and money on the interactions that need the tight band, and you deliberately relax the ones that do not. This is the same deliberate-choice discipline I apply to message delivery guarantees: decide per message type instead of forcing one global rule that is wrong for half your traffic.

Where the latency actually goes

To budget, you have to know where time is spent, because teams routinely optimize the wrong segment. The path is network transit to your infrastructure, processing and authorization on the server, fan-out across nodes to the right subscribers, and transit back to the receiving client. Real latency is usually dominated by geography and by your own fan-out, not by the socket protocol.

If your server sits in one region and your users are global, no socket cleverness fixes the physics of distance. That is a placement decision about where your real-time infrastructure runs, which ties into the broader case for infrastructure you actually control. And if your fan-out to subscribers is slow, that is your pub/sub backbone, which is exactly the scaling work in scaling WebSockets without it falling over. Measure the segments before you optimize, or you will tune the one that was never the problem.

Design the fallbacks on purpose

A latency budget also tells you what to do when you blow it, which will happen. When an update is late, does the app wait for it, show a stale value, or show an optimistic guess and reconcile later. Optimistic updates, showing the result immediately and correcting if the server disagrees, are how you make an interaction feel instant even when the round trip is not. That is a product decision too, made per interaction, not a default you inherit from a library.

This is the kind of end-to-end thinking a complete real-time backend should support rather than fight. AltoHost exists to own the delivery, fan-out, and connection guarantees so you can spend your attention on the product-level question of what has to feel instant, instead of reassembling the plumbing every time you want to tune it.

Closing

Stop asking how low your latency can go. Ask which interactions have to feel instant, which can feel fast, and which can lag unnoticed. Write a budget per interaction, spend your engineering where human perception actually lives, and design your fallbacks deliberately. Latency is not a metric you minimize. It is a product decision you make on purpose, one interaction at a time.