Why Least Connections Beats Round Robin for Long Sessions

When someone asks me, “Which load-balancing algorithm should I use?” they usually expect a one-liner. They want me to say “round robin” because that’s the default everywhere. But the real answer is: it depends on your traffic. And if you have long-lived connections, round robin is actively hurting you. Let’s bust that myth and get you to the algorithm that fits your actual workload.

Is round robin really the best default?

Round robin is the default in nginx and a common choice in HAProxy, and it’s fine for short, stateless connections like typical web requests. It distributes each new request to the next server in line, so it’s simple and fair in a naive sense. But “fair” means every server gets the same number of connections, not the same amount of work. If one request holds a connection open for ten seconds and another finishes in ten milliseconds, round robin will pile up connections on servers that got the slow ones. That’s the misconception: fairness by count, not by load.

Why does least connections work better for long sessions?

Least connections sends each new request to the server with the fewest active connections. That’s a much better proxy for load when connections are long-lived, like WebSockets, streaming, or database connections. HAProxy’s documentation explicitly recommends leastconn for long sessions such as LDAP or SQL, and warns it’s not well suited to short HTTP sessions because the connection count doesn’t reflect request cost. For short requests, round robin is simpler and avoids the overhead of tracking connection counts. But for anything where a client stays connected for a while, least connections is the right call.

What about sticky sessions and hashing?

Sometimes you need a client to hit the same server every time, either because of session state or to improve cache hit ratios. That’s where hashing comes in. IP hash (or source hash) maps a client’s IP to a server consistently, and nginx’s hash directive with the consistent parameter uses ketama consistent hashing so that adding or removing a server only remaps a few keys—great for caching layers where you want to keep a user’s cache warm. HAProxy offers source, uri, url_param, and hdr for similar purposes. The general guidance from the system-design-primer: use hashing or cookies for session persistence. So if you have sticky sessions, don’t default to round robin; pick a hash that matches your session mechanism.

Is weighted round robin worth the complexity?

Weighted round robin is useful when you have heterogeneous servers—maybe one box is beefier than another. You can assign weights so the bigger server gets more requests. Both nginx and HAProxy support weighting, and it’s a natural extension of round robin. But it still has the same flaw as plain round robin: it doesn’t account for connection duration. If you’re going to bother with weights, you might as well use least connections with weights, which HAProxy supports via balance leastconn plus server weight. Don’t add complexity unless you have a real difference in capacity.

What about least response time?

Least response time is the algorithm that tries to send requests to the server that’s responding fastest. It’s a great idea, but it requires the load balancer to measure response times, which adds overhead and complexity. The system-design-primer lists it as one of the common algorithms, but nginx and HAProxy don’t implement it as a built-in balance method in the same way (HAProxy has fair? no—it doesn’t have a direct “least response time” either; it has random and others). So in practice, you’re often choosing between round robin, least connections, and hashing. For most web services, least connections is a safe middle ground, but if your requests are uniformly fast and stateless, round robin is perfectly fine.

How do I pick the right algorithm for my API?

Here’s a concrete scenario. You’re running an API with a mix of short REST calls and occasional long-polling or streaming requests. If you use round robin, the server that gets a long-polling request will hold that connection for minutes, while other servers are idle. Switch to least connections and the next request goes to a server with fewer active connections, balancing the load more evenly. For a typical web API with short requests, round robin is fine and simpler. But if you have any long-lived connections, go with least connections. And if you need sticky sessions, use a hash—ideally consistent hashing if you’re scaling your cache.

There’s no one-size-fits-all. The myth that round robin is the “default” and therefore the best is exactly that—a myth. It’s the default because it’s simple, not because it’s optimal. Take a minute to look at your connection duration and session needs, then pick the algorithm that matches. Your servers will thank you.

Sources

• system-design-primer - https://github.com/donnemartin/system-design-primer
• Nginx (HTTP load balancing) - https://nginx.org/en/docs/http/load_balancing.html
• HAProxy (configuration manual) - https://docs.haproxy.org/3.4/configuration.html