Least Connections #

Least connections is a load balancing algorithm that sends each new request to the server with the fewest active connections at that moment. Unlike round robin, which just takes turns without paying attention to the actual server state, least connections is more adaptive — it dynamically adjusts distribution based on the real load each server is currently carrying.

The Problem Round Robin Doesn’t Solve #

Imagine this scenario where round robin starts to misbehave:

Server pool: A, B, C — all plain round robin

t=0: Request "large CSV export" → Server A (starts processing, takes 30 seconds)
t=1: Regular request → Server B (done in 50ms)
t=2: Regular request → Server C (done in 50ms)
t=3: Regular request → Server A  ← round robin's turn for A again!
                                  But A is still busy processing the export!
t=4: Regular request → Server B
t=5: Regular request → Server C
t=6: Regular request → Server A  ← A is still busy!
...

For 30 seconds, a third of all requests are sent to Server A, which is busy processing one large export. Users whose requests land on Server A experience high latency or timeouts.

With least connections, the same scenario:

t=0: Request "large CSV export" → Server A (1 active connection)
t=1: Regular request → Server B (0 active connections) ← least conn picks B
t=2: Regular request → Server C (0 active connections) ← or C
t=3: Regular request → Server B or C             ← A has 1, B/C have 0
t=4: All requests keep going to B and C while A is busy
...
Server A finishes the export → connections = 0, back to receiving new requests

Configuration #

upstream app_servers {
    # Just add this directive to enable least connections
    least_conn;

    server 10.0.0.1:3000;
    server 10.0.0.2:3000;
    server 10.0.0.3:3000;

    # All other parameters still apply
    keepalive 32;
    zone app_upstream 64k;
}

server {
    listen 443 ssl;
    server_name example.com;

    location / {
        proxy_pass         http://app_servers;
        proxy_http_version 1.1;
        proxy_set_header   Connection    "";
        proxy_set_header   Host          $host;
        proxy_set_header   X-Real-IP     $remote_addr;
        proxy_set_header   X-Forwarded-For   $proxy_add_x_forwarded_for;
        proxy_set_header   X-Forwarded-Proto $scheme;
    }
}

How Nginx Chooses a Server #

When a new request arrives, Nginx looks at the number of active connections to each server and picks the one with the fewest:

flowchart TD
    REQ["New request arrives"] --> READ["Read the active connection counts\non all servers"]
    READ --> TABLE["Server A: 12 connections\nServer B: 3 connections\nServer C: 8 connections"]
    TABLE --> MIN{"Which server has\nthe fewest\nconnections?"}
    MIN --> B["Server B (3 connections)\n← pick this one"]
    B --> SEND["Send the request to Server B\nB's connections become 4"]
    SEND --> DONE["After the request finishes:\nB's connections return to 3"]

What counts as an active connection:

Requests currently being processed by the backend (response not yet sent)
Keepalive connections waiting for the next request

Tiebreaker: If several servers have exactly the same number of connections, Nginx uses weighted round robin among them as the tiebreaker.

Least Connections with Weights #

least_conn can be combined with weight to account for servers with different capacities:

upstream app_servers {
    least_conn;

    # Strong server: considered "busy" only after 3x more connections
    server 10.0.0.1:3000 weight=3;

    # Weak server: considered "busy" sooner
    server 10.0.0.2:3000 weight=1;
}

With weights, the selection formula becomes active_connections / weight. The server with the smallest value is chosen:

Situation: Server A (weight=3) has 12 active connections
           Server B (weight=1) has 3 active connections

Effective values:
  Server A: 12/3 = 4
  Server B:  3/1 = 3

Pick Server A? No → A's value (4) > B's value (3)
Pick Server B? Yes → B's value (3) < A's value (4)

Even though in absolute terms Server A has 12 connections and Server B only 3,
proportionally to capacity, Server B is "busier".

In-Depth Comparison: Round Robin vs Least Connections #

flowchart LR
    subgraph RR["Round Robin"]
        direction TB
        R1["Request 1 → A"]
        R2["Request 2 → B"]
        R3["Request 3 → C"]
        R4["Request 4 → A (even though A is still busy!)"]
    end

    subgraph LC["Least Connections"]
        direction TB
        L1["Request 1 → A (A: 1 connection)"]
        L2["Request 2 → B (B: 0 connections)"]
        L3["Request 3 → C (C: 0 connections)"]
        L4["Request 4 → B or C (A still at 1, B and C back to 0)"]
    end

Aspect	Round Robin	Least Connections
Distribution	Even by rotation	Even by actual load
Adaptive	No	Yes — automatically avoids busy servers
Complexity	Very simple	Slightly more complex
Fast requests (<100ms)	Nearly identical results	Nearly identical results
Slow requests (>1 second)	Can pile up	Automatically avoided
WebSocket	Not ideal	Better — handles long-lived connections
Best for	Uniform, fast requests	Requests with varying durations

Ideal Scenarios for Least Connections #

1. APIs with Mixed Endpoints #

# An app with both fast and slow endpoints:
# GET /products → 20ms (simple query)
# GET /analytics/report → 10 seconds (heavy query)
# POST /export → 30 seconds (file generation)

upstream api_backend {
    least_conn;  # Automatically avoids servers processing reports/exports

    server 10.0.0.1:3000 max_fails=3 fail_timeout=30s;
    server 10.0.0.2:3000 max_fails=3 fail_timeout=30s;
    server 10.0.0.3:3000 max_fails=3 fail_timeout=30s;

    keepalive 32;
    zone api_upstream 64k;
}

2. WebSocket and Long-Lived Connections #

http {
    map $http_upgrade $connection_upgrade {
        default upgrade;
        ''      close;
    }

    upstream ws_backend {
        least_conn;  # WebSockets can last for hours → least conn helps a lot

        server 10.0.0.1:4000;
        server 10.0.0.2:4000;
        server 10.0.0.3:4000;

        keepalive 16;
        zone ws_upstream 32k;
    }

    server {
        location /ws/ {
            proxy_pass          http://ws_backend;
            proxy_http_version  1.1;
            proxy_set_header    Upgrade    $http_upgrade;
            proxy_set_header    Connection $connection_upgrade;
            proxy_read_timeout  3600s;
        }
    }
}

With WebSocket, a single connection can last for hours. Round robin can leave one server full of old WebSocket connections while another sits empty because new WebSocket requests keep rotating to different servers. Least connections automatically balances this.

3. Microservices with Different Loads #

# Service A: lightweight API
upstream service_a {
    # plain round robin is enough — all requests are fast
    server 10.0.1.1:3000;
    server 10.0.1.2:3000;
    keepalive 32;
}

# Service B: Heavy processing (video transcoding, ML inference, etc.)
upstream service_b {
    least_conn;  # Essential — per-request duration varies wildly (1s - 5 min)

    server 10.0.2.1:3001;
    server 10.0.2.2:3001;
    keepalive 8;
}

Monitoring Connection Distribution #

# See active connections to each upstream server (if stub_status is available)
curl http://localhost/nginx_status

# With nginx-module-vts or a prometheus exporter, you can see per-server:
# nginx_upstream_peers_active{upstream="app_servers", server="10.0.0.1:3000"} 12
# nginx_upstream_peers_active{upstream="app_servers", server="10.0.0.2:3000"} 3
# nginx_upstream_peers_active{upstream="app_servers", server="10.0.0.3:3000"} 8

# From the access log — average response time per server
awk '{print $5, $8}' /var/log/nginx/access.log | \
    awk '{sum[$1]+=$2; count[$1]++} END {for(k in sum) print k, sum[k]/count[k], "avg_rt"}' | \
    sort -k2 -n

Load Testing: Proving the Difference Between Least Connections and Round Robin #

The best way to understand when least_conn is better is to measure it directly with a workload reflecting production conditions:

# Install wrk for load testing
brew install wrk  # macOS
apt install wrk   # Ubuntu

# Scenario 1: All requests fast and uniform
# Expectation: round robin and least_conn are nearly identical
wrk -t12 -c400 -d30s --latency http://example.com/api/products

# Scenario 2: A mix of fast and slow requests
# Add one slow endpoint during the test:
# /api/report (5 seconds per request)
# Expectation: least_conn is much better
wrk -t12 -c400 -d30s --latency \
    -s mixed_workload.lua \
    http://example.com/

-- mixed_workload.lua: 90% fast requests, 10% slow requests
math.randomseed(os.time())

request = function()
    local r = math.random()
    if r < 0.9 then
        return wrk.format("GET", "/api/products")
    else
        return wrk.format("GET", "/api/report")  -- 5-second endpoint
    end
end

Comparing Results #

# wrk output shows:
# Latency distribution: 50th, 75th, 90th, 99th percentile
# Request/sec
# Transfer/sec

# Round robin with mixed workload:
# Latency distribution:
#   50%    45ms
#   75%   890ms    ← drastic spike
#   90%  4320ms    ← many requests land on a busy server
#   99%  8910ms

# Least connections with mixed workload:
# Latency distribution:
#   50%    43ms
#   75%    89ms    ← far more consistent
#   90%   156ms
#   99%   890ms    ← p99 much lower

Production-Ready Configuration with Least Connections #

A complete configuration template for production with least_conn:

http {
    upstream api_backend {
        # Algorithm: least connections
        least_conn;

        # Shared memory for cross-worker data
        zone api_upstream 128k;

        # Production servers
        server 10.0.0.1:3000 max_fails=3 fail_timeout=30s;
        server 10.0.0.2:3000 max_fails=3 fail_timeout=30s;
        server 10.0.0.3:3000 max_fails=3 fail_timeout=30s;

        # Backup server: shown if all main servers are down
        server 10.0.0.4:8080 backup;

        # Keepalive pool to the backend
        keepalive 64;
        keepalive_timeout 60s;
        keepalive_requests 1000;
    }

    server {
        listen 443 ssl;
        http2 on;
        server_name api.example.com;

        location /api/ {
            proxy_pass         http://api_backend;
            proxy_http_version 1.1;
            proxy_set_header   Connection    "";
            proxy_set_header   Host          $host;
            proxy_set_header   X-Real-IP     $remote_addr;
            proxy_set_header   X-Forwarded-For   $proxy_add_x_forwarded_for;
            proxy_set_header   X-Forwarded-Proto $scheme;

            # Retry another server on errors
            proxy_next_upstream error timeout http_502 http_503 http_504;
            proxy_next_upstream_tries 3;
            proxy_next_upstream_timeout 10s;

            # Timeouts
            proxy_connect_timeout  5s;
            proxy_read_timeout    30s;

            # Debug header for monitoring (can be disabled in production)
            add_header X-Upstream-Server $upstream_addr always;
        }
    }
}

Best Practices and Anti-Patterns #

What to Do #

# ✓ Always use the zone directive for shared memory
upstream app_servers {
    least_conn;
    zone app_upstream 128k;  # ← Essential for multi-worker clusters
    server 10.0.0.1:3000;
    server 10.0.0.2:3000;
}

# ✓ Combine with max_fails and fail_timeout
upstream app_servers {
    least_conn;
    zone app_upstream 128k;
    server 10.0.0.1:3000 max_fails=3 fail_timeout=30s;
    server 10.0.0.2:3000 max_fails=3 fail_timeout=30s;
}

# ✓ Add keepalive for optimal performance
upstream app_servers {
    least_conn;
    zone app_upstream 128k;
    server 10.0.0.1:3000 max_fails=3 fail_timeout=30s;
    server 10.0.0.2:3000 max_fails=3 fail_timeout=30s;
    keepalive 32;   # ← Maintain the connection pool
    keepalive_requests 1000;
}

Anti-Patterns to Avoid #

# ✗ Without the zone directive: data isn't shared between workers
upstream app_servers {
    least_conn;
    # No zone!
    # Worker process 1 has its own connection data
    # Worker process 2 has different data
    # Distribution is inaccurate at the cluster level
    server 10.0.0.1:3000;
    server 10.0.0.2:3000;
}

# ✗ least_conn for very fast, uniform requests
# (not wrong, but unnecessary overhead)
# If all responses are < 10ms, round robin produces the same result
# and is more predictable
upstream static_files {
    least_conn;  # ← Not needed for fast static file serving
    server 10.0.0.1:3000;
    server 10.0.0.2:3000;
    # Use plain round robin for this
}

# ✓ What's right for static files:
upstream static_files {
    # No directive — round robin default
    server 10.0.0.1:3000;
    server 10.0.0.2:3000;
    keepalive 64;  # But still use keepalive
}

FAQ and Troubleshooting #

Q: Is least_conn safe for all request types?

Yes, least_conn doesn’t affect request content — it only chooses the destination server. Requests are still forwarded intact to the backend. There’s no risk of data corruption or session mixing.

Q: Why is the connection distribution never perfectly even?

Because requests finish at different times. When a new request arrives, the distribution reflects currently active connections, not the total requests ever received. If 10,000 requests have finished and only 5 are active, it’s the distribution of those 5 active connections that matters.

Q: What happens if two servers have exactly the same number of connections?

Nginx uses weighted round robin as the tiebreaker. If all servers have the same weight, it picks round robin among the tied servers.

Q: Does least_conn work with a keepalive pool?

Yes, and this combination is highly recommended. Keepalive reduces TCP handshake overhead, while least_conn ensures the existing keepalive connections are distributed evenly based on actual load.

# Troubleshooting: traffic keeps going to one server
# Likely: another server is in max_fails status
nginx -s reload  # Reload to reset fail counters (not ideal in production)

# Or check the logs:
tail -f /var/log/nginx/error.log | grep -i "fail\|down\|unavailable"

# Check the actual distribution from the access log:
awk '{print $5}' /var/log/nginx/access.log | sort | uniq -c
# If one server gets 0 requests, it's likely marked down

Operational Tip: When switching from round robin to least_conn on a running production environment, no connections are cut. nginx -s reload is graceful — old workers finish active connections, new workers start with the new least_conn algorithm. Users don’t notice the difference.

Summary #

least_conn sends requests to the server with the fewest active connections — more adaptive than round robin for varying workloads.
Can be combined with weight to account for servers with different capacities; the effective formula = active_connections / weight.
The main advantage: automatically avoids servers currently busy processing heavy requests, without extra configuration.
Ideal for APIs with mixed endpoints (some light, some heavy) and WebSocket or other long-lived connections.
For requests with uniform, short durations (< 100ms), round robin and least connections produce nearly identical distribution.
Always use it with the zone directive so active connection data is shared across all Nginx worker processes, not per-worker.

← Previous: Round Robin Next: IP Hash →