Buffering & Timeouts #

Buffering and timeouts are two aspects of the reverse proxy most often overlooked — yet their impact can be very noticeable. Incorrectly configured buffering affects performance when the backend is slow or the client has a slow connection. Timeouts that are too short cause 504 errors for operations that should succeed; too long lets connections hang for too long and drains resources.

This article covers both in depth so you can tune them properly for different workload types.

How Buffering Works: Two Modes #

When Nginx receives a response from the backend, it can work in two fundamentally different modes.

Mode 1: Buffering Enabled (Default) #

Backend ──────► Nginx Buffer (RAM/disk) ──────► Client
          fast                           slow OK

Nginx reads the entire response from the backend into a memory buffer first. After the backend sends all data, Nginx handles delivery to the client independently — even if the client downloads very slowly.

Advantages:

  • The backend is free faster — it can handle other requests
  • Nginx optimally manages slow client connections
  • More efficient for applications returning many small responses

When to use: Almost all regular web applications — REST APIs, websites, dashboards.

Mode 2: Buffering Disabled #

Backend ──────► Nginx ──────► Client
          real-time      real-time

Nginx directly forwards data from the backend to the client in real time, byte by byte. The backend and client are connected synchronously.

Advantages:

  • Minimal latency — data reaches the client immediately
  • Right for streaming and event-driven responses

When to use: Server-Sent Events (SSE), video streaming, log streaming, long-polling.

flowchart LR
    subgraph BUFFER["Mode: Buffering Enabled"]
        direction LR
        B1["Backend"] -- "send fast" --> N1["Nginx Buffer\nRAM or Disk"]
        N1 -- "send at the\nclient's speed" --> C1["Client\nslow OK"]
    end

    subgraph NOBUFFER["Mode: No Buffering"]
        direction LR
        B2["Backend"] -- "sync" --> N2["Nginx\njust relays"]
        N2 -- "real-time" --> C2["Client\nSSE or streaming"]
    end

Buffering Directives #

Standard Buffer Configuration #

http {
    # Enable/disable response buffering (default: on)
    proxy_buffering on;

    # Buffer size for response headers (and a little bit of the body start)
    # Increase if the backend sends many headers
    proxy_buffer_size 4k;

    # Number and size of buffers for the response body
    # Total buffer = 16 × 4k = 64k per connection
    # Increase if the backend sends large responses
    proxy_buffers 16 4k;

    # Limit of data ready to be sent to the client (buffered but not yet sent)
    # Must be ≥ proxy_buffer_size and ≤ total proxy_buffers
    proxy_busy_buffers_size 8k;

    # If the response exceeds all buffers (above), temporarily store it to a file
    # 0 = disable temp files (response errors if buffers fill up)
    proxy_max_temp_file_size 1024m;

    # Directory for temp files
    proxy_temp_path /var/cache/nginx/temp 1 2;
}

Tuning Buffers for Large Responses #

For APIs returning large JSON (e.g., data exports):

location /api/export/ {
    proxy_pass http://backend;

    # Increase buffers for large responses
    proxy_buffer_size       16k;
    proxy_buffers           8 16k;   # Total 128k
    proxy_busy_buffers_size 32k;

    # Allow temp files for very large responses
    proxy_max_temp_file_size 500m;
}

Disabling Buffering for Streaming and SSE #

# Server-Sent Events
location /events {
    proxy_pass http://sse_backend;

    # Disable buffering — forward directly to the client
    proxy_buffering off;
    proxy_cache off;

    # HTTP/1.1 for chunked transfer
    proxy_http_version 1.1;
    proxy_set_header Connection '';
    chunked_transfer_encoding on;

    # Long timeouts — SSE connections can last a long time
    proxy_read_timeout 3600s;
    proxy_send_timeout 3600s;
}

# Video streaming
location /stream/ {
    proxy_pass http://streaming_backend;
    proxy_buffering off;
    proxy_cache off;
    proxy_read_timeout 3600s;

    # No need for access logs on streaming
    access_log off;
}

Five Timeout Types — the Critical Differences #

Nginx has several different timeouts for backend connections. Understanding what each one measures is the key to debugging 502/504 errors.

sequenceDiagram
    participant N as Nginx
    participant B as Backend

    Note over N,B: proxy_connect_timeout
    N->>B: Open TCP connection
    Note over N,B: If not connected within the limit → 502

    Note over N,B: proxy_send_timeout
    N->>B: Send request headers
    N->>B: Send request body
    Note over N,B: Idle between two writes? → 504

    Note over N,B: proxy_read_timeout
    B->>N: Response header
    B->>N: Response body (chunk 1)
    Note over N: Waiting for the next chunk
    B->>N: Response body (chunk 2)
    Note over N,B: Idle between two packets? → 504

proxy_connect_timeout #

The maximum time to open a connection to the backend. If the backend doesn’t respond to the TCP handshake within this time, Nginx returns 502.

# Default: 60s — usually too long
proxy_connect_timeout 10s;

# When this triggers:
# - The backend isn't running
# - The firewall blocks the port
# - The backend is overloaded and can't accept new connections

proxy_send_timeout #

The maximum idle time between two writes to the backend. Not the total time to send a request — this is a per-“packet” timeout. As long as Nginx keeps sending data, the timeout doesn’t trigger.

proxy_send_timeout 60s;

# When to increase:
# - Requests with very large bodies (file uploads)
# - Slow connections to the backend

proxy_read_timeout #

The maximum idle time between two packets received from the backend. This is the most often misunderstood — it is not the total time the backend spends processing a request.

proxy_read_timeout 60s;

# What it means:
# - The backend MAY process for as long as it wants
# - As long as it occasionally sends data (even just 1 byte)
# - The timeout only triggers if the backend is SILENT for 60s

# Valid example:
# Backend processes a large report for 10 minutes, then sends everything at once
# → proxy_read_timeout 600s is needed

# When it triggers:
# - The backend hangs/deadlocks
# - A database query runs very long
# - The backend is overwhelmed and doesn't respond

Client-Side Timeouts #

server {
    # Timeout for reading the request body from the client
    # (between two packets received from the client)
    client_body_timeout 30s;

    # Timeout for reading request headers from the client
    client_header_timeout 30s;

    # How long keep-alive connections are left idle before closing
    keepalive_timeout 75s;

    # Timeout for sending a response to the client
    send_timeout 60s;
}

Proper Timeout Values for Various Scenarios #

Regular REST API #

location /api/ {
    proxy_pass http://api_backend;

    proxy_connect_timeout  5s;    # The backend should respond quickly
    proxy_read_timeout    30s;    # API responses are usually < 5 seconds
    proxy_send_timeout    30s;
}

Heavy Operations (Reports, Exports, ETL) #

location /api/reports/ {
    proxy_pass http://backend;

    proxy_connect_timeout   5s;
    proxy_read_timeout    300s;   # 5 minutes to generate large reports
    proxy_send_timeout    300s;

    # Large buffers for export responses
    proxy_buffer_size       64k;
    proxy_buffers           8 64k;
    proxy_max_temp_file_size 500m;
}

File Uploads #

location /upload/ {
    proxy_pass http://backend;

    # Large file uploads need a long send time
    proxy_connect_timeout   10s;
    proxy_send_timeout     300s;   # 5 minutes for large file uploads
    proxy_read_timeout      60s;

    # Allowed request body size
    client_max_body_size    500m;

    # Buffer the request body before forwarding to the backend
    client_body_buffer_size  10m;
}

WebSocket and Long-Polling #

location /ws/ {
    proxy_pass http://ws_backend;
    proxy_http_version 1.1;
    proxy_set_header Upgrade    $http_upgrade;
    proxy_set_header Connection $connection_upgrade;

    proxy_connect_timeout   10s;
    proxy_read_timeout    3600s;   # 1 hour for long-lived connections
    proxy_send_timeout    3600s;

    # Disable buffering for WebSocket
    proxy_buffering off;
}

Keepalive Pool to the Backend #

Every time Nginx opens a new connection to the backend, there’s TCP handshake overhead (3-way handshake) that takes time. With keepalive, Nginx maintains a pool of already-open connections that can be reused for subsequent requests.

upstream backend {
    server localhost:3000;

    # Keep a maximum of 32 idle connections to this backend
    # Connections exceeding 32 are closed after the request finishes
    keepalive 32;

    # How long keepalive connections are maintained
    keepalive_timeout 60s;

    # How many requests per connection before closing
    keepalive_requests 1000;
}

server {
    location / {
        proxy_pass         http://backend;
        proxy_http_version 1.1;          # Keepalive needs HTTP/1.1
        proxy_set_header   Connection ""; # Remove the Connection header from the client
    }
}

The Impact of Keepalive #

Without keepalive (every request):
  TCP Handshake (1 RTT) → Request → Response → TCP Close
  Total: ~2-3 RTT overhead per request

With keepalive (connection reuse):
  Request → Response (connection already exists, no handshake needed)
  Total: almost 0 overhead

For an application with 1000 requests/second to a local backend,
keepalive can save thousands of handshakes per second.
Error CodeCauseHow to Debug
502 Bad GatewayThe backend can’t be reached, connection refused, or proxy_connect_timeout exceededCheck whether the backend is running: curl localhost:3000
504 Gateway TimeoutThe backend took too long to respond — proxy_read_timeout exceededIncrease proxy_read_timeout or optimize the backend
413 Request Entity Too LargeRequest body exceeds client_max_body_sizeIncrease client_max_body_size for upload endpoints
499 Client Closed RequestThe client cut the connection before the backend finishedClient timeout on its side, or an unstable connection
# How to debug 502/504:

# 1. Check whether the backend is running
curl -v http://localhost:3000/api/health

# 2. Check the Nginx error log
tail -f /var/log/nginx/error.log | grep -E "502|504|timeout|connect"

# 3. Measure the backend response time directly
time curl http://localhost:3000/api/heavy-endpoint
# If > proxy_read_timeout, it will time out

# 4. Check whether this is a load problem (backend responsive alone but not under traffic)
ab -n 100 -c 10 http://localhost:3000/api/test

# 5. Add $upstream_response_time logging to see how long the backend took
log_format detailed '$remote_addr [$time_local] "$request" '
                    '$status $upstream_status '
                    '$upstream_response_time '  # backend time
                    '$request_time';            # total time including sending to client

Complete Buffer & Timeout Configuration #

http {
    # ─── Defaults for all proxies ────────────────────────────────────────────
    proxy_buffering         on;
    proxy_buffer_size       4k;
    proxy_buffers           16 4k;
    proxy_busy_buffers_size 8k;
    proxy_max_temp_file_size 1024m;
    proxy_temp_path         /var/cache/nginx/temp 1 2;

    proxy_connect_timeout   10s;
    proxy_read_timeout      60s;
    proxy_send_timeout      60s;

    # ─── Client timeouts ──────────────────────────────────────────────────────
    client_body_timeout     30s;
    client_header_timeout   30s;
    keepalive_timeout       75s;
    send_timeout            60s;

    upstream app_backend {
        server localhost:3000;
        keepalive 32;
        keepalive_timeout 60s;
        keepalive_requests 1000;
    }

    server {
        listen 443 ssl;

        # ─── Regular API ────────────────────────────────────────────────────────
        location /api/ {
            proxy_pass http://app_backend;
            proxy_http_version 1.1;
            proxy_set_header Connection "";
            proxy_read_timeout 30s;  # Override the default for APIs
        }

        # ─── Heavy operations ────────────────────────────────────────────────────
        location /api/export/ {
            proxy_pass http://app_backend;
            proxy_http_version 1.1;
            proxy_set_header Connection "";
            proxy_read_timeout     300s;
            proxy_buffer_size       64k;
            proxy_buffers           8 64k;
        }

        # ─── SSE / Streaming ──────────────────────────────────────────────────
        location /events/ {
            proxy_pass http://app_backend;
            proxy_http_version 1.1;
            proxy_set_header Connection '';
            proxy_buffering off;
            proxy_read_timeout 3600s;
        }

        # ─── Upload ───────────────────────────────────────────────────────────
        location /upload/ {
            proxy_pass http://app_backend;
            client_max_body_size    100m;
            client_body_buffer_size  10m;
            proxy_send_timeout      300s;
            proxy_read_timeout       60s;
        }
    }
}

Buffer Tuning for Special Cases #

Applications with Large Response Headers #

Some frameworks send many headers (cookies, custom headers, debug info) that can exceed the default proxy_buffer_size (4k):

# Symptom: "upstream sent too big header" error
# Solution: increase proxy_buffer_size

location / {
    proxy_pass http://backend;

    # Increase the header buffer (default: 4k, usually sufficient)
    # Increase if you see "upstream sent too big header" errors
    proxy_buffer_size 16k;

    # Also adjust proxy_buffers if the response body is large
    proxy_buffers 8 16k;    # 8 buffers @ 16k = 128k total
    proxy_busy_buffers_size 32k;
}

# Diagnose: check whether buffer-related errors appear in the log
grep -i "upstream sent too big header" /var/log/nginx/error.log

# Measure the backend response header size
curl -v http://localhost:3000/api/test 2>&1 | grep -E "^<"
# Count the total header size

Micro-Buffering for Low Latency #

For very latency-sensitive APIs (fintech, gaming), you can optimize so data is immediately forwarded to the client:

location /api/realtime/ {
    proxy_pass http://realtime_backend;

    # Minimal buffering — forward to the client immediately
    proxy_buffer_size 4k;
    proxy_buffers 2 4k;      # Only 8k total
    proxy_busy_buffers_size 4k;

    # Disable temp files (no disk write delays allowed)
    proxy_max_temp_file_size 0;

    # TCP_NODELAY: send packets immediately without waiting for the Nagle algorithm
    # (usually on by default)
    tcp_nodelay on;
}

Monitoring Buffer Usage #

# Check whether Nginx is using temp files (a sign of full buffers)
ls -la /var/cache/nginx/temp/
# If there are many files here, proxy_buffers needs to be increased
# or there are very slow clients downloading

# Check active connections
nginx -s status
# Or if stub_status is available:
curl http://localhost/nginx_status

# Monitor Nginx memory usage (related to buffering)
cat /proc/$(pgrep -o nginx)/status | grep -E "VmRSS|VmPeak"

Summary #

  • Buffering on (default): the backend is free faster, Nginx handles delivery to slow clients — good for all regular web apps. Disable with proxy_buffering off for SSE, streaming, or WebSocket.
  • proxy_connect_timeout: connection-open timeout — triggers when the backend isn’t running. proxy_read_timeout: idle timeout between packets from the backend, not total time — increase for heavy operations.
  • proxy_read_timeout 300s for endpoints processing reports/exports; the default 60s is too short for these.
  • Keepalive in the upstream block (keepalive 32) + proxy_http_version 1.1 + proxy_set_header Connection "" saves thousands of TCP handshakes per second.
  • 502 = backend unreachable; 504 = backend too slow; 499 = client disconnected first — each needs different handling.
  • Use $upstream_response_time in your log format to measure how long the backend actually takes to respond.

← Previous: Proxy Headers   Next: Proxy Cache →

About | Author | Content Scope | Editorial Policy | Privacy Policy | Disclaimer | Contact