Access Log #

Every time a client sends an HTTP request to our server — whether it’s fetching an HTML page, uploading a file, calling an API, or loading a small image — Nginx processes it and automatically records the event’s details into the Access Log.

The access log is one of the most important instruments for a system administrator and developer. Through the access log, we can monitor traffic volume, detect suspicious activity (like security hole scanning by bots), analyze user behavior, and diagnose application performance problems. In this article, we’ll dissect in depth how Nginx handles the access log, the anatomy of the default format, modular storage configuration per virtual host, memory optimization techniques using buffering, and how to dynamically filter log entries with conditional logging.

How Does Nginx Record the Access Log? #

Architecturally, Nginx processes incoming connections using the non-blocking event-driven model. When a request finishes processing and the response is sent back to the client, Nginx doesn’t just stop the workflow there. The Nginx worker process immediately formulates a log line matching the defined format, then writes it to the target log file in our local storage system.

This log write happens at the end of the request cycle. This is important to understand because the recorded variables (like the size of data sent or processing time) can only be known with certainty after the entire response data has been successfully transmitted.

Anatomy of the Default Log Format: combined #

By default, Nginx uses a format named combined. This format is a replica of the industry standard popularized by the Apache Web Server, so almost all third-party log analysis tools (like GoAccess, AWStats, Logstash, or Splunk) can read and interpret it directly without additional parser configuration.

Here’s the combined format definition declared in Nginx’s global http configuration block:

http {
    log_format combined '$remote_addr - $remote_user [$time_local] '
                        '"$request" $status $body_bytes_sent '
                        '"$http_referer" "$http_user_agent"';
}

Dissecting the combined Format Variables #

Let’s break down each variable used in that default format:

• $remote_addr: Stores the real client IP address that made the direct connection to the server. If the client is behind a proxy without additional configuration, this variable will contain the proxy’s IP.
• $remote_user: Records the username if the page or location is protected by the HTTP Basic Authentication module. If there’s no authentication, this part is filled with a dash character (-).
• $time_local: Shows the server’s local time when the log entry was written, using the standard Common Log Format (e.g., 15/Jun/2026:13:18:13 +0700).
• $request: Contains the client’s full original HTTP request line, covering the HTTP method (GET, POST, etc.), the requested URI/path, and the protocol used (e.g., "GET /index.html HTTP/1.1").
• $status: The HTTP response status code returned by the server to the client (like 200 for success, 404 for not found, or 500 for internal server error).
• $body_bytes_sent: The number of bytes sent to the client as the response body. This value doesn’t include HTTP header sizes. This value is more accurate than the $bytes_sent variable because it reflects the actual payload load the client downloaded.
• $http_referer: Contains the origin page URL (referrer) where the client found the link to our current page. Very useful for traffic origin analysis.
• $http_user_agent: Contains information about the browser application, operating system, and device used by the client to access our server.

Example of Reading a Log Line #

Suppose we see the following line in the /var/log/nginx/access.log file:

203.0.113.88 - budi [16/Jun/2026:13:20:00 +0700] "GET /api/v1/profile HTTP/2.0" 200 4096 "https://blog.unisbadri.com/home" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36..."

From the line above, we can deduce the following information:

1. The request was sent from IP address 203.0.113.88.
2. The user was authenticated as budi via Basic Auth.
3. The request was processed on 16 June 2026 at 13:20:00 server local time (GMT+7 timezone).
4. The client accessed the /api/v1/profile endpoint using the GET method over HTTP/2.0.
5. The server returned status 200 (OK).
6. The profile payload size sent was 4096 bytes (4 KB).
7. The user came from the external page https://blog.unisbadri.com/home.
8. The user was using a Chrome browser on the macOS operating system.

Configuring the access_log Directive #

The access_log directive is used to enable, disable, or redirect log recording to a specific file path using the desired format. This directive is very flexible because it can be placed at several context levels: http, server, location, as well as inside limit_except or if blocks.

Basic Syntax #

access_log path [format [buffer=size] [gzip[=level]] [flush=time] [if=condition]];

Here are some common scenarios for applying the access_log directive:

http {
    # 1. Enable global logging with the default 'combined' format
    access_log /var/log/nginx/access.log;

    server {
        listen 80;
        server_name example.com;

        # 2. Redirect logging specific to this virtual host to a separate file
        access_log /var/log/nginx/example.com-access.log combined;
        
        # 3. Turn off the access log entirely for this server block (Not recommended in production)
        # access_log off;
    }
}

Disabling Logging to Reduce Disk I/O #

On servers with limited storage capacity or high disk write loads, we often want to avoid logging requests considered less important or occurring very frequently. The most common scenario is turning off logging for static asset loads (like images, CSS files, and JavaScript files) as well as load balancer healthcheck endpoints.

Here’s the recommended configuration for disabling logging on certain locations:

server {
    listen 80;
    server_name myapp.com;
    access_log /var/log/nginx/myapp-access.log;

    # Ignore logging for healthcheck requests from Load Balancers / Kubernetes
    location /healthz {
        access_log off;
        default_type text/plain;
        return 200 'OK';
    }

    # Ignore logging for static assets to save disk capacity
    location ~* \.(js|css|png|jpg|jpeg|gif|ico|woff|woff2|svg)$ {
        access_log off;
        expires 30d;
        add_header Cache-Control "public, no-transform";
    }
}

By configuring access_log off; on static file locations, we not only save disk space, but also reduce input/output operation load (I/O bottlenecks) on our server’s storage media.

Isolating Logs per Virtual Host (Server Block) #

In production environments serving multiple domain names (multi-tenant), piling all log entries into one main file /var/log/nginx/access.log is a fatal mistake. When one site experiences a failure or suspicious traffic surge, we’ll struggle to separate the data without heavy regex processing help.

Therefore, we should always isolate logs for each virtual host into their own files.

# Virtual Host 1 Configuration: blog.unisbadri.com
server {
    listen 443 ssl;
    server_name blog.unisbadri.com;

    ssl_certificate /etc/letsencrypt/live/blog.unisbadri.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/blog.unisbadri.com/privkey.pem;

    # Isolate the Access Log and Error Log specifically for the blog
    access_log /var/log/nginx/blog.unisbadri.com-access.log combined;
    error_log /var/log/nginx/blog.unisbadri.com-error.log warn;

    root /var/www/blog;
    index index.html;
}

# Virtual Host 2 Configuration: api.unisbadri.com
server {
    listen 443 ssl;
    server_name api.unisbadri.com;

    ssl_certificate /etc/letsencrypt/live/api.unisbadri.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/api.unisbadri.com/privkey.pem;

    # Isolate the Access Log with a custom format for the API
    access_log /var/log/nginx/api.unisbadri.com-access.log combined;
    error_log /var/log/nginx/api.unisbadri.com-error.log error;

    location / {
        proxy_pass http://api_backend;
    }
}

Separating logs like this makes it easier to monitor logs in real-time on a specific domain using commands like tail -f /var/log/nginx/blog.unisbadri.com-access.log.

Log Buffering for High Traffic (High Performance Logging) #

By default, every time Nginx finishes a request, it immediately performs a write system call to the log file on disk. On servers serving thousands of requests per second, continuous synchronous disk writes can cause long disk I/O queues (disk I/O wait), which ultimately hampers the server’s overall response performance.

To solve this problem, Nginx provides the Memory Buffering feature. By enabling the buffer, Nginx holds log entries in RAM first before writing them collectively to disk.

Here’s an illustration of how Nginx log buffering works:

flowchart LR
    Client1("Client Request 1") --> Nginx("Nginx Worker")
    Client2("Client Request 2") --> Nginx
    Client3("Client Request 3") --> Nginx
    Nginx --> Buffer{"Memory Buffer (32KB)"}
    Buffer -->|Not Full & < 5 Seconds| Buffer
    Buffer -->|Full OR 5s Time Limit Exceeded| Disk[("Disk Storage (access.log)")]

    classDef default fill:#f9f9f9,stroke:#d1d5db,stroke-width:1px,color:#111827;
    classDef nodeStyle fill:#eff6ff,stroke:#3b82f6,stroke-width:2px,color:#1e3a8a;
    class Nginx,Buffer nodeStyle;

Buffering Configuration #

We can enable the buffer by adding the buffer parameter to the access_log directive. We also highly recommend including the flush parameter to guarantee log entries are still written periodically even if the buffer capacity hasn’t been fully reached.

http {
    # Define the custom or default log format
    log_format combined '$remote_addr - $remote_user [$time_local] '
                        '"$request" $status $body_bytes_sent '
                        '"$http_referer" "$http_user_agent"';

    # Enable a 32 Kilobyte buffer
    # If the buffer isn't full yet, force a disk write after 5 seconds
    access_log /var/log/nginx/access.log combined buffer=32k flush=5s;
}

• buffer=32k: Nginx holds log entries in RAM and only writes them to disk when the collected log data reaches 32 Kilobytes.
• flush=5s: Guarantees that if traffic is quiet and the 32KB buffer isn’t full, Nginx forces writing all buffered log data to disk every 5 seconds. This prevents log data from lingering too long in memory, which could make real-time monitoring inaccurate.

Buffering Trade-Off Analysis #

Applying log buffering has pros and cons we must weigh based on our system’s needs:

[!TIP] For medium to large-scale production environments, enabling buffer=32k flush=5s is a very logical decision because the CPU and disk I/O efficiency gains are far more valuable than the potential loss of the last few seconds of logs during a total hardware crash.

Conditional Logging with the if Parameter #

Sometimes, turning off logging by statically matching location blocks isn’t enough. We might need more dynamic logic to decide whether a request deserves to be logged or not. Nginx accommodates this need through the if parameter on the access_log directive.

The if parameter only accepts variables whose value is evaluated by Nginx. If the variable’s value is an empty string ("") or the number 0, the request won’t be logged. If it’s anything else, the request will be logged.

We usually combine the map directive to dynamically set this decision variable’s value based on other Nginx request variables.

Scenario 1: Only Log Error Responses (Status 4xx and 5xx) #

If we only want to monitor traffic anomalies (like broken pages or API failures) without recording regular successful transactions, we can create a status code filter:

http {
    # Map HTTP response status codes to the $log_only_errors variable
    map $status $log_only_errors {
        ~^[23]  0;  # If the status starts with 2 or 3 (success/redirect), set to 0 (don't log)
        default 1;  # Otherwise (4xx, 5xx), set to 1 (log it)
    }

    server {
        listen 80;
        server_name static.example.com;

        # Use the if parameter for conditional logging
        access_log /var/log/nginx/error-traffic.log combined if=$log_only_errors;
    }
}

Scenario 2: Only Log Slow Requests (Slow-Log Performance) #

This scenario is very useful for detecting our API endpoints that take a long processing time on the backend side. We can detect requests based on the $request_time variable (total request processing time in seconds with millisecond precision):

http {
    # Map the request time to the $is_slow_request variable
    map $request_time $is_slow_request {
        ~^[0]\.[0-8] 0; # If the request time is below 0.9 seconds, set to 0 (ignore)
        default      1; # If the request time >= 0.9 seconds (almost 1 second or more), set to 1 (log)
    }

    log_format performance_log '$remote_addr - [$time_local] "$request" '
                               '$status rt=$request_time urt=$upstream_response_time';

    server {
        listen 80;
        server_name api.unisbadri.com;

        # Log all requests to the main log file
        access_log /var/log/nginx/api-access.log;

        # Log slow requests to a special log file for developer team optimization needs
        access_log /var/log/nginx/api-slow.log performance_log if=$is_slow_request;
    }
}

Through this method, our developer team can directly open the /var/log/nginx/api-slow.log file to identify slow database queries or memory problems in the application without being disturbed by millions of lines of fast successful transaction logs.

Practical Access Log Analysis via CLI #

After successfully configuring and collecting data in the access log, we need to know how to read and extract valuable information from the file quickly directly from the terminal. We can use combinations of built-in Unix text utility commands like awk, grep, sort, uniq, and head.

Here’s a collection of very useful CLI commands for quick troubleshooting on the server:

1. Finding the 10 Most Active IP Addresses (Indication of Potential DDoS/Scraping Attacks) #

# The first column ($1) in the combined format is the client IP ($remote_addr)
awk '{print $1}' /var/log/nginx/access.log | sort | uniq -c | sort -rn | head -n 10

• How it works: This command takes the first column from the log file, sorts it so the same IPs group together, counts unique occurrences per IP (uniq -c), then sorts them in reverse by highest hit count (sort -rn), and shows the top 10 lines.

2. Finding the 10 Most Frequently Accessed Endpoints/URIs #

# The seventh column ($7) in the combined format usually contains the request path (URI)
awk '{print $7}' /var/log/nginx/access.log | sort | uniq -c | sort -rn | head -n 10

3. Monitoring HTTP Status Code Occurrence Counts (Checking Application Health) #

# The ninth column ($9) contains the HTTP status code ($status)
awk '{print $9}' /var/log/nginx/access.log | sort | uniq -c | sort -rn

If we see the number of 500 or 502 statuses suddenly spike, that’s a strong indicator that our backend application is experiencing crashes or running out of database connections.

4. Tracking the Most Active User Agents (Identifying Bots / Scrapers) #

# The 12th column and beyond contains the User Agent, we use the quote marker
awk -F'"' '{print $6}' /var/log/nginx/access.log | sort | uniq -c | sort -rn | head -n 10

5. Monitoring Logs in Real-time While Filtering Errors #

If we’re doing maintenance and want to see only failed requests live:

tail -f /var/log/nginx/access.log | grep --line-buffered -E " 4[0-9]{2} | 5[0-9]{2} "

Summary and Best Practices #

• Separate Logs per Domain: Always declare the access_log directive in every virtual host server block so logs don’t mix into one confusing giant file.
• Use Buffering in Production: Apply the buffer=32k flush=5s parameters to reduce disk I/O write load on high-traffic servers.
• Turn Off Unneeded Logs: Use access_log off; inside location blocks for static asset files and healthcheck endpoints to minimize log spam.
• Use Conditional Logging: Use the if= parameter combined with the map directive to dynamically filter data, like isolating slow request or error request recording.
• Secure File Access Permissions: Make sure Nginx log files can only be read by administrative users. By default, log directories are usually given 0750 permissions with adm or www-data group ownership.

← Previous: Security Headers   Next: Error Log →