Log files can be a source of invaluable information for debugging and diagnosing problems. They are usually written in plain text, which is easier to read than binary logs from other servers. This article will cover the basics of what log file analysis should entail, how they’re used by web developers and experts, where to find them on your server and why not just log everything yourself with access_logs.
The “nginx access log example” is a good way to understand what the logs contain. It is also a great idea to review your logs on a regular basis so that you can keep track of new issues and changes in your web server.
NGINX logging is sometimes neglected as a key component of the web service, and it is typically used only when an issue occurs. However, it’s critical to understand how to set NGINX logs and what information is deemed the most significant from the start.
The error log and the access log are two sorts of logs accessible in NGINX. What format would you use for the error and access logs, and how would you set them? Continue reading to discover all there is to know about NGINX logging!
PowerShell Logging: Everything You Need to Know About Recording and Auditing
Prerequisites
To follow along with this guide, you’ll need a functional NGINX installation, preferably version 1.21.1 or above. The NGINX installation is hosted by Ubuntu in this guide. The jq program may be used to display structured JSON log file output in the terminal.
Getting to Know NGINX’s Logging System
There are a lot of moving elements in the NGINX logging system. Logging consists of log formats (how logs are saved) and an NGNIX configuration file (nginx.conf) that enables and tunes log generation.
Let’s start with the NGINX configuration file. Within the NGINX documentation, an NGNIX configuration file specifies a hierarchy of parts known as contexts. Although not all potential contexts are given below, these contexts are made up of a mix of the following.
- The root of the nginx.conf file is the “main” context.
- The context of http
- There are many server contexts.
- Various geographical situations
You may specify access log and error log configuration items, or directives, inside one or more of these contexts. A logging directive specifies how NGINX should handle logs in each context.
Getting Started with NGINX on Docker is a related article.
Logging Directive Structure in NGINX
The log name, the place to store the log, and the level of log data to save are all configured under each context.
<log name> <log location> <logging level>;
- Log Storage – Logs may be saved to a file, such as /var/log/nginx/error.log, a syslog, such as syslog:server=unix:/var/log/nginx.sock, or a cyclic memory buffer, such as memory:32m.
- Debug, info, notice, warn, error, crit, alert, or emerg are the logging levels available, with error being the default. Unless NGINX was built with the —with-debug switch, the debug level may not be accessible.
Contexts for Logging Directives That Are Allowed
Only some scenarios enable the use of the error log and access log directives. In the main, http, mail, stream, server, and location contexts, error log is permitted. In the http, server, location, if in location, and limit except contexts, the access log directive is permitted.
Logging directives override higher-up directives. For example, the error_log directive specified in a location context will override the same directive specified in The context of http.
Below is an example setup that includes a number of specified directives.
# Log all warning and higher level errors to a file on disk error log /var/log/nginx/error.log warn; http access log /var/log/nginx/access.log combined; server access log /var/log/nginx/domain1.access.log combined; location access log /var/log/nginx/domain1.access.log combined # syslog:server=unix:/var/log/nginx.sock,facility=local7,tag=nginx notice; syslog:server=unix:/var/log/nginx.sock,facility=local7,tag=nginx notice; syslog:server=unix:/var/log/nginx.sock,facility=local7,tag=nginx /var/log/nginx/domain2.access.log combined; server access log; location # Store all information and higher error messages in memory, but only up to 32 Mb error log memory:32m info;
The Access Log Directive and Log Formats
Each access request to NGINX is reported in addition to NGINX error logs. An access request might range from requesting a particular web page to requesting a single picture. As you may expect, the logged requests might include a great deal of information.
The access log directive is used by NGNIX to keep track of generic NGINX request activity. You may set NGINX access logs to store in a specific format, unlike the error log directive, which has a standard format.
The Access log Log Format by Default
NGNIX can save data from access logs in a variety of log formats. That log format is known as mixed by default. If you don’t specify a log format in your NGINX configuration file, NGNIX will log all requests using the structure below.
‘”$request” $status $body bytes sent””$http referer” “$http user agent”‘
An example of the combined format in action is shown below.
127.0.0.1 – – [10/Oct/2020:15:10:20 -0600] “HEAD / HTTP/1.1” 200 0 “<https://example.com>” “Mozilla/5.0…”
Customizing access log Formats with the log format Directive
The default combined NGINX log format may suit your requirements fine, but what if you want to include more data, such as upstream service details, or utilize JSON format instead? You’ll need to use the log format directive to create a custom log format.
The log format directive enables you to declare many distinct access log formats in the configuration file, which will be utilized across all situations.
Below is an example of a log format definition that specifies several distinct fields and variables. You may select to show different fields in this example’s JSON logging format.
Check out the NGINX manual for a complete list of accessible variables.
The json text following the log format directive is just the name that any access log directive that chooses to utilize this format would reference. Multiple logging output formats may be established and utilized by any combination of access log directives in the NGINX configuration file using log format.
log format json escape=json “”time”: “$time iso8601″,””remote addr”: “$remote addr”,””remote user”: “$remote user”,””ssl protocol cipher”: “$ssl protocol/$ssl cipher”,””body bytes sent”:
The log_format may only be used in The context of http, but referenced by any access_log directive regardless of location.
Getting Rid of Log Output
You may need to escape variables specified in JSON to be considered as literal parts in the NGNIX configuration file when defining log format through JSON, for example. You may achieve this by using escape formats like default, json, and none. The default format is utilized if the escape command is not used.
- default – All characters with ISO values less than 32 and larger than 126 will be escaped as “x##.” A hyphen (-) will be reported if no variable value is detected.
- json — In the JSON string format, all forbidden characters will be escaped.
- none – All value escaping is disabled.
In the sample above, you’ll notice NGNIX uses the json escape format (escape=json) to escape all JSON variables.
Access log Directives Configuration
You must activate the access log directive for NGNIX to start recording access activities in the fancy log format you created before.
After you’ve specified the log format, you’ll need to activate it in the NGINX configuration file, similar to how you’d enable the error log directive.
An example of a common access log directive is provided below, which writes access logs to a file (/var/log/nginx/access.log) in the previously stated json log format. Then, in a particular scenario when the directive is used, the special off argument disables access logging.
access log off; access log /var/log/nginx/domain.access.log json; access log /var/log/nginx/domain.access.log json; access log /var/log/nginx
Maybe you’ve set up an access log for a domain. What would you do if you wanted to inspect the output of the directive below?
/var/log/nginx/domain.access.log json; access log /var/log/nginx/domain.access.log json;
Run the Linux cat command to retrieve the file contents and pass the output to the tail command to display just a single line to illustrate NGINX transmitting log output as stated by the access log directive. Then, to neatly style the JSON output, feed the single line to the jq program.
jq /var/log/nginx/domain.access.log | cat /var/log/nginx/domain.access.log | tail -n 1
Both the memory and syslog formats, like the error log, operate in addition to the regular file output.
An NGINX access log generates a JSON log file.
NGINX Buffering Disk Writes Configuration
Because access logging often produces significantly more data than error logging, extra capabilities for compression and buffering of log data to disk writes are offered, although disabled by default. Tell NGINX to buffer disk writes to prevent continual disk writes and possibly request blockage while waiting for disk IO.
Below is an example of an access log directive that defines the gzip, buffer, and flush parameters.
gzip=7 buffer=64k flush=3m access log /var/log/nginx/domain.access.log;
- A buffer is a device that momentarily holds data before transmitting it to another location. The default buffer size is 64k, but you may change it by including a value in the directive, such as buffer=32k instead of simply buffer.
- gzip — Sets the compression level of GZIP from 1 to 9, with 9 being the slowest but maximum compression level. For example, gzip’s default compression level is 1, but you’ll increase it to 9 (gzip=9).
If you use gzip without buffering, the writes will be buffered by default. Because log entries cannot be sent to disk due to the nature of GZIP compression, disk buffering is necessary.
- flush – To prevent keeping in-memory logs for seldom viewed sites forever, you’ll set a flush time after which any logging data will be written to disk. Flush=5m, for example, forces all recorded data to be written to disk, even if the buffer has not yet been full.
Conditional Logging of Access Entries
There will be occasions when you simply need to record a single access request. Instead of recording all requests, including HTTP/200 (successful requests), you could simply want to report HTTP/404 requests (file not found requests). If this is the case, you may use the if option in the access log directive to establish a logging condition.
To continue logging, the access log directive’s if= argument searches for values handed in by the related variable that are not “0” or an empty string.
For example, you could want to tell NGNIX to only record HTTP access requests that begin with a 4 in the HTTP code.
In the NGINX configuration file, add the following:
Create a map directive to assign a value of 0 or 1 to a variable based on the evaluated condition. The first regular expression searches for HTTP status codes that do not begin with 4. For any values that do not fulfill that criteria, the default condition is used as a fallback.
$status $logged 0; default 1; map $status $logged 0; default 1;
The map directive must be defined at The context of http level. You may use the map directive output variable, shown below as $logged, further in the configuration file and not confined to The context of http level.
After you’ve created the map directive, which will give the $logged variable a value of 0 or 1, you may use it in conditions like the ones below. Using the if argument, you’re directing NGINX to only report activity to the access 404.log file if a request begins with the number 4.
/var/log/nginx/access 404.log json if=$logged; access log /var/log/nginx/access 404.log json;
Conclusion
You may start monitoring your NGINX installation for faults and user-facing problems now that you know how to report errors and access requests in a number of methods.
What comes next? Try feeding the logs’ data into a SIEM program for examination!
NGINX logs are a great way to troubleshoot and analyze your website. NGINX logs can be found in the “nginx” directory at “/var/log”. Reference: nginx logs ubuntu.
Frequently Asked Questions
Related Tags
- nginx log format
- nginx logs location ubuntu
- nginx error logs
- nginx log to stdout
- nginx combined log format
