This article will show you how to set up a centralized logging rsyslog configuration. This is helpful if you have many servers or need a central log management location for things like the Apache web server and other important systems. For this example, we are going to use Linux as our target operating system.,
The “rsyslog configuration example” is a document that describes how to create a centralized logging rsyslog configuration. The document also includes an explanation of what the different fields are for.
Although logs are an important aspect of applications and server architecture, examining them may be difficult if they are disorganized. Why don’t you set up a central logging system? Rsyslog might be just what you’re searching for!
A centralized logging system collects logs from numerous servers/applications and sends them to a central log server. In this video, you’ll learn how to set up centralized logging on Linux servers using the Rsyslog setup.
Prepare yourself and dive right in!
Prerequisites
This will be a hands-on presentation in this course. If you want to follow along, make sure you have the following items on hand:
- Two or more Linux servers are required — The central-rsyslog, client01, and client02 Debian 11 servers used in this tutorial are central-rsyslog, client01, and client02.
Related: [Step-by-Step] How to Install Ubuntu 20.04
- On all servers, a user has sudo/root capabilities.
Performing a Rsyslog Service Check on All Servers
Rsyslog is a high-performance log processing system for Linux distributions that comes pre-installed on Debian- and RHEL-based systems.
Rsyslog is a modernized version of the Syslog protocol that is both dependable and extensible. Multiple inputs and outputs, modularity, and comprehensive filtering capabilities are among the extra features.
The most recent version of Rsyslog is v8.2112.0 as of this writing. As a result, you’ll be verifying the version of Rsyslog installed on your system as well as the health of the Rsyslog service. This guarantees that you have the most up-to-date features of Rsyslog and that it is functioning smoothly.
1. Open a terminal window, log in to all servers, and perform the sudo su command to get root access. When asked, enter your user’s password.
2. Check the Rsyslog version installed on your system using the rsyslogd command.
Make sure you have Rsyslog v8.xxx installed on your system since certain options are only accessible with this version.
# rsyslogd -v rsyslogd -v rsyslogd -v rsyslogd -v rsy
The default Rsyslog v8.2102.0 is installed on the Debian 11 system, as shown below.
Checking the version of Rsyslog
3. Finally, test the Rsyslog service using the systemctl command below. Ensure that the Rsyslog service is operating (running).
# sudo systemctl status rsyslog to check the status of the rsyslog service
The Rsyslog service is operational, as seen below (running).
Checking to see whether the Rsyslog service is up and running
Controlling Systemd services using Ubuntu systemctl is a related topic.
Using Rsyslog to Create a Centralized Logging System
Now that you’ve installed the most recent version of Rsyslog, you may use the Rsyslog setup on the central-rsyslog server to set up centralized logging.
You’ll activate the Rsyslog UDP input module imudp and establish the Rsyslog template for accepting log messages from other servers to set up centralized logging. The imudp input module enables the central-rsyslog server to accept Syslog messages over the UDP protocol.
1. Using your chosen editor, open the Rsyslog configuration file (/etc/rsyslog.conf) and activate the settings indicated in the picture below, then save and quit the editor.
Uncomment each setting to enable it by deleting the # symbol at the beginning of each line throughout this guide.
This setting allows the imudp input module to use the default UDP port 514.
The Rsyslog imudp input module is enabled.
2. Using your chosen editor, build a new Rsyslog template (/etc/rsyslog.d/50-remote-logs.conf) and fill it with the settings below. Exit the editor after saving your changes.
The central-rsyslog server may store logs from distant servers in the /var/log/remotelogs/ directory using the Rsyslog template below.
# design a template for remote login # logs will be kept in the /var/log/remotelogs directory # each host will have its own directory depending on the system percent HOSTNAME percent # the log file’s name will be percent PROGRAMNAME percent.log such as sshd.log, su.log # The Rsyslog message attributes template (name=”RemoteLogs” type=”string” string=”/var/log/remotelogs/ percent HOSTNAME percent / percent PROGRAMNAME percent.log”) has both percent HOSTNAME percent and percent PROGRAMNAME percent. # copy all log messages from all facilities to the RemoteLogs template at all severity levels *.* -? RemoteLogs # after the file is written, halt the process
3. Use the following commands to establish a new log directory (/var/log/remotelogs/) for the root user with the adm group. The Ryslog service will now be able to write logs to the /var/log/remotelogs directory.
# mkdir -p /var/log/remotelogs/ # construct the /var/log/remotelogs/ directory To change the ownership to root, do the following: sudo chown -R root:adm /var/log/remotelogs/ adm sudo chown -R root:adm /var/log/remotelogs/
4. Verify the Rsyslog settings (/etc/rsyslog.conf and /etc/rsyslog.d/50-remote-logs.conf) using the rsyslogd instructions following.
Always review and validate the Rsyslog settings after making changes to ensure there are no errors.
rsyslogd -N1 -f /etc/rsyslog.conf # validate /etc/rsyslog.conf # rsyslogd -N1 -f /etc/rsyslog.d/50-remote-logs.conf rsyslogd -N1 -f /etc/rsyslog.d/50-remote-logs.conf rsyslogd -N1 -f /etc/rsyslog.d/50-remote-logs.conf
If there are no errors, the output will look like this.
Rsyslog Configuration Checking and Verification
5. Run the command below to restart the rsyslog service and apply the updated configurations after checking the settings.
The Rsyslog service is already up and running, with the imudp input module enabled and the Syslog protocol’s default UDP port 514 open. The central-rsyslog server may now accept log messages from other servers.
sudo systemctl restart rsyslog # restart rsyslog
6. Finally, use the ss command to check that your system’s ports are open.
# ss -plntu ss -plntu ss -plntu ss
The Rsyslog service with the imudp input module enabled is operating on the default UDP port 514, as shown below.
Checking if the Rsyslog service is running on UDP Port 514
Using a Centralized Rsyslog Server to Send Logs
You’ve already set up centralized log management using the central-rsyslog server’s Rsyslog configuration. How do you know the central-rsyslog server is receiving logs, though? By activating and setting the Rsyslog output module on the client computer, you’ll be able to submit logs remotely to the centralized Rsyslog server (central-rsyslog).
The client computer (with the hostname client01) sends logs to the central-rsyslog server using the Rsyslog output module omfwd in this sample.
The omfwd module is a built-in module that uses Rsyslog templates to handle messages and logs. Using the Rsyslog action object, the module then sends the messages to different destinations through UDP and TCP.
1. Set up and deliver logs to the central-rsyslog server on the client01 computer.
2. Using your favourite editor, create a new Rsyslog configuration (/etc/rsyslog.d/20-forward-logs.conf) and fill the following settings.
This setup uses the SendRemote template to prepare log messages and sends them to the target central-rsyslog server over UDP protocol (172.16.1.10). Make careful to replace the IP address with the IP address of your central Rsyslog server.
An Rsyslog template is a collection of Rsyslog message characteristics that define log message forms. To examine several sorts of templates, look at the documentation for the Rsyslog template.
# process all log messages before sending # with the SendRemote template template( name=”SendRemote” type=”string” string=”<%PRI%>%TIMESTAMP:::date-rfc3339% %HOSTNAME% %syslogtag:1:32%%msg:::sp-if-no-1st-sp%%msg%” ) # forward log messages using omfwd module # to the target server 172.16.1.10 # via UDP porotocol on port 514 # log messages is formatted using the SendRemote template # setup queue for remote log action( type=”omfwd” Target=”172.16.1.10″ Port=”514″ Protocol=”udp” template=”SendRemote” queue.SpoolDirectory=”/var/spool/rsyslog” queue.FileName=”remote” queue.MaxDiskSpace=”1g” queue.SaveOnShutdown=”on” queue.Type=”LinkedList” ResendLastMSGOnReconnect=”on” ) # stop process after the file is written stop
3. Run the rsyslogd command below to check the Rsyslog setup and make sure there are no errors.
# double-check the rsyslog configuration rsyslogd -N1 -f /etc/rsyslog.d/20-remote-logs.conf rsyslogd -N1 -f /etc/rsyslog.d/20-remote-logs.conf
Rsyslog Configuration Verification
4. To restart the Rsyslog service and apply a new configuration, use the systemctl command below.
# rsyslog restart on client01 sudo systemctl restart rsyslog
5. Return to the central-rsyslog server and check the logs from the client01 computer.
To verify that the client01 directory exists, use the following commands to go to the /var/log/remotelogs directory and display the files and folders within that directory.
# Go to the /var/log/remotelogs folder. cd /var/log/remotelogs/ /var/log/remotelogs/ /var/log/remotelog # verifying the folders that are available ls -lah ls -lah ls -lah
The client01 directory exists in the /var/log/remotelogs directory, as seen below.
Verifying the existence of the client01 directory
6. Switch to the client01 directory and list (ls -lah) the log files within it using the commands shown below.
# cd client01/ to go to the client01 directory # ls -lah checking files in client01 directory
Administrators may use a centralized logging system to evaluate log data from hundreds of servers on a single workstation.
Select one of the logs to examine and evaluate.
Getting a List of Log Files in the client01 Directory
7. Finally, use the cat command to see the contents of your target log. However, this sample returns the content of the sshd.log file. Replace logfile with the name of the log file.
The sshd.log file on the Rsyslog server contains logs from the client01 computer, which you can see below (central-rsyslog).
On the Rsyslog Server, verifying logs from client01
So far, you’ve seen how a centralized logging system protects log files by storing them on a central server even if apps or servers are stopped.
Logs from a specific program are sent to the Rsyslog server.
You’ve just forwarded all of the client01 machine’s application logs to the central Rsyslog server (central-rsyslog). But what if you just want logs from a single program, such as ssh or su? On the Rsyslog action object, implement the Rsyslog rainerscript control structure.
The client02 computer is used in this demonstration to deliver solely sshd logs to the Rsyslog server (central-rsyslog).
1. Set up and deliver logs to the central-rsyslog server on the client02 computer.
2. Using your chosen editor, create a new Rsyslog configuration (/etc/rsyslog.d/50-sshd.conf) and fill in the following information.
This setup only sends logs to the Rsyslog server for the sshd application (programname) (172.16.1.10). The SendRemote template also formats the log messages.
# create new Rsyslog tempate for formatted logs template( name=”SendRemote” type=”string” string=”<%PRI%>%TIMESTAMP:::date-rfc3339% %HOSTNAME% %syslogtag:1:32%%msg:::sp-if-no-1st-sp%%msg%” ) # implementation of if conditional on rsyslog # to send specific logs for programname sshd # stop the process after sending logs if $programname == “sshd” then { action(type=”omfwd” target=”172.16.1.10″ port=”514″ protocol=”udp” template=”SendRemote”) stop }
3. Finally, execute the rsyslogd command to check the Rsyslog setup and make sure there are no errors.
# double-check the rsyslog configuration rsyslogd -N1 -f /etc/rsyslog.d/50-sshd.conf rsyslogd -N1 -f /etc/rsyslog.d/50-sshd.conf
Rsyslog Configuration Verification
4. To produce a fresh sshd log, use the following command to restart the Rsyslog service, apply a new configuration, and log out of the client02 server.
The central-rsyslog server has only received the sshd log so far, rather than all system logs.
# rsyslog service should be restarted sudo systemctl rsyslog restart
5. Return to the central-rsyslog server and check the logs from the client02 computer.
To list (ls -lah) accessible log files in the /var/log/remotelogs/ directory, use the scripts following.
# cd /var/log/remotelogs/ # ls -lah /var/log/remotelogs/ # validate log directory for client02
As you can see in the screenshot below, a new directory called client02 has been created to hold log messages from the client02 computer.
The /var/log/remotelogs/ directory contains a list of available log files.
6. Use the instructions below to browse to the client02 directory and check the available log files there.
For the software sshd, you’ll see the log file sshd.log. Also, use the cat command to see the contents of the sshd.log log file.
Because you utilized the /etc/rsyslog.d/50-sshd.conf setting, there is only one log file accessible, sshd.log.
On the Rsyslog server, verify that only sshd logs are received from the client02 machine.
7. Finally, to read and analyze the sshd.log file, use the cat program.
Viewing the Content of the sshd.log File
Using the Rsyslog Server to Send Syslog Message-Converted Logs
You’ve seen how to transmit logs to your Rsyslog server so far, and that’s sufficient. However, if you want to make your logs simpler to read and analyze, you could opt to have them in Syslog message format. If that’s the case, the Rsys imfile input module will suffice.
Any text messages/logs may be converted to Syslog message format using the imfile input module. In this demonstration, you’ll transfer Apache2 web server logs from the client01 PC to the Rsyslog server (central-rsyslog).
You may transmit any Syslog message-converted logs to the Rsyslog server by combining the Rsyslog input module (imfile) with the output module (omfwd) (central-rsyslog).
1. Return to the client01 computer and build a new Rsyslog configuration, /etc/rsyslog.d/50-apache2.conf, with the following settings. For your logs, make sure to modify the template name, ruleset name, and input file.
With the imfile module, the template below translates Apache logs to Syslog message format and delivers them to the Rsyslog server (172.16.1.10).
# deifne global workDirectory for saving the state file of log messages. global(workDirectory=”/var/spool/rsyslog”) # enable the Rsyslog imfile module processing text files or logs. module(load=”imfile” PollingInterval=”10″) # define template for apache2 for processing log messages. # that will be forwarded to rsyslog server template( name=”TestLog” type=”string” string=”<%PRI%>%TIMESTAMP:::date-rfc3339% %HOSTNAME% %syslogtag:1:32%%msg:::sp-if-no-1st-sp%%msg%” ) # define ruleset “ApacheLogs” with action object to send logs to rsyslog server # define the queue ruleset(name=”ApacheLogs”) { action( type=”omfwd” target=”172.16.1.10″ port=”514″ protocol=”udp” template=”TestLog” queue.SpoolDirectory=”/var/spool/rsyslog” queue.FileName=”remote” queue.MaxDiskSpace=”1g” queue.SaveOnShutdown=”on” queue.Type=”LinkedList” ResendLastMSGOnReconnect=”on” ) stop } # define input files Apache2 logs to send to the Rsyslog server # and apply ruleset “ApacheLogs” input(type=”imfile” ruleset=”ApacheLogs” Tag=”apache2″ File=”/var/log/apache2/*.log”)
2. Verify the /etc/rsyslog.d/50-apache2.conf setup using the rsyslogd command.
# double-check the rsyslog configuration rsyslogd -N1 -f /etc/rsyslog.d/50-apache2.conf rsyslogd -N1 -f /etc/rsyslog.d/50-apache2.conf
Rsyslog Configuration Verification
3. To restart the rsyslog service and apply changed settings, use the systemctl command below. All Apache2 logs on the client01 computer will be routed to the Rsyslog server at this point (central-rsyslog).
# rsyslog service should be restarted sudo systemctl rsyslog restart
4. To produce Apache2 logs, execute the curl command below. New Apache2 log messages will be sent to the file /var/log/apache2/access.log using this command.
# by using the curl command curl localhost curl localhost/testnotfound # to produce a new log for Apache
5. Return to the central-rsyslog Rsyslog server and execute the commands below to list accessible log files in the /var/log/remotelogs/client01 directory.
# cd /var/log/remotelogs/client01/ # go to the /var/log/remotelogs/client01/ directory # ls -a ls -a ls -a ls -a ls
The apache.log file from the client01 computer was received by the Rsyslog server (central-rsyslog), as shown below.
Available logs from the client01 machine are listed below.
6. Finally, use the cat command to see the contents of the apache2.log log file.
# cat apache2.log displays the contents of the apache2.log file
If /etc/rsyslog.d/50-apache2.conf is enabled, the contents of the log file will be shown in Syslog message format, as seen below.
Receiving Syslog Message-converted Logs from the client01 Machine and viewing them
Conclusion
You learnt how to set up centralized logging using Rsyslog setup in this tutorial. You’ve also used several sorts of Rsyslog input and output plugins to send server logs to the centralized Rsyslog server. You no longer need to look for logs in locations other than your Rsyslog server.
What more applications do you have in mind for Rsyslog? Start looking at additional Rsyslog input and output modules. Or connect Rsyslog to a different logging system like Fluentd or Logstash?
The “how to configure syslog server in linux” is a tutorial that will teach you how to set up a centralized logging Rsyslog configuration.
Frequently Asked Questions
How do I create a centralized log server?
A: This is not possible since this was removed from the Steam API.
How do you center logs with rsyslog Logstash and Elasticsearch?
A: You can use the pager plugin when setting up rsyslog.
How do I send logs to rsyslog?
A: You can use the following commands to send logs to rsyslog for analysis.
-1 icanotsendtoasyslog
-2 ihavealtsenttoasyslog
Related Tags
- configure rsyslog to send to remote server
- rsyslog remote logging to different files
- rsyslog forward specific log file
- rsyslog logs
- configure rsyslog to receive remote logs ubuntu
