Introduction
Earlier this year I was tasked with the project of creating a central logging server within our domain environment. The idea behind this is to have all of our server’s logs forwarded to one central server that will store, parse, and archive the logs. Through research, I found many different solutions. One of the best solutions happened to be the ELK stack by elastic. Although this solution is very pretty and offers heavy-duty log parsing abilities, I decided to take a different route with something more simple. I eventually stumbled across nxlog, which is a small application that runs as a service and ships logs from a source and repeats them to a destination of your choosing.
Once I had nxlog configured to ship logs to my central logging server, I went searching for a solid product that will display these logs and manage them automatically. I came across Kiwi Syslog Server created by SolarWinds which worked perfectly well for my task. The program is, unfortunately, a paid product, but does some heavy lifting. It has the ability to store all the syslog traffic, archive and compress the old logs, and send notifications through email/text/or SNMP. Within a couple of hours of designing and configuring this project, I was able to complete everything necessary to no longer worry about my logs.
Please note that this guide is written for Windows machines only.
NXLOG
Product information
Price: Free!
Documentation: here
Download: NXLOG community edition: here
NXLOG supports windows and a variety of Linux distros, but Max OS support is nonexistent.
Installation
Installation of NXLOG is as simple as running the executable, accepting their terms and conditions, and finishing the wizard. Once installed we now need to configure it to work as we wish.
Configuration
- Navigate to nxlog’s installation directory, usually: ‘C:\Program Files (x86)\nxlog’.
- Open the ‘conf’ folder (C:\Program Files (x86)\nxlog\conf) where you will see a nxlog.conf file. Open this in an editor.
As you can see this is a sample file that is the default configuration for the program. This is where we will configure the program to ship our specified logs to the specified location. Once we have the configuration a way we like, we save the .conf file, restart the nxlog service and have syslog messages being delivered to the destination. It is important to keep in mind that not all servers needing logs shipped are the same. We typically have different requirements in terms of what logs to ship and how they will be sent. With this being said, not one configuration file fits all. We need to reshape the configuration of nxlog to work depending on the server.
The example configuration will be my print server. My idea was to send window event logs and print jobs logs to my central logging server. I will explain how each part works.
This is the header portion of the configuration file. If you have nxlog installed in a directory different than the ROOT specified below, you may need to change it. For most cases, leave it as it is.
## This is a sample configuration file. See the nxlog reference manual about the ## configuration options. It should be installed locally and is also available ## online at http://nxlog.org/nxlog-docs/en/nxlog-reference-manual.html ## Please set the ROOT to the folder your nxlog was installed into, ## otherwise it will not start. #define ROOT C:\Program Files\nxlog define ROOT C:\Program Files (x86)\nxlog Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log ## Eventlog
The code below is an extension module for nxlog. These modules can enhance the features of nxlog in different ways such as exporting new functions and procedures1. This module’s name is ‘json’. This is mostly for specifying the output format of the logs. You can choose options such as JSON, CSV, XML, GELF, etc.
<Extension json>
Module xm_json
</Extension>
Now we are moving onto the Input module. Input modules are responsible for collecting event log data from various sources1. This is where we specify what logs we want to collect. This module’s name is ‘eventlog’.
From the code below I specified that I want to receive Windows event logs from a number of sources like Application, System, and Security. I have also limited what logs in each category by their severity level. This is useful because being spammed by information logs is generally not helpful.
I also added Windows event logs for PrintService. This is what enables me to capture logs for Print Jobs and what is being sent to our print server. You can choose other event logs by traversing the tree as shown here.
The QueryList is very similar to Window event log’s filter.
<Input eventlog>
Module im_msvistalog
Query <QueryList>\
<Query Id="0">\
<Select Path="Application">*[Application/Level=2]</Select>\
<Select Path="System">*[System/Level=4]</Select>\
<Select Path="Security">*[Security/Level=3]</Select>\
<Select Path="Microsoft-Windows-AppLocker/EXE and DLL">*</Select>\
<Select Path="Microsoft-Windows-AppLocker/MSI and Script">*</Select>\
<Select Path="Microsoft-Windows-PrintService/Admin">*</Select>\
<Select Path="Microsoft-Windows-PrintService/Operational">*</Select>\
</Query>\
</QueryList>
</Input>
The Output modules are responsible for delivering the collected logs to the destination. This is where we specify where the logs will going to. This module’s name is ‘out’.
There is minor configuration to do here. Firstly, we need to set the method of how we will transport. I chose to use UDP with the standard port of 514. We need to make sure that we set the ‘host’ attribute to the IP address of the server that will be receiving the logs.
<Output out>
Module om_udp
Host IP ADDRESS HERE
Port 514
Exec $EventReceivedTime = integer($EventReceivedTime) / 1000000; \
to_json();
</Output>
The last portion of nxlog’s configuration is the Route module. This module’s purpose is to map the desired Input module to the desired Output module. Below we have a Route named ‘1’ with its Path set to our Input module’s name ‘eventlog’ to the Output module’s name ‘out’.
<Route 1>
Path eventlog => out
</Route>
Here is the compiled code of all of the above snippets.
## This is a sample configuration file. See the nxlog reference manual about the
## configuration options. It should be installed locally and is also available
## online at http://nxlog.org/nxlog-docs/en/nxlog-reference-manual.html
## Please set the ROOT to the folder your nxlog was installed into,
## otherwise it will not start.
#define ROOT C:\Program Files\nxlog
define ROOT C:\Program Files (x86)\nxlog
Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log
## Eventlog
<Extension json>
Module xm_json
</Extension>
<Input eventlog>
Module im_msvistalog
# For windows 2003 and earlier use the following:
# Module im_mseventlog
Query <QueryList>\
<Query Id="0">\
<Select Path="Application">*[Application/Level=2]</Select>\
<Select Path="System">*[System/Level=4]</Select>\
<Select Path="Security">*[Security/Level=3]</Select>\
<Select Path="Microsoft-Windows-AppLocker/EXE and DLL">*</Select>\
<Select Path="Microsoft-Windows-AppLocker/MSI and Script">*</Select>\
<Select Path="Microsoft-Windows-PrintService/Admin">*</Select>\
<Select Path="Microsoft-Windows-PrintService/Operational">*</Select>\
</Query>\
</QueryList>
</Input>
<Output out>
Module om_udp
Host IP ADDRESS HERE
Port 514
Exec $EventReceivedTime = integer($EventReceivedTime) / 1000000; \
to_json();
</Output>
<Route 1>
Path eventlog => out
</Route>
Now that we have the configuration file complete how we want it, we will overwrite the default configuration file (.conf) in the same directory.
nxlog is now sending logs, based on your specifications, to the pointed IP address.
Kiwi Syslog Server
Product information
Price: One-time payment of $295 with a free 14-day trial
Documentation: here
Download: 14 Day Free Trial
Kiwi Syslog Server only supports Windows 7, Windows 8, Windows 10, Windows 2008 R2, Windows Server 2012 and 2012 R2, and Windows Server 2016.
Installation
Installation of Kiwi Syslog is fairly simple. Just follow the wizard mostly, but I recommend turning off the ‘Kiwi Syslog Web Access’ shown below. It would be nice to be able to view your logs in a web interface, but the product does not handle this very well. It is slow, ugly, and doesn’t filter logs correctly. The original product is much better.
Configuration
Once the product is installed we are ready for configuration. Launch the Kiwi Syslog Server Console from the start menu or your desktop shortcut. You should see a window that resembles a blank excel spreadsheet once loaded.
Setting Displays
The first step we should do is set our displays correctly instead of the default ‘Display 1’, ‘Display 2’, etc.
In order to do this:
- Go to File -> Setup
- Open the ‘Display’ category on the left pane.
- On the right pane, choose ‘Display 1’ from the ‘Modify display names’ section. In the input edit to the right enter the name you would like this display window to be named.
- Click Update
- Click Apply/OK
Setting Inputs
We need to make sure that we are using the same protocol as our nxlog log shipper. I will be using my example of UDP with port 514 as mentioned earlier.
- Go to File -> Setup
- Open the ‘Inputs’ category and drill down to ‘UDP’
- Make sure ‘Listen for UDP Syslog messages’ is checked/enabled
- Configure ‘UDP Port’ edit to ‘514’.
- Click Apply/OK.
Creating Rules
Now that we have laid the groundwork, we need to filter our messages somehow. If this server is going to be centralized we want to keep logs separate for each server shipping them to us.
-
- Go to File -> Setup
- Right-click the ‘Rules’ category on the left pane
- Select ‘Add Rule’
- You will see a new rule was created
- Right-click on the ‘New Rule’ and rename it to whatever
- right-click on ‘Filter’s and select ‘Add filter’
- Select ‘IP address’ from the ‘Field’ dropdown and ‘Simple’ from the ‘Filter Type’ drop-down
- Enter the IP address of the server sending the logs to the central server in double quotes i.e. “172.16.50.111”
- Back on the left pane of the window, right-click on the new filter created and rename it to something reasonable. I recommend using the IP address we just entered.
- Back on the left pane of the window, right-click on the ‘Actions’ portion of the rule and click ‘Add action’
- From the ‘Action’ drop-down select ‘Display’
- Choose the Display that we created earlier in the first step of ‘Setting Display’
- Back on the left pane of the window, right-click on the new action we created and select ‘Auto-name action’. This will rename the action to ‘Display’
- We will now create another action, so right-click ‘Actions’ and select ‘ Add action’
- Select the newly added action, and on the right pane select ‘Log to file’ from the ‘Action’ drop-down
- Enter the path of where you would like the logs from this source to be saved
- Back on the left pane of the window, right-click on the new action we created and select ‘Auto-name action’. This will rename the action to ‘Log to file’
- Click Apply/OK
- You should now start seeing logs from your server (assuming there are logs) appearing on your display created on the application’s ‘home screen’.
Configuring Email
It is important to know what is going on within your log environment so Kiwi offers email notifications on events such as tasks completion or disk space warnings.
Configuring email for notifications is simple. I had to create an email mailbox on my Exchange server in order to have the email sent through SMTP. This section is self-explanatory, if you do not have the necessary information then ask your system administrator.
Configuring Auto-archive
A powerful feature of this program is the ability to set it and forget it. No worrying about logs filling up your drives, no laborious tasks needing to be done manually. We will now configure Kiwi Syslog Server to compress, archive, and password protect our logs automatically on a schedule.
Schedule tab:
On the left pane select the category ‘Schedules’, right-click it and create a new schedule. Change the ‘Task Type’ to ‘Archive’ and the ‘Task Trigger’ to ‘On a schedule’.
You can select when the task will start, how often it will run, and manually run it from this tab.
Source tab:
This tab has settings specifying where the logs you want to be archived are located.
Destination tab:
This tab is used to choose where the logs will be archived to.
Archive Options tab:
This tab lets you set some options for the task. As you can see, I set it so all logs will be zipped, compressed, and password protected.
If you want to get fancy, you can also set some scripts to run when this event takes place.
Archive Notifications tab:
Once the archive runs, do you want to be notified about it? Currently, I have mine set up so I get an email with the statistics of the archive i.e. original log size -> new file size.
I mainly do this to remind myself the function is still working.
Once you have some logs and this function setup, it wouldn’t be a bad idea to test it out. Apply the settings and go back to the ‘Schedule tab’. Click the ‘Run Now’ button in the bottom right corner and you will receive instructions that the task is running. Wait a few moments and you should see your logs in a new location with an email sent to your inbox.
1Referenced nxlog’s documentation located here.