Question

what is syslog? what does is use it for? what are the most important parts of it regarding firewall and intrusion detection ? and what are examples of it?

Answer 1

what is syslog? what does is use it for? what are the most important parts of it regarding firewall and intrusion detection ? and what are examples of it?

what is syslog?

Syslog stands for System Logging Protocol and is a standard protocol used to send system log or event messages to a specific server, called a syslog server. It is primarily used to collect various device logs from several different machines in a central location for monitoring and review.

what does is use it for?

Syslog, is a standardized way (or Protocol) of producing and sending Log and Event information from Unix/Linux and Windows systems (which produces Event Logs) and Devices (Routers, Firewalls, Switches, Servers, etc) over UDP Port 514 to a centralized Log/Event Message collector which is known as a Syslog Server.

One of the main reasons Syslog was so widely accepted throughout the industry was because of its simplicity – There is little to no uniformity or standardization when it comes to the content that a Device, Server or Operating system is written and sends log information.

It simply sends and transports messages over Syslog protocol with no acknowledgement of receipt (hence the use of UDP).

Furthermore, there is no complex requirements between the Sending device and Receiving/Collecting Server, as Syslog messages are simply just sent regardless if there is a Receiver configured on the other end or not.

what are the most important parts of it regarding firewall and intrusion detection ? and what are examples of it?

How It Works

There are three different layers within the Syslog standard, which are:

• Syslog content (information contained in an event message)
• Syslog application (generates, interprets, routes and stores messages)
• Syslog transport (transmits the messages)

In addition, applications can be configured to send messages to several destinations. There are also alarms that give instant notifications for events such as:

• Hardware errors
• Application failures
• Lost contact
• Mis-configuration

Furthermore, alarms can be set up to send notifications via SMS, pop-up messages, email, HTTP and more. Since the process is automated, the IT team will get immediate notifications of there is a sudden failure of any of the devices. (Image via Wikimedia Commons)

Syslog Servers

Syslog servers are used to send diagnostic and monitoring data. The data can then be analyzed for system monitoring, network maintenance and more. Since the Syslog protocol is supported by a wide swath of devices, they can conveniently log information into the Syslog server.

SNMP data can be used to assess any failure points quickly. Syslog servers can also have automated events to trigger alerts that help to prevent downtime or outages. Here is a list of a few Windows-based Syslog servers:

1. Kiwi Syslog Server. This server is simple to install and generates reports in plain text or HTML. The software handles Syslog and SNMP, even from Linux and UNIX hosts. It is compatible with Win XP 32/64, Win 2003 32/64, Windows Vista 32/64, Win7 32/64, Windows 2008 R2 32/64, Windows 8, Windows Server 2012 & 2012 R2.

2. PRTG. This adds a sensor to the PRTG monitoring to enable Syslog ability. It focuses on SNMP and Syslog protocol data. It is compatible with any Windows 64-bit environment with Windows Server 2012 R2.

3. SNMPSoft Sys-log Watcher. This is a dedicated syslog server for a wide variety of devices. It can also parse and manage non-standard Syslog. It is compatible with Windows XP through Windows 10.

4. The Dude. This system is used for general network management with a built-in syslog server. Furthermore, it comes with functionality for remote logging through the RouterOS. It is compatible with Windows 2000 or newer. Yet, it also runs on Linux or MacOS using Wine/Darwine.

5. Visual Syslog Server. This is a lighter syslog option that looks at alerts in real time. Thresholds can be configured to trigger both scripts and programs. It is compatible with Windows XP, Vista, 7, 8, 8.1, as well as Windows Server 2003, 2008, 2012.

6. Datagram. This program offers enterprise-level functionality. It works well in large environments. It obtains and stores Syslog data. Moreover, it is compatible with Windows 2000 and newer.

For the Mac OS X, you can use Splunk–which enables system monitoring and syslog events. In fact, Splunk is known as the tool for operational intelligence. Also, you can configure Splunk as a forwarder to your central monitoring server. To configure the Mac OS X Syslogd you would:

1. Open a terminal window

2. Make a backup copy of syslogd.conf into the /tmp folder by typing

 $ cp /etc/syslog.conf /tmp/syslog.conf.bkp

3. Open the configuration file in the editor of your choice

 $ sudo vi /etc/syslog.conf

Password: The ‘sudo’ command is used to execute vi with “root” privileges.

4. Replace the IP address 192.168.1.12 with the IP address if your Splunk server’s network interface

5. Type ‘i’ in vi to enter the insert mode (text entry). Make sure to use tabs and not spaces between the selector and action fields.

6. Save and Exit. Save the file by typing

 ’:wq <enter>’

7 Restart the ‘syslogd’ service, and test it by typing

$ ps -e | grep syslogd 5070 ?? 2:33.75 /usr/sbin/syslogd

These are the commands that restart the service:

$ sudo launchctl unload /System/Library/LaunchDaemons/com.apple.syslogd.plist $ sudo launchctl load /System/Library/LaunchDaemons/com.apple.syslogd.plist

To log an event, open a new Terminal window and type:

$ logger -s -p user.info Testing splunk syslog forwarding

The Syslog Format

Syslog has a standard definition and format of the log message defined by RFC 5424. As a result, it is composed of a header, structured-data (SD) and a message. Within the header, you will see a description of the type such as:

• Priority
• Version
• Timestamp
• Hostname
• Application
• Process id
• Message id

Then, you will see structured-data which have data blocks in the “key=value” format within square brackets. After the SD, you will see the detailed log message, which is encoded in UTF-8.

For example, the following message:

<34>1 2003-10-11T22:14:15.003Z mymachine.example.com su - ID47 - BOM'su root' failed for lonvick on /dev/pts/8

Corresponds to the following format:

<priority>VERSION ISOTIMESTAMP HOSTNAME APPLICATION PID MESSAGEID STRUCTURED-DATA MSG 

Syslog Messages

Syslog messages are used to report levels of Emergency and Warnings with regards to software or hardware issues. To illustrate, a system restart will be sent through the Notice level. A system reload will be sent through the Informational level. If debug commands are outputted, it is conveyed through the Debug level.

Screenshot via Cisco

Here are the Syslog Message Levels:

• Emergency Messages–System is unavailable and unusable (Could be a “panic” condition due to a natural disaster)
• Alert Messages–Action needs to be taken immediately (an example is loss of backup ISP connection)
• Critical Messages–Critical conditions (this could be a loss of primary ISP connection)
• Error Messages–Error conditions (must be resolved within a specified time frame)
• Warning Messages–Warning conditions (indicates an error may occur if action is not taken)
• Notification Messages–Things are normal, but this is still a significant condition (immediate action is usually not required)
• Informational Messages–Informational messages (for reporting and measuring)
• Debugging Messages–Debug-level messages (Offers information around debugging apps)

Here are some additional examples from TechZone. This system error message:

May 11 10:40:48 scrooge disk-health-nurse[26783]: [ID 702911 user.error] m:SY-mon-full-500 c:H : partition health measures for /var did not suffice - still using 96% of partition space 

Can be split into the following columns:

Column 1 = "May 11 10:40:48" > Timestamp Column 2 = "scrooge" > Loghost Column 3 = "disk-health-nurse[26783]:" > Application/Process Column 4 = "[ID 702911 user.error]" > Syslog facility.level Column 5 = "m:SY-mon-full-500" > Message ID Column 6 = "c:H : partition health..." > Message [possibly including rid, sid, ip] 

And, here’s an example of a summary message:

May 11 10:00:39 scrooge SG_child[808]: [ID 748625 user.info] m:WR-SG-SUMMARY c:X  vhost:iscrooge61.seclutions.com:80 (http) GET / => http://bali/ , status:200 , redirection URL:<n/a> , referer:<n/a> , mapping:bali , request size: 421 , backend response size: 12960 , audit token:- , time statistics (microseconds): [request total 16617 , allow/deny filters 1290 , backend responsiveness 11845 , response processing 1643 , ICAP reqmod <n/a> , ICAP respmod <n/a> ] timestamp: [2012-05-11 10:00:39] [ rid:T6zHJ38AAAEAAAo2BCwAAAMk sid:910e5dd02df49434d0db9b445ebba975 ip:172.18.61.2 ] 

Which contains the following columns:

Column 1 = "May 11 10:00:39" > Timestamp Column 2 = "scrooge" > Loghost Column 3 = "SG_child[808]:" > Application/Process Column 4 = "[ID 748625 user.info]" > Syslog facility.level Column 5 = "m:WR-SG-SUMMARY" > Message ID Column 6 = "c:X vhost:..." > Message [including time statistics and rid, sid, ip] 

The Most Important Log Files to Track and Monitor

Monitoring log files is critical because it helps you manage any errors in the functioning of your OS. Some the types of relevant information you will receive includes:

• User issues
• Security breaches
• Hard disk crashes or power outages

Of course, there are log files of high priority that you should always keep track of. The log files include:

• /var/log/messages–Contains most system messages
• /var/log/secure–Authentication messages
• /var/log/cron–Logs Cron job activities
• /var/log/maillog–Mail transactions

If you were to look into /var/log/messages, you would find:

• The timestamp
• The hostname of the executing program
• Name of the utility that prompted the message
• The action that took place

Pros and Cons of Syslog

One problematic scenario is when your /var/log/messages file fills up due to logging misconfiguration. Plus, there will be times when your system’s logging will cause unforeseen issues. This is why it is imperative to understand how to control your logging and where your logs are saved. Plus, there may be some packet loss if there is a large burst of network traffic.

Also, the fact that Syslog is UDP-based means there can be issues with reliability. On the other hand, as systems become more complex, it becomes increasingly important to collect and monitor all relevant data produced by applications.

This data can be analyzed to determine how systems are behaving. Plus, logs are considered a reliable data source for understanding current system statistics and making trend predictions. Not to mention, logs are used for activities such as troubleshooting such or rolling back the system after a failure accident.