To get better at anything you should be willing to go slow before you go fast.

Deliberate Practice

What’s better, few or many input ports in Logstash/Graylog?

This thought often cross our mind when configuring log collection inputs in Logstash or Graylog. The arguments can go countless and every network have a unique argument based on their specific configuration. I won’t be debate on the myriad arguments but list down key logical factors to help us make our decision.

Log enrichment is evident

Unless we are planning to just dump the logs to get rid of centralized logging compliance requirement, we will be working with each unique log type to filter, transform, and add new data to make it useful.

We must need to identify and pick unique log types to apply the enrichment procedures.

Parsing is costly

If we are reading the strings of log messages to identify unique log types, we are wasting precious CPU cycles which can be put to better use. Syslog is the most widely encountered log format. It needs to be parsed to extract individual units of information like severity, timestamp, facility, host, and actual log message from the string.

If we are receiving multiple type of logs from multiple kind of devices on a single port, we need to parse-out each unique log type for future processing. The processing resources will take a significant hit as we scale.

Avoid parsing for log identification

We can increase our processing efficiency by skipping the need to parse logs for identification of type. This can be done in two ways

Engineering detections can be just as interesting as finding bypasses/exploits.

― Casey Smith

Installing Suricata 5.0.0 from source on CentOS 8

Following insturctions will get you a fully working Suricata 5.0.0 by installing from the source. These steps are aimed at setting up Suricata 5.0.0 quickly for a test environment and isn’t recommended for a production server.

Installing pre-requisite

$ sudo dnf config-manager --set-enabled PowerTools
$ sudo dnf -y install gcc libpcap-devel pcre-devel libyaml-devel file-devel zlib-devel jansson-devel nss-devel libcap-ng-devel libnet-devel tar make libnetfilter_queue-devel lua-devel python3-PyYAML libmaxminddb-devel rustc cargo lz4-devel

Computers are useless. They can only give you answers.

Pablo Picasso

Understanding Suricata Config – append

Config Example

outputs:
  - fast:
    enabled: yes
    filename: fast.log
    append: no

Purpose

Suricata generates multiple log files e.g.

 -rw-r--r--. 1 root root 4.3G Aug 13 12:47 eve.json
 -rw-r--r--. 1 root root  17K Aug 13 15:01 suricata.log
 -rw-r--r--. 1 root root 1.8G Aug 13 18:11 stats.log
 -rw-r--r--. 1 root root 2.0M Aug 13 18:11 fast.log

When we restart or re-run suricata deamon it has to decide what to do with the existing files. It has two options to decide from.

One of the first principals in solving crime is never to disregard anything no matter how trivial.

― Sherlock Holmes

Installing Suricata 4.1.2 from source on CentOS 7

Suricata IDS binary package is available in the EPEL repository for CentOS 7 but it’s not always the latest stable release. At the time of writing the v4.1.2 is the latest stable release and v4.0.6 is available in the EPEL repo.

We’ll proceed with installing from the source tar.gz.

Installing pre-requisite

Prepare the system by installing all the dependencies required for a full working Suricata v4.2 installation.

$ sudo yum -y install epel-release
$ sudo yum -y install jq cargo openssl-devel PyYAML lz4-devel gcc libpcap-devel pcre-devel libyaml-devel file-devel zlib-devel jansson-devel nss-devel libcap-ng-devel libnet-devel tar make libnetfilter_queue-devel lua-devel

Download & Unpack Suricata v4.2

$ wget https://www.openinfosecfoundation.org/download/suricata-4.1.2.tar.gz
$ tar xzvf suricata-4.1.2.tar.gz
$ cd suricata-4.1.2

A trusted tool is one that you understand what it does.

― Chris Pogue

Suricata 4.1 + Ubuntu 18.04 – Binary Installation

Suricata IDS is available in the default repository and the package is maintained by members of Ubuntu MOTU Developers community. Unfortunately, it’s not always the latest stable release. As of writing, it offered v3.2 and the official stable release is v4.1, that’s a huge delay in packaging. 

ubuntu@ubuntu:~$ sudo apt list -a suricata
Listing… Done
suricata/bionic,now 3.2-2ubuntu3 amd64

“Check out the Ubuntu Packaging Guide if you are interested in contributing to the community by maintaining packages.”

However, Open InfoSec Foundation (OISF) the developers of Suricata do maintain an official repository for Ubuntu and that is the preferred way to source the latest stable release. Installing the repository is simple.