rsyslog logstash graylog setup

Install rsyslog

yum -y install rsyslog

Install logstash

First download and install the public signing key:

rpm --import

The add logstash repository

cat > /etc/yum.repos.d/logstash.repo<<EOF
name=Elastic repository for 5.x packages

And your repository is ready for use. You can install it with:

sudo yum install logstash


sudo yum install java-1.8.0-openjdk
curl -L -O
sudo rpm -i logstash-5.4.1.rpm

Install rabbitmq

yum install -y epel-release
yum -y install rabbitmq-server
systemctl start rabbitmq-server

Configure firewalld

firewall-cmd --add-port=5672/tcp --permanent
firewall-cmd --reload

Configure rabbitmq for graylog by creating a user for log delivery

rabbitmqctl add_user graylog strong_password
rabbitmqctl set_permissions -p / graylog ".*" ".*" ".*"

Configure rsyslog to logstash log shipping

Let’s now look at how to configure rsyslog to send messages to logstash running locally.

vim /etc/rsyslog.d/90-logstash.conf


PreserveFQDN on
         option.json="on") {
             constant(value="\"@timestamp\":\"")     property(name="timereported" dateFormat="rfc3339")
             constant(value="\",\"message\":\"")     property(name="msg")
             constant(value="\",\"host\":\"")        property(name="hostname")
             constant(value="\",\"severity\":\"")    property(name="syslogseverity-text")
             constant(value="\",\"facility\":\"")    property(name="syslogfacility-text")
             constant(value="\",\"programname\":\"") property(name="programname")
             constant(value="\",\"procid\":\"")      property(name="procid")

*.* @;ls_json
  • To identify the messages with the Full Qualified Domain Name of the System that has created the message we use the Option PreserveFQDN - but you will need to have a clean working hostname resolution.
  • rsyslog will send the message via UDP to the local running logstash.

Rsyslog needs to be restarted

systemctl restart rsyslog

route messages with logstash

  • We need to use logstash to send messages to AMQP
  • Logstash will listen on localhost port udp/5514 for the messages that are coming from rsyslog and forward them to the rabbitMQ Server.

vim /etc/logstash/conf.d/graylog.conf

 input {
    UDP {
        port => 5514
        host => ""
        type => syslog
        codec => "json"

filter {
  # This replaces the host field (UDP source) with the host that generated the message (sysloghost)
  if [sysloghost] {
      mutate {
          replace => [ "host", "%{sysloghost}" ]
          remove_field => "sysloghost" # prune the field after successfully replacing "host"

output {
    rabbitmq {
      exchange => "log-messages"
        exchange_type => "fanout"
        key => "log-messages"
        host => ""
        workers => 1         # if you have alot of messages raise this slowly
        durable => true
        persistent => true
        port => 5672
        user => "graylog"
        password => "Password@321!"
        ssl => false         # over unsecure network do not use plain!
        verify_ssl => false  # we assume that you have a valid certificate!

Run logstash with above Configuration

#/bin/logstash -f logstash-filter.conf
sudo service logstash start

Consume messages with graylog

  • Now the Data need to be consumed by graylog.
  • Create an input with the Input Syslog AMQP.
  • Add the Information that is configured in the former steps (exchange, username, password, hostname).
  • Set the Option Allow overwrite date.

Configuring rsyslog to Send Data Remotely

  • In a default rsyslog setup on Ubuntu, you’ll find two files in /etc/rsyslog.d:

20-ufw.conf 50-default.conf

  • On the rsyslog-client, edit the default configuration file:


  • Add the following line at the top of the file before the log by facility section, replacing private_ip_of_ryslog_server with the private IP of your centralized server:
*.*                         @private_ip_of_ryslog_server:514
  • The @ symbol before the IP address tells rsyslog to use UDP to send the messages.
  • Change this to @@ to use TCP. This is followed by the private IP address of rsyslog-server with rsyslog and Logstash installed on it
  • The number after the colon is the port number to use.
  • Restart rsyslog to enable the changes:
service rsyslog restart

Run the following command to validate your rsyslog Configuration

sudo rsyslogd -N1
Share Comments
comments powered by Disqus