Forwarding logs with rsyslog

rsyslog never seems to get as much attention as it should now that everyone seems to use logstash. I like rsyslog - it's always in the system repositories and it's been about for ages.

In this scenario I wanted to forward log files from a local file system verbatim to a remote host.  Here is what I came up with.  Both the configurations show the rsyslog.conf in it's entirety.  Adding TLS would also be easy and might be the subject of a future post. Centos 7 and the system rsyslog 7.4.7 is used

We use the tag name to set the target filename on the sever.  Notice how by using the dynaFile module we can set the filesystem layout.  In this example where the tag name is set to icecast-error and is shipped from host relay3. On the server we end up with the file being created as /var/log/relay3/icecast-error.log

Notice queue parameters are used to provide a local filesystem based queue.  This configuration will store messages in a queue and write the queue to disk should the system get rebooted.  This queue is useful for keeping messages locally when the remote logging server goes down or cannot be contacted to.  I played with this quite a lot and it seems entirely robust.

Also note how in the server configuration that each line from the imported file is now wrapped in a new rsyslog item and we just want the message portion (our original shipped line) from it.

Client configuration

global(WorkDirectory="/var/spool/rsyslog")

ruleset(name="forward"){    
    action(type="omfwd" 
    target="logs.example.com"
    port="601" 
    protocol="tcp"
    queue.type="LinkedList" 
    queue.filename="srvrfwdQueue" 
    queue.saveonshutdown="on" 
    action.resumeRetryCount="-1")
    }

input(type="imfile" 
    file="/var/log/icecast/error.log"
    tag="icecast-error"  
    ruleset="forward")

input(type="imfile" 
    file="/var/log/icecast/access.log"
    tag="icecast-access"
    ruleset="forward")

Server Configuration

$umask 0000 # Reset the umask so the CreateMode values work as expected 
$MainMsgQueueSize 1000000 
$MainMsgQueueDequeueBatchSize 1000

# http://www.rsyslog.com/doc/master/configuration/properties.html
template(name="directoryPerHost"
    type="string"
    string="/var/log/%source%/%syslogtag%.log") 

# Strip the leading space induced by RFC3164, mmrm1stspace module can be used# in >= 8.24.0
template(name="messageOnly" 
    type="string" 
    string="%msg:2:$:%\n")

ruleset(name="remoteInbound"){
    action(type="omfile"
    template="messageOnly"
    dynaFile="directoryPerHost" 
    fileCreateMode="0644"
    dirCreateMode="0755")
}

input(type="imptcp" 
    port="601"
    ruleset="remoteInbound")