Multiple configuration files for LogStash

If you have multiple applications running on a box with the logstash client, you probably want to split the config file up a bit. According to this thread, you can use a directory as the value for the config input:

/opt/logstash/bin/logstash agent --config /etc/logstash.d --log /var/log/logstash/logstash.log

and the files in that directory will be conflated in lexicographical order.

This means you need some sort of scheme to keep your inputs before your outputs, etc. I’m not too bothered about the order of the individual inputs, so I prefixed each category with a different number:

$ ls -la /etc/logstash.d
total 32
drwxr-xr-x  2 logstash logstash 4096 Apr 14 13:42 .
drwxr-xr-x 98 root     root     4096 Apr 16 01:28 ..
-rwxr-xr-x  1 logstash logstash  140 Jan  8 13:34 1_foo_inputs.conf
-rwxr-xr-x  1 logstash logstash  122 Jan  8 12:55 1_bar_inputs.conf
-rwxr-xr-x  1 logstash logstash   99 Jan  8 12:06 1_syslog_inputs.conf
-rwxr-xr-x  1 logstash logstash  237 Jan  8 13:34 2_foo_filters.conf
-rwxr-xr-x  1 logstash logstash  229 Jan  8 12:55 2_bar_filters.conf
-rwxr-xr-x  1 logstash logstash  170 Apr 14 13:42 3_output.conf

ensuring that inputs will be before filters, and both will be before outputs.

This means that the logstash config for each app can be deployed with it, without needing to amend a shared file.

Advertisements

3 thoughts on “Multiple configuration files for LogStash

  1. Adam September 5, 2014 / 9:34 am

    thanks!

  2. Gopalakrishna Palem (@GPalem) August 11, 2015 / 5:34 am

    Is it possible to have each config as a self-contained one (with input, filter, output completely inside it) and multiple config files (one for each input type, say file, http, websocket etc.) ?

  3. Cédric September 17, 2015 / 10:05 am

    thanks !

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s