logstash
my logsash config
ELK (Elasticsearch + Logstash + Kibana) is fun!
Logstash is super flexible, most operations can be.
memo
Start separately Java process, shipper indexer. (divided into two by copying the startup script that is distributed in the package version)
reference
postfix grok patterns :
- https://github.com/whyscream/postfix-grok-patterns
- https://gist.github.com/poolski/9911628
- https://gist.github.com/jamtur01/4385667
- https://gist.github.com/randywallace/6983588
sshd grok patterns :
Lightweight log shipper : logstash-forwarder (aka lumberjack)
- https://github.com/elasticsearch/logstash-forwarder
- https://www.digitalocean.com/community/tutorial_series/centralized-logging-with-logstash-and-kibana-on-ubuntu-14-04
- https://www.digitalocean.com/community/tutorials/adding-logstash-filters-to-improve-centralized-logging
grok filter ruby :
- https://groups.google.com/forum/#!topic/logstash-users/iEYRv7bCqdM
- http://stackoverflow.com/questions/20512416/adding-tags-to-logstash-events-based-on-the-md5-of-the-filename
kibana geoip BetterMap :
- https://beingasysadmin.wordpress.com/2014/04/07/near-realtime-dashboard-with-kibana-and-elasticsearch/
- http://dev.maxmind.com/geoip/legacy/geolite/
grok apache User-Agent :
- http://untergeek.com/2013/09/11/getting-apache-to-output-json-for-logstash-1-2-x/
- https://github.com/ua-parser/uap-core/blob/master/regexes.yaml
Integrating DataDog
zimbra mailbox.log & zimbra.log (amavis)
- http://blog.itlinux.cl/blog/2015/05/25/buscando-mensajes-de-correo-con-kibana/
- https://github.com/ITLinuxCL/zimbra_logstash
- http://antisp.in/2014/04/01/useful-logstash-grok-patterns/
- https://github.com/Autobase/Zimbra/blob/4bf3dc250c68a38e38286bdd972c8d5469d40e34/ZimbraCommon/src/java/com/zimbra/common/util/ZimbraLog.java
- https://wiki.zimbra.com/wiki/Centralized_Logs_-_Elasticsearch,_Logstash_and_Kibana
- https://blog.zimbra.com/2007/05/mailboxlog-the-king-of-zimbra-log-files/
- https://www.zimbra.com/docs/os/5.0.19/administration_guide/9_Monitoring.11.1.html