All Projects → defenxor → Dsiem

defenxor / Dsiem

Licence: gpl-3.0
Security event correlation engine for ELK stack

Programming Languages

go
31211 projects - #10 most used programming language

Projects that are alternatives of or similar to Dsiem

Elk Stack
ELK Stack ... based on Elastic Stack 5.x
Stars: ✭ 148 (-41.96%)
Mutual labels:  logstash, elk, elasticsearch
Spring Boot Microservice Eureka Zuul Docker
Spring-Boot rest microservices using Eureka, Zuul, Docker. Monitoring with logstash, logback, elasticsearch, kibana
Stars: ✭ 45 (-82.35%)
Mutual labels:  logstash, elk, elasticsearch
Docker monitoring logging alerting
Docker host and container monitoring, logging and alerting out of the box using cAdvisor, Prometheus, Grafana for monitoring, Elasticsearch, Kibana and Logstash for logging and elastalert and Alertmanager for alerting.
Stars: ✭ 479 (+87.84%)
Mutual labels:  logstash, elk, elasticsearch
Json Logging Python
Python logging library to emit JSON log that can be easily indexed and searchable by logging infrastructure such as ELK, EFK, AWS Cloudwatch, GCP Stackdriver
Stars: ✭ 143 (-43.92%)
Mutual labels:  logstash, elk, elasticsearch
Elastiflow
Network flow analytics (Netflow, sFlow and IPFIX) with the Elastic Stack
Stars: ✭ 2,322 (+810.59%)
Mutual labels:  logstash, elk, elasticsearch
Elk Docker
Docker configuration for ELK monitoring stack with Curator and Beats data shippers support
Stars: ✭ 342 (+34.12%)
Mutual labels:  logstash, elk, elasticsearch
Synesis lite suricata
Suricata IDS/IPS log analytics using the Elastic Stack.
Stars: ✭ 167 (-34.51%)
Mutual labels:  logstash, elk, elasticsearch
Elk
搭建ELK日志分析平台。
Stars: ✭ 688 (+169.8%)
Mutual labels:  logstash, elk, elasticsearch
Mozdef
DEPRECATED - MozDef: Mozilla Enterprise Defense Platform
Stars: ✭ 2,164 (+748.63%)
Mutual labels:  elk, elasticsearch, siem
Redelk
Red Team's SIEM - tool for Red Teams used for tracking and alarming about Blue Team activities as well as better usability in long term operations.
Stars: ✭ 1,692 (+563.53%)
Mutual labels:  logstash, elasticsearch, siem
Ansible Elk
📊 Ansible playbook for setting up an ELK/EFK stack and clients.
Stars: ✭ 284 (+11.37%)
Mutual labels:  logstash, elk, elasticsearch
Openuba
A robust, and flexible open source User & Entity Behavior Analytics (UEBA) framework used for Security Analytics. Developed with luv by Data Scientists & Security Analysts from the Cyber Security Industry. [PRE-ALPHA]
Stars: ✭ 127 (-50.2%)
Mutual labels:  elk, elasticsearch, siem
Helk
The Hunting ELK
Stars: ✭ 3,097 (+1114.51%)
Mutual labels:  logstash, elk, elasticsearch
Elk Docker
Elasticsearch, Logstash, Kibana (ELK) Docker image
Stars: ✭ 1,973 (+673.73%)
Mutual labels:  logstash, elk, elasticsearch
Elkstack
The config files and docker-compose.yml files of Dockerized ELK Stack
Stars: ✭ 96 (-62.35%)
Mutual labels:  logstash, elk, elasticsearch
Elastic
Elastic Stack (6.2.4) 을 활용한 Dashboard 만들기 Project
Stars: ✭ 121 (-52.55%)
Mutual labels:  logstash, elk, elasticsearch
Docker Elk
The Elastic stack (ELK) powered by Docker and Compose.
Stars: ✭ 12,327 (+4734.12%)
Mutual labels:  logstash, elk, elasticsearch
Ansible Role Logstash
Ansible Role - Logstash
Stars: ✭ 136 (-46.67%)
Mutual labels:  logstash, elk
Terraform Aws Elasticsearch
Terraform module to provision an Elasticsearch cluster with built-in integrations with Kibana and Logstash.
Stars: ✭ 137 (-46.27%)
Mutual labels:  elk, elasticsearch
Elk Hole
elasticsearch, logstash and kibana configuration for pi-hole visualiziation
Stars: ✭ 136 (-46.67%)
Mutual labels:  logstash, elasticsearch

Dsiem

CircleCI Codecov Go Report Card License: GPL v3

Dsiem is a security event correlation engine for ELK stack, allowing the platform to be used as a dedicated and full-featured SIEM system.

Dsiem provides OSSIM-style correlation for normalized logs/events, perform lookup/query to threat intelligence and vulnerability information sources, and produces risk-adjusted alarms.

Example Kibana Dashboard

Features

  • Runs in standalone or clustered mode with NATS as messaging bus between frontend and backend nodes. Along with ELK, this made the entire SIEM platform horizontally scalable.
  • OSSIM-style correlation and directive rules, bridging easier transition from OSSIM.
  • Alarms enrichment with data from threat intel and vulnerability information sources. Builtin support for Moloch Wise (which supports Alienvault OTX and others) and Nessus CSV exports. Support for other sources can easily be implemented as plugins.
  • Instrumentation supported through Metricbeat and/or Elastic APM server. No need extra stack for this purpose.
  • Builtin rate and back-pressure control, set the minimum and maximum events/second (EPS) received from Logstash depending on your hardware capacity and acceptable delays in event processing.
  • Loosely coupled, designed to be composable with other infrastructure platform, and doesn't try to do everything. Loose coupling also means that it's possible to use Dsiem as an OSSIM-style correlation engine with non ELK stack if needed.
  • Batteries included:
    • A directive conversion tool that reads OSSIM XML directive file and translate it to Dsiem JSON-style config.
    • A SIEM plugin creator tool that will read off an existing index pattern from Elasticsearch, and creates the necessary Logstash configuration to clone the relevant fields' content to Dsiem. The tool can also generate basic directive required by Dsiem to correlate received events and generate alarm.
    • A helper tool to serve Nessus CSV files over the network to Dsiem.
    • A light weight Angular web UI just for basic alarms management (closing, tagging), and easy pivoting to the relevant indices in Kibana to perform the actual analysis.
  • Obviously a cloud-native, twelve-factor app, and all that jazz.

How It Works

Simple Architecture

On the diagram above:

  1. Log sources send their logs to Syslog/Filebeat, which then sends them to Logstash with a unique identifying field. Logstash then parses the logs using different filters based on the log sources type, and sends the results to Elasticsearch, typically creating a single index pattern for each log type (e.g. suricata-* for logs received from Suricata IDS, ssh-* for SSH logs, etc.).

  2. Dsiem uses a special purpose logstash config file to clone incoming event from log sources, right after logstash has done parsing it. Through the same config file, the new cloned event is used (independently from the original event) to collect Dsiem required fields like Title, Source IP, Destination IP, and so on.

  3. The output of the above step is called Normalized Event because it represent logs from multiple different sources in a single format that has a set of common fields. Those events are then sent to Dsiem through Logstash HTTP output plugin, and to Elasticsearch under index name pattern siem_events-*.

  4. Dsiem correlates incoming normalized events based on the configured directive rules, perform threat intel and vulnerability lookups, and then generates an alarm if the rules conditions are met. The alarm is then written to a local log file, that is harvested by a local Filebeat configured to send its content to Logstash.

  5. At the logstash end, there's another Dsiem special config file that reads those submitted alarms and push them to the final SIEM alarm index in Elasticsearch.

The final result of the above processes is that now we can watch for new alarms and updates to an existing one just by monitoring a single Elasticsearch index.

Installation

You can use Docker Compose or the release binaries to install Dsiem. Refer to the Installation Guide for details.

Alternatively, there's also a Docker Compose or virtual machine-based demo environment that you can use to evaluate all Dsiem integration from one simple web interface.

Documentation

Currently available docs are located here.

Reporting Bugs and Issues

Please submit bug and issue reports by opening a new Github issue. Security-sensitive information, like details of a potential security bug, may also be sent to [email protected]. The GPG public key for that address can be found here.

How to Contribute

Contributions are very welcome! Submit PR for bug fixes and additional tests, gist for Logstash config files to parse device events, SIEM directive rules, or a new threat intel/vulnerability lookup plugins.

If you're not sure on what to do on a particular matter, feel free to open an issue and discuss first.

License

The project is licensed under GPLv3. Contributors are not required to sign any form of CAA/CLA or a like: We consider their acceptance of this Github terms of service clause to be sufficient.

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].