All Projects → robcowart → Elastiflow

robcowart / Elastiflow

Licence: other
Network flow analytics (Netflow, sFlow and IPFIX) with the Elastic Stack

Programming Languages

shell
77523 projects
Dockerfile
14818 projects

Projects that are alternatives of or similar to Elastiflow

flowanalyzer
Manito Networks Flow Analyzer
Stars: ✭ 42 (-98.19%)
Mutual labels:  kibana, sflow, netflow, ipfix
Elk Docker
Elasticsearch, Logstash, Kibana (ELK) Docker image
Stars: ✭ 1,973 (-15.03%)
Mutual labels:  logstash, elk, elasticsearch, kibana
Helk
The Hunting ELK
Stars: ✭ 3,097 (+33.38%)
Mutual labels:  logstash, elk, elasticsearch, kibana
Docker Elk
The Elastic stack (ELK) powered by Docker and Compose.
Stars: ✭ 12,327 (+430.88%)
Mutual labels:  logstash, elk, elasticsearch, kibana
Elastic
Elastic Stack (6.2.4) 을 활용한 Dashboard 만들기 Project
Stars: ✭ 121 (-94.79%)
Mutual labels:  logstash, elk, elasticsearch, kibana
Json Logging Python
Python logging library to emit JSON log that can be easily indexed and searchable by logging infrastructure such as ELK, EFK, AWS Cloudwatch, GCP Stackdriver
Stars: ✭ 143 (-93.84%)
Mutual labels:  logstash, elk, elasticsearch, kibana
Elk Stack
ELK Stack ... based on Elastic Stack 5.x
Stars: ✭ 148 (-93.63%)
Mutual labels:  logstash, elk, elasticsearch, kibana
Ansible Elk
📊 Ansible playbook for setting up an ELK/EFK stack and clients.
Stars: ✭ 284 (-87.77%)
Mutual labels:  logstash, elk, elasticsearch, kibana
Elk
搭建ELK日志分析平台。
Stars: ✭ 688 (-70.37%)
Mutual labels:  logstash, elk, elasticsearch, kibana
Docker monitoring logging alerting
Docker host and container monitoring, logging and alerting out of the box using cAdvisor, Prometheus, Grafana for monitoring, Elasticsearch, Kibana and Logstash for logging and elastalert and Alertmanager for alerting.
Stars: ✭ 479 (-79.37%)
Mutual labels:  logstash, elk, elasticsearch, kibana
Elk Docker
Docker configuration for ELK monitoring stack with Curator and Beats data shippers support
Stars: ✭ 342 (-85.27%)
Mutual labels:  logstash, elk, elasticsearch, kibana
Dynamite Nsm
DynamiteNSM is a free Network Security Monitor developed by Dynamite Analytics to enable network visibility and advanced cyber threat detection
Stars: ✭ 92 (-96.04%)
Mutual labels:  logstash, elasticsearch, netflow, kibana
Spring Boot Microservice Eureka Zuul Docker
Spring-Boot rest microservices using Eureka, Zuul, Docker. Monitoring with logstash, logback, elasticsearch, kibana
Stars: ✭ 45 (-98.06%)
Mutual labels:  logstash, elk, elasticsearch, kibana
Synesis lite suricata
Suricata IDS/IPS log analytics using the Elastic Stack.
Stars: ✭ 167 (-92.81%)
Mutual labels:  logstash, elk, elasticsearch, kibana
Docker Elastic Stack
ELK Stack Dockerfile
Stars: ✭ 175 (-92.46%)
Mutual labels:  logstash, elasticsearch, kibana
Elastic Stack
Aprenda Elasticsearch, Logstash, Kibana e Beats do jeito mais fácil ⭐️
Stars: ✭ 135 (-94.19%)
Mutual labels:  logstash, elasticsearch, kibana
Spring Cloud Microservices Development
Spring Cloud Microservices Development.《Spring Cloud 微服务架构开发实战》
Stars: ✭ 106 (-95.43%)
Mutual labels:  logstash, elasticsearch, kibana
Vagrant Elastic Stack
Giving the Elastic Stack a try in Vagrant
Stars: ✭ 131 (-94.36%)
Mutual labels:  logstash, elasticsearch, kibana
Elk Hole
elasticsearch, logstash and kibana configuration for pi-hole visualiziation
Stars: ✭ 136 (-94.14%)
Mutual labels:  logstash, elasticsearch, kibana
Logtrail
Kibana plugin to view, search & live tail log events
Stars: ✭ 1,343 (-42.16%)
Mutual labels:  logstash, elasticsearch, kibana

This legacy version of ElastiFlow™ is now deprecated.

Try the new ElastiFlow™!!!

PLEASE NOTE: This repository remains for historical purposes, however no part of this legacy solution is shared with the next generation of ElastiFlow™. To install the new solution please refer to the documentation at docs.elastiflow.com. For assistance with the new solution, please join the ElastiFlow Community Slack.

Now that the new ElastiFlow™ Unified Flow Collector is generally available, the legacy Logstash-based solution in this repository is deprecated. It will remain here for historical purposes, but will not receive updates. You are encouraged to try out the new collector. See more details below.

We have released the next generation of ElastiFlow™ which introduces the new ElastiFlow Unified Flow Collector for Netflow, IPFIX and sFlow.

Some of the enhancements included in the all-new collector include:

  • Proper management templates from different devices. - NO MORE template conflicts!
  • Improved Scalability - Over 10x more flows/sec than Logstash, over 3x more than Filebeat!
  • Support for Option Templates - Dynamic enrichment of network interface name and application names and more!
  • Telemetry Support - sFlow Counter Samples and other metrics from Cisco, Calix and more!
  • Gracefully handles “unknown” Fields - No more dropped flows when a single unsupported field is encountered.
  • Fully decodes and translates all available data - DSCP, TCP Options, ECN, Fragmentation Flags, and more.

Get Started with the next generation of ElastiFlow... HERE

You can also join the ElastiFlow community Slack... HERE

ElastiFlow™ provides network flow data collection and visualization using the Elastic Stack (Elasticsearch, Logstash and Kibana). It supports Netflow v5/v9, sFlow and IPFIX flow types (1.x versions support only Netflow v5/v9).

ElastiFlow™

I was inspired to create ElastiFlow™ following the overwhelmingly positive feedback received to an article I posted on Linkedin... WTFlow?! Are you really still paying for commercial solutions to collect and analyze network flow data?

User Testimonials

Organization Feedback
Uber "ElastiFlow has significantly reduced our network flow monitoring costs. The flexibility and extensibility inherent to ElastiFlow was essential for an easy integration with our infrastructure monitoring suite."
ESnet “Right now this is my personal favorite analytics tool. I use it extensively and am always finding a new way to leverage it."
Payback "We're using it since two months in our new datacenter and our network admins are very happy and impressed."
Catapult Systems "Of all the netflow tools I’ve tested it has, by far, the best visualizations."
Imagine Software "We absolutely love ElastiFlow and recently stood it up in production. Looking forward to new functionality and dashboards."

Getting Started

NOTE - Please refer to INSTALL.md for installation instructions.

ElastiFlow™ is built using the Elastic Stack, including Elasticsearch, Logstash and Kibana. Please refer to INSTALL.md for instructions on how to install and configure ElastiFlow™.

If you are new to the Elastic Stack, this video goes beyond a simple default installation of Elasticsearch and Kibana. It discusses real-world best practices for hardware sizing and configuration, providing production-level performance and reliability.

0003_es_install

Additionally local SSD storage should be considered as mandatory! For an in-depth look at how different storage options compare, and in particular how bad HDD-based storage is for Elasticsearch (even in multi-drive RAID0 configurations) you should watch this video...

0001_es_storage

NOTE: Please make sure that have reviewed KNOWN_ISSUES.md prior to getting started.

Kibana App Compatibility

SIEM

SIEM: Hosts

SIEM: Network Map

SIEM: Network Statistics

Logs

Logs

Provided Dashboards

The following dashboards are provided.

NOTE: The dashboards are optimized for a monitor resolution of 1920x1080.

Overview

Overview

Top-N

There are separate Top-N dashboards for Top Talkers, Services, Conversations and Applications.

Top Talkers

Top Talkers

Top Services

Top Services

Top Conversations

Top Conversations

Top Applications

Top Applications

Threats

ElastiFlow™ includes a dictionary of public IP addresses that are known to have a poor reputation. This dictionary is built from many OSINT data sources, normalized to a common taxonomy. The Threats dashboard uses this IP reputation information to highlight three threat/risk types.

  1. Public Threats - Public clients with a poor IP reputation that are reaching private addresses.
  2. At-Risk Servers - Private Servers that are being reached by clients with a poor IP reputation.
  3. High-Risk Clients - Private clients that are accessing public servers which have a poor reputation.

Threats

Flows

There are separate Sankey dashboards for Client/Server, Source/Destination and Autonomous System perspectives. The sankey visualizations are built using the new Vega visualization plugin.

Client/Server Flows

Client/Server Flows

Source/Destination Flows

Source/Destination Flows

Autonomous System Flows

Autonomous System Flows

Geo IP

There are separate Geo Location dashboards for Client/Server and Source/Destination perspectives.

Client/Server Geo IP

Client/Server Geo IP

Source/Destination Geo IP

Source/Destination Geo IP

AS Traffic

Provides a view of traffic to and from Autonomous Systems (public IP ranges)

AS Traffic

Flow Exporters

Flow Exporters

Traffic Details

Provides more detailed breakdown of various network traffic characteristics.

Traffic Types

Traffic Types

Traffic Attributes

Traffic Attributes

Traffic Locality

Traffic Locality

Flow Records

Client/Server

Client/Server

Source/Destination

Source/Destination

Ziften ZFlow

ElastiFlow™ v3.4.0 added support for IPFIX records from Ziften's ZFlow agent. In addition to being fully integrated with the standard dashboards, a stand-alone ZFlow dashboards displays network traffic based on user and command data provided by ZFlow. Ziften ZFlow

Attribution

This product includes GeoLite2 data created by MaxMind, available from (http://www.maxmind.com)

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].