All Projects → adibendahan → Mysqlbeat

adibendahan / Mysqlbeat

Licence: other
Fully customizable Beat for MySQL server - this beat will ship the results of any query defined in the config file to Elasticsearch.

Programming Languages

go
31211 projects - #10 most used programming language

Projects that are alternatives of or similar to Mysqlbeat

Szt Bigdata
深圳地铁大数据客流分析系统🚇🚄🌟
Stars: ✭ 826 (+734.34%)
Mutual labels:  mysql, elasticsearch, kibana
Microservices Sample
Sample project to create an application using microservices architecture
Stars: ✭ 167 (+68.69%)
Mutual labels:  mysql, elasticsearch, kibana
Elasticsearch Cli
Command line interface for ElasticSearch
Stars: ✭ 70 (-29.29%)
Mutual labels:  elasticsearch, kibana
Ebean
Ebean ORM
Stars: ✭ 1,172 (+1083.84%)
Mutual labels:  mysql, elasticsearch
Helm Elasticstack
Kubernetes Helm Charts and Tools to run Elastic Stack(ELK) on Azure Container Service(AKS)
Stars: ✭ 76 (-23.23%)
Mutual labels:  elasticsearch, kibana
Aws Config To Elasticsearch
Generates an AWS Config Snapshot and ingests it into ElasticSearch for further analysis using Kibana
Stars: ✭ 62 (-37.37%)
Mutual labels:  elasticsearch, kibana
Mall Learning
mall学习教程,架构、业务、技术要点全方位解析。mall项目(40k+star)是一套电商系统,使用现阶段主流技术实现。涵盖了SpringBoot 2.3.0、MyBatis 3.4.6、Elasticsearch 7.6.2、RabbitMQ 3.7.15、Redis 5.0、MongoDB 4.2.5、Mysql5.7等技术,采用Docker容器化部署。
Stars: ✭ 10,236 (+10239.39%)
Mutual labels:  mysql, elasticsearch
Logstash
OSSEC + Logstash + Elasticsearch + Kibana
Stars: ✭ 74 (-25.25%)
Mutual labels:  elasticsearch, kibana
Mysql Es
Sync MySQL to ElasticSearch, Support Relationship
Stars: ✭ 53 (-46.46%)
Mutual labels:  mysql, elasticsearch
Supermarket
设计精良的网上商城系统,包括前端、后端、数据库、负载均衡、数据库缓存、分库分表、读写分离、全文检索、消息队列等,使用SpringCloud框架,基于Java开发。该项目可部署到服务器上,不断完善中……
Stars: ✭ 1,278 (+1190.91%)
Mutual labels:  mysql, elasticsearch
Sentinl
Kibana Alert & Report App for Elasticsearch
Stars: ✭ 1,233 (+1145.45%)
Mutual labels:  elasticsearch, kibana
Systemdlogger
Exports systemd logs to an external service, eg cloudwatch, elasticsearch
Stars: ✭ 91 (-8.08%)
Mutual labels:  elasticsearch, kibana
Pivot Kibana
Flexmonster Pivot plugin for Kibana
Stars: ✭ 58 (-41.41%)
Mutual labels:  elasticsearch, kibana
Flume Canal Source
Flume NG Canal source
Stars: ✭ 56 (-43.43%)
Mutual labels:  mysql, elasticsearch
Docker Elk Tutorial
docker-elk-tutorial + django + logging
Stars: ✭ 69 (-30.3%)
Mutual labels:  elasticsearch, kibana
Aspnetcorenlog
ASP.NET Core NLog MS SQL Server PostgreSQL MySQL Elasticsearch
Stars: ✭ 54 (-45.45%)
Mutual labels:  mysql, elasticsearch
Jmeter Elasticsearch Backend Listener
JMeter plugin that lets you send sample results to an ElasticSearch engine to enable live monitoring of load tests.
Stars: ✭ 72 (-27.27%)
Mutual labels:  elasticsearch, kibana
Search Guard Docs
Official documentation for Search Guard, the Elasticsearch security suite
Stars: ✭ 92 (-7.07%)
Mutual labels:  elasticsearch, kibana
Phalcon Vm
Vagrant configuration for PHP7, Phalcon 3.x and Zephir development.
Stars: ✭ 43 (-56.57%)
Mutual labels:  mysql, elasticsearch
Spring Boot Microservice Eureka Zuul Docker
Spring-Boot rest microservices using Eureka, Zuul, Docker. Monitoring with logstash, logback, elasticsearch, kibana
Stars: ✭ 45 (-54.55%)
Mutual labels:  elasticsearch, kibana

mysqlbeat

Fully customizable Beat for MySQL server - this beat will ship the results of any query defined in the config file to Elasticsearch.

Current status

First beta release available here.

Features

  • Connect to any MySQL server and run queries
  • single-row queries will be translated as columnname:value.
  • two-columns will be translated as value-column1:value-column2 for each row.
  • multiple-rows each row will be a document (with columnname:value) NEW: Added DELTA support.
  • show-slave-delay will only send the "Seconds_Behind_Master" column from SHOW SLAVE STATUS;
  • Any column that ends with the delatwildcard (default is __DELTA) will send delta results, extremely useful for server counters. ((newval - oldval)/timediff.Seconds())
  • MySQL Performance Dashboard (more details below)

How to Build

mysqlbeat uses Glide for dependency management. To install glide see: https://github.com/Masterminds/glide

$ glide update --no-recursive
$ make

Default Configuration

Edit mysqlbeat configuration in mysqlbeat.yml . You can:

  • Add queries to the queries array
  • Add query types to the querytypes array
  • Define Username/Password to connect to the MySQL
  • Define the column wild card for delta columns
  • Define the column wild card for delta key columns
  • Password can be saved in clear text/AES encryption

If you choose to use the mysqlbeat as is, just run the following on your MySQL Server:

 GRANT REPLICATION CLIENT, PROCESS ON *.* TO 'mysqlbeat_user'@'%' IDENTIFIED BY 'mysqlbeat_pass';

Notes on password encryption: Before you compile your own mysqlbeat, you should put a new secret in the code (defined as a const), secret length must be 16, 24 or 32, corresponding to the AES-128, AES-192 or AES-256 algorithm. I recommend deleting the secret from the source code after you have your compiled mysqlbeat. You can encrypt your password with mysqlbeat-password-encrypter just update your secret (and commonIV if you choose to change it) and compile.

Template

The default template is provided, if you add any queries you should update the template accordingly.

To apply the default template run: curl -XPUT http://<host>:9200/_template/mysqlbeat [email protected]/mysqlbeat-template.json

How to use

Just run mysqlbeat -c mysqlbeat.yml and you are good to go.

MySQL Performance Dashboard by mysqlbeat

This dashboard created as an addition to the MySQL dashboard provided by packetbeat, use them both. Run the default configuration provided to get the dashboard below (you should import dashboard/mysql_performance_dashboard_by_mysqlbeat.json to create the dashboard in Kibana).

mysql_performance_by_mysqlbeat__dashboard__kibana

License

GNU General Public License v2

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].