GitPlanet

Cheap and reliable Node.js hosting starts at $3/month, and $1/month static HTML hosting

Created with love in Canada, visit hostnodejs.com today

Feel like to post an Ad? Learn Details
All Projects → PBWebMedia → airflow-prometheus-exporter

PBWebMedia / airflow-prometheus-exporter

Licence: MIT license
Export Airflow metrics (from mysql) in prometheus format

Programming Languages

go
31211 projects - #10 most used programming language
Dockerfile
14818 projects
Makefile
30231 projects

Labels

mysqlairflowmetricsexporterapacheprometheusapache-airflow

Projects that are alternatives of or similar to airflow-prometheus-exporter

Airflow
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
Stars: ✭ 24,101 (+96304%)
Mutual labels:  airflow, apache, apache-airflow
Druid Exporter
A Golang based exporter captures druid API related metrics and receives druid-emitting HTTP JSON data.
Stars: ✭ 54 (+116%)
Mutual labels:  exporter, apache, prometheus
airflow-client-python
Apache Airflow - OpenApi Client for Python
Stars: ✭ 172 (+588%)
Mutual labels:  airflow, apache, apache-airflow
airflow-user-management-plugin
A plugin for Apache Airflow that allows you to manage the users that can login
Stars: ✭ 13 (-48%)
Mutual labels:  airflow, apache-airflow
airflow-code-editor
A plugin for Apache Airflow that allows you to edit DAGs in browser
Stars: ✭ 195 (+680%)
Mutual labels:  airflow, apache-airflow
ap-airflow
Astronomer Core Docker Images
Stars: ✭ 87 (+248%)
Mutual labels:  airflow, apache
Clickhouse exporter
This is a simple server that periodically scrapes ClickHouse stats and exports them via HTTP for Prometheus(https://prometheus.io/) consumption.
Stars: ✭ 193 (+672%)
Mutual labels:  exporter, prometheus
Awesome Apache Airflow
Curated list of resources about Apache Airflow
Stars: ✭ 2,755 (+10920%)
Mutual labels:  airflow, apache-airflow
Beyond Jupyter
🐍💻📊 All material from the PyCon.DE 2018 Talk "Beyond Jupyter Notebooks - Building your own data science platform with Python & Docker" (incl. Slides, Video, Udemy MOOC & other References)
Stars: ✭ 135 (+440%)
Mutual labels:  airflow, apache
docker-pulls
A simple compose script to use in conjunction with the prometheus stack to monitor Docker pulls.
Stars: ✭ 19 (-24%)
Mutual labels:  exporter, prometheus
yarn-prometheus-exporter
Export Hadoop YARN (resource-manager) metrics in prometheus format
Stars: ✭ 44 (+76%)
Mutual labels:  exporter, apache
openverse-catalog
Identifies and collects data on cc-licensed content across web crawl data and public apis.
Stars: ✭ 27 (+8%)
Mutual labels:  airflow, apache-airflow
Github Exporter
Prometheus exporter for github metrics
Stars: ✭ 231 (+824%)
Mutual labels:  exporter, prometheus
Awesome Prometheus Alerts
🚨 Collection of Prometheus alerting rules
Stars: ✭ 3,323 (+13192%)
Mutual labels:  exporter, prometheus
airflow-site
Apache Airflow Website
Stars: ✭ 95 (+280%)
Mutual labels:  airflow, apache
fairflow
Functional Airflow DAG definitions.
Stars: ✭ 38 (+52%)
Mutual labels:  airflow, apache-airflow
Airflow Exporter
Airflow plugin to export dag and task based metrics to Prometheus.
Stars: ✭ 161 (+544%)
Mutual labels:  airflow, prometheus
Jenkins exporter
Prometheus Metrics exporter for Jenkins
Stars: ✭ 152 (+508%)
Mutual labels:  exporter, prometheus
Event exporter
Exporter for kubernetes events
Stars: ✭ 179 (+616%)
Mutual labels:  exporter, prometheus
Insight-GDELT-Feed
A way for home buyers to know about factors affecting a state
Stars: ✭ 43 (+72%)
Mutual labels:  airflow, apache
View All Similar Projects ➔

Airflow prometheus exporter

Docker Hub Docker Pulls Docker Stars

Export airflow metrics in Prometheus format.

Build

Requires Go. Tested with Go 1.9+.

go get
go build -o airflow-prometheus-exporter .

Run

The exporter can be configured using environment variables. These are the defaults:

AIRFLOW_PROMETHEUS_LISTEN_ADDR=:9112
AIRFLOW_PROMETHEUS_DATABASE_BACKEND=mysql
AIRFLOW_PROMETHEUS_DATABASE_HOST=localhost
AIRFLOW_PROMETHEUS_DATABASE_PORT=3306
AIRFLOW_PROMETHEUS_DATABASE_USER=airflow
AIRFLOW_PROMETHEUS_DATABASE_PASSWORD=airflow
AIRFLOW_PROMETHEUS_DATABASE_NAME=airflow

When using postgres, SSL can be enabled and configured using:

AIRFLOW_PROMETHEUS_POSTGRES_SSL_MODE=verify-full
AIRFLOW_PROMETHEUS_POSTGRES_SSL_CERT=/path/to/certificate
AIRFLOW_PROMETHEUS_POSTGRES_SSL_KEY=/path/to/key
AIRFLOW_PROMETHEUS_POSTGRES_SSL_ROOT_CERT=/path/to/root_cert

Run the exporter:

./airflow-prometheus-exporter

The metrics can be scraped from:

http://localhost:9112/metrics

Run using docker

Run using docker:

docker run -p 9112:9112 pbweb/airflow-prometheus-exporter

Or using docker-compose:

services:
    image: pbweb/airflow-prometheus-exporter
    restart: always
    environment:
        - "AIRFLOW_PROMETHEUS_DATABASE_HOST=mysql.airflow.lan"
    ports:
        - "9112:9112"

License

See LICENSE.md

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].