All Projects → mesosphere-backup → Etcd Mesos

mesosphere-backup / Etcd Mesos

Licence: apache-2.0
self-healing etcd on mesos!

Programming Languages

go
31211 projects - #10 most used programming language

Projects that are alternatives of or similar to Etcd Mesos

Dcos
DC/OS - The Datacenter Operating System
Stars: ✭ 2,316 (+3305.88%)
Mutual labels:  mesos, dcos, cluster
container-orchestration
A Benchmark for Container Orchestration Systems
Stars: ✭ 19 (-72.06%)
Mutual labels:  cluster, mesos, dcos
dcos-deploy
Deploy, manage and orchestrate services and apps on DC/OS
Stars: ✭ 21 (-69.12%)
Mutual labels:  mesos, dcos
dcos-autoscaler
Autoscaler for DC/OS hosted in a cloud provider
Stars: ✭ 12 (-82.35%)
Mutual labels:  mesos, dcos
Cook
Fair job scheduler on Kubernetes and Mesos for batch workloads and Spark
Stars: ✭ 314 (+361.76%)
Mutual labels:  mesos, cluster
microservices-demo.github.io
The Microservices Demo website.
Stars: ✭ 65 (-4.41%)
Mutual labels:  mesos, dcos
dcos-k8s-rust-skaffold-demo
A demo of pipelining Rust application development to Kubernetes on DC/OS with Skaffold.
Stars: ✭ 40 (-41.18%)
Mutual labels:  mesos, dcos
Postgresql cluster
PostgreSQL High-Availability Cluster (based on "Patroni" and "DCS(etcd)"). Automating deployment with Ansible.
Stars: ✭ 294 (+332.35%)
Mutual labels:  etcd, cluster
Mr Redis
Redis on Apache Mesos
Stars: ✭ 127 (+86.76%)
Mutual labels:  framework, mesos
Marathon Lb
Marathon-lb is a service discovery & load balancing tool for DC/OS
Stars: ✭ 449 (+560.29%)
Mutual labels:  mesos, dcos
Minimesos
The experimentation and testing tool for Apache Mesos - NO LONGER MAINTANED!
Stars: ✭ 429 (+530.88%)
Mutual labels:  mesos, cluster
Gonet
go分布式服务器,基于内存mmo
Stars: ✭ 804 (+1082.35%)
Mutual labels:  etcd, cluster
marathon-slack
Integration for Marathon's Event Bus with Slack
Stars: ✭ 42 (-38.24%)
Mutual labels:  mesos, dcos
urb-k8s
Kubernetes adapter for Universal Resource Broker
Stars: ✭ 19 (-72.06%)
Mutual labels:  cluster, mesos
Actionhero
Actionhero is a realtime multi-transport nodejs API Server with integrated cluster capabilities and delayed tasks
Stars: ✭ 2,280 (+3252.94%)
Mutual labels:  framework, cluster
Reshifter
Kubernetes cluster state management
Stars: ✭ 292 (+329.41%)
Mutual labels:  etcd, cluster
Dcos Commons
DC/OS SDK is a collection of tools, libraries, and documentation for easy integration of technologies such as Kafka, Cassandra, HDFS, Spark, and TensorFlow with DC/OS.
Stars: ✭ 162 (+138.24%)
Mutual labels:  mesos, dcos
Traefik
The Cloud Native Application Proxy
Stars: ✭ 36,089 (+52972.06%)
Mutual labels:  etcd, mesos
Hyperf
🚀 A coroutine framework that focuses on hyperspeed and flexibility. Building microservice or middleware with ease.
Stars: ✭ 4,206 (+6085.29%)
Mutual labels:  etcd, framework
Linkerdcosdockerfile
Linker Dcos DockerFile&DockerCompose yml file
Stars: ✭ 8 (-88.24%)
Mutual labels:  mesos, dcos

[ALPHA] etcd-mesos

This is an Apache Mesos framework that runs an etcd cluster. It performs periodic health checks to ensure that the cluster has a stable leader and that raft is making progress. It replaces nodes that die.

Guides:

features

  • [x] runs, monitors, and administers an etcd cluster of your desired size
  • [x] recovers from n/2-1 failures by reconfiguring the etcd cluster and launching replacement nodes
  • [x] recovers from up to n-1 simultaneous failures by picking a survivor to re-seed a new cluster (ranks survivors by raft index, prefering the replica with the highest commit)
  • [x] etcd proxy configurer (etcd-mesos-proxy) and optional SRV record support via mesos-dns

running

Marathon spec:

{
  "id": "etcd",
  "container": {
    "docker": {
      "forcePullImage": true,
      "image": "mesosphere/etcd-mesos:0.1.0-alpha-target-23-24-25"
    },
    "type": "DOCKER"
  },
  "cpus": 0.2,
  "env": {
    "FRAMEWORK_NAME": "etcd",
    "WEBURI": "http://etcd.marathon.mesos:$PORT0/stats",
    "MESOS_MASTER": "zk://master.mesos:2181/mesos",
    "ZK_PERSIST": "zk://master.mesos:2181/etcd",
    "AUTO_RESEED": "true",
    "RESEED_TIMEOUT": "240",
    "CLUSTER_SIZE": "3",
    "CPU_LIMIT": "1",
    "DISK_LIMIT": "4096",
    "MEM_LIMIT": "2048",
    "VERBOSITY": "1"
  },
  "healthChecks": [
    {
      "gracePeriodSeconds": 60,
      "intervalSeconds": 30,
      "maxConsecutiveFailures": 0,
      "path": "/healthz",
      "portIndex": 0,
      "protocol": "HTTP"
    }
  ],
  "instances": 1,
  "mem": 128.0,
  "ports": [
    0,
    1,
    2
  ]
}

building

First, check out the tagged release for your version of Mesos.

For Mesos versions 23, 24, or 25, check out v0.1.0-alpha-target-23-24-25

For Mesos versions 22 and below (the farther below, the less the chances of compatibility), check out v0.1.0-alpha-target-22

Next, built it!

make

The important binaries (etcd-mesos-scheduler, etcd-mesos-proxy, etcd-mesos-executor) are now present in the bin subdirectory.

A typical production invocation will look something like this:

/path/to/etcd-mesos-scheduler \
    -log_dir=/var/log/etcd-mesos \
    -master=zk://zk1:2181,zk2:2181,zk3:2181/mesos \
    -framework-name=etcd \
    -cluster-size=5 \
    -executor-bin=/path/to/etcd-mesos-executor \
    -etcd-bin=/path/to/etcd \
    -etcdctl-bin=/path/to/etcdctl \
    -zk-framework-persist=zk://zk1:2181,zk2:2181,zk3:2181/etcd-mesos

If you'd like to build a new docker container, change the DOCKER_ORG and VERSION in the Makefile, and then run:

make docker

service discovery

Options for finding your etcd nodes on mesos:

  • Run the included proxy binary locally on systems that use etcd. It retrieves the etcd configuration from mesos and starts an etcd proxy node. Note that this it not a good idea on clusters with lots of tasks running, as the master will iterate through each task and spit out a fairly large chunk of JSON, so this approach should be avoided in favor of mesos-dns on larger clusters.
etcd-mesos-proxy --master=zk://localhost:2181/mesos --framework-name=etcd
  • Use mesos-dns or another system that creates SRV records and have an etcd proxy use SRV discovery:
etcd --proxy=on --discovery-srv=etcd.mesos
  • Use Mesos DNS or another DNS SRV system and have clients resolve _etcd-server._client.<framework name>.mesos

  • Use another system that builds configuration from mesos's state.json endpoint. This is how #1 works, so check out the code for it in cmd/etcd-mesos-proxy/app.go if you want to go this route. Be sure to minimize calls to the master for state.json on larger clusters, as this becomes an expensive operation that can easily DDOS your master if you are not careful.

  • Current membership may be queried from the etcd-mesos-scheduler's /members http endpoint that listens on the --admin-port (default 23400)

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].