All Projects → hseeberger → Constructr

hseeberger / Constructr

Licence: apache-2.0
Coordinated (etcd, ...) cluster construction for dynamic (cloud, containers) environments

Programming Languages

scala
5932 projects

Labels

Projects that are alternatives of or similar to Constructr

Akka
Build highly concurrent, distributed, and resilient message-driven applications on the JVM
Stars: ✭ 11,938 (+5351.14%)
Mutual labels:  akka
Swarmstack
A Docker swarm-based starting point for operating highly-available containerized applications.
Stars: ✭ 181 (-17.35%)
Mutual labels:  etcd
Kafka With Akka Streams Kafka Streams Tutorial
Code samples for the Lightbend tutorial on writing microservices with Akka Streams, Kafka Streams, and Kafka
Stars: ✭ 204 (-6.85%)
Mutual labels:  akka
Discovery.etcd.io
etcd discovery service
Stars: ✭ 160 (-26.94%)
Mutual labels:  etcd
Akka Stream Contrib
Add-ons to Akka Stream
Stars: ✭ 173 (-21%)
Mutual labels:  akka
Jvm Bloggers
JVM Bloggers - website and newsletter with JVM blogs from Poland
Stars: ✭ 193 (-11.87%)
Mutual labels:  akka
Dotnet Etcd
A C# .NET (dotnet) GRPC client for etcd v3 +
Stars: ✭ 157 (-28.31%)
Mutual labels:  etcd
Skipper
An HTTP router and reverse proxy for service composition, including use cases like Kubernetes Ingress
Stars: ✭ 2,606 (+1089.95%)
Mutual labels:  etcd
Otoroshi
Lightweight api management on top of a modern http reverse proxy
Stars: ✭ 177 (-19.18%)
Mutual labels:  akka
Remco
remco is a lightweight configuration management tool
Stars: ✭ 200 (-8.68%)
Mutual labels:  etcd
Pifpaf
Python fixtures and daemon managing tools for functional testing
Stars: ✭ 161 (-26.48%)
Mutual labels:  etcd
Lighthouse
Lighthouse - a simple service discovery platform for Akka.Cluster (Akka.NET)
Stars: ✭ 164 (-25.11%)
Mutual labels:  akka
Cookim
Distributed web chat application base websocket built on akka.
Stars: ✭ 198 (-9.59%)
Mutual labels:  akka
Etcd Watcher
Etcd watcher for Casbin
Stars: ✭ 157 (-28.31%)
Mutual labels:  etcd
Source Code Reading Notes
源码阅读笔记
Stars: ✭ 207 (-5.48%)
Mutual labels:  etcd
Safe Chat
IRC-style chat demo featuring full-stack F#, Akka.Streams, Akkling, Fable, Elmish, Websockets and .NET Core
Stars: ✭ 157 (-28.31%)
Mutual labels:  akka
Go Flagz
Dynamic flag management for Go.
Stars: ✭ 191 (-12.79%)
Mutual labels:  etcd
Akka Management
Akka Management is a suite of tools for operating Akka Clusters.
Stars: ✭ 218 (-0.46%)
Mutual labels:  akka
Dbtester
Distributed database benchmark tester
Stars: ✭ 214 (-2.28%)
Mutual labels:  etcd
Dcmp
基于etcd的配置管理系统 (etcd v2)
Stars: ✭ 200 (-8.68%)
Mutual labels:  etcd

ConstructR

Join the chat at https://gitter.im/hseeberger/constructr

ConstructR is for bootstrapping (construction) an Akka cluster by using a coordination service.

Disambiguation: Despite the similar name, ConstructR is not related to Lightbend ConductR.

ConstructR utilizes a key-value coordination service like etcd to automate bootstrapping or joining a cluster. It stores each member node under the key /constructr/$clusterName/nodes/$address where $clusterName is for disambiguating multiple clusters and $address is a Base64 encoded Akka Address. These keys expire after a configurable time in order to avoid stale information. Therefore ConstructR refreshes each key periodically.

In a nutshell, ConstructR is a state machine which first tries to get the nodes from the coordination service. If none are available it tries to acquire a lock, e.g. via a CAS write for etcd, and uses itself or retries getting the nodes. Then it joins using these nodes as seed nodes. After that it adds its address to the nodes and starts the refresh loop:

                  ┌───────────────────┐              ┌───────────────────┐
              ┌──▶│   GettingNodes    │◀─────────────│BeforeGettingNodes │
              │   └───────────────────┘    delayed   └───────────────────┘
              │             │     │                            ▲
  join-failed │   non-empty │     └──────────────────────┐     │ failure
              │             ▼               empty        ▼     │
              │   ┌───────────────────┐              ┌───────────────────┐
              └───│      Joining      │◀─────────────│      Locking      │
                  └───────────────────┘    success   └───────────────────┘
                            │
              member-joined │
                            ▼
                  ┌───────────────────┐
                  │    AddingSelf     │
                  └───────────────────┘
                            │     ┌────────────────────────────┐
                            │     │                            │
                            ▼     ▼                            │
                  ┌───────────────────┐              ┌───────────────────┐
                  │ RefreshScheduled  │─────────────▶│    Refreshing     │
                  └───────────────────┘              └───────────────────┘

If something goes finally wrong when interacting with the coordination service, e.g. a permanent timeout after a configurable number of retries, ConstructR terminates its ActorSystem in the spirit of "fail fast".

// All releases including intermediate ones are published here,
// final ones are also published to Maven Central.
resolvers += Resolver.bintrayRepo("hseeberger", "maven")

libraryDependencies ++= Vector(
  "de.heikoseeberger" %% "constructr" % "0.19.0",
  "de.heikoseeberger" %% "constructr-coordination-etcd" % "0.19.0", // in case of using etcd for coordination
  ...
)

Simply add the ConstructrExtension to the extensions configuration setting:

akka.extensions = [de.heikoseeberger.constructr.ConstructrExtension]

This will start the Constructr actor as a system actor. Alternatively start it yourself as early as possible if you feel so inclined.

The following listing shows the available configuration settings with their defaults:

constructr {
  coordination {
    host = localhost
    port = 2379
  }

  coordination-timeout    = 3 seconds  // Maximum response time for coordination service (e.g. etcd)
  join-timeout            = 15 seconds // Might depend on cluster size and network properties
  abort-on-join-timeout   = false      // Abort the attempt to join if true; otherwise restart the process from scratch
  max-nr-of-seed-nodes    = 0          // Any nonpositive value means Int.MaxValue
  nr-of-retries           = 2          // Nr. of tries are nr. of retries + 1
  refresh-interval        = 30 seconds // TTL is refresh-interval * ttl-factor
  retry-delay             = 3 seconds  // Give coordination service (e.g. etcd) some delay before retrying
  ttl-factor              = 2.0        // Must be greater or equal 1 + ((coordination-timeout * (1 + nr-of-retries) + retry-delay * nr-of-retries)/ refresh-interval)!
  ignore-refresh-failures = false      // Ignore failures once machine is already in "Refreshing" state. It prevents from FSM being terminated due to exhausted number of retries.

}

Coordination

ConstructR comes with out-of-the-box support for etcd: simply depend on the "constructr-coordination-etcd" module. If you want to use some other coordination backend, e.g. Consul, simply implement the Coordination trait from the "constructr-coordination" module and make sure to provide the fully qualified class name via the constructr.coordination.class-name configuration setting.

Community Coordination Implementations

There are some implementations for other coordination backends than etcd:

Testing

etcd must be running, e.g.:

docker run \
  --detach \
  --name etcd \
  --publish 2379:2379 \
  quay.io/coreos/etcd:v2.3.8 \
  --listen-client-urls http://0.0.0.0:2379 \
  --advertise-client-urls http://192.168.99.100:2379

Contribution policy

Contributions via GitHub pull requests are gladly accepted from their original author. Along with any pull requests, please state that the contribution is your original work and that you license the work to the project under the project's open source license. Whether or not you state this explicitly, by submitting any copyrighted material via pull request, email, or other means you agree to license the material under the project's open source license and warrant that you have the legal authority to do so.

Please make sure to follow these conventions:

  • For each contribution there must be a ticket (GitHub issue) with a short descriptive name, e.g. "Respect seed-nodes configuration setting"
  • Work should happen in a branch named "ISSUE-DESCRIPTION", e.g. "32-respect-seed-nodes"
  • Before a PR can be merged, all commits must be squashed into one with its message made up from the ticket name and the ticket id, e.g. "Respect seed-nodes configuration setting (closes #32)"

License

This code is open source software licensed under the Apache 2.0 License.

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].