younisshah / jakob

jakob

Jakob is fault-tolerant, distributed cluster of Redis servers with built-in load-balancing and fall-backs to provide data availability. Jakob is specifically meant to start a cluster of Tile38 servers to store geo-spatial data.

Jakob relies on Apache Kafka to store the logs and for log replication. It also relies on the amazing Machinery to sync data(logs) between servers in background. Jakob's Machinery setup uses RabbitMQ as broker and Redis as result-backend.

Jakob has two types of servers - setters and getters.

A setter server will receive all the Tile38 setter commands like (SET, NEARBY, FENCE, etc), while a getter server will always receive Tile38 getter commands like (GET, MATCH, etc).

The two clusters are servers are arranged in a consistent-hashed ring. A setter or getter server is selected using consistent hashing.

It just exposes two HTTP endpoints. Both of them are POST HTTP endpoints.

Installation:

go get -u github.com/younisshah/jakob

or using glide

glide get github.com/younisshah/jakob

and then glide up

Next install and start Apache Kafka. Refer to this for a quick start. Apache Kafka Quick Start. Download and install RabbitMQ from here RabbitMQ. Finally download and start Redis server from here Redis.

Sample usage:

Boot jakob:

boot.Up()

Initialize the cluster, use /init

curl -X POST localhost:30000/init -d '{"setter":"localhost:9000", "getter":"localhost:9001"}'

To join a cluster, use /join

curl -X POST localhost:30000/join -d '{"setter":"localhost:9002", "getter":"localhost:9003"}'

To get a setter peer from the cluster

ring := jring.New(jfs.SETTER)
peer, err := ring.Get("some_id")
if err != nil {
    log.Println(err)
}

To get getter peer from the cluster

ring := jring.New(jfs.GETTER)
peer, err := ring.Get("some_id")
if err != nil {
    log.Println(err)
}

To send a Tile38 command to jakob

cmd = command.NewCommand(peer, "SET", "my", "home", "POINT", "12.4", "23.45")
cmd.Execute()
if cmd.Error != nil {
    log.Println(cmd.Error)
} else {
    log.Println(cmd.Result)
}
// Do something with c.Result

Pipelined SET

cmds := []string{"SET", "SET"}
a1 := []interface{}{"fleet", "home", "POINT", "12.2", "32.56"}
a2 := []interface{}{"fleet", "office", "POINT", "20.2", "52.56"}
args := []interface{}{a1, a2}
c := command.NewPipelinedCommand("localhost:9851", cmds, args)
c.PipelinedExecute()
if c.Error != nil {
    // Handle error
}
// do something with c.Result

Jakob is still in infancy which means API is subject change.

TODO

more comprehensive documentation