ServiceQ 
ServiceQ is a fault-tolerant gateway for HTTP clusters. It employs probabilistic routing to distribute load during partial cluster shutdown (k/n nodes experiencing downtimes, timeouts, connection loss etc) and queues requests during total cluster shutdown (n nodes down). The queued requests are forwarded in FIFO order when the cluster is available next.
Below graph shows the routing probability (P) on a down node (D) in a 8-node cluster with respect to number of requests (r). Notice how quickly the routing probability on D reduces as the requests on D start to fail. Depending on the rate of request, it will only take a few seconds (sometime even milliseconds) to move all requests away from D, thus ensuring more requests are routed to healthier nodes.
Note that, even when requests keep failing on D (however less), ServiceQ retries them on other nodes until they succeed. If they do not succeed on any of the nodes, they are queued and periodically retried on the cluster (using the same approach above), until they succeed.
Noticeable features
- HTTP Load Balancing
- Probabilistic node selection based on error feedback
- Failed request queueing and deferred forwarding
- Upfront request queueing
- Request retries
- Concurrent connections limit
- Complete TLS/SSL support (automatic and manual)
Here are the steps to run ServiceQ -
Download
Clone the project into any directory in your workspace
$ git clone https://github.com/gptankit/serviceq
Change into directory serviceq
How to Build
$ make ('make build' will also work)
Optional: make with debug symbols removed (~25% size reduction)
$ make build-nodbg
This will create a Go binary serviceq in the current directory
How to Install
Make sure the current user has root privileges, then -
$ make install
This will create a folder serviceq in /usr/local directory and copy the serviceq binary (generated in the build step) to /usr/local/serviceq and sq.properties (serviceq configuration file) to /usr/local/serviceq/config.
How to Run
Before running, make sure the mandatory configurations in /usr/local/serviceq/config/sq.properties are set (LISTENER_PORT, PROTO, ENDPOINTS, CONCURRENCY_PEAK). The configuration file closely resembles a typical INI file so its fairly easy to understand and make changes -
#sq.properties #Port on which serviceq listens on LISTENER_PORT=5252 #Protocol the endpoints listens on -- 'http' for both http/https PROTO=http #Endpoints seperated by comma (,) -- no spaces allowed, can be a combination of http/https ENDPOINTS=http://my.server1.com:8080,http://my.server2.com:8080,http://my.server3.com:8080 #Concurrency peak defines how many max concurrent connections are allowed to the cluster CONCURRENCY_PEAK=2048
Also, verify timeout value (default is set to 5s). Low value is preferable as it allows retries to be faster -
#Timeout (s) is added to each outgoing request to endpoints, the existing timeouts are overriden, value of -1 means no timeout OUTGOING_REQUEST_TIMEOUT=5
By default deferred queue is enabled with all methods and routes allowed. These options can be controlled as -
#Enable deferred queue for requests on final failures (cluster down) ENABLE_DEFERRED_Q=true #Request format allows given method/route on deferred queue -- picked up if ENABLE_DEFERRED_Q is true #Q_REQUEST_FORMATS=POST /orders,PUT,PATCH,DELETE #Q_REQUEST_FORMATS=ALL Q_REQUEST_FORMATS=POST,PUT,PATCH,DELETE
(Note that Q_REQUEST_FORMATS is also considered if ENABLE_UPFRONT_Q is true)
After all is set -
$ sudo /usr/local/serviceq/serviceq
Refer wiki for more details: https://github.com/gptankit/serviceq/wiki
Feel free to play around and post feedbacks