All Projects → lipanski → slow-down

lipanski / slow-down

Licence: MIT License
A centralized Redis-based lock to help you wait on throttled resources

Programming Languages

ruby
36898 projects - #4 most used programming language
shell
77523 projects

Projects that are alternatives of or similar to slow-down

distlock
The universal component of distributed locks in golang , support redis and postgresql
Stars: ✭ 60 (+185.71%)
Mutual labels:  lock, locks, distributed
Redislock
Simplified distributed locking implementation using Redis
Stars: ✭ 370 (+1661.9%)
Mutual labels:  lock, distributed
Redisson
Redisson - Redis Java client with features of In-Memory Data Grid. Over 50 Redis based Java objects and services: Set, Multimap, SortedSet, Map, List, Queue, Deque, Semaphore, Lock, AtomicLong, Map Reduce, Publish / Subscribe, Bloom filter, Spring Cache, Tomcat, Scheduler, JCache API, Hibernate, MyBatis, RPC, local cache ...
Stars: ✭ 17,972 (+85480.95%)
Mutual labels:  lock, distributed
Redlock Php
Redis distributed locks in PHP
Stars: ✭ 651 (+3000%)
Mutual labels:  lock, distributed
Foundatio
Pluggable foundation blocks for building distributed apps.
Stars: ✭ 1,365 (+6400%)
Mutual labels:  lock, distributed
Diplomat
A HTTP Ruby API for Consul
Stars: ✭ 358 (+1604.76%)
Mutual labels:  lock, distributed
Pottery
Redis for humans. 🌎🌍🌏
Stars: ✭ 204 (+871.43%)
Mutual labels:  lock, distributed
mutexsafe
MutexSafe will help you use mutex more effectively. Different mutex for different components are presented. In addition, you can add your own lockers and use within the library.
Stars: ✭ 15 (-28.57%)
Mutual labels:  lock, redis-lock
wechat gate
隔壁老李的微信开发Ruby Gem
Stars: ✭ 53 (+152.38%)
Mutual labels:  gem
aiorwlock
Read/Write Lock - synchronization primitive for asyncio
Stars: ✭ 90 (+328.57%)
Mutual labels:  lock
dtm
A distributed transaction framework that supports multiple languages, supports saga, tcc, xa, 2-phase message, outbox patterns.
Stars: ✭ 6,110 (+28995.24%)
Mutual labels:  distributed
priora
An Object Prioritization Utility for Ruby
Stars: ✭ 30 (+42.86%)
Mutual labels:  gem
filtered
Filters ActiveRecord queries in a nice way
Stars: ✭ 28 (+33.33%)
Mutual labels:  gem
historical-bank-ruby
A Ruby Bank that serves historical exchange rates
Stars: ✭ 14 (-33.33%)
Mutual labels:  gem
http bench
golang HTTP stress test tool, support single and distributed
Stars: ✭ 142 (+576.19%)
Mutual labels:  distributed
visual-call-graph
Visual Call Graph will help you visualize how a method works by creating a call graph of all the other methods called.
Stars: ✭ 46 (+119.05%)
Mutual labels:  gem
angular-lock
No description or website provided.
Stars: ✭ 16 (-23.81%)
Mutual labels:  lock
delta
DDD-centric event-sourcing library for the JVM
Stars: ✭ 15 (-28.57%)
Mutual labels:  distributed
miniDFS
Simple version of Distributed File System on multiple threads
Stars: ✭ 38 (+80.95%)
Mutual labels:  distributed
graphql-rails logger
Display GraphQL queries in a more readable format
Stars: ✭ 102 (+385.71%)
Mutual labels:  gem

SlowDown

Gem Version Build Status

Why would you want to slow down your requests?!

Some external APIs might be throttling your requests (or web scraping attempts) or your own infrastructure is not able to bear the load. It sometimes pays off to be patient...

SlowDown delays a call up until the point where you can afford to trigger it. It relies on a Redis lock so it should be able to handle a cluster of servers all going for the same resource. It's based on the PX and NX options of the Redis SET command, which should make it thread-safe. Note that these options were introduced with Redis version 2.6.12.

Installation

Add it to your Gemfile:

gem "slow_down"

...and call bundle install.

Usage

Basic

require "slow_down"

SlowDown.config do |c|
  c.requests_per_second = 10
  c.retries = 50 # times
  c.timeout = 5 # seconds
  c.raise_on_timeout = true # will raise SlowDown::Timeout
  c.redis_url = "redis://localhost:6379/0" # or set the REDIS_URL environment variable
end

SlowDown.run do
  some_throttled_api_call # accepting only 10 req/sec
end

Groups

SlowDown can be configured for individual groups, which can be run in isolation:

SlowDown.config(:github) do |c|
  c.requests_per_second = 50
  c.timeout = 10
end

SlowDown.config(:twitter) do |c|
  c.requests_per_second = 10
  c.timeout = 1
end

# Acquire a lock for the :github group
SlowDown.run(:github) { ... }

# Acquire a lock for the :twitter group
SlowDown.run(:twitter) { ... }

Retrieve configuration

When called without a block, SlowDown.config will return the configuration of the default group. In order to fetch the configuration of a different group use SlowDown.config(:group_name).

Inline configuration

SlowDown may also be configured directly within the SlowDown.run call:

# Configure the :default group and run a call
SlowDown.run(requests_per_second: 5, timeout: 15, raise_on_timeout: true) do
  # ...
end

# Configure a different group and run a call within that group
SlowDown.run(:my_group, requests_per_second: 2, timeout: 1) do
  # ...
end

Defaults & available options

SlowDown.config do |c|
  # The allowed number of calls per second.
  c.requests_per_second = 10

  # The number of seconds during which SlowDown will try and acquire the resource
  # for a given call.
  c.timeout = 5

  # Whether to raise an error when the timeout was reached and the resource could
  # not be acquired.
  # Raises SlowDown::Timeout.
  c.raise_on_timeout = false

  # How many retries should be performed til the timeout is reached.
  c.retries = 30

  # The algorithm used to schedule the amount of time to wait between retries.
  # Available strategies: :linear, :inverse_exponential, :fibonacci or a class
  # extending SlowDown::Strategy::Base.
  c.retry_strategy = :linear

  # Redis can be configured either directly, by setting a Redis instance to this
  # variable, or via the REDIS_URL environment variable or via the redis_url
  # setting.
  c.redis = nil

  # Configure Redis via the instance URL.
  c.redis_url = nil

  # The Redis namespace to apply to all locks.
  c.redis_namespace = :slow_down

  # The namespace to apply to the default group.
  # Individual groups will overwrite this with the group name.
  c.lock_namespace = :default

  # Set this to a path or file descriptor in order to log to file.
  c.log_path = STDOUT

  # By default, the SlowDown logger is disabled.
  # Set this to Logger::DEBUG, Logger::INFO or Logger::ERROR for logging various
  # runtime information.
  c.log_level = Logger::UNKNOWN
end

Non-blocking checks

A call to .run will halt until the resource is either acquired or the timeout kicks in. In order to make a non-blocking check, you can use the SlowDown.free? method.

SlowDown.config do |c|
  c.requests_per_second = 2
end

SlowDown.free? # true
SlowDown.free? # true
SlowDown.free? # false (won't wait)
sleep(1)
SlowDown.free? # true

The SlowDown.free? method also works with groups and inline configuration:

def register_user(name, address, phone)
  user.name = name

  # Optional: geocode address, if we didn't exceed the request limit
  if SlowDown.free?(:geocoding, requests_per_second: 5)
    user.coordinates = geocode(address)
  end

  # Optional: send SMS, if we didn't exceed the request limit
  if SlowDown.free?(:sms, requests_per_second: 10)
    send_sms(phone)
  end

  user.save
end

Resetting the locks

If you ever need to reset the locks, you can do that for any group by calling:

SlowDown.reset(:group_name)

Polling strategies

When a request is placed that can't access the lock right away, SlowDown puts it to sleep and schedules it to wake up & try again for the amount of retries configured by the user (defaulting to 30 retries).

The spread of these retry sessions can be linear (default behaviour) or non-linear - in case you want to simulate different strategies:

  1. FIFO: Inverse exponential series - set SlowDown.config { |c| c.retry_strategy = :inverse_exponential }
  2. LIFO: Fibonacci series - set SlowDown.config { |c| c.retry_strategy = :fibonacci }

These polling strategies are just a proof of concept and their behaviour relies more on probabilities.

Inspiration

Quotes

  • It's like sleep but more classy
  • It's like sleep but over-engineered
  • SlowHand

Development

Run tests:

bundle exec rake

Contributing

  1. Fork it ( https://github.com/lipanski/slow_down/fork )
  2. Create your feature branch (git checkout -b my-new-feature)
  3. Commit your changes (git commit -am 'Add some feature')
  4. Push to the branch (git push origin my-new-feature)
  5. Create a new Pull Request
Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].