All Projects → didactic-drunk → concurrent.cr

didactic-drunk / concurrent.cr

Licence: other
Modern concurrency tools for Crystal

Programming Languages

crystal
512 projects

Projects that are alternatives of or similar to concurrent.cr

core.horse64.org
THIS IS A MIRROR, CHECK https://codeberg.org/Horse64/core.horse64.org
Stars: ✭ 3 (-91.89%)
Mutual labels:  concurrent
goroutines
provides utilities to perform common tasks on goroutines
Stars: ✭ 19 (-48.65%)
Mutual labels:  concurrent
mapreduce
A in-process MapReduce library to help you optimizing service response time or concurrent task processing.
Stars: ✭ 93 (+151.35%)
Mutual labels:  concurrent
practice
Java并发编程与高并发解决方案:http://coding.imooc.com/class/195.html Java开发企业级权限管理系统:http://coding.imooc.com/class/149.html
Stars: ✭ 39 (+5.41%)
Mutual labels:  concurrent
java-multithread
Códigos feitos para o curso de Multithreading com Java, no canal RinaldoDev do YouTube.
Stars: ✭ 24 (-35.14%)
Mutual labels:  concurrent
jdk-source-code-reading
JDK source code reading
Stars: ✭ 19 (-48.65%)
Mutual labels:  concurrent
YACLib
Yet Another Concurrency Library
Stars: ✭ 193 (+421.62%)
Mutual labels:  concurrent
vmlens
unit-testing multi-threaded applications on the JVM made easy
Stars: ✭ 88 (+137.84%)
Mutual labels:  concurrent
talepy
📚Coordinate "transactions" across a number of services in python
Stars: ✭ 20 (-45.95%)
Mutual labels:  concurrent
concurrent-data-structure
Concurrent Data Structure for Rust
Stars: ✭ 18 (-51.35%)
Mutual labels:  concurrent
gini
A fast SAT solver
Stars: ✭ 139 (+275.68%)
Mutual labels:  concurrent
mango
Core utility library & data connectors designed for simpler usage in Scala
Stars: ✭ 41 (+10.81%)
Mutual labels:  concurrent
web-scraping-engine
A simple web scraping engine supporting concurrent and anonymous scraping
Stars: ✭ 27 (-27.03%)
Mutual labels:  concurrent
funboost
pip install funboost,python全功能分布式函数调度框架,。支持python所有类型的并发模式和全球一切知名消息队列中间件,python函数加速器,框架包罗万象,一统编程思维,兼容50% python编程业务场景,适用范围广。只需要一行代码即可分布式执行python一切函数。旧名字是function_scheduling_distributed_framework
Stars: ✭ 351 (+848.65%)
Mutual labels:  concurrent
sharded-slab
a lock-free concurrent slab (experimental)
Stars: ✭ 116 (+213.51%)
Mutual labels:  concurrent
Hunch
Hunch provides functions like: All, First, Retry, Waterfall etc., that makes asynchronous flow control more intuitive.
Stars: ✭ 94 (+154.05%)
Mutual labels:  concurrent
go-soda
Socrata Open Data API (SODA) GET client for Golang
Stars: ✭ 40 (+8.11%)
Mutual labels:  concurrent
space
A SCI-FI community game server simulating space(ships). Built from the ground up to support moddable online action multiplayer and roleplay!
Stars: ✭ 25 (-32.43%)
Mutual labels:  concurrent
retil
The React Utility Library
Stars: ✭ 46 (+24.32%)
Mutual labels:  concurrent
treap
A thread-safe, persistent Treap (tree + heap) for ordered key-value mapping and priority sorting.
Stars: ✭ 23 (-37.84%)
Mutual labels:  concurrent

concurrent.cr

Crystal CI GitHub release Docs

Modern Adequate Any New opportunities for concurrency tools in Crystal.
Large empty lots spacious directories available to build your dream home algorithm!
Space is filling up at (24k code bytes / 2 months) 0.004 bytes per second. Register your PR today!
©️ Real estate marketing association

Inspired by Erlang, Clojure, Scala, Haskell, F#, C#, Java, and classic concurrency patterns which inspired Ruby, which inspired this library.

Available classes:

TODO:

  • Change Enumerable/Channel in to generic stream processing.
  • Enumerable/Channel custom error handling.

More algorithms are coming. Contributions welcome.

Installation

  1. Add the dependency to your shard.yml:

    dependencies:
      concurrent:
        github: didactic-drunk/concurrent.cr
  2. Run shards install

Usage

Parallel map (experimental)

require "concurrent/enumerable"

(1..50).parallel.select(&.even?).map { |n| n + 1 }.serial.sum
                 ^               ^                 ^ Results joined.
                 |               | Spawns separate fiber pool
                 | Spawns fiber pool

Batches

(1..50).parallel.map { |n|
  # Parallel processing in a fiber pool
  Choose::A::ORM.new(id: n)
}.batch(10).run { |array_of_records|
  # Run 10 Inserts inside a transaction for faster db writes
  # Real applications should choose ~~~100-100000 depending on the database, schema, data & hardware
  ORM.transaction { array_of_records.each &.save! }
}.wait

Stream processing from a Channel (experimental).

require "concurrent/channel"

# Same interface and restrictions as concurrent/enumerable.

ch = Channel(Int32).new

spawn do
  10.times { |i| ch.send 1 }
  ch.close
end

# map is processed in a Fiber pool.
# All other fibers will shut down after all messages are processed.
# Any errors in processing are raised here.
ch.parallel.map { |n| n + 1 }.serial.sum

Open ended stream processing aka simplified fiber pools (experimental)

require "concurrent/channel"

# Same interface and restrictions as concurrent/enumerable.

ch = Channel(Int32).new
# Messages may be processed in parallel within each `tee` and `run`.
# Make sure to use immutable objects or concurrency safe data structures.
run = ch.parallel.tee { |n| Log.info { "n=#{n}" } }.batch(2).run { |n| p n }

10.times { |i| ch.send 1 }
ch.close

# Wait until all messages/errors are processed.
run.wait

Stream error handling

ary = (1..10).to_a.parallel.select { |i|
  raise "select error" if i == 2
  true
}.parallel.map { |i|
  raise "map error" if i.even?
  i.to_s
# All errors in prior blocks handled here
}.errors { |ex, obj|
  puts "#{obj} #{ex}"
}.map { |s|
  s.to_i
}.to_a

p ary => [1, 3, 5, 7]

CountDownLatch

require "concurrent/count_down_latch"

fiber_count = 10
latch = Concurrent::CountDownLatch.new
10.times do
  spawn do
    # Do work
    latch.count_down
  end
end

latch.wait_count = fiber_count
latch.wait

Semaphore

require "concurrent/semaphore"

sem = Concurrent::Semaphore.new n

# spawn a lot of fibers
2000.times do
  spawn do
    sem.acquire do
      ...
    end
  end
end

Development

TODO: Write development instructions here

Contributing

  1. Fork it (https://github.com/didactic-drunk/concurrent.cr/fork)
  2. Install a formatting check git hook (ln -sf ../../scripts/git/pre-commit .git/hooks)
  3. Create your feature branch (git checkout -b my-new-feature)
  4. Commit your changes (git commit -am 'Add some feature')
  5. Push to the branch (git push origin my-new-feature)
  6. Create a new Pull Request

Contributors

  • Click or Run git shortlog --summary --numbered --email
Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].