All Projects → HenrikBengtsson → future.callr

HenrikBengtsson / future.callr

Licence: other
🚀 R package future.callr: A Future API for Parallel Processing using 'callr'

Programming Languages

r
7636 projects
Makefile
30231 projects
shell
77523 projects

Projects that are alternatives of or similar to future.callr

Future
🚀 R package: future: Unified Parallel and Distributed Processing in R for Everyone
Stars: ✭ 735 (+1313.46%)
Mutual labels:  promises, programming, asynchronous, parallel-computing, parallelization, futures, parallel-processing
Future.apply
🚀 R package: future.apply - Apply Function to Elements in Parallel using Futures
Stars: ✭ 159 (+205.77%)
Mutual labels:  programming, asynchronous, parallel-computing, parallelization, parallel-processing
Tomorrowland
Lightweight Promises for Swift & Obj-C
Stars: ✭ 106 (+103.85%)
Mutual labels:  promises, asynchronous, futures
Swiftcoroutine
Swift coroutines for iOS, macOS and Linux.
Stars: ✭ 690 (+1226.92%)
Mutual labels:  promises, asynchronous, futures
Lwt
OCaml promises and concurrent I/O
Stars: ✭ 505 (+871.15%)
Mutual labels:  promises, asynchronous, futures
Asynchronous
Implementation-agnostic asynchronous code
Stars: ✭ 13 (-75%)
Mutual labels:  promises, asynchronous, futures
Radon
Object oriented state management solution for front-end development.
Stars: ✭ 80 (+53.85%)
Mutual labels:  promises, asynchronous
Promis
The easiest Future and Promises framework in Swift. No magic. No boilerplate.
Stars: ✭ 110 (+111.54%)
Mutual labels:  promises, futures
swift-futures
Demand-driven asynchronous programming in Swift
Stars: ✭ 32 (-38.46%)
Mutual labels:  promises, futures
Hydra
⚡️ Lightweight full-featured Promises, Async & Await Library in Swift
Stars: ✭ 1,954 (+3657.69%)
Mutual labels:  promises, futures
Easyfutures
Easy Swift Futures & Promises.
Stars: ✭ 40 (-23.08%)
Mutual labels:  promises, futures
Promise
Promise / Future library for Go
Stars: ✭ 149 (+186.54%)
Mutual labels:  promises, futures
Q
A platform-independent promise library for C++, implementing asynchronous continuations.
Stars: ✭ 179 (+244.23%)
Mutual labels:  promises, futures
Pinfuture
An Objective-C future implementation that aims to provide maximal type safety
Stars: ✭ 74 (+42.31%)
Mutual labels:  promises, futures
Tascalate Concurrent
Implementation of blocking (IO-Bound) cancellable java.util.concurrent.CompletionStage and related extensions to java.util.concurrent.ExecutorService-s
Stars: ✭ 144 (+176.92%)
Mutual labels:  promises, asynchronous
Php Ion
Asynchronous PHP
Stars: ✭ 65 (+25%)
Mutual labels:  promises, asynchronous
Brightfutures
Write great asynchronous code in Swift using futures and promises
Stars: ✭ 1,890 (+3534.62%)
Mutual labels:  promises, futures
java-red
Effective Concurrency Modules for Java
Stars: ✭ 25 (-51.92%)
Mutual labels:  asynchronous, futures
tsukuyomi
Asynchronous Web framework for Rust
Stars: ✭ 81 (+55.77%)
Mutual labels:  asynchronous, futures
corebench
corebench - run your benchmarks against high performance computing servers with many CPU cores
Stars: ✭ 29 (-44.23%)
Mutual labels:  parallel-computing, parallelization
CRAN check status R CMD check status future.tests checks status Coverage Status

future.callr: A Future API for Parallel Processing using 'callr'

Introduction

The future package provides a generic API for using futures in R. A future is a simple yet powerful mechanism to evaluate an R expression and retrieve its value at some point in time. Futures can be resolved in many different ways depending on which strategy is used. There are various types of synchronous and asynchronous futures to choose from in the future package.

This package, future.callr, provides a type of futures that utilizes the callr package.

For example,

> library("future.callr")
> plan(callr)
>
> x %<-% { Sys.sleep(5); 3.14 }
> y %<-% { Sys.sleep(5); 2.71 }
> x + y
[1] 5.85

This is obviously a toy example to illustrate what futures look like and how to work with them. For further examples on how to use futures, see the vignettes of the future package as well as those of future.apply and doFuture.

Using the callr backend

The future.callr package implements a future wrapper for callr.

Backend Description Alternative in future package
callr parallel evaluation in a separate R process (on current machine) plan(multisession)

Each callr future uses a fresh R session

When using callr futures, each future is resolved in a fresh background R session which ends as soon as the value of the future has been collected. In contrast, multisession futures are resolved in background R worker sessions that serve multiple futures over their life spans. The advantage with using a new R process for each future is that it is that the R environment is guaranteed not to be contaminated by previous futures, e.g. memory allocations, finalizers, modified options, and loaded and attached packages. The disadvantage, is an added overhead of launching a new R process. (At the moment, I am neither aware of formal benchmarking of this extra overhead nor of performance comparisons of callr to alternative future backends.)

More than 125 parallel callr futures

Another advantage with callr futures compared to multisession futures is that they do not communicate via R (socket) connections. This avoids the limitation in the number of parallel futures that can be active at any time that multisession futures and cluster futures in general have, which they inherit from SOCKcluster clusters as defined by the parallel package. The number of parallel futures these can serve is limited by the maximum number of open connections in R, which currently is 125 (excluding the three reserved by R itself). Note that these 125 slots have to be shared with file connections etc. To increase this limit, R has to be rebuilt from source. However, since callr futures rely on the callr package which does not make use of R-specific connections, there is no limit in the number of background R processes that can be used simultaneously.

No ports are used - no port clashes or firewall issues

A third advantage with callr futures, is that there is not risk for port-clashing with other processes on the system when clusters are set up (**), because callr does not rely on ports. Furthermore, on Windows, the firewall triggers an alert that the user needs to approve whenever a not-previously-approved port is requested by R - which happens also for local, non-public ports that are used by SOCKcluster:s. When using callr futures, no sockets and therefore no ports are involved.

(**) To lower the risk for such clashes SOCKcluster:s (of the parallel package) request random ports, but clashes still occur at times.

Demos

The future package provides a demo using futures for calculating a set of Mandelbrot planes. The demo does not assume anything about what type of futures are used. The user has full control of how futures are evaluated. For instance, to use callr futures, run the demo as:

library("future.callr")
plan(callr)
demo("mandelbrot", package = "future", ask = FALSE)

Installation

R package future.callr is available on CRAN and can be installed in R as:

install.packages("future.callr")

Pre-release version

To install the pre-release version that is available in Git branch develop on GitHub, use:

remotes::install_github("HenrikBengtsson/future.callr", ref="develop")

This will install the package from source.

Contributing

To contribute to this package, please see CONTRIBUTING.md.

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].