All Projects → jishnub → ParallelUtilities.jl

jishnub / ParallelUtilities.jl

Licence: MIT license
Fast and easy parallel mapreduce on HPC clusters

Programming Languages

julia
2034 projects

Projects that are alternatives of or similar to ParallelUtilities.jl

t8code
Parallel algorithms and data structures for tree-based AMR with arbitrary element shapes.
Stars: ✭ 37 (+32.14%)
Mutual labels:  hpc, parallel, parallel-computing, high-performance-computing
Future.apply
🚀 R package: future.apply - Apply Function to Elements in Parallel using Futures
Stars: ✭ 159 (+467.86%)
Mutual labels:  hpc, parallel, parallel-computing, distributed-computing
hp2p
Heavy Peer To Peer: a MPI based benchmark for network diagnostic
Stars: ✭ 17 (-39.29%)
Mutual labels:  hpc, parallel, parallel-computing, hpc-applications
Easylambda
distributed dataflows with functional list operations for data processing with C++14
Stars: ✭ 475 (+1596.43%)
Mutual labels:  hpc, parallel, distributed-computing
Pwrake
Parallel Workflow extension for Rake, runs on multicores, clusters, clouds.
Stars: ✭ 57 (+103.57%)
Mutual labels:  parallel, parallel-computing, distributed-computing
cruise
User space POSIX-like file system in main memory
Stars: ✭ 27 (-3.57%)
Mutual labels:  hpc, parallel, parallel-computing
Future
🚀 R package: future: Unified Parallel and Distributed Processing in R for Everyone
Stars: ✭ 735 (+2525%)
Mutual labels:  hpc, parallel-computing, distributed-computing
Core
parallel finite element unstructured meshes
Stars: ✭ 124 (+342.86%)
Mutual labels:  hpc, parallel, parallel-computing
Mfem
Lightweight, general, scalable C++ library for finite element methods
Stars: ✭ 667 (+2282.14%)
Mutual labels:  hpc, parallel-computing, high-performance-computing
Batchtools
Tools for computation on batch systems
Stars: ✭ 127 (+353.57%)
Mutual labels:  hpc, parallel-computing, high-performance-computing
Opencoarrays
A parallel application binary interface for Fortran 2018 compilers.
Stars: ✭ 151 (+439.29%)
Mutual labels:  hpc, parallel-computing, high-performance-computing
Sundials
SUNDIALS is a SUite of Nonlinear and DIfferential/ALgebraic equation Solvers. This is a mirror of current releases, and development will move here eventually. Pull requests are welcome for bug fixes and minor changes.
Stars: ✭ 194 (+592.86%)
Mutual labels:  hpc, parallel-computing, high-performance-computing
Taskflow
A General-purpose Parallel and Heterogeneous Task Programming System
Stars: ✭ 6,128 (+21785.71%)
Mutual labels:  parallel, parallel-computing, high-performance-computing
future.batchtools
🚀 R package future.batchtools: A Future API for Parallel and Distributed Processing using batchtools
Stars: ✭ 77 (+175%)
Mutual labels:  hpc, parallel, distributed-computing
Geni
A Clojure dataframe library that runs on Spark
Stars: ✭ 152 (+442.86%)
Mutual labels:  parallel-computing, distributed-computing, high-performance-computing
Dash
DASH, the C++ Template Library for Distributed Data Structures with Support for Hierarchical Locality for HPC and Data-Driven Science
Stars: ✭ 134 (+378.57%)
Mutual labels:  hpc, parallel-computing, high-performance-computing
vpic
Vector Particle-In-Cell (VPIC) Project
Stars: ✭ 124 (+342.86%)
Mutual labels:  hpc, high-performance-computing, hpc-applications
Foundations of HPC 2021
This repository collects the materials from the course "Foundations of HPC", 2021, at the Data Science and Scientific Computing Department, University of Trieste
Stars: ✭ 22 (-21.43%)
Mutual labels:  hpc, parallel-computing, hpc-applications
ParMmg
Distributed parallelization of 3D volume mesh adaptation
Stars: ✭ 19 (-32.14%)
Mutual labels:  hpc, parallel
pooljs
Browser computing unleashed!
Stars: ✭ 17 (-39.29%)
Mutual labels:  parallel, distributed

ParallelUtilities.jl

Build status codecov Stable Dev DOI

Parallel mapreduce and other helpful functions for HPC, meant primarily for embarassingly parallel operations that often require one to split up a list of tasks into subsections that may be processed on individual cores.

Installation

Install the package using

pkg> add ParallelUtilities
julia> using ParallelUtilities

Quick start

Just replace mapreduce by pmapreduce in your code and things should work the same.

julia> @everywhere f(x) = (sleep(1); x^2); # some expensive calculation

julia> nworkers()
2

julia> @time mapreduce(f, +, 1:10) # Serial
 10.021436 seconds (40 allocations: 1.250 KiB)
385

julia> @time pmapreduce(f, +, 1:10) # Parallel
  5.137051 seconds (863 allocations: 39.531 KiB)
385

Usage

See the documentation for examples and the API.

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].