Parallel Computing in Python with Dask @ MadPy

This repository contains the materials for my "Parallel Computing in Python with Dask" talk at the Madison Python (MadPy) Meetup.

An interactive version of the notebook from this talk is available by clicking the "launch binder" button below:

Setup

Step 1: Create Conda environment

A Conda environment with the dependencies needed to run the notebook from this talk can be created with:

conda env create --name madpy-dask --file binder/environment.yml

Step 2: Activate Conda environment

Activate the Conda environment:

conda activate madpy-dask

(Optional) Step 3: Install JupyterLab extension

The Dask JupyterLab extension can be installed with:

jupyter labextension install dask-labextension

inside the activated Conda environment.

Step 4: Run Jupyter

The notebook can then be launched with:

jupyter lab dask-demo.ipynb

Additional Resources

• Dask links:

  • GitHub repository: https://github.com/dask/dask

  • Documentation: https://docs.dask.org

  • Dask examples repository: https://github.com/dask/dask-examples

• There are lots of great Dask tutorial from various conference on YouTube. For example:

  • "Parallelizing Scientific Python with Dask" @ SciPy 2018: YouTube

  • "Scalable Machine Learning with Dask" @ SciPy 2018: YouTube

• If you have a Dask usage questions, please ask it on Stack Overflow with the #dask tag. Dask developers monitor this tag and will answer questions.

• If you run into a bug, feel free to file a report on the Dask GitHub issue tracker.