Quick Reference · parallel & distributed computing in Python

dask cheat sheet

Dask has three moving parts: a collection (a DataFrame, array, or bag split into many small pieces) records the operations you ask for into a lazy task graph, and a scheduler runs that graph in parallel. Nothing actually runs until you call .compute(). Learn that one loop and the whole API stops being a list to memorize.

setup / import create / load transform (lazy) execute / trigger cluster / scheduler gotcha ★ most common

Distilled & cross-checked against: docs.dask.org · distributed.dask.org · ml.dask.org · Dask cheat sheet (dask/dask) · Anaconda · Coiled · Saturn Cloud  — Verified 2026-08-26 against Dask 2026.8 (CalVer)

The mental model · collection → lazy task graph → scheduler → result
Your data CSV · Parquet · arrays plain Python code Collection dataframe · array bag · delayed split into partitions Task Graph a lazy DAG of tasks .visualize() Scheduler threads · processes distributed Client() runs tasks in parallel Result pandas · numpy list · scalar read_csv from_array lazy ops groupby… submit .compute() materialize .persist() — run now, but keep the result in memory as a Dask object (not a concrete one) LAZY — nothing has run yet, only a graph is built EAGER
01Setup & Importonce per environment
★Pick a Collectionwhich API do I want?
03DataFrame · Load & Saveread with Dask
04DataFrame · Transformpandas API, lazy
05Array · CreateNumPy in chunks
06Array · OperateNumPy API, lazy
07Bag · Messy DataJSON, logs, objects
08Delayed · Any Codecustom task graphs
09Trigger Executionturn graph into data
10Cluster & Schedulerthe execution engine
11Futures · Real-timeeager · needs Client
12Partitions & Chunksthe performance lever
13Inspect & Diagnosesee what's happening
14Gotchascommon anti-patterns

Four ideas that make Dask click

The distinctions that trip people up most — laziness, partitioning, compute vs persist, and choosing a scheduler. Based on the official Dask docs diagrams.

lazy build, then one compute

Every op just adds a node to the graph. Work happens only at .compute() — so batch your work and call it once.

a b + ×2 graph grows · no work yet .compute() result

one Dask object = many small ones

A DataFrame is a stack of pandas partitions; an array is a grid of NumPy chunks. Dask coordinates the pieces.

dask.dataframe pandas · part 0 pandas · part 1 pandas · part 2 pandas · part 3 dask.array npnp npnp chunks

compute vs persist

compute pulls one concrete result back to your client. persist runs the graph but leaves results spread across worker memory as a Dask object.

.compute() 1 pandas on client .persist() kept in worker RAM

which scheduler?

Threads share memory (great for NumPy/pandas, which release the GIL). Processes suit text & dicts. The distributed Client scales to many machines — and works on one.

threads shared memory NumPy · pandas GIL-releasing processes isolated memory text · lists pure-Python Client() 1 → 1000s cores dashboard :8787 the "advanced" one default: threads (arrays/frames/delayed) · processes (bag)

Worth memorizing

lazy by defaultnothing runs until .compute()
compute ≠ persistcompute → concrete on client; persist → Dask object in RAM
~100 MBtarget size of each partition / chunk
one computedask.compute(*xs) — never compute in a loop
threads vs processesthreads for numeric; processes for text/dicts
delayed ≠ futuresdelayed is lazy; futures are eager (need a Client)
read with Daskdd.read_csv — don't pandas-read then hand it over
set_indexexpensive shuffle — do it once, then persist
:8787the dashboard is your best debugging tool
start smallif pandas/NumPy already fit in RAM, skip Dask