Dagger.jl
A Julia framework that automatically parallelizes code across CPU cores, GPUs, and multiple machines by scheduling tasks as a directed-acyclic graph, without manual coordination.
Dagger.jl speeds up Julia code by running tasks in parallel across CPU cores, GPUs, and multiple computers, without the developer managing the coordination directly. Wrapping a function call with its spawn macro, similar to Julia's async, lets Dagger decide how to distribute the work and move data between workers, which is useful for nested loops, large datasets, and GPU-accelerated computation.
At its core is a scheduler, inspired by Dask, that represents computations as directed-acyclic graphs and runs them across Julia worker processes, through Distributed.jl or MPI, and threads, as well as GPUs. A separate distributed table type that once lived in this repository has since moved to its own package.
It installs through Julia's package manager and is used by loading the Dagger package, optionally alongside Distributed.jl for multi-process parallelism.