Weld

Weld addresses a common problem in data-analytics applications: individual library functions can each be fast, but combining several of them into a workflow often runs an order of magnitude below hardware limits because of data movement between functions. Weld expresses the core computation of each library in a common intermediate representation, lazily builds up the computation for an entire workflow, and only optimizes and evaluates it once a result is actually needed, optimizing across the combined functions rather than each one in isolation.

The project includes Grizzly, an integration that runs Pandas operations through Weld. Building it requires a current stable Rust toolchain and LLVM/Clang 6.0, and it is distributed under the BSD 3-Clause license. It originated at Stanford, and discussion happens on a Google Group and mailing list.