DataFlow
A visual, low-code system for generating, cleaning and preparing high-quality training data for large language models from noisy sources such as PDFs and low-quality QA pairs.
DataFlow is a data preparation and training system that generates, refines, evaluates and filters high-quality data for AI from noisy sources such as PDFs, plain text and low-quality question-answer pairs, aimed at improving LLM performance in specific domains such as healthcare, finance and law through targeted pre-training, supervised fine-tuning, reinforcement learning or retrieval-augmented generation.
It offers visual, low-code pipelines that can be orchestrated flexibly across domains and use cases, a WebUI launched with a single command for building and running pipelines visually or through MCP, and DataFlow-Harness, which lets a coding agent build DataFlow pipelines on a user's behalf. DataFlow-Skills provides a related collection of skills and tutorials.
The project's approach has been described in a technical report and an accepted paper at ACM SIGMOD 2027.