DataEvolver: Automatic data preparation for LLMs via multi-level self-evolving pipelines

Paper · Demo · Quick Start · Usage · Results

_{Turn noisy raw data + a few seed examples into training-ready, seed-aligned datasets.}

_{Give us a ⭐ if DataEvolver helps your data-prep workflow.}

DataEvolver is a seed-driven, multi-level self-evolving system for LLM training data preparation. Starting from raw data and only a handful of seed examples, it automatically understands target data characteristics, builds and repairs executable operator DAGs, trial-runs on samples, and iteratively refines the pipeline until outputs align with seeds — then runs full preparation.

DataEvolver supports:

• 🌱 Seed-guided understanding — distill schema, format, style, and quality constraints from seed data (not just task descriptions)
• 🔧 Operator-level self-evolving — orchestrate DAGs, detect structural gaps, and synthesize bridging / task-specific operators when needed
• 🔄 Pipeline-level self-evolving — trial runs, Pilot LLM judging, experience reflow, and next-round understanding + orchestration updates
• 🖥 Three aligned interfaces — Web UI (evolution canvas), CLI, and HTTP API share the same workflow semantics
• 📦 Fully open & deployable — git clone for full stack, or pip install dataevolver for CLI/API; all stage artifacts are inspectable

Upload raw data and seed examples — DataEvolver automatically performs structured understanding, DAG orchestration, instantiation, trial evaluation, and experience reflow across rounds.

▶ Watch full demo (2m50s) · Download HD .mov

• Executable + seed-aligned — jointly optimizes can the pipeline run end-to-end? and does output match seed supervision style?
• DAG-native orchestration — three-stage LLM planning with automatic structural check + LLM assessment after each orchestration
• Evolvable operator registry — auto-evolve operators on demand; manually add custom operators via CLI/API and re-orchestrate
• Observable workflow — per-step artifacts, orchestration retry tabs, token ledger, and round/iteration archives
• Automation with control — advance-all for hands-off runs, or step-by-step understand / orchestrate / trial with rerun and --force
• Cross-platform — Linux / macOS / Windows setup scripts; PyPI package for portable CLI workspaces

Full deployment guide: docs/INSTALL.md

From source (Web UI + CLI + API)

git clone https://github.com/ruc-datalab/DataEvolver.git
cd DataEvolver
python scripts/setup_env.py    # or: bash setup_env.sh / setup_env.ps1

From PyPI (CLI + API only)

pip install dataevolver
mkdir my_project && cd my_project
dataevolver init
dataevolver --help

config/api_config.json   # provider, base URL, model
config/api_keys.json     # API key (gitignored — do not commit)

dataevolver session-start my_pipeline \
  --raw tmp/samples/finance_raw.jsonl \
  --seed tmp/samples/finance_seed.jsonl \
  --description tmp/samples/finance_description.txt

dataevolver workflow advance-all my_pipeline --max-steps 32
dataevolver state my_pipeline

Open http://127.0.0.1:5173 for the evolution canvas, or continue in the terminal with dataevolver advance my_pipeline.

DataEvolver exposes the same workflow through three interfaces.

1. Create or select a pipeline session
2. Upload raw data, seed data, and optional task description
3. Advance step-by-step or run continuously
4. Inspect DAG tabs, instantiation code, trial scores, and experience
5. Trigger full run only after quality gates pass

dataevolver state my_pipeline
dataevolver advance my_pipeline
dataevolver workflow advance-all my_pipeline --max-steps 32
dataevolver lang en    # CLI output: zh / en

dataevolver rerun my_pipeline orchestration
dataevolver tokens my_pipeline
dataevolver op add my_op -p my_pipeline -d "Clean records" -c structure

dataevolver op list -p my_pipeline
dataevolver op add -p my_pipeline --from-file examples/operator_template.json
dataevolver workflow orchestrate my_pipeline

DataEvolver/
├── subsystems/     # understanding, orchestration, workflow, …
├── web/            # FastAPI
├── frontend/       # React evolution canvas
├── cli/            # Typer CLI (dataevolver)
├── config/         # api_config, api_keys, operator registry
└── data/           # runtime artifacts (gitignored)

Empirical evaluation from our paper — DataEvolver improves downstream training data quality and model performance across diverse task types.

Downstream performance across 7 benchmarks from 4 task categories.

vs. vanilla SFT on raw data and strong data-prep baselines — fewer, better-prepared samples can match larger weakly-prepared sets.

understanding → orchestration → operator_evolution → instantiation
             → trial_run → quality_check → experience → (refine or full run)

Contributions welcome — operators, dataset adapters, UI polish, docs, and benchmark scripts.

If you use DataEvolver in research, please cite our paper:

@misc{deng2026dataevolverautomaticdatapreparation,
      title={DataEvolver: Automatic Data Preparation for Large Language Models through Multi-Level Self-Evolving}, 
      author={Chao Deng and Shaolei Zhang and Ju Fan and Xiaoyong Du},
      year={2026},
      eprint={2606.07001},
      archivePrefix={arXiv},
      url={https://arxiv.org/abs/2606.07001}, 
}

📄 Paper: arXiv:2606.07001

_{Built for teams who want executable and seed-aligned data pipelines — not one-shot prompts.}