DataPilot
A ReAct data agent that turns natural-language asks into validated SQL/transform DAGs — with self-correction when intermediate checks fail.
The gap
Analysts still burn most of their week on mechanical ETL: clean the CSV, join the tables, drop test accounts, compute the monthly rollup. BI tools break when schemas drift. Chat-to-SQL demos look clever until a join is wrong and nobody notices. What is missing is an agent that plans a DAG, validates each step, and repairs itself before it ships a wrong number.
What I built
DataPilot accepts a natural-language instruction, decomposes it into atomic data ops (ingest → validate → transform → aggregate → report), executes against PostgreSQL, and scores intermediate outputs against schema + business constraints. On failure it revises the plan (ReAct loop) instead of returning a confident lie. A Next.js UI shows the live DAG and conversation.
How it fits together
Technical choices
- ●DAG-first planning — the model proposes nodes; the runtime owns execution order and retries.
- ●Validate every hop — never trust a single end-to-end SQL generation.
- ●Keep ops inspectable in SQL — every intermediate is a named relation, not a black-box dataframe.
System features
- NL → executable DAG
Instructions become a graph of typed data ops, not one opaque mega-query.
- Self-correcting loop
Failed constraint checks feed back into the planner before the final report.
- Live DAG view
Conversational UI streams node status so you see what the agent is doing.
- Schema-aware checks
Null rates, keys, and column contracts gate every transform.