← Selected work

Case 03 / 04

AI / Data pipelines

Audit-Grade AI

Project brief

The data pipeline behind an audit-AI platform — raw domain material in, task-specific datasets out, ready for LLM retrieval or small-language-model fine-tuning.

Timeline
2024 —
Scope
4 responsibilities
Outcome
One pipeline, any dataset the next model needs
An engineer cross-checking source documents against model outputs in a data workspace
Editorial project visualization · reconstructed from the delivered workflow

The problem

Audit work demands answers that are precise and defensible, so general-purpose models are only a starting point. The platform needed its own data — and the shape of that data kept changing: one week a retrieval corpus, the next a fine-tuning set for a small language model.

What I owned: the data pipeline

I built the pipeline that turns raw domain material into datasets on demand:

  • Ingestion and cleaning of raw domain data into a consistent intermediate format.
  • Transformation into whatever the downstream task needs — data an LLM can consume, or training data for fine-tuning.
  • Dataset generation for small-language-model fine-tuning with Unsloth, so the team could request a new dataset instead of hand-building one.

Also on the platform

Around the models I built product modules — marketplace, training, certification and model management — and supported the team’s self-hosted SLM and GraphRAG workflows.

Notes

The interesting part is less any single model and more the plumbing: clean, reproducible datasets in, models out into modules people actually use.

Stack

Python · OpenAI · Gemini · LlamaIndex · Unsloth · FastAPI · PostgreSQL · Kubernetes · GCP · Azure