The problem
Most fine-tuning work is a notebook that ran once. The config drifts from what actually trained, the eval is whatever cell was executed last, and nobody — including the author — can reproduce the number in the writeup. LetsFineTune is structured against that failure mode: every model gets its own pipeline from dataset preparation through evaluation, with configs and notebooks committed so a run can be re-executed, not just remembered.
Architecture
The repo separates what usually gets tangled: YAML training configs in configs/, training and data-prep scripts in scripts/, reusable modules under src/ (data loading, training loops, eval metrics), and exploratory notebooks kept apart from the reproducible path. Checkpoints and datasets are gitignored with download instructions — the repo versions the recipe, not the artifacts.
The stack is the current open-source fine-tuning standard: PyTorch and Hugging Face Transformers with PEFT and TRL, Unsloth for throughput, bitsandbytes for quantization, LoRA/QLoRA adapters in bf16 mixed precision. Runs log to Weights & Biases and TensorBoard.
The first model on the bench is a reasoning fine-tune: Qwen3.5-9B on OpenThoughts-114k with LoRA at r=32, alpha=64 in bf16. Evaluation is wired for code and math — HumanEval via EvalPlus and GSM8K — alongside ROUGE and BLEU for text tasks.
What's measured
Per-model status is tracked in the README's model table, and W&B holds the training curves. The eval harness scores checkpoints against fixed processed datasets from a single entry point, so a benchmark number is a command, not a screenshot.
Honest limitations
The first model is still marked Training in the repo, and no benchmark results are published yet — so, consistent with the rest of this site, no performance number appears on this page. The project is also the raw material for the @letsfinetune build-in-public series on Instagram, which documents runs as they happen rather than after the story is tidy. When the evals land, they will be quoted from the committed results, not from memory.