A TYPESET PDF LANDS HERE SOON — UNTIL THEN, PRINT / SAVE AS PDF PRODUCES A CLEAN COPY.

Yash Bishnoi

ML Engineer · Edinburgh, UK · Open to roles — no sponsorship needed

Profile

Builds language models from scratch and verifies them bit-exact against the official weights before claiming anything on top: max |Δlogits| = 0.0 vs HuggingFace, for both SmolLM2-135M and Qwen3-0.6B.
Holds every cross-run claim to a paper-grade evidence standard — single-variable, 3-seed, iso-FLOP, across-seed 95% CIs, a held-out noise floor — and publishes the nulls at the same prominence as the wins.
Runs the small-scale LLM lifecycle end to end (pretraining → data studies → mid-training → SFT → RL) on a single NVIDIA GB10 box with a ~119 GB unified memory pool; no rented compute.
Every number on this page traces to a results file at a pinned commit — see yashbishnoi.io/research.

Single-file PyTorch reproductions of SmolLM2-135M (134,515,008 params) and Qwen3-0.6B-Base (596,049,920 params), each verified bit-exact against the official HuggingFace weights: max |Δlogits| = 0.0.
Evidence standard: every attributed effect is single-variable, 3-seed, iso-FLOP, scored on held-out bits-per-byte with across-seed 95% CIs; single-run numbers are tagged descriptive (n=1), never presented as headline.
Largest measured effect is a pure data swap: dclm-edu vs FineWeb-Edu at fixed tokens improved code BPB by +0.7034 [95% CI +0.6639, +0.7430] — bigger than the optimizer effect (NorMuon +0.4743 BPB [+0.4435, +0.5052], early-training regime, ~42M tokens/arm) or the architecture effect (+0.1179 BPB wikitext-2 [+0.1005, +0.1354]); a 50/50 mix kept ~84% of the code win while holding English.
Publishes negative results at equal prominence: WSD schedule and z-loss not significant on the canonical metric; a response-masked-SFT "win" of 0.68 PPL exposed as an eval-token confound and collapsed to +0.009; a pre-registered GRPO/RLVR null at 0.6B confirmed exactly as predicted.
Descriptive context (n=1): the faithful reproduction trained from scratch lands within 2.14× of the original Qwen3-0.6B's perplexity on ~30,000× less data (1.19B vs 36T tokens).

max (TypeScript, Jun 2026) — Max Assistant, a personal AI assistant; the same Max persona answers questions on this site.
multi-agent-patterns (Python, Mar–May 2026) — multi-agent orchestration patterns, plus JobPulse daily automation and MindGraph knowledge visualization.
Prism (Python, Mar 2026) — a research agent that uses recursive language models to decompose million-token corpora into verifiable reasoning chains.
LetsBuild (Python, Mar 2026) — Autonomous Portfolio Factory: job description in, published GitHub repo out.
Velox AI (TypeScript, Feb–Mar 2026) — enterprise-grade AI voice-agent platform for real phone-call agents on Twilio + Gemini + Deepgram: visual flow design, multi-agent LLM routing, hybrid RAG, billing, analytics, production observability.
talent-agent (TypeScript, Jun 2026) — AI talent-intelligence agent powered by Kimi K2.6.
LetsFineTune (Jupyter, Apr 2026) — a collection of fine-tuned language models built with LoRA, Unsloth, and Hugging Face Transformers.

ML / training: PyTorch; from-scratch transformer implementation (RoPE, GQA, SwiGLU, RMSNorm, QK-Norm); pretraining, continued pretraining, low-LR anneals; response-masked SFT; GRPO/RLVR (Dr.GRPO, DAPO dynamic sampling, rejection-sampling and random-reward controls); LoRA / Unsloth; Hugging Face Transformers, datasets, safetensors; lm-evaluation-harness; BPB / perplexity / passkey-retrieval evals; multi-seed CI + iso-FLOP experiment design; FLOP and MFU accounting.
Agents / orchestration: multi-agent orchestration patterns; recursive-LM research agents (Prism); autonomous repo generation (LetsBuild); MCP tooling (code-graph-mcp); voice agents with multi-agent LLM routing and hybrid RAG (Velox AI); Kimi K2.6 (talent-agent).
Infra / serving: single-box GB10 (Grace Blackwell, aarch64, DGX OS) training operations — unified-memory guards, watchdog sentinels, idempotent resumable training supervisors; Twilio + Deepgram + Gemini telephony stack with production observability (Velox AI).
Languages: Python, TypeScript/JavaScript, Bash; Jupyter-notebook workflows.

Self-directed ML research alongside part-time retail work — the work this site documents.

Reproduced SmolLM2-135M and Qwen3-0.6B from scratch in single-file PyTorch, each verified bit-exact against the official Hugging Face weights (max |Δlogits| = 0.0).
Matched-compute experiment on Qwen3-0.6B: a modernized architecture bundle beat the faithful baseline by 17.9% in-loop (23.52 vs 28.65 val PPL, n=1) — then de-confounded into attributed, 3-seed, CI-backed effects; a second method lost and is reported as a clean negative result.
Single-variable, 3-seed ablations with across-seed 95% CIs (e.g. NorMuon vs AdamW: +0.4743 BPB wikitext-2 [+0.4435, +0.5052], early-training regime).
Continued pretraining of SmolLM2-135M on TinyStories: validation perplexity 6.89 → 3.79.
Honest scope: self-directed research on a single machine — not production or distributed-scale deployment experience.

Run day-to-day shifts in a busy retail store, supervising a team of 8+: task allocation, cover planning, rotas around peak trade.
Use the store's sales and stock systems to monitor stock levels and reduce shelf gaps; train newer team members on tills, processes, and standards.
Previously Customer Team Member (Apr – Aug 2025), same store.

Turned sales, supplier, and customer data into buying and sourcing decisions: market trends, customer segments, and competitor pricing analysed in Excel and SQL.
Built dashboards tracking supplier performance, lead times, and inventory turnover; maintained CRM data and retention reporting.
Automated repetitive reporting with Python (pandas, openpyxl) — the work that led into machine learning.

E4CC — Full Stack Engineer (Jan – Mar 2024, Gurgaon): full-stack apps in Node.js/React with PostgreSQL/MySQL; schema design and query optimization (~30% page-load improvement via indexing); Agile with Git/Jira/CI-CD.
Bharat Intern — Application Developer (Nov – Dec 2023): backend services and REST APIs in Node.js/Python/SQL; deployments to AWS/GCP.

University of Dundee — MSc Computer Science (Jan 2025 – Jan 2026). Coursework includes the Machine Learning module documented in the public Machine-Learning_University_of_Dundee repo.
JECRC University — MBA, Accounting and Finance (2019 – 2021).
Amity University, Jaipur — BSc Biotechnology (2015 – 2019).

IBM Machine Learning Specialization · Data Cleaning · Feature Engineering · SQL Essential Training