Train a small GPT — the kind of model behind ChatGPT, a million times smaller — from scratch, right here in this tab. No server, no install. Watch the loss curve fall, then ask it to write you a sentence. Source ↗
Decision, forecasting & learned-terminal modes ↓ · one Mac-local specialist workspace
Train
Hyperparameters — preset above sets these. Click to edit individually.
Advanced
Loss curve wasm system: nominal
More metrics
Sample live
You can sample while training is still running — this is the trick. The model is a single instance in a Web Worker; Generate runs a forward pass against the same weights training is mutating, between steps. Sample twice during a run to watch what the model has picked up.
Inspect & evaluate
Benchmark
Score the loaded model against a registered held-out set.
Logit lens interpretability
Per-layer predictions for the current prompt. Reveals when specific knowledge appears in the residual stream. Optionally upload a tuned-lens .lenses sidecar trained via "posttrainllm tuned-lens" for calibrated probes.
Ablation interpretability
Zero out one layer's attention, MLP, or whole-block contribution, then sample. Reveals how load-bearing that component is.
Activation patching interpretability
Intervene on the residual stream at (layer, position) of the recipient prompt above. Zero: drop that token's representation at that depth — exposes whether it's load-bearing. Swap: substitute a hidden state captured from a separate donor prompt at the same coords — the full Meng et al. (2022) variant that reveals which downstream outputs causally depend on the donor's representation.
Watch the model think
Click any byte below to see what the model considered at that
position — the top-10 next-byte probabilities the
last forward pass produced, and the per-head attention
weights from the final transformer block (which earlier
bytes each head was looking at).
This is honest: it's the actual distribution that token was sampled
from, and the actual softmax weights from the last attention layer.
Nothing prettified. · marks bytes that are whitespace
or control characters.
Top-10 candidates
Attention from last layer · per head
Your machine detecting… computing…
computing…
4 Resources curated rabbit holes — 3Blue1Brown, Karpathy, papers, and this repo's own docs
Start here (visual + intuitive)
- 3Blue1Brown — Neural Networks — the maths, visually
- Jay Alammar — The Illustrated Transformer — the diagrams everyone has seen
- Karpathy — Let's build GPT — a 2 h build, from zero
- Karpathy — Neural Networks: Zero to Hero — the full lecture series
Go deeper (code + papers)
- karpathy/nanoGPT — the cleanest small GPT in PyTorch
- karpathy/minGPT — the original minimal build
- Attention Is All You Need — Vaswani et al., 2017
- LoRA: Low-Rank Adaptation — Hu et al., 2021
- AdamW — Loshchilov & Hutter, 2017
This project's own write-ups
- docs/notes.md — how every layer was built & tested
- docs/learn/README.md — the active learning index
- docs/techniques/lora_guide.md — LoRA in detail
- docs/performance/performance.md — the SIMD & WebGPU speed work
- docs/browser_notes.md — WASM, Workers, OPFS, WebGPU
Related projects
- mlc-ai/web-llm — run real LLMs in the browser (inference)
- ml-explore/mlx — Apple Silicon ML framework
- Hugging Face datasets — where the corpus picker pulls from
- MDN — WebGPU — the API powering the GPU path
5 WebGPU matmul benchmark Same matmul on WebGPU + WASM, side-by-side, with a live speedup number
Runs the same matmul through the WebGPU compute kernel and the WASM kernel, parity-checks they agree, and reports the per-shape speedup on your hardware. Needs Chrome / Edge 113+. For the historical end-to-end curve and its evidence limits, see /speedup.
Three ways to teach a small model.
Choose an answer, forecast a series, or predict a terminal screen. One workspace for inputs, training, evaluation, and saved models.
Start your local runtime
The specialist modes run on your Mac through Python. Your inputs stay between this page and the local runtime.
uv sync --project scripts/specialists --all-extras --frozen
pnpm --dir browser run build
uv run --project scripts/specialists python scripts/specialists/lab.py serveDownload the selected model first using the setup commands. The runtime loads cached weights. Training defaults to one step; a training smoke proves wiring, not model quality.
Make a choice in one forward pass.
Compare every allowed answer. Fit temperature on separate examples before interpreting the probabilities.
Decision execution · Python or WASM
Run a locally exported Qwen3 model and optional q/v LoRA adapter in this tab. Tokenizer checks and three Python logit probes must pass before activation. Portable float32 scores are uncalibrated; original MLX calibration cannot be reused.
Select manifest.json, model.onnx, model.onnx.data, tokenizer.json and tokenizer_config.json together. The export sets an explicit prompt limit; longer input is rejected without truncation. Model files stay local. A 1.7B package needs several GiB of browser memory.
Terminal execution · Python, WASM or WebGPU
A browser artifact predicts screens entirely in this tab. Export your trained terminal checkpoint, then select its JSON file. Three Python logit probes must pass before activation. Training and qualification use Python.
Export with terminal export --model <checkpoint> --dtype f32 --out terminal-browser.json through the lab CLI. f16 is an optional compression candidate with a separate numerical gate.
Forecast execution · Python or WASM
Run an exported Timestral model entirely in this tab. Its manifest fixes the history length, horizon and covariate counts. Three Python probes must pass before activation. Training and qualification use Python.
Export with forecast export-browser --model <checkpoint> --data example.jsonl --horizon 24 --out forecast-browser through the lab CLI. Load model.onnx and manifest.json from that directory.
Pinned model choices and cached weights appear after connecting.
Bring your own data · CSV, JSON or JSONL
Preview your data, map its columns, and freeze separate training and evaluation windows. Files stay in the local lab.
Decision imports need at least 20 distinct labelled questions. Forecast data stays chronological. Terminal imports use recorded episode JSON.
Result
Unqualified modelConnect your local runtime to run this example.
Every frame is predicted. No shell runs here. A tiny typing artifact does not know shell commands.
Inspect full output
Saved runs
Configs, checkpoints and reports stay on your Mac. Reopen a completed run after restarting; cancelled runs remain in history.
Training and qualification use your local Python runtime. Exported small terminal models predict in this tab through WASM or WebGPU; exported Timestral forecasts and portable Qwen decisions use WASM. Portable decision probabilities remain uncalibrated.