Train a 125‑M‑Parameter LLM on Your Laptop in Under 10 Minutes

NanoGPT speedrun – the minimalist codebase that lets you train a functional GPT‑2‑style model on a consumer laptop (RTX 4060, Apple M2/M3, or even a high‑end CPU) without spending a dime on cloud credits.

Introduction

Imagine turning on your laptop, running a single command, and watching a 125 M‑parameter language model finish training in ≈ 8 minutes. That’s the reality with NanoGPT. This guide shows why the buzz matters, walks you through the exact steps to reproduce the speedrun, and gives you concrete code snippets so you can start experimenting right now.

Quick‑Start: One‑Command Training

# Clone the repo and install dependencies
git clone https://github.com/karpathy/nanoGPT.git && cd nanoGPT
pip install -r requirements.txt

# Download the WikiText‑103 dataset (≈ 103 M tokens)
python data/wikitext.py --download

# Train a 125 M‑parameter model on a RTX 4060 (or Apple M2/M3)
python train.py \
    --dataset wikitext \
    --batch_size 64 \
    --seq_len 1024 \
    --n_layer 12 \
    --n_head 12 \
    --n_embd 768 \
    --max_iters 5000 \
    --lr_decay_iters 5000 \
    --learning_rate 6e-4 \
    --weight_decay 0.1 \
    --grad_clip 1.0 \
    --device cuda   # use "metal" for M‑series Macs or "cpu" as fallback

The script automatically enables mixed‑precision (torch.cuda.amp), gradient checkpointing, and dynamic batching. On a RTX 4060 the run completes in ≈ 8 min; on an Apple M2/M3 it finishes in ≈ 12 min.

Frequently Asked Questions

Why This Matters Right Now

1. Cost savings – Training a 125 M‑parameter model on a laptop consumes ~0.1 kWh, costing ≈ $0.10 in electricity versus $15‑$30 for a comparable cloud spot instance.
2. Speed of iteration – The “train‑once‑iterate‑fast” loop eliminates the cloud‑deployment lag, letting you test prompts, fine‑tune on domain data, and measure latency on the exact hardware you’ll ship.
3. Democratization – Students, hobbyists, and developers in regions with limited cloud access can now build LLMs locally, expanding the talent pool and fostering open‑source innovation.
4. Environmental impact – Running experiments on a laptop cuts the carbon footprint by ~80 % per run compared with multi‑node cloud training, aligning with ESG goals.

How NanoGPT Works

1. Minimalist Architecture

All of this fits in ≈ 200 KB of Python code, making it easy to read, modify, and debug.

2. Speed‑run Optimizations

End‑to‑End Tutorial

Step 1: Prepare the Environment

# (Optional) Create a clean virtual environment
python -m venv venv && source venv/bin/activate

# Install PyTorch with the right CUDA version
pip install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cu118

Step 2: Download a Dataset

# WikiText‑103 (default)
python data/wikitext.py --download

# Tiny Shakespeare (for sub‑minute runs)
python data/tinyshakespeare.py --download

Step 3: Train

python train.py \
    --dataset wikitext \
    --batch_size 64 \
    --seq_len 1024 \
    --n_layer 12 \
    --n_head 12 \
    --n_embd 768 \
    --max_iters 5000 \
    --lr_decay_iters 5000 \
    --learning_rate 6e-4 \
    --device cuda      # "metal" on macOS, "cpu" otherwise

You’ll see a live loss curve printed every 100 steps:

step 100 | loss 2.34 | lr 5.9e-04
step 200 | loss 2.01 | lr 5.8e-04
...
step 5000 | loss 1.23 | lr 0.0

Step 4: Generate Text

python sample.py \
    --out_dir out-wikitext \
    --start "The future of AI" \
    --max_new_tokens 200

Sample output (first 50 tokens):

The future of AI looks promising as researchers continue to push the boundaries of language modeling. With smaller, faster models like NanoGPT, developers can experiment locally, iterate quickly, and bring innovations to production without the overhead of cloud resources.

Step 5: Fine‑Tune on Your Own Data

1. Put your text files in data/custom/.
2. Run the preprocessing script:

   python data/preprocess.py --input_dir data/custom --output_dir data/custom-bin

1. Train with the same command, pointing to the new dataset:

   python train.py --dataset custom-bin --batch_size 32 ...

Real‑World Benchmarks

All costs assume US average electricity price of $0.12/kWh.

Takeaways

• You don’t need a cloud GPU to train a usable LLM; a modest laptop is enough for rapid prototyping.
• NanoGPT’s tiny codebase makes it trivial to experiment, tweak hyper‑parameters, or add new features (e.g., LoRA adapters).
• The cost, speed, and environmental benefits create a compelling loop for startups, researchers, and hobbyists alike.

Give it a try—clone the repo, run the one‑liner, and start generating your own text today. Happy hacking!

Herramienta mencionada: Groq Cloud