Track 10: Training LLMs, How Training Works

Pre-training at Scale, Explained

Explain data mixtures, tokens seen and compute budgets.

Status: Coming soon IntermediateVideo lessonAbout 13 min

Part of Train and Fine-Tune Models (20 of 45)

Get notified on YouTube

Course

Part of the free syllabus, not yet in a certificate course.

Udemy: coming soon

Watch first

Then try

In this module

  1. Fine-Tune vs Prompt vs RAG
  2. How LLM Training Works: Tokens, Loss, Pre-training vs Fine-tuning
  3. GPUs for Training: Colab, Kaggle, Cloud GPUs Safely
  4. Pre-training at Scale, Explained
  5. Scaling Laws and Compute-Optimal Training