Track 10: Training LLMs, How Training Works

Scaling Laws and Compute-Optimal Training

Estimate model size vs data for a budget.

Status: Coming soon AdvancedVideo lessonAbout 12 min

Part of Train and Fine-Tune Models (21 of 45)

Get notified on YouTube

Course

Part of the free syllabus, not yet in a certificate course.

Udemy: coming soon

Watch first

In this module

  1. Fine-Tune vs Prompt vs RAG
  2. How LLM Training Works: Tokens, Loss, Pre-training vs Fine-tuning
  3. GPUs for Training: Colab, Kaggle, Cloud GPUs Safely
  4. Pre-training at Scale, Explained
  5. Scaling Laws and Compute-Optimal Training