Track 10: Training LLMs, Small Models From Scratch

Build a BPE Tokenizer

Train a tokenizer on your own text.

Status: Coming soon IntermediateVideo lessonAbout 13 min Free tools
Get notified on YouTube Udemy certificate: coming soon

Course

Transformers & GPT From Scratch for AI Engineers

Udemy: coming soon

Tools

Watch first

Then try

In this module

  1. How a GPT Is Built, End to End
  2. Build a BPE Tokenizer
  3. Build and Train a Toy GPT
  4. Train a Small Language Model on Colab

The course this lesson belongs to