Track 11: ML and Deep Learning Foundations for AI Engineers, Transformers and Attention

Code a Transformer Block

Implement a decoder block in PyTorch.

Status: Coming soon AdvancedVideo lessonAbout 15 min Free tools

Part of Train and Fine-Tune Models (13 of 45)

Get notified on YouTube Udemy certificate: coming soon

Course

Transformers & GPT From Scratch for AI Engineers

Udemy: coming soon

Tools

Watch first

Then try

In this module

  1. Embeddings: Meaning as Numbers
  2. The Transformer in One Picture
  3. Self-Attention, Multi-Head and Positions, Step by Step
  4. Code a Transformer Block
  5. KV Cache and Why Long Context Is Expensive
  6. Mixture of Experts, Beyond Transformers, and Looking Inside Models
  7. Interpretability Basics: Look Inside a Model

The course this lesson belongs to