Track 11: ML and Deep Learning Foundations for AI Engineers, Transformers and Attention

Self-Attention, Multi-Head and Positions, Step by Step

Compute attention for a 3-word sentence.

Status: Coming soon IntermediateVideo lessonAbout 13 min Free tools

Part of Train and Fine-Tune Models (12 of 45)

Get notified on YouTube Udemy certificate: coming soon

Course

Transformers & GPT From Scratch for AI Engineers

Udemy: coming soon

Tools

  • NumPy (open-source, free)

Watch first

Then try

In this module

  1. Embeddings: Meaning as Numbers
  2. The Transformer in One Picture
  3. Self-Attention, Multi-Head and Positions, Step by Step
  4. Code a Transformer Block
  5. KV Cache and Why Long Context Is Expensive
  6. Mixture of Experts, Beyond Transformers, and Looking Inside Models
  7. Interpretability Basics: Look Inside a Model

The course this lesson belongs to