Transformers & GPT From Scratch for AI Engineers
Coming soon BeginnerIntermediateAdvanced
Get notified on YouTube Udemy certificate: coming soon
- Udemy
- Coming soon
- Lessons
- 24
- Planned length
- About 8 h 35 min
- Projects
- 4
- Practicals
- Planned: a hands-on practical and a Udemy assignment in every section, plus a capstone
What you will be able to do
- Explain every stage from text to chatbot
- Train a tokenizer on your own text
- Train a tiny GPT that writes in a style
- Pre-train and evaluate a small LM on a curated corpus
- Compute cosine similarity and read a token distribution
- Read a matrix multiply as "mixing features"
Syllabus
Training LLMs: Small Models From Scratch
- How a GPT Is Built, End to End Explain every stage from text to chatbot
- Build a BPE Tokenizer Train a tokenizer on your own text
- Build and Train a Toy GPT Train a tiny GPT that writes in a style
- Train a Small Language Model on Colab Pre-train and evaluate a small LM on a curated corpus
ML and Deep Learning Foundations for AI Engineers: Maths Intuition
- Vectors, Similarity and Probability, Visually Compute cosine similarity and read a token distribution
- Matrices: What a Layer Really Does Read a matrix multiply as "mixing features"
- Gradients, Loss Functions and Gradient Descent Explain how a model learns by going downhill
- Optimisers: SGD, Adam, Learning-Rate Schedules Tune a learning rate from a loss curve
ML and Deep Learning Foundations for AI Engineers: Classic ML
- Supervised vs Unsupervised Learning Frame a business problem as an ML task
- Your First Model With scikit-learn Train and evaluate a classifier
- Metrics and Overfitting: Precision, Recall, F1, Validation Pick a metric and spot overfitting
- Trees and Gradient Boosting Beat a baseline on tabular data
ML and Deep Learning Foundations for AI Engineers: Neural Networks
- From One Neuron to a Network Build a neuron in NumPy
- Backpropagation, Step by Step Trace gradients through a tiny network
- PyTorch Basics Train a model with a PyTorch training loop
- Project: Digit Recogniser Train and test a network on MNIST
- Before and Around Transformers: CNNs, RNNs, Regularisation Know why transformers replaced RNNs and stabilise training
ML and Deep Learning Foundations for AI Engineers: Transformers and Attention
- Embeddings: Meaning as Numbers Explain why king - man + woman is close to queen
- The Transformer in One Picture Name every block of a transformer
- Self-Attention, Multi-Head and Positions, Step by Step Compute attention for a 3-word sentence
- Code a Transformer Block Implement a decoder block in PyTorch
- KV Cache and Why Long Context Is Expensive Estimate memory for a context length
- Mixture of Experts, Beyond Transformers, and Looking Inside Models Explain MoE, the alternatives and interpretability basics
- Interpretability Basics: Look Inside a Model Inspect attention and activations in a small model to see what drives an answer
Take before
Lesson list from the ADITEJUS curriculum. Planned lengths are estimates per lesson; Udemy shows the final length.