Track 4: Working With LLM APIs
Open Models in Practice: Hugging Face, Ollama, llama.cpp & vLLM
9 lessons
Track 15: Frontier and Advanced, Reasoning Models and Inference
Speculative Decoding and Continuous Batching
Turn on speculative decoding and batching and measure the speed-up.
Get notified on YouTube Udemy certificate: coming soon
Course
Open Models in Practice: Hugging Face, Ollama, llama.cpp & vLLM
Udemy: coming soon
Tools
Watch first
Then try
In this module
- Reasoning Models in Production: When They Pay Off
- Thinking Budgets: Cost, Latency and Quality
- Long Context vs RAG: Measure Cost and Accuracy
- Small and On-Device Models: Laptops, Phones and Browsers
- Inference Optimisation: KV Cache and Prefix Caching
- Speculative Decoding and Continuous Batching
- Project: Benchmark Your Model Serving Stack
The course this lesson belongs to