Track 15: Frontier and Advanced, Reasoning Models and Inference

Thinking Budgets: Cost, Latency and Quality

Tune reasoning effort against an eval set (verify price).

Status: Coming soon IntermediateVideo lessonAbout 13 min Needs a paid tool
Get notified on YouTube Udemy certificate: coming soon

Course

LLM APIs Masterclass: OpenAI, Claude, Gemini & Local Models

Udemy: coming soon

Tools

Watch first

In this module

  1. Reasoning Models in Production: When They Pay Off
  2. Thinking Budgets: Cost, Latency and Quality
  3. Long Context vs RAG: Measure Cost and Accuracy
  4. Small and On-Device Models: Laptops, Phones and Browsers
  5. Inference Optimisation: KV Cache and Prefix Caching
  6. Speculative Decoding and Continuous Batching
  7. Project: Benchmark Your Model Serving Stack

The course this lesson belongs to