Track 9: LLM Engineering and Production, Cost, Caching and Routing
Model Routing, Cascades and Latency Engineering
Send each request to the cheapest model that passes, faster.
Get notified on YouTube Udemy certificate: coming soon
Course
Agent Observability & Cost Control
Udemy: coming soon
Tools
- LiteLLM (open-source, free)
Watch first
Then try
In this module
- Where the Money Goes: Token Accounting
- Caching: Prompt Caching and Semantic Caching
- Model Routing, Cascades and Latency Engineering
- AI Gateways, Budgets and Kill Switches
The course this lesson belongs to