Track 9: LLM Engineering and Production, Cost, Caching and Routing
AI Gateways, Budgets and Kill Switches
Centralise keys, logs, limits and spend caps.
Get notified on YouTube Udemy certificate: coming soon
Course
Agent Observability & Cost Control
Udemy: coming soon
Tools
- LiteLLM (open-source, free)
- Portkey (freemium)
- Cloudflare AI Gateway (freemium)
Watch first
In this module
- Where the Money Goes: Token Accounting
- Caching: Prompt Caching and Semantic Caching
- Model Routing, Cascades and Latency Engineering
- AI Gateways, Budgets and Kill Switches
The course this lesson belongs to