Track 9: LLM Engineering and Production
Agent Testing & Evaluation v2 (update of the live DeepEval course)
35 lessons
Track 9: LLM Engineering and Production, Eval Tools (One Lesson Each)
OpenAI Evals: Open-Source Framework and Paid Evals API
Run a graded eval on the framework and on the API (verify price).
Get notified on YouTube Udemy certificate: coming soon
Course
Agent Testing & Evaluation v2 (update of the live DeepEval course)
Udemy: coming soon
Tools
- OpenAI Evals (open-source framework + Evals API) (open-source, paid)
- OpenAI API (paid)
Watch first
In this module
- promptfoo: Your First Eval Suite in YAML
- Inspect (UK AISI): Tasks, Solvers and Scorers
- OpenAI Evals: Open-Source Framework and Paid Evals API
- LangSmith Evaluations: Datasets and Experiments
- Langfuse Evals: Datasets, Experiments, LLM Judges
- Braintrust: Experiments, Scorers and autoevals
- Arize Phoenix Evals: Trace First, Then Evaluate
- TruLens: Feedback Functions and the RAG Triad
- DeepEval Advanced: Custom Metrics and Agent Tracing
- RAGAS Advanced: Agent, Tool-Call and Multi-Turn Metrics
- One Eval, Six Tools: Pick Your Eval Stack (Free vs Paid)
The course this lesson belongs to