One Eval, Six Tools: Pick Your Eval Stack (Free vs Paid)

Course

Agent Testing & Evaluation v2 (update of the live DeepEval course)

Udemy: coming soon

Tools

• DeepEval (open-source, freemium)
• RAGAS (open-source, free)
• promptfoo (open-source, free)
• Inspect (UK AI Security Institute) (open-source, free)
• Langfuse (open-source, freemium)
• Braintrust (freemium)

Watch first

• promptfoo: Your First Eval Suite in YAML
• Langfuse Evals: Datasets, Experiments, LLM Judges
• Braintrust: Experiments, Scorers and autoevals

In this module

1. promptfoo: Your First Eval Suite in YAML
2. Inspect (UK AISI): Tasks, Solvers and Scorers
3. OpenAI Evals: Open-Source Framework and Paid Evals API
4. LangSmith Evaluations: Datasets and Experiments
5. Langfuse Evals: Datasets, Experiments, LLM Judges
6. Braintrust: Experiments, Scorers and autoevals
7. Arize Phoenix Evals: Trace First, Then Evaluate
8. TruLens: Feedback Functions and the RAG Triad
9. DeepEval Advanced: Custom Metrics and Agent Tracing
10. RAGAS Advanced: Agent, Tool-Call and Multi-Turn Metrics
11. One Eval, Six Tools: Pick Your Eval Stack (Free vs Paid)