Track 9: LLM Engineering and Production, Testing and Evals

Evaluating Agents End to End

Score tool use, trajectories and outcomes.

Status: Coming soon AdvancedVideo lessonAbout 13 min Free tools

Part of Get an AI Engineering Job (72 of 83) and Build AI Agents and Apps (82 of 88)

Get notified on YouTube Udemy certificate: coming soon

Course

Agent Testing & Evaluation v2 (update of the live DeepEval course)

Udemy: coming soon

Tools

Watch first

In this module

  1. Why Evals Are the Unit Tests of AI
  2. Build an Eval Dataset (and Compare Prompts With promptfoo)
  3. LLM-as-Judge, Done Right
  4. DeepEval Tutorial
  5. Regression Gates in CI
  6. Evaluating Agents End to End

The course this lesson belongs to