Track 9: LLM Engineering and Production, Testing and Evals

Build an Eval Dataset (and Compare Prompts With promptfoo)

Collect cases and run config-driven comparisons.

Status: Coming soon IntermediateVideo lessonAbout 12 min Free tools

Part of Get an AI Engineering Job (68 of 83) and Build AI Agents and Apps (78 of 88)

Get notified on YouTube Udemy certificate: coming soon

Course

Agent Testing & Evaluation v2 (update of the live DeepEval course)

Udemy: coming soon

Tools

Watch first

Then try

In this module

  1. Why Evals Are the Unit Tests of AI
  2. Build an Eval Dataset (and Compare Prompts With promptfoo)
  3. LLM-as-Judge, Done Right
  4. DeepEval Tutorial
  5. Regression Gates in CI
  6. Evaluating Agents End to End

The course this lesson belongs to