Track 10: Training LLMs, Evaluate, Compress and Ship

Benchmarks Explained, and lm-evaluation-harness

Read benchmark tables critically and run standard benchmarks.

Status: Coming soon BeginnerVideo lessonAbout 11 min

Part of Train and Fine-Tune Models (39 of 45)

Get notified on YouTube

Course

Part of the free syllabus, not yet in a certificate course.

Udemy: coming soon

Watch first

Then try

In this module

  1. Benchmarks Explained, and lm-evaluation-harness
  2. Evaluate Your Fine-Tune Properly
  3. Quantization: GGUF, GPTQ, AWQ, bitsandbytes
  4. Merge, Export, Run Your Model (and Model Merging)
  5. Distillation: Teach a Small Model From a Big One
  6. Capstone: Fine-Tune for a Real Work Task
  7. Model Merging With mergekit: SLERP, TIES, DARE