Track 10: Training LLMs, Evaluate, Compress and Ship

Model Merging With mergekit: SLERP, TIES, DARE

Merge two fine-tunes into one model on a free GPU and test that it improved.

Status: Coming soon AdvancedVideo lessonAbout 18 min Free tools

Part of Train and Fine-Tune Models (45 of 45)

Get notified on YouTube

Course

Part of the free syllabus, not yet in a certificate course.

Udemy: coming soon

Tools

Watch first

In this module

  1. Benchmarks Explained, and lm-evaluation-harness
  2. Evaluate Your Fine-Tune Properly
  3. Quantization: GGUF, GPTQ, AWQ, bitsandbytes
  4. Merge, Export, Run Your Model (and Model Merging)
  5. Distillation: Teach a Small Model From a Big One
  6. Capstone: Fine-Tune for a Real Work Task
  7. Model Merging With mergekit: SLERP, TIES, DARE