week-4

Fine-tuning LLMs

Lecture slides: LLM-Course Lecture 4

Lab Exercise

1. Run the supervised_finetuning.ipynb notebook in Google Colab.
2. Change the base model used (search for small <7B parameter models in Hugging Face).
3. Change the dataset used in fine-tuning.
4. Bonus challenge:
   • Change the fine-tuning method from supervised fine-tuning to DPO.
   • Change the code accordingly, see: Hugging Face DPO Trainer Documentation
   • Select an appropriate DPO dataset. Search Hugging Face Datasets.