The three questions to ask before you fine-tune
Priya Raman ·
Fine-tuning is the most expensive way to find out your eval set was wrong. Ask these first.
What we changed
Do you have 200 examples you would be happy to be graded against? If not, start there.
Has a better prompt been tried, measured against those examples?
Is the problem knowledge, or behaviour? Knowledge is a retrieval problem and fine-tuning will not fix it.
The full walkthrough is in the documentation.