Unsloth: Fine-Tuning on Hardware You Already Have
Before you fine-tune anything, read the part about why your problem is probably a retrieval problem.
Before anything else in this article: your problem is probably not a fine-tuning problem.
The request I get is usually "train it on our documents." That is retrieval, not fine-tuning, and confusing the two costs weeks. Fine-tuning teaches a model how to behave. Retrieval gives it what to know.
When fine-tuning is genuinely right
- You need a consistent output format that prompting keeps drifting from
- You need a specific tone or domain register that examples in the prompt cannot reliably enforce
- You want a small model to handle a narrow task well, to cut cost or run locally
- You have hundreds of good input-output examples already
When it is not
- You want the model to know your company's facts — that is retrieval
- Your documents change regularly — retraining on every change is absurd
- You have fifty examples — not enough
- You have not seriously tried prompting yet
I would guess four out of five fine-tuning requests are actually retrieval requests, and the fifth would have been solved by a better prompt.
What Unsloth changes
If you do need it, the practical barrier used to be hardware. Unsloth makes LoRA fine-tuning meaningfully faster and lighter on memory, which puts a 7B model within reach of a single consumer GPU or a modest cloud instance.
LoRA is the reason this is affordable: instead of updating every weight, you train a small set of adapter matrices. The output is a small adapter file, not a new multi-gigabyte model, and you can swap adapters per task.
The part that determines success
Data quality, entirely. A hundred carefully written examples beat a thousand scraped ones. Every example should demonstrate exactly the behaviour you want, in exactly the format you want, with no inconsistency between them — because the model will faithfully learn your inconsistencies too.
Hold out a test set before you start. Fine-tuning can make a model better at your task and worse at everything else, and without an evaluation you will not notice until a user does.
The maintenance question nobody asks
A fine-tuned model is a dependency you now own. When the base model is superseded in six months, your adapter does not automatically move with it. Budget for redoing the work, and keep the training data in version control so redoing it is cheap.
Resources
- Repo: unslothai/unsloth
- Docs: docs.unsloth.ai
- Video walkthroughs: YouTube: Unsloth fine-tuning
- Try this first: LlamaIndex RAG
Need this built properly?
I build secure, fast, bilingual platforms for clients across Egypt, Saudi Arabia, the UAE and Kuwait.
