A team wants their AI to answer support questions using the company's own help docs. Someone says the word "fine-tune." Three weeks and a pile of GPU spend later, the model still gets facts wrong — and now it's also expensive to update.

This happens constantly. It's the most common AI build mistake I see, and it comes from mixing up three tools that look interchangeable but aren't: prompting, RAG, and fine-tuning. Pick the wrong one and you burn months solving a problem the right one would've handled in an afternoon.

So let's sort them out. The trick is to stop thinking about them as "levels of effort" and start thinking about what each one actually changes.

Prompting is exactly what it sounds like: the instructions you give the model. The system prompt, the examples you paste in, the way you phrase the request. You're not changing the model at all. You're changing what you hand it.

This is where you should always start, and most people quit on it far too early. A surprising number of "the model can't do this" problems are really "we asked badly" problems. Give it a couple of worked examples, tell it the format you want, spell out the edge cases, and behaviour you thought needed training often just appears.

Take a support bot that keeps replying in long, fluffy paragraphs when you want three tight bullet points. The instinct is "train it to be concise." You don't need to. Show it two examples of the exact format you want, add the line "answer in at most three bullets, no preamble," and the behaviour changes on the next message — for free.

Prompting is free, instant, and reversible. You can change it a hundred times a day. If a better prompt gets you there, you're done. Don't reach for anything heavier.

Here's the one people get wrong. When you want the model to answer using your information — your help docs, your product catalogue, last night's order data — the tool is RAG, not fine-tuning.

RAG (retrieval-augmented generation) is an open-book exam. The model itself stays frozen. When a question comes in, your system goes and fetches the relevant documents, pastes them into the prompt, and lets the model read them before answering. The knowledge lives outside the model, in a store you control, and gets pulled in fresh each time.

That "fresh each time" bit is the whole point. Your help docs changed this morning? RAG picks up the new version instantly, because it's just reading a document. Nothing about the model needs to change.

Think of the model as a sharp new hire and RAG as the reference binder you slide across the desk. They don't need to memorise the binder. They just need to know how to read it and answer well. When the binder updates, you swap the pages — you don't send the hire back to school.

Almost every "answer questions about our stuff" product is a RAG product. If knowledge changes daily or weekly, RAG is basically your only sane option.

So what's fine-tuning actually for? Behaviour, not knowledge.

Fine-tuning means taking the model and continuing its training on your own examples, nudging its internal weights so it becomes something slightly different. It's a closed-book exam: whatever it learns is baked in, and it answers from that, not from documents you hand it at the moment.

That makes fine-tuning excellent at things prompting can't reliably pin down: a very specific tone or brand voice, a rigid output format it must hit every single time, a specialised reasoning style, a narrow classification task at huge volume. When the model basically knows the right answer but keeps expressing it in the wrong shape, and no amount of prompting fixes it. That's the fine-tuning signal.

Fine-tuning to add knowledge is paying the highest price for the worst version of what RAG does cheaply.

A clean example of when it is right: say you're auto-routing millions of support tickets into forty precise categories, and the labels have to be exact every time. You've prompted it, you've given examples, and it still drifts on the tricky ones. That's a behaviour problem at scale — a fixed task, a rigid output, huge volume, no fresh facts needed. Fine-tune it on a few thousand correctly-labelled tickets and it locks in. Notice what that example is not: it isn't "answer questions about our docs." It's a repeated shape you want burned in, not a body of knowledge you want looked up.

For a while, fine-tuning meant retraining an enormous model, slow and painfully expensive. That's why the old advice was "avoid it unless you're a big lab."

Then came LoRA and the family of methods called PEFT (parameter-efficient fine-tuning). Instead of retraining the whole model, LoRA freezes the original and trains a tiny set of add-on adjustments that graft new behaviour onto the frozen weights. It's a small clip-on lens rather than a new camera. That dropped the cost enough that fine-tuning is now realistic for normal teams.

Which is good, and also why the mistake got more common. Because fine-tuning is cheap now, people reach for it as a first move instead of a last one. Cheap doesn't mean correct. It's still the wrong tool for adding knowledge; it just got wrong faster and cheaper.

Put it together and there's a clean sequence. Start with prompting. It costs nothing and solves more than you'd expect. Add RAG when the model needs your data — especially data that changes, or facts a user could challenge you on. Fine-tune last, and only for behaviour the first two can't nail down.

And these aren't rivals. The strongest production systems stack them: a model lightly fine-tuned to hold a consistent voice, with RAG feeding it current facts and citations on top, all steered by a sharp prompt. Behaviour from fine-tuning, knowledge from RAG, direction from the prompt — each doing the one job it's actually good at.

The mistake was never picking one tool over another. It was using a tool for a job it was never built to do, and finding out three weeks and a GPU bill later.

Thats it
See you next time,
—Sid

P.S. small announcement: I’ve started creating reels on Instagram, where I explain how popular tech products work behind the scenes. I decode decisions and unpack products that impact how they function in the real world. Users love some decisions, and absolutely hate others. I show them all. If that is a space that interests you, please follow me, and drop a hi @sidwillexplain