RAG or Fine Tuning: Put the Information in the Right Place
You have a folder of reliable information and an AI assistant that keeps giving unreliable answers about it. The temptation is to ask someone to train the assistant on the folder.
Sometimes that is appropriate. Sometimes it is an expensive answer to a problem that needs retrieval. Start by separating access to information from changes to a model's behaviour.
Retrieval supplies material during use
Retrieval augmented generation, usually shortened to RAG, combines a generation system with a process that retrieves external material. Relevant passages are supplied as context for the response.
Fine tuning continues training a model using selected examples. It changes learned parameters. It can be used to adapt behaviour or performance for an intended task, although it is not a reliable substitute for a searchable, current record of every fact.
Microsoft Research's survey examines these and related approaches to bringing external information into language models. There is no universal winner independent of the application.
My first distinction would be operational. Do you need the system to consult information that changes, follow a particular output pattern or both Those are different requirements, and they may lead to different components.

A project archive makes the difference visible
Imagine a fictional research project whose partners, events and published reports change over time. If the assistant answers questions about next month's event, it needs access to the current approved event record.
I would first examine retrieval: which documents can be searched, how their versions are identified, how access is controlled and whether the answer links to the specific passage it used. Updating a document should have a defined route into the searchable index.
Now imagine a second problem. Every answer needs to follow an unusual approved structure. Relevant examples and clear instructions may be sufficient. If they are not, a carefully evaluated fine tuning experiment might be worth considering.
That does not mean fine tuning guarantees obedience, or that retrieval guarantees accuracy. A retrieval system can return the wrong document. A model can misunderstand the right passage. An apparently helpful answer can combine material from incompatible versions.
For this hypothetical project, I would include an expired event announcement in the test collection. The test is whether the complete system distinguishes the current record from an attractive but outdated result.

Test the complete route
Before adding complexity, trace one question through the system. What query is generated Which passages are retrieved Are their dates and permissions correct What information reaches the model Which claims appear in the answer
Check retrieval and generation separately. If the relevant document never arrives, changing the writing prompt may not solve the problem. If the document arrives but the model adds an unsupported claim, improving the search alone may not solve it either.
I would make unsupported answers visible and allow the assistant to say that the available material does not establish an answer. For important tasks, a reviewer still needs to inspect the source.
Keep a small evaluation set with current documents, obsolete documents, missing answers and ambiguous names. Repeat it when changing the model, index or instructions.
The right architecture should make the information easier to maintain and the failures easier to diagnose. A complicated system that hides where an answer came from is difficult to improve.
Put changing facts where they can be updated and inspected. Treat behavioural adaptation as a separate question. Then choose the smallest combination that meets the actual requirements.
Sources and further reading
Research checked on October 5, 2026. Original explanatory illustrations by Sami Haraketi, prepared for this article.
Sources and further reading
Research checked on October 5, 2026. Original explanatory illustrations by Sami Haraketi, prepared for this article.
Have a project behind the idea?
I work on websites, branding and film. If you need help turning your next idea into something people can see and use, tell me what you have in mind.
What did you think?
A thought, a different perspective, a little appreciation. You’re welcome here.





