What this post covers

In this short post we look at why ChatGPT and Gemini can sound sure and still be wrong, and how you can get answers from pages you trust. It is for anyone who uses those chatbots, and for junior developers who keep hearing the words RAG and fine-tuning. You only need to have asked a chatbot a question.

We start with the short explanation most people repeat and the two things it gets wrong about training. Then we name RAG and its three steps, see why it still needs a check, and finish with what to change when a page changes, which is where RAG and fine-tuning part ways.

By the end you can say why a sure answer is not a checked one, and whether a changed page needs RAG or retraining.

Why a sure answer can be wrong

Picture asking a chatbot how many days you have to return a sale item. The answer comes back calm and fluent, with no doubt in it. Most people explain that confidence with a short version of how these models are made.

What people say: it learned from the whole internet, and it was only trained to answer, never to say "I do not know".

Closer to true: it learned from a huge mix, not everything, and guesses still get rewarded.

First, the training mix. It is huge, but it is not the whole internet. It holds text you would trust and text you would not, and the model does not keep a label on which was which. So the same smooth sentence can come from a careful source or a careless one.

Second, models are also trained to admit uncertainty. They can still sound sure and be wrong, because a lot of training rewards a plausible answer instead of checking every fact. OpenAI described this in September 2025, in Why language models hallucinate: many tests still score a guess higher than "I do not know". A model that learns from those scores learns that guessing pays.

Teal pages from the left and crumpled grey scraps from the right pour into the open chest of one clay robot and mix. — Figure 1. The training mix holds text you would trust and text you would not. Both go into the same model, so a fluent answer can come from either.

For a quick look, that is often fine. When the answer has to be right, it is not.

A fluent sentence is not proof the source was trustworthy.

What RAG does, step by step

When the answer has to be right, you want the model to look the answer up in pages you chose, not only use what it absorbed in training. That lookup is called RAG, short for retrieval augmented generation. The name comes from a 2020 research paper by Lewis and colleagues, and each word is one step.

Here is the return question again, with RAG in place. The store's returns page is the example document.

Step	What it does	In the example
Retrieve	Searches the documents you trust and brings back the pages that match the question	Finds the returns page
Augment	Places those pages beside the question, so the model reads them as context	Your question plus the returns page
Generate	The model writes the answer from that context	"Sale items can be returned within 14 days" (example wording)

With RAG, the pages stay outside the model. That is what lets you check them and change them.

RAG still needs a check

The short version also leaves out a third thing. RAG does not guarantee the answer. The model can skip a page you gave it, or lean on what it absorbed in training instead. So you ask it to name the source, and then you open that page yourself.

A clay robot holds an answer card, a teal thread points to a source page under a magnifier, and one skipped page lies on the floor. — Figure 2. The answer is tied to a source page, but the model can still skip one. You open the source and find the line yourself.

A link is not a check. Opening the page and finding the sentence is.

The page says it
The page says something different
There is no page

The three results you can get when you open the page.

A page changed: RAG or fine-tuning?

That check only helps if the pages are current. Say the store changes its return rules next month, and a teammate says to retrain the model on the new page. That sounds thorough, but it is the wrong turn.

Update the documents it searches, and update the search index: the list that retrieve actually looks through. If the index stays old, retrieve still returns the old page.

Retraining a model on your own examples is called fine-tuning. It trains the model on many examples of a behavior, so that behavior stays inside the model. It is not a lookup. Editing a page later does not change what it learned, and there is no page to open when you want to check.

	RAG	Fine-tuning
Where the knowledge lives	Pages outside the model	Inside the model
When something changes	Update the documents and the index	Train again on new examples
Can you check the source?	Yes, open the page	No page to open
Best for	Facts on pages that change	A steady tone, format, or judgment

So the return rules that change a few times a year are a RAG job. If you want every support reply to open with the customer's name, one sentence of empathy, and then the fix, and you have 200 replies written that way, that is a fine-tuning job, because the shape does not live on a page. And one updated policy page is still a RAG job: update the page and the index, not the model.

Facts on a page: RAG. A way of answering, learned from examples: fine-tuning.

What you can say now

The companion tip, A sure answer is not a checked one, has a short quiz on the three steps and a challenge that sorts real situations into RAG or fine-tuning.

This post named the lookup. If you want to build it, RAG You Can Run is the course that walks you through it. You crawl pages you choose, store them, and answer from those pages, the same retrieve, augment, and generate steps you just read.

A sure answer is not a checked answer. RAG makes a chatbot answer from pages you chose, but you still open the page it cites. When a page changes, you update the documents and the search index. Fine-tuning is for a way of answering, not for a fact on a page.

A sure answer is not a checked one

More from the blog.

Software job posts, five years

The Origin header

The Cookie header