<p>This tip covers why ChatGPT and Gemini can sound sure and still be wrong, and how to get answers from pages you trust. If you use those chatbots, this is for you.</p><p>By the end, you can check a sure answer instead of trusting it.</p><p>A short version says they learned from the whole internet, and that they were only trained to answer, never to say they do not know. That version leaves three things out. Two are about training. The third is about RAG.</p><p>First, the training mix is huge, and it is not the whole internet. It includes text you would trust and text you would not. So a fluent sentence is not proof the source was trustworthy.</p><p>Second, models are also trained to admit uncertainty. They can still sound sure and be wrong, because a lot of training rewards a plausible answer instead of checking every fact. OpenAI described that in September twenty twenty-five, in Why language models hallucinate. Many tests still score a guess higher than I do not know.</p><p>A quick look can live with a fluent wrong answer. When the answer has to be right, you want the model to look up pages you chose, not only what it absorbed in training.</p><p>That lookup is RAG, retrieval augmented generation. Three words, three steps. Retrieve, augment, generate.</p><p>Retrieve searches the documents you trust and brings back the pages that match the question.</p><p>Augment places those pages beside the question, so the model reads them as context.</p><p>Generate is the model writing the answer from that context.</p><p>Here is the third thing the short version leaves out. RAG does not guarantee the answer. The model can skip a page you gave it, so you ask it to name the source, and then you read that page yourself.</p><p>That check only helps if the pages are current. When one document changes next month, training the model again on that one file is the wrong turn. You update the documents it searches, and you update the search index, the list retrieve actually looks through. If that list stays old, retrieve still returns the old pages.</p><p>So leave with this. A sure answer is not a checked answer. You still read the page it cites, and when a page changes you update the search, including the index.</p><p>Now take the quiz on this tip. It checks which RAG step does which job, and what you update when a page changes. Then try the challenge. Pick a question you already asked a chatbot, ask it for the source, and open that page to see if the answer is really there.</p><p>I will meet you in the next tip.</p>

Why chatbots sound sure and still get it wrong, what RAG's retrieve, augment, and generate steps do, and why you still read the page it cites.

A sure answer is not a checked one | Tech Tips by Moh

They are never trained to say I do not know

A lot of training rewards a plausible answer instead of checking every fact

Tests always score I do not know higher than a guess

Update the documents and the search index

Update only the documents and leave the index as it is

Nothing, because RAG notices the change by itself

A sure answer is not a checked one

<div class="course-slide cs-center">
  <p class="cs-kicker">This tip</p>
  <h2 class="cs-title cs-at-3-1">Sure, and still <span class="cs-bad">wrong</span></h2>
  <p class="cs-pill cs-at-1 cs-until-5-8">ChatGPT</p>
  <p class="cs-pill cs-at-2-1 cs-until-5-8">Gemini</p>
  <p class="cs-pill cs-at-5-8">Answers from pages you trust</p>
  <p class="cs-pill cs-at-11-4">By the end: check a sure answer</p>
</div>

<div class="course-slide">
  <p class="cs-kicker">The short version</p>
  <h2 class="cs-title">What people usually say</h2>
  <div class="cs-grid cs-cols-2">
    <div class="cs-card cs-at-1-2"><p class="cs-card-text">They learned from the whole internet</p></div>
    <div class="cs-card cs-at-3-3"><p class="cs-card-text">They were only trained to answer, never to say “I do not know”</p></div>
  </div>
  <p class="cs-pill cs-at-7-6">It leaves <span class="cs-hl">three things</span> out</p>
  <div class="cs-row"><span class="cs-chip cs-at-9">1 · The training mix</span> <span class="cs-chip cs-at-9">2 · Uncertainty</span> <span class="cs-chip cs-at-11-4">3 · RAG</span></div>
</div>

<div class="course-slide cs-center">
  <p class="cs-kicker">1 · The training mix</p>
  <h2 class="cs-title cs-at-1-4">Huge, not the <span class="cs-hl">whole internet</span></h2>
  <p class="cs-pill cs-at-4-6">Text you would trust</p>
  <p class="cs-pill cs-at-6">Text you would not</p>
  <p class="cs-pill cs-at-7-6">Fluent is not proof</p>
</div>

<div class="course-slide">
  <p class="cs-kicker">2 · Uncertainty</p>
  <h2 class="cs-title cs-at-0-7">Also trained to <span class="cs-hl">admit uncertainty</span></h2>
  <ul class="cs-crosses">
    <li class="cs-at-3-2">Can still sound sure and be wrong</li>
    <li class="cs-at-6-4">Training often rewards a plausible answer</li>
    <li class="cs-at-8-7">Instead of checking every fact</li>
  </ul>
</div>

<div class="course-slide cs-center">
  <p class="cs-kicker">OpenAI · September 2025</p>
  <h2 class="cs-title cs-at-2-9">Why language models hallucinate</h2>
  <p class="cs-text cs-at-5-1">Many tests score <span class="cs-hl">a guess</span> higher than “I do not know”</p>
</div>

<div class="course-slide cs-center">
  <p class="cs-kicker">When it has to be right</p>
  <h2 class="cs-title cs-at-5-7">Look up <span class="cs-hl">pages you chose</span></h2>
  <p class="cs-pill cs-at-0-5 cs-until-3-3">Quick look: fluent is fine</p>
  <p class="cs-pill cs-at-3-3">Has to be right?</p>
  <p class="cs-pill cs-at-7-4">Not only what it absorbed in training</p>
</div>

<div class="course-slide cs-center">
  <p class="cs-kicker">That lookup</p>
  <p class="cs-display cs-at-0-7"><span class="cs-hl">RAG</span></p>
  <p class="cs-lead cs-at-1-5">Retrieval augmented generation</p>
  <p class="cs-note cs-at-3-7">Three words, three steps</p>
  <div class="cs-row"><span class="cs-chip cs-at-5-8">Retrieve</span> <span class="cs-chip cs-at-6-7">Augment</span> <span class="cs-chip cs-at-7-2">Generate</span></div>
</div>

<div class="course-slide">
  <p class="cs-kicker">Three steps</p>
  <h2 class="cs-title">What each step does</h2>
  <ol class="cs-steps">
    <li><strong>Retrieve</strong> searches the documents you trust and brings back the matching pages</li>
    <li class="cs-at-4-8"><strong>Augment</strong> places those pages beside the question, as context</li>
    <li class="cs-at-9-3"><strong>Generate</strong> writes the answer from that context</li>
  </ol>
</div>

<div class="course-slide cs-center">
  <p class="cs-kicker">3 · Left out</p>
  <h2 class="cs-title cs-at-2-9">RAG does not <span class="cs-bad">guarantee</span> the answer</h2>
  <p class="cs-pill cs-at-5-7">It can skip a page you gave it</p>
  <p class="cs-pill cs-at-7-8">Ask it to name the source</p>
  <p class="cs-pill cs-at-9-1">Read that page yourself</p>
</div>

<div class="course-slide cs-center">
  <p class="cs-kicker">When a page changes</p>
  <p class="cs-pill cs-at-1-8 cs-until-4-6">Only helps if the pages are current</p>
  <h2 class="cs-title cs-at-4-6 cs-until-7-8"><span class="cs-strike">Retrain on that one file</span></h2>
  <p class="cs-pill cs-at-7 cs-until-7-8"><span class="cs-bad">Wrong turn</span></p>
  <h2 class="cs-title cs-at-7-8">Update what it <span class="cs-hl">searches</span></h2>
  <p class="cs-pill cs-at-8-9">The documents</p>
  <p class="cs-pill cs-at-11-3">The search index</p>
  <p class="cs-pill cs-at-14-1"><span class="cs-bad">Old index returns old pages</span></p>
</div>

<div class="course-slide cs-center">
  <p class="cs-kicker">Leave with this</p>
  <p class="cs-display cs-at-1-4">A sure answer is not a <span class="cs-hl">checked</span> answer</p>
  <ul class="cs-checks">
    <li class="cs-at-4-5">Read the page it cites</li>
    <li class="cs-at-5-8">Page changed? Update the search, index included</li>
  </ul>
</div>

<div class="course-slide cs-center">
  <p class="cs-kicker">On this tip</p>
  <h2 class="cs-title cs-at-0-4">Take the <span class="cs-hl">quiz</span></h2>
  <p class="cs-pill cs-at-2-4">Which step does which job</p>
  <p class="cs-pill cs-at-4-6">What you update when a page changes</p>
  <p class="cs-pill cs-at-6-7">Then: the challenge</p>
</div>

<div class="course-slide cs-center">
  <p class="cs-kicker">The challenge</p>
  <h2 class="cs-title">Check one <span class="cs-hl">real answer</span></h2>
  <p class="cs-pill cs-at-0-4">A question you already asked a chatbot</p>
  <p class="cs-pill cs-at-3">Ask it for the source</p>
  <p class="cs-pill cs-at-4-2">Open that page. Is the answer there?</p>
  <p class="cs-pill cs-at-7-4">See you in the next tip</p>
</div>