What this post covers

In this short post we walk through the eight stages inside a basic RAG chatbot, written as eight lines of code. RAG stands for retrieval augmented generation. It is for you if you have heard the word but cannot yet list what actually runs.

We start with all eight lines in one block, then the four that run before anyone asks, and the four that run on every question. Last, we send a follow-up the chatbot cannot handle and fix it by changing one line. You do not need to code. You only need to read a line like text = gather(sources) as "run gather on the sources, and keep the result as text".

By the end you can name the eight stages in order, and say where a chatbot keeps the conversation.

The eight lines

Picture one help page you trust, the page that explains how to reset a password. You want a chatbot that answers from that page, not from whatever it absorbed in training. Here is everything that chatbot does, one line per stage. In this block, sources is that help page.

text = gather(sources)
chunks = split(text)
vectors = embed(chunks)
db.add(vectors)
q = embed(question)
hits = find_similar(q)
messages = build(question, hits)
raw = llm(messages)

This is pseudocode, so you will not find a gather function in a library. Each name stands for one real job. Lines 1 to 4 run once, before anyone asks anything. Lines 5 to 8 run every time someone asks.

Each line hands its result to a later line. Keep this block in view. The rest of the post points back at it by line number.

Lines 1 to 4: before anyone asks

The first four lines prepare the help page ahead of time, so it is ready the moment a question arrives.

Line	Stage	What it does
1	Gather	Your help page, or a PDF, a scan, or a recording, becomes clean text.
2	Split	That text is broken into chunks, short pieces of a few sentences each.
3	Embed	Each chunk becomes a vector, a list of numbers. Chunks with similar meaning get similar numbers.
4	Store	Those vectors go into a database.

After line 4, the chunks of the password page sit in the database as numbers, waiting for a question.

Lines 5 to 8: every time someone asks

Now a user types: How do I reset my password? The last four lines run for that one question.

Line	Stage	What it does
5	Embed the question	The same embed from line 3, run on the question. Because it is the same step, the question's numbers can be compared with the chunks' numbers.
6	Find similar	The database returns the chunks closest to the question.
7	Messages	The question and those chunks go together into one request for the model.
8	The model	It reads the messages and returns raw text. Your app turns that raw text into the reply the user sees.

Line 6 is where people get confused. The closest chunks look like an answer, because they come from the right page. They are not one.

A teal magnet lifts the closest cards out of a drawer and lays them in front of a clay robot that has not answered yet. — Figure 1. Find similar pulls out the closest chunks. The model still has to read them and write the answer.

The closest chunks are not the answer. They are the reading the model gets in line 7.

If you read the last tip in this series, you met RAG as three steps. They sit on these same lines: lines 5 and 6 are retrieve, line 7 is augment, and line 8 is generate.

Retrieve: lines 5–6
Augment: line 7
Generate: line 8

The follow-up it forgets

These eight lines start over on every question. That works for the first question and breaks on the second. The same user writes back: And what if the email never arrives? Lines 5 to 8 run on that sentence alone, so the chatbot has no idea they mean the password reset email.

There are two tempting places to add memory, and both are wrong.

Push the old chat into Gather. Gather is line 1. It is for your documents, and it runs before anyone asks.

Retrain the model so it remembers. Training is slow, and the chat changes on every turn.

Chat bubbles clog a funnel meant for a stack of pages, under a red warning. Beside it, a queue of bubbles waits on slow gears and an hourglass. — Figure 2. Two wrong places for memory: the document intake, and a slow training run.

Memory means the earlier turns of this chat: the password question and the reply it got. They belong where the model reads, which is line 7. So line 7 gets one more input, history, next to the question and the chunks.

messages = build(history, question, hits)

Now the model sees the password question before the follow-up, and it can tell they mean the reset email.

Only line 7 changed. The other seven lines stay as they are.

Where to go next

To check yourself, open the companion tip, RAG in 8 lines, and where memory goes. It shows these eight lines on one screen, then gives you a short quiz and a practice where you change Messages for a follow-up yourself.

The step before this one is A sure answer is not a checked one, as a tip and as a post. It explains why a chatbot can sound sure and still be wrong, and when RAG beats fine-tuning. When you want to build these eight stages for real, the course RAG You Can Run crawls pages you choose, stores them, and answers from them.

A basic RAG chatbot is eight stages, one line each. The closest chunks are the reading, not the answer. And conversation memory goes in Messages, so only one line changes.

RAG in 8 lines, and where memory goes

More from the blog.

A sure answer is not a checked one

Software job posts, five years

The Origin header