RAG You Can Run: Crawl Your Site, Store the Pages, Answer from Them

Decide whether this course is for you, confirm the prerequisites, and write a one-sentence acceptance test for the system you will build: a named page on a site you choose, one question that page can answer, and the evidence you expect to inspect after the crawl. Nothing is installed here. Keep that acceptance card; chapter 5 completes it.

Use one question from your acceptance card to trace all eight RAG stages. For each stage, record its input, output, choice, and likely failure in a one-page pipeline sheet. You do not install or build here; the completed sheet is the chapter win and the diagnostic map you will reuse in chapters 4–6.

The first artefact on your machine. Git, Docker on your OS, clone `rag_tutorial`, the three env files, `docker compose up -d --build`, then a high-level folder map. You do not crawl in this chapter.

You cloned the finished repo and started it in chapter 3. This chapter walks `api/` and Postgres in the order the folder grew, on the real lines, so you can explain and change every file. Four checkpoints on your running stack: Compose connects `api` and `db`; the container waits, migrates, and serves; the pgvector tables exist; the six `/rag` routes answer. Every lesson ends with one command or request you run to see that layer working. The scraper stays idle.

The scraper is how rows get in. Point it at a site you choose. Honour `robots.txt`. Crawl. List pages and chunks. Ask one question on your pages. Then read `responses/` as proof the same path already worked on the sample site.

Capstone first, roadmap second. Open on pride, then grade the evidence from chapter 5 — listed pages, listed chunks, your answer, and the saved sample answers — on a pass/fail scorecard. Then map the next layers a real visitor needs and close on pride. Nothing here is built in this repo: no chat UI, Redis, nginx, hybrid search, or eval harness.