RAG Chatbots & Knowledge Bases
A chatbot trained on a company's actual documents instead of the open internet — the automation equivalent of an employee who's read every FAQ, contract, and SOP the business owns and never forgets a word of it.
What RAG solves
Retrieval-Augmented Generation exists because a model's knowledge is frozen at training time and its context window is too small (and too expensive) to paste an entire knowledge base into every prompt. RAG instead retrieves only the handful of relevant passages for a given question at the moment it's asked, and hands the model those passages as grounding — turning "what do you know about X" into "given these specific documents, what do they say about X."
Two pipelines: ingestion and query
Every RAG system is really two separate workflows that happen to share a database:
| Pipeline | Runs | Steps |
|---|---|---|
| Ingestion | Whenever source documents change | Load file → extract text → chunk → embed → store |
| Query | On every user question | Embed the question → retrieve nearest chunks → hand to the model → generate an answer |
Chunking
A chunk is the smallest unit your system can retrieve — never more precise than that. n8n's Recursive Character Text Splitter is the sensible default: it breaks documents along paragraph and sentence boundaries rather than at a fixed character count, so a single answer (e.g. "can I negotiate pricing?") doesn't get sliced across three unrelated chunks. Start around 500 tokens per chunk with 10–20% overlap, and adjust only once you're seeing real retrieval misses — not before.
Build: a simple vector store bot
- Trigger → Manual (for ingestion, run on demand) → Google Drive → Download File (a company FAQ, SOP, or policy PDF).
- Extract From File → Extract from PDF, then Vector Store (Simple) → Add Documents.
- Attach an Embeddings node (OpenAI's
text-embedding-3-smallis the standard default) and a Default Data Loader using the Recursive Character Text Splitter from step 02. - Execute — a single FAQ typically splits into dozens of stored chunks.
- Add a second trigger, On Chat Message, into an AI Agent with a Simple Memory and a Vector Store (Simple) → Retrieve tool pointed at the same store.
- Ask it a question covered in the source document and confirm the answer echoes the document's actual wording, not a generic model guess.
A retrieval-first system prompt
Without an explicit instruction, an agent with a knowledge-base tool will often just answer from its own training data and never touch the tool. Force retrieval first:
You are a customer support assistant. Never answer from general
knowledge alone. Always call the knowledge base tool first and base
your answer only on what it returns. If the knowledge base has no
relevant answer, say so honestly rather than guessing.
Upgrading to Pinecone for production
n8n's Simple Vector Store lives in memory and resets with the workflow — fine for testing, not for a client's production chatbot. Pinecone (free tier available) persists permanently:
- Create a Pinecone account and an API key under API Keys.
- Create an index matching your embedding model's dimensions exactly —
1536fortext-embedding-3-small. Get this wrong and you'll need to delete and recreate the index; it can't be changed after creation. - Swap the Simple Vector Store nodes for Pinecone Vector Store (same Add Documents / Retrieve pattern), pointing both the ingestion and query workflows at the same index.
Metadata & namespaces
Tag chunks with metadata at ingestion time (a document's source, category, or date) to filter retrieval later — think of an index as a library, a namespace as one shelf in it (Sales docs, Support docs, HR docs), and each stored chunk as one book on that shelf. This is what lets one Pinecone index serve multiple departments, or multiple clients, without their content bleeding into each other's answers.
Data hygiene: garbage in, garbage out
Before pointing this at a client's Drive folder wholesale, get their source documents deduplicated and dated. This single step prevents more support tickets than any prompt tweak ever will.
Variations worth selling
- Trigger ingestion automatically on Google Drive → On Changes Involving a Specific Folder instead of running it manually, so a client's team can just drop new docs in.
- Package this as an internal support-desk deflection tool, priced against the support hours it saves.
- Combine with the personal assistant from Layer 06 so the agent can both take action and answer questions from company knowledge.
Sources
| Source | Used for |
|---|---|
| Jono Catliff (Automatable), "n8n 8 Hour Masterclass — Build & Sell AI Agents (Automate Anything)" | Primary source for this layer's concepts & build |
| n8n docs, Simple Vector Store node | In-memory ingest/retrieve build |
| Pinecone docs, "Get started" | Upgrading to a persistent production index |