RAG and document search
What Is RAG? How AI Answers from Your Documents
RAG (Retrieval-Augmented Generation) is an approach in which AI retrieves relevant information from external sources before generating an answer. For business documents, it means finding passages in approved manuals, policies or catalogues and using them to produce a response with a checkable source. The documents supply context for the question; updating them does not require retraining the model each time.
Alexander Chakmakov 
What RAG means for your business documents
A customer asks which information to include in a warranty request. The answer exists in your procedure, but someone needs to find the current document, read the relevant section and explain it. RAG connects that search to a language model, the software that interprets the question and writes the response.
Retrieval-Augmented Generation describes the sequence: retrieve information, add it to the model's context, then generate an answer. The original RAG research paper explores combining a language model with retrieved external passages. In a business application, that external source can be your approved document collection.
A useful result includes enough provenance for a person to verify it: a document title, version and relevant section. RAG is an approach to building the system. The chat window is only the place where someone uses it; document preparation, search and evaluation determine whether the answer is useful.
How RAG works: prepare the archive, then retrieve for each question
There are two processes. First, the documents become searchable. Text is extracted from files, headings and source references are preserved, and longer content is divided into passages. This is called chunking. A passage needs enough surrounding context to retain the meaning of its rules and exceptions.
Second, each question triggers retrieval. The system selects useful passages and supplies them to the model. OpenAI's file search tool is one hosted implementation that searches uploaded files using semantic and keyword search. Your choice of implementation depends on sources, permissions and operating requirements.
01 / Preparation and answering happen separately
- WHEN ADDEDDocumentExtract text, version and access permissions.
- PREPARATIONPassagesKeep meaningful sections with source references.
- STORAGESearch indexStore information ready for retrieval.
- FOR EACH QUESTIONSearchRetrieve only from sources the person may access.
- CONTEXTSelected passagesSupply relevant evidence for this question.
- RESPONSEAnswer and sourceExplain the evidence or report missing information.
A scanned PDF requires OCR, or optical character recognition, to turn its page images into text. A table extracted in the wrong reading order can reverse the meaning of a condition. Inspecting the extracted content is therefore an early quality check, before changing prompts or choosing a larger model.
A document-search example from question to cited answer
A lighting retailer keeps its warranty procedure and delivery terms in separate documents. The user asks: “What should I send with a warranty request?”.
02 / Warranty procedure and delivery terms
What should I send with a warranty request?
Include the order number, a description of the problem and a photograph of the product.
The delivery address can be changed before the order is handed to the courier.
Send your order number, a description of the problem and a photograph of the product. Source: “Warranty procedure v3”, §2.
In a working application, the source reference opens the permitted document at its section or page. Its version helps establish whether the answer used the current procedure. A citation needs to support the claim next to it; displaying a document name alone does not establish accuracy.
If the user asks whether their repair has been approved, the general procedure cannot answer. That requires a live lookup of their case. If the documents do not cover a question, the application should say what information is missing and provide a route to a person.
Semantic search, keyword search and exact identifiers
Semantic search finds related meaning. “My lamp stopped working” can match a passage about “lighting equipment faults” despite different wording. Many implementations use embeddings, numerical representations that make similarity comparison possible.
OpenAI's retrieval guide explains semantic matching and vector-store indexing. Exact matching still matters for product codes, agreement numbers and unfamiliar abbreviations. Evaluate the search method with actual customer questions, including typos and identifiers that differ by a single character.
We developed meaning-based search over a 1.7-million-image archive for a platform with 2.3 million-plus searches. Finding an appropriate result is a substantial task in its own right. A RAG application adds answer generation and checks that the resulting claims are supported by the retrieved content.
RAG does not require one particular database or framework. A small, well-structured collection may be served by straightforward search. Larger or less structured archives can need richer retrieval. The questions and measured results should justify each added component.
Where RAG fails and how to evaluate it
RAG can supply relevant evidence without guaranteeing a correct answer. Search may retrieve an old policy, omit an exception or confuse two products. The model may also misread a correct passage. These failure points need separate checks.
Build a set of questions with expected sources and acceptable answers. Check retrieval first: did the right passage reach the model? Then check the generated response: does each claim follow from that evidence? Changing the answer prompt will not repair a search that consistently retrieves the wrong manual.
- A straightforward question: does the relevant section appear?
- A rule with an exception: does the answer retain the condition?
- A question outside the documents: does the system acknowledge missing evidence?
- An outdated or unauthorised source: is it excluded from the answer?
Measure response time and usage cost on the same questions. A single accuracy percentage is hard to assess without knowing the test cases. Repeat the evaluation when source handling, retrieval settings or models change.
Documents can also contain misleading instructions. Treat retrieved text as source material and keep permission enforcement outside the model. A paragraph in an uploaded file must not grant the user access to other records or authorise a write operation.
Keep public documents and private customer records separate
A public chatbot may answer from general instructions. A signed agreement or a customer's discount belongs behind different permissions. Apply those boundaries before a passage reaches the model. Asking the model not to reveal a document is not access control.
03 / Permissions determine the search space
Public instructions and catalogue. Personal documents are excluded.
Filter permitted sources before providing context to the model.
Public material and their own records. Other customers' agreements are excluded.
Establish which content reaches the model provider and what the application records. Removed access, deleted documents and cached results must follow the same rules. A permission change in the source system is incomplete until retrieval respects it.
RAG versus fine-tuning, and where MCP fits
Fine-tuning uses training examples to change model behaviour. RAG supplies retrieved information at request time. They can be combined. OpenAI's model optimization guide treats providing context and training as different ways to improve results.
For frequently updated instructions, retrieval allows the document and index to change without training the model for each edit. It still needs a policy for current versions, deletion and source references. An old passage remains old information even when a capable model reads it.
An MCP server exposes capabilities to an AI application. One of those capabilities can be RAG search. For live stock or order status, a direct call to the source system is often more suitable than retrieval from a PDF catalogue.
Starting with a RAG chatbot for your business
Choose a repeated question and a bounded set of current documents. Include a correct answer, an unsupported question and an unauthorised document in the first evaluation. This makes the initial release small enough to inspect and substantial enough to test the real problem.
For customer conversations, see our custom AI chatbot service. When retrieval belongs in an internal workflow or product, our AI application development covers that integration. Document quality, updates, permissions and connected systems determine the scope and price.
Technical sources and tool references were checked on 28 October 2026.
Frequently asked questions
What is RAG in simple terms?
RAG retrieves relevant information before an AI model writes its answer. The retrieved passages become context for that question, so the answer can use your approved sources.
Does RAG prevent hallucinations?
No. Retrieval can find the wrong passage or an outdated version, and a model can still misinterpret a correct source. Source checks, evaluation questions and a clear response when evidence is missing remain necessary.
Can a RAG chatbot use PDF documents?
Yes, when their content can be extracted reliably. Scanned pages need optical character recognition. Tables and complex layouts need separate checks, and each passage should retain a reference to its document and page.
Does RAG require a vector database?
Not in every implementation. RAG requires useful retrieved context. Keyword search, semantic vector search or a combination can supply it, depending on the data and the questions.
What happens when a source document changes?
The changed document is processed and its relevant passages are updated in the index. Superseded versions should be excluded from current answers. Deletions and changed permissions also need to reach the retrieval system.
