The Skill Explained
What Are RAG and Retrieval?
Retrieval-augmented generation (RAG) is a method where a system first searches a knowledge source for passages relevant to a question, then gives them to a language model to write an answer based on them, often with citations. It lets a model use private or recent information and reduces hallucinations.
Why AI Employers Ask for Them
Companies want AI assistants that answer from their policies, manuals and records, not from general internet knowledge, and they want answers that cite sources. RAG is the most common way to build that without retraining a model, which is why it appears so often in AI engineer and solutions roles.
How It Works
First, documents are prepared: split into chunks of a few paragraphs, turned into embeddings (vectors of numbers that capture meaning) and stored in a vector database or search index. This is called indexing.
When a user asks a question, the question is embedded the same way and the system retrieves the closest chunks, often combining meaning-based search with keyword search and a reranking step that orders results more precisely. The best chunks are inserted into the prompt with instructions to answer only from them and to cite them.
Quality depends on every step: how documents are chunked, which embedding model is used, how many passages are retrieved, and how the prompt handles missing information. Teams measure retrieval and answers separately with test questions.
What You Do With It at Work
- Build an ingestion pipeline that chunks documents and stores their embeddings.
- Tune retrieval: chunk size, hybrid search, reranking, metadata filters.
- Write prompts that make the model cite sources and admit when it does not know.
- Create an evaluation set and measure retrieval and answer quality.
- Keep the index up to date as documents change, with access rights respected.
Tools and Terms You Will Meet
- Chunking
- Splitting documents into passages small enough to retrieve and fit in a prompt.
- Vector database
- A store that finds texts with similar meaning by comparing embeddings.
- Hybrid search
- Combining meaning-based search with classic keyword search.
- Reranker
- A model that reorders retrieved passages by how well they answer the question.
- Grounding
- Making answers depend on given sources rather than the model's memory.
Industry Example: Pharma
An assistant answers operators' questions from the site's standard operating procedures (SOPs). Every answer quotes the SOP number and section, so an operator can open the official document before acting.
A Common Mix-Up
RAG does not teach the model anything permanently. The knowledge stays in your documents and is fetched at every question, which is why updating a document updates the answers immediately.
How Deep Do You Need to Go?
- Know itExplain why retrieval grounds answers and what chunks and embeddings are.
- Use itBuild and evaluate a RAG pipeline with hybrid search, reranking and citations.
- Master itDesign RAG for large, changing document sets with access control and measured quality.