The Skill Explained
What Are Embeddings and Vector Search?
An embedding is a list of numbers, a vector, that represents the meaning of a piece of text, an image or another item. An embedding model places similar meanings close together, so "car" and "automobile" get nearby vectors. Embeddings power semantic search, recommendations and retrieval-augmented generation.
Why AI Employers Ask for Them
Embeddings power semantic search, recommendations, duplicate detection and the retrieval step of RAG assistants. Employers ask for them because building a good AI search or assistant depends on choosing an embedding model, storing vectors efficiently and measuring whether the right items come back.
How It Works
An embedding model, usually a Transformer, reads an input and outputs a vector, often several hundred numbers long. The model is trained so that texts with similar meaning end up near each other. "How do I reset my password" and "I forgot my login" land close together even though they share few words.
To search, you embed every document once and store the vectors. A query is embedded the same way, and the system returns the documents whose vectors are most similar, typically measured by cosine similarity. With millions of vectors, approximate nearest neighbour indexes find close matches fast without comparing every pair.
Vector databases such as Pinecone, Weaviate, Qdrant or Milvus, or PostgreSQL with the pgvector extension, store vectors with metadata so you can filter by date, document type or user permissions.
What You Do With It at Work
- Choose and test an embedding model on your own data.
- Embed and index a document collection in a vector database.
- Combine vector search with filters and keyword search.
- Measure search quality with labelled test queries.
- Re-embed content when the model or documents change.
Tools and Terms You Will Meet
- Vector
- A list of numbers; here, a point in a space where distance means difference in meaning.
- Cosine similarity
- A measure of how closely two vectors point in the same direction.
- Approximate nearest neighbour (ANN)
- Indexes that find close vectors fast, trading a little accuracy for speed.
- pgvector
- An extension that adds vector search to PostgreSQL.
- Semantic search
- Search by meaning rather than exact words.
Industry Example: Pharma
A medical information team embeds thousands of past answers to doctors' questions, so when a new question arrives, the most similar approved answers appear first for the specialist to reuse.
A Common Mix-Up
An embedding is not a summary a person can read. It only becomes useful when compared with other embeddings from the same model; vectors from different models cannot be compared.
How Deep Do You Need to Go?
- Know itExplain that embeddings place similar meanings close together.
- Use itEmbed, index and search documents in a vector database and measure results.
- Master itChoose or fine-tune embedding models and design large-scale vector search.