Keyword search returns documents that contain the query's terms; semantic search tries to return those that mean something relevant, even when they share no words. It was one of the first industrial applications of language models: months after BERT was published, Google announced it was using it in its search engine (Alammar & Grootendorst, 2024, p. 225).
Tiny example. Query: “how do I fix a flat?”. Document: “A guide to repairing a bicycle inner tube.” They share not a single word, but their embeddings lie close together, and the document shows up among the top results.
The core technique is dense retrieval: split the archive into chunks, compute an embedding for each, store them in an index or vector database, and, given a query, look for its nearest neighbors, typically by cosine similarity (Alammar & Grootendorst, 2024, pp. 226–229). It is often followed by a reranker, a model that reads query and document together and reorders the top candidates; and it is the first half of retrieval-augmented systems (Retrieval-augmented generation (RAG)), in which what is retrieved is handed to a generator (Alammar & Grootendorst, 2024, pp. 226–227). Sentence-BERT made this scheme practical by producing, for each sentence, a directly comparable vector (Reimers & Gurevych, 2019).
Limits that a theory of meaning helps to see. Closeness is not relevance: a question and its answer are not always semantically similar, which is why retrieval models are trained on question–answer pairs (Alammar & Grootendorst, 2024, p. 229). There is always a nearest neighbor, even when no document answers: asking an article about a film for the mass of the Moon still returns three sentences (Alammar & Grootendorst, 2024, p. 234). Exact-phrase searches work better with keywords, hence hybrid systems, and performance drops outside the training domain, for example on legal texts (Alammar & Grootendorst, 2024, p. 235). And there is no truth check here: semantic search finds similar texts, not correct ones.