To search by meaning rather than by keywords, documents are first split into chunks and each chunk is turned into an embedding: a point in a high-dimensional space where texts with similar meanings lie close together. These vectors are stored in an index. When a query arrives, it is turned into a vector with the same model and its nearest neighbors are retrieved, for instance by cosine similarity (Alammar & Grootendorst, 2024, pp. 228–230).
With thousands of vectors, computing every distance is enough. With millions, one uses approximate nearest-neighbor libraries such as Annoy or FAISS, which return results in milliseconds at the price of not guaranteeing the exact nearest neighbor. Vector databases proper (Weaviate, Pinecone and others) add what one expects from a database: adding and deleting vectors without rebuilding the index, and filtering by other criteria (Alammar & Grootendorst, 2024, pp. 238–239). The original RAG system used a FAISS index over 21 million Wikipedia passages (P. Lewis et al., 2020, sec. 3). Today they are the standard infrastructure for retrieval augmentation and semantic search, with models such as Sentence-BERT producing the vectors.
For a logician, the contrast with a classical database is instructive. An SQL query is, at bottom, a first-order formula: it returns exactly the rows that satisfy it, and the answer is true or false for each one. A vector query has no truth conditions: it returns what is most similar, it always returns something, and "similar" is defined by the geometry a model has learned. It is a theory of meaning as position and distance (distributional hypothesis), with the virtues and risks of similarity: two texts can be close because they say the same thing or because they say opposite things about the same topic.