An embedding maps words, sentences or documents into a continuous vector space where distance reflects meaning. Mikolov et al. (2013) showed that such vectors, learned from raw text, capture surprisingly rich relationships; Pennington et al. (2014) derived them from global word co-occurrence statistics in the GloVe method.
Modern LLMs build contextual embeddings — the same word gets different vectors depending on its surroundings. Embeddings are the bridge between human language and the numerical operations a model performs.
Why it matters
Embeddings power semantic search, retrieval-augmented generation and clustering. They are how a knowledge base finds the passage that actually answers a question, even when no keywords match.