Overview
In an embedding space, the 'distance' and 'direction' between vectors represent relationships. For example, the vector for 'Paris' minus 'France' plus 'Italy' might land very close to the vector for 'Rome'.
Dimensionality
Modern LLMs often use embedding spaces with thousands of dimensions (e.g., 1536 or 4096), allowing them to capture incredibly subtle nuances of meaning.
Visualization
Because humans can't see in 1000D, techniques like t-SNE or UMAP are used to project these spaces down to 2D or 3D for analysis.