Roy Schwartz (HUJI)- On the Secret Language of Large Language Models

Abstract: Despite what their name suggests, large language models (LLMs) do not process natural language directly, but rather operate on word vectors that represent textual units. In this talk, I will present surprising findings about this vector space, which I call the “inner language” of LLMs. I will demonstrate that LLMs can “understand” vectors representing words outside their vocabulary—words they have never encountered during training—suggesting that the learned vector space captures linguistic structures that generalize beyond specific training tokens. I will then show that this inner language exhibits systematic structural regularity, where complex words can be decomposed into meaningful components (e.g., “dogs” can be expressed as v_dog + v_plural). These findings reveal that LLMs implicitly learn algebraic operations mirroring grammatical and semantic relationships, opening pathways toward dramatically more efficient vocabulary utilization. This approach has major implications for low-resource languages, which remain severely underrepresented in LLM vocabularies, by enabling the construction of representations for novel terms through principled vector operations without costly model retraining. This is joint work with Guy Kaplan, Yuval Reif, and Matanel Oren.