What Is an LLM? Large Language Models Explained

Summary

A Large Language Model (LLM), at its core, is an incredibly well-trained prediction machine, that can consistently predict the next appropriate word in a sentence. LLMs are trained on massive amounts of data, from books, to internet forums, to articles. GPT-2 for example, was trained on 8 million sources that comprised of 40gb of text. Once given this information, they are trained to recognise patterns: which words follow which, how are sentences structured, what sentences read normally versus abnormally. Over time they gain an exceptional understanding of linguistic structures to the point they can form full sentences that make complete sense given the question/prompt they are asked.

This isn't our content. “Large Language Models explained briefly” was created by 3Blue1Brown and is embedded from YouTube. All credit goes to them.

Key points

• 01An LLM is fundamentally a next-word predictor: it assigns a probability to every possible next word, samples one, and repeats -- and that loop alone generates fluent text.
• 02The 'large' in LLM refers to parameters (weights): hundreds of billions of tunable numbers, learned automatically during training rather than set by hand.
• 03Training feeds the model vast amounts of text and uses backpropagation to adjust every parameter -- GPT-3's training data would take a human thousands of years to read at normal speed.
• 04LLMs run on the transformer architecture, whose 'attention' mechanism lets words pass context to each other so meaning depends on surrounding words, and which processes all words in parallel.
• 05Reinforcement learning from human feedback (RLHF) turns the raw predictor into a helpful assistant -- but because behaviour emerges from billions of tuned parameters, even its builders cannot fully explain any given output.