What is a large language model (LLM)?

How large language models work

At its core, an LLM does one thing: given a stretch of text, it predicts what comes next. Input text is first broken into tokens—words or word fragments—and each token is mapped to a numeric vector called an embedding. The model then processes these vectors and outputs a probability distribution over its entire vocabulary for the next token. By repeating this prediction step and feeding each new token back in, the system produces fluent sentences, paragraphs, and code.

The engine behind this is the transformer, introduced by Google researchers in "Attention Is All You Need" (Vaswani et al., 2017, arXiv:1706.03762). Its key innovation is the self-attention mechanism, which lets the model weigh the relevance of every other token when interpreting a given one—capturing long-range context such as which noun a pronoun refers to. Unlike earlier recurrent networks, transformers process a sequence in parallel, which is what made training on internet-scale data computationally feasible.

What looks like reasoning is, mechanically, repeated next-token prediction over a model with billions of learned weights. Capabilities such as summarising, translating, or writing code emerge because those tasks are latent in the statistical structure of the text the model was trained on. This is also why an LLM has no built-in notion of truth: it generates the most probable continuation, not the verified one.

How LLMs are trained

Training happens in stages. First comes pre-training: the model is exposed to enormous text corpora—web pages, books, code, and reference material—and learns by repeatedly predicting masked or next tokens, adjusting its parameters through gradient descent and backpropagation. GPT-3, for example, was trained on hundreds of billions of tokens drawn largely from a filtered version of Common Crawl plus curated datasets (Brown et al., 2020, arXiv:2005.14165). This self-supervised phase is where the bulk of the model's knowledge is formed.

Scale matters in a measurable way. Kaplan et al. (2020, arXiv:2001.08361) documented "scaling laws": model performance improves predictably as parameters, dataset size, and compute increase together. This finding drove the industry's race toward ever-larger models and is the reason the word "large" sits in the name.

Pre-trained models are then aligned to be useful and safe. Fine-tuning on curated examples, followed by reinforcement learning from human feedback (RLHF), teaches the model to follow instructions, refuse harmful requests, and adopt a helpful tone. This alignment step is what turns a raw next-token predictor into a usable assistant such as ChatGPT or Claude.

What LLMs are used for

Because language is the interface to so much knowledge work, LLMs are broadly applicable. Common uses include drafting and editing text, summarising long documents, answering questions, translating between languages, writing and debugging code, extracting structured data from unstructured text, and powering conversational agents and customer-support bots (IBM; AWS).

A widely used pattern for grounding LLMs in current or proprietary facts is retrieval-augmented generation (RAG), where the model is given relevant documents retrieved from a database before it answers. This reduces hallucination and lets an LLM cite up-to-date sources it never saw during training. Increasingly, LLMs also act as the reasoning core of AI agents that call tools, browse, and execute multi-step tasks.

Adoption has moved quickly from research into the economy. Stanford's HAI AI Index reports sharply rising enterprise use of generative AI and record private investment in the field, with foundation-model releases accelerating year over year (Stanford HAI, AI Index Report).

Key types and examples

LLMs differ by architecture objective and by how they are released. Decoder-only models such as OpenAI's GPT series, Anthropic's Claude, Meta's Llama, and Google's Gemini are optimised for text generation and dominate today's chat assistants. Encoder models such as Google's BERT (Devlin et al., 2018, arXiv:1810.04805) are built to understand and classify text rather than generate it, and remain widely used for search and analysis.

A second axis is openness. Proprietary models like GPT-4 and Gemini are accessed through APIs, while open-weight models such as Meta's Llama family can be downloaded and run or fine-tuned locally—an important distinction for cost, privacy, and control.

Many leading models are now multimodal, extending beyond text to accept images, audio, or video, which blurs the line between an LLM and the broader category of generative AI. GPT-4, Claude, and Gemini all process more than plain text, though their linguistic capability remains rooted in the same language-modelling foundation.

Limitations and risks

The best-known failure mode is hallucination: an LLM can produce confident, fluent statements that are factually wrong, because it is optimising for plausible text rather than truth (NIST AI glossary; IBM). This makes independent verification essential for any high-stakes use, particularly in finance, law, and health.

LLMs also inherit biases present in their training data, can leak or memorise sensitive information, and are vulnerable to prompt-injection and jailbreak attacks. Their knowledge is frozen at the training cutoff unless augmented with retrieval, and they lack a reliable internal model of what they do and do not know. The U.S. NIST AI Risk Management Framework was published specifically to help organisations identify and manage these risks.

Finally, scale carries real costs: training frontier models requires large amounts of compute and energy, and the resulting systems are difficult to fully audit or interpret. These constraints shape how responsibly LLMs can be deployed.

The bigger picture

LLMs are the language-focused branch of a wider shift toward generative AI—systems that create new content rather than merely classify it. They belong to the class of foundation models: large, general-purpose systems trained once and adapted to many downstream tasks, a framing popularised by Stanford researchers.

The trajectory since 2017 has been one of rapid capability gains driven mainly by scale, better data, and alignment techniques, alongside growing attention to safety, evaluation, and governance. Whether the next leap comes from still-larger models, more efficient architectures, or tighter integration with tools and retrieval remains the central open question in the field.

Frequently asked questions

Is ChatGPT an LLM?

ChatGPT is an application, and the large language model is the engine that powers it. Behind the chat interface run OpenAI's GPT models (such as GPT-4), which are LLMs fine-tuned with human feedback to follow instructions and hold conversations.

How many parameters does an LLM have?

It varies enormously. OpenAI's GPT-3 has 175 billion parameters (Brown et al., 2020, arXiv:2005.14165), while smaller open models run in the single-digit billions. Parameter counts for the largest frontier models are often not disclosed, but the trend since 2020 has been toward larger models before efficiency techniques pushed back.

What is the difference between an LLM and generative AI?

Generative AI is the broad category of systems that create new content—text, images, audio, video, or code. An LLM is the subset of generative AI specialised in language. Every LLM is generative AI, but generative AI also includes image models like diffusion systems that are not language models.

What data are LLMs trained on?

Mostly large collections of public text—web pages, books, code repositories, and reference works. GPT-3, for instance, was trained largely on a filtered version of the Common Crawl web dataset plus curated corpora (Brown et al., 2020). Exact training mixes for the newest proprietary models are generally not published.

Why do LLMs make things up?

Because an LLM generates the statistically most likely continuation of text, not verified fact. When it lacks the right information, it still produces fluent, confident output—a phenomenon called hallucination (NIST; IBM). Techniques like retrieval-augmented generation and human review reduce, but do not eliminate, the problem.

What is the transformer in an LLM?

The transformer is the neural-network architecture nearly all modern LLMs are built on, introduced by Google in 2017 (Vaswani et al., arXiv:1706.03762). Its self-attention mechanism lets the model weigh how relevant every word is to every other word, which is what enables it to handle long-range context efficiently.

Related

Explainers: GenAI

Glossary: Transformer (architecture), Token / tokenization, Fine-tuning, RAG (retrieval-augmented generation), Hallucination, Foundation model, Parameters (model), Attention / self-attention

Sources

• arXiv — Attention Is All You Need (Vaswani et al., 2017)
• arXiv — Language Models are Few-Shot Learners / GPT-3 (Brown et al., 2020)
• arXiv — Scaling Laws for Neural Language Models (Kaplan et al., 2020)
• IBM — What is a large language model?
• NIST — Trustworthy & Responsible AI Glossary