The AI Glossary

Browse by category

Core concepts 22

# Artificial intelligence (AI)

Artificial intelligence is the field of computer science that builds systems able to perform tasks normally requiring human intelligence, such as reasoning, perception, language understanding, and decision-making. It spans rule-based systems and modern data-driven approaches like machine learning, and underpins applications from spam filters to self-driving cars and large language models.

Source: IBM · NIST AIRC · Related: Machine learning, Deep learning, Neural network, Large language model (LLM)

# Attention / self-attention

Attention is a mechanism that lets a model weigh the relevance of different parts of an input when producing each output element, focusing on the most informative tokens. In self-attention, every token in a sequence computes weighted relationships to all other tokens using query, key, and value vectors, which is the core operation of the transformer architecture.

Source: arXiv (Vaswani et al.) · IBM · Related: Transformer (architecture), Large language model (LLM), Embedding (vector), Token / tokenization

# Backpropagation

Backpropagation is the core training algorithm for neural networks that computes how much each weight contributed to the model's error by applying the chain rule of calculus backward through the layers. These gradients are then used by an optimizer such as gradient descent to update the weights, iteratively reducing the loss over many training passes.

Source: IBM · Google ML Glossary · Related: Gradient descent, Neural network, Model weights, Deep learning

# Context window

The context window is the maximum amount of text, measured in tokens, that a language model can consider at once, spanning both the input prompt and the generated output. It functions as the model's working memory: content beyond the limit is ignored or truncated. Windows range from a few thousand tokens in early models to over one million in recent ones.

Source: IBM · Google ML Glossary · Related: Token / tokenization, Large language model (LLM), Prompt, Inference

# Deep learning

Deep learning is a subset of machine learning that uses neural networks with many stacked layers to learn hierarchical representations directly from raw data such as images, audio, or text. The word deep refers to the number of hidden layers, often dozens or hundreds, which let the model capture increasingly abstract features and drive breakthroughs in vision and language.

Read the full explainer →

Source: IBM · Google ML Glossary · Related: Neural network, Machine learning, Transformer (architecture), Backpropagation

# Embedding (vector)

An embedding is a dense vector of numbers that represents a piece of data, such as a word, sentence, or image, in a continuous space where semantic similarity corresponds to geometric closeness. By mapping tokens to vectors of hundreds or thousands of dimensions, embeddings let models measure meaning with distance metrics like cosine similarity, powering search, clustering, and retrieval.

Source: IBM · Google ML Glossary · Related: Vector database, Token / tokenization, Transformer (architecture), RAG (retrieval-augmented generation)

# Generative AI

Generative AI is a class of artificial intelligence that creates new content, such as text, images, audio, code, or video, by learning the statistical structure of training data and sampling from it. Unlike discriminative models that classify inputs, generative models like large language models and diffusion models produce original outputs in response to a prompt.

Read the full explainer →

Source: IBM · NIST AIRC · Related: Large language model (LLM), Prompt, Transformer (architecture), Inference

# Gradient descent

Gradient descent is an optimization algorithm that minimizes a model's loss function by iteratively adjusting parameters in the direction opposite to the gradient, the direction of steepest increase. The step size is controlled by a learning rate, and variants such as stochastic and mini-batch gradient descent make training large neural networks computationally feasible.

Source: IBM · Google ML Glossary · Related: Backpropagation, Model weights, Parameters (model), Neural network

# Inference

Inference is the process of running a trained model on new input to produce a prediction or generated output, as opposed to training, which sets the model's parameters. For a large language model, inference means taking a prompt and generating tokens one at a time; it is the phase that consumes compute every time the model is used in production.

Source: IBM · Google ML Glossary · Related: Large language model (LLM), Parameters (model), Prompt, Context window

# Large language model (LLM)

A large language model is a deep learning model trained on massive text corpora to predict the next token in a sequence, enabling it to generate and understand human language. Built on the transformer architecture, modern LLMs such as GPT-4, Claude, and Llama contain billions to trillions of parameters and learn statistical patterns of language from trillions of tokens of training data.

Read the full explainer →

Source: IBM · AWS · Related: Transformer (architecture), Token / tokenization, Parameters (model), Generative AI

# Machine learning

Machine learning is a branch of artificial intelligence in which algorithms learn patterns from data rather than following explicitly programmed rules, improving their performance on a task as they process more examples. Its three main paradigms are supervised, unsupervised, and reinforcement learning, powering systems from fraud detection to recommendation engines and modern generative models.

Source: IBM · Google ML Glossary · Related: Supervised learning, Unsupervised learning, Reinforcement learning, Deep learning

# Model weights

Model weights are the numerical values assigned to connections in a neural network that control how strongly each input influences the next layer. They are the largest group of a model's parameters, adjusted through backpropagation and gradient descent during training, and together they encode everything the model has learned; releasing them is what makes a model open-weight.

Source: IBM · Google ML Glossary · Related: Parameters (model), Neural network, Backpropagation, Gradient descent

# Natural language processing (NLP)

Natural language processing is the field of artificial intelligence concerned with enabling computers to understand, interpret, and generate human language. It combines linguistics with machine learning to power tasks such as translation, sentiment analysis, named-entity recognition, and question answering, and modern NLP is dominated by transformer-based large language models.

Source: IBM · Google ML Glossary · Related: Large language model (LLM), Transformer (architecture), Token / tokenization, Embedding (vector)

# Neural network

A neural network is a machine learning model composed of interconnected layers of nodes, or artificial neurons, loosely inspired by the human brain. Each connection carries a weight that is adjusted during training, allowing the network to transform inputs into outputs through weighted sums and nonlinear activation functions. Networks with many hidden layers form the basis of deep learning.

Read the full explainer →

Source: IBM · AWS · Related: Deep learning, Model weights, Backpropagation, Machine learning

# Parameters (model)

Parameters are the internal variables of a model, chiefly the weights and biases, that are learned from data during training and determine how inputs are transformed into outputs. The parameter count measures model scale: GPT-3 has 175 billion parameters, and larger counts generally allow a model to capture more complex patterns at greater computational cost.

Source: IBM · Google ML Glossary · Related: Model weights, Large language model (LLM), Neural network, Gradient descent

# Prompt

A prompt is the input text or instruction given to a generative AI model to elicit a desired response. It can include a question, examples, context, or formatting rules, and its wording strongly shapes the output. Crafting effective prompts, known as prompt engineering, is a practical skill for steering large language models without changing their underlying weights.

Source: IBM · Google ML Glossary · Related: Large language model (LLM), Context window, Inference, Generative AI

# Reinforcement learning

Reinforcement learning is a machine learning paradigm in which an agent learns to make decisions by taking actions in an environment and receiving rewards or penalties, optimizing its behavior to maximize cumulative reward over time. It powers game-playing systems like AlphaGo and, through reinforcement learning from human feedback, is used to align large language models with human preferences.

Source: IBM · AWS · Related: Supervised learning, Unsupervised learning, Machine learning, Large language model (LLM)

# Supervised learning

Supervised learning is a machine learning paradigm in which a model learns from labeled data, where each training example pairs an input with the correct output. The algorithm minimizes the error between its predictions and the known labels, making it well suited to classification and regression tasks such as spam detection, image labeling, and price prediction.

Source: IBM · Google ML Glossary · Related: Unsupervised learning, Reinforcement learning, Machine learning, Gradient descent

# Token / tokenization

A token is the basic unit of text a language model processes, often a word, subword, or character, and tokenization is the process of splitting raw text into these units. Modern LLMs use subword schemes like byte-pair encoding, where roughly one token equals about four English characters, so a 1,000-word document is around 1,300 tokens.

Read the full explainer →

Source: IBM · Google ML Glossary · Related: Large language model (LLM), Embedding (vector), Context window, Transformer (architecture)

# Transformer (architecture)

The transformer is a neural network architecture introduced in the 2017 paper Attention Is All You Need, which processes entire sequences in parallel using self-attention instead of recurrence. By letting each token attend to every other token, it captures long-range dependencies efficiently and became the foundation for large language models like GPT, BERT, and Claude.

Read the full explainer →

Source: arXiv (Vaswani et al.) · IBM · Related: Attention / self-attention, Large language model (LLM), Embedding (vector), Deep learning

# Unsupervised learning

Unsupervised learning is a machine learning paradigm in which a model finds structure in unlabeled data without being told the correct answers. Techniques such as clustering and dimensionality reduction group similar examples or compress features, and the approach underlies pretraining of large language models, which learn from raw text with no explicit labels.

Source: IBM · Google ML Glossary · Related: Supervised learning, Reinforcement learning, Machine learning, Embedding (vector)

# Vector database

A vector database is a database designed to store and query high-dimensional embedding vectors, retrieving items by similarity rather than exact match. Using approximate nearest-neighbor indexes, it finds the vectors closest to a query in milliseconds across millions of records, making it a key component of semantic search and retrieval-augmented generation for large language models.

Source: IBM · AWS · Related: Embedding (vector), RAG (retrieval-augmented generation), Large language model (LLM), Natural language processing (NLP)

↑ Top

Training & optimization 18

# Distillation (knowledge distillation)

Knowledge distillation is a compression technique in which a smaller student model is trained to reproduce the behavior of a larger, more capable teacher model, learning from the teacher's output probabilities rather than only hard labels. Formalized by Hinton, Vinyals, and Dean in 2015, it transfers much of the teacher's accuracy into a model that is cheaper and faster to run. It is widely used to produce lightweight versions of large language models.

Source: Distillation paper (arXiv) · IBM · Related: Quantization, Pretraining, Fine-tuning, Loss function

# Epoch (training)

An epoch is one complete pass of the training algorithm over the entire training dataset, during which the model has seen every example once. Training usually runs for multiple epochs so the model can gradually refine its weights, with each epoch composed of many smaller batch updates. Too many epochs risk overfitting, while too few can leave the model underfit.

Source: Google ML Glossary · IBM · Related: Learning rate, Loss function, Overfitting, Training data / dataset

# Fine-tuning

Fine-tuning is the process of taking a pretrained model and continuing training on a smaller, task-specific or domain-specific dataset so its weights adapt to a narrower objective. It reuses the general knowledge captured during pretraining, requiring far less data and compute than training from scratch. Common variants include supervised fine-tuning on labeled examples and parameter-efficient methods such as LoRA.

Read the full explainer →

Source: IBM · Google ML Glossary · Related: Pretraining, PEFT (parameter-efficient fine-tuning), LoRA (low-rank adaptation), RLHF (reinforcement learning from human feedback)

# Learning rate

The learning rate is a hyperparameter that controls how large a step the optimizer takes when updating a model's weights on each training iteration. Too high a value can cause training to diverge or overshoot a good solution, while too low a value makes training slow and prone to getting stuck. It is often varied over training through a schedule such as warmup followed by decay.

Source: Google ML Glossary · IBM · Related: Loss function, Epoch (training), Overfitting, Fine-tuning

# LoRA (low-rank adaptation)

LoRA, low-rank adaptation, is a parameter-efficient fine-tuning method that freezes a pretrained model's weights and injects small trainable low-rank matrices into its layers, so only those matrices are updated. Introduced by Hu et al. in 2021, it can cut trainable parameters by orders of magnitude while matching full fine-tuning quality, and the compact adapters can be swapped or merged. It has become the default approach for cheaply adapting large language models.

Source: LoRA paper (arXiv) · Hugging Face PEFT · Related: PEFT (parameter-efficient fine-tuning), Fine-tuning, Quantization, Pretraining

# Loss function

A loss function is a mathematical function that quantifies how far a model's predictions are from the correct answers, producing a single number the training process seeks to minimize. Its gradient tells the optimizer how to adjust weights to reduce error, making it the objective that steers learning. Common examples are cross-entropy for classification and mean squared error for regression.

Source: Google ML Glossary · IBM · Related: Learning rate, Epoch (training), Overfitting, Underfitting

# Mixture of experts (MoE)

Mixture of experts (MoE) is a neural network architecture that divides the model into many specialized sub-networks called experts, with a learned gating network routing each input to only a small subset of them. This sparse activation lets a model hold a very large total parameter count while keeping per-token compute low, improving efficiency and scale. Many frontier large language models use MoE layers.

Source: Sparsely-Gated MoE paper (arXiv) · Hugging Face · Related: Reasoning model / chain-of-thought, Pretraining, Quantization, Distillation (knowledge distillation)

# Overfitting

Overfitting is a failure mode in which a model learns the training data too closely, capturing noise and idiosyncrasies rather than general patterns, so it performs well on training examples but poorly on unseen data. It signals excess model capacity or too little data, and is countered with regularization, more data, early stopping, or simpler models. It is diagnosed by a widening gap between training and validation performance.

Source: Google ML Glossary · AWS · Related: Underfitting, Training data / dataset, Loss function, Epoch (training)

# PEFT (parameter-efficient fine-tuning)

PEFT, parameter-efficient fine-tuning, is a family of methods that adapt a large pretrained model by training only a small number of extra or selected parameters while keeping most weights frozen. This dramatically lowers the compute and storage cost of specialization while yielding performance comparable to full fine-tuning. Common PEFT techniques include LoRA, prefix tuning, and adapter layers.

Source: Hugging Face PEFT · IBM · Related: LoRA (low-rank adaptation), Fine-tuning, Quantization, Transfer learning

# Pretraining

Pretraining is the initial, compute-intensive phase in which a model learns general patterns from a large, unlabeled corpus before any task-specific adaptation. For large language models this is typically self-supervised next-token prediction over trillions of tokens of text, producing a base model with broad linguistic and world knowledge. The resulting foundation model is later specialized through fine-tuning.

Source: Google ML Glossary · IBM · Related: Fine-tuning, Training data / dataset, Transfer learning, Reasoning model / chain-of-thought

# Quantization

Quantization is a model-compression technique that reduces the numerical precision of a model's weights and activations, for example from 32-bit or 16-bit floating point to 8-bit or 4-bit integers. This shrinks memory footprint and speeds up inference, letting large models run on constrained hardware, usually with only minor accuracy loss. It can be applied after training (post-training quantization) or during it (quantization-aware training).

Source: Hugging Face · IBM · Related: Distillation (knowledge distillation), LoRA (low-rank adaptation), PEFT (parameter-efficient fine-tuning), Fine-tuning

# RAG (retrieval-augmented generation)

RAG, retrieval-augmented generation, is a technique that connects a language model to an external knowledge source so it retrieves relevant documents at inference time and conditions its generation on them. Introduced by Lewis et al. in 2020, it grounds outputs in up-to-date, verifiable data without retraining the model, reducing hallucination. A typical pipeline embeds a query, searches a vector database, and injects the retrieved passages into the prompt.

Read the full explainer →

Source: RAG paper (arXiv) · AWS · Related: Fine-tuning, Pretraining, Training data / dataset, Reasoning model / chain-of-thought

# Reasoning model / chain-of-thought

A reasoning model is a language model designed to work through problems in explicit intermediate steps before answering, using chain-of-thought, a prompting and training technique in which the model generates a sequence of reasoning steps. Introduced by Wei et al. in 2022, chain-of-thought markedly improves performance on arithmetic, logic, and multi-step tasks. Modern reasoning models are trained to produce and refine these step-by-step traces automatically.

Source: Chain-of-thought paper (arXiv) · IBM · Related: RLHF (reinforcement learning from human feedback), Fine-tuning, Mixture of experts (MoE), RAG (retrieval-augmented generation)

# RLHF (reinforcement learning from human feedback)

RLHF, reinforcement learning from human feedback, is a training method that aligns a language model with human preferences by using human comparisons of model outputs to train a reward model, which then guides fine-tuning via reinforcement learning. Introduced at scale in OpenAI's 2022 InstructGPT work, it made a 1.3B-parameter model's outputs preferred over the 175B GPT-3. RLHF underpins the helpfulness and safety of modern chat assistants.

Source: InstructGPT paper (arXiv) · Hugging Face · Related: Fine-tuning, Reasoning model / chain-of-thought, Loss function, Synthetic data

# Synthetic data

Synthetic data is artificially generated data that mimics the statistical properties of real data, produced by algorithms, simulations, or generative models rather than collected from real-world events. It is used to augment scarce datasets, protect privacy, and cover rare or hard-to-sample cases, and increasingly to train or fine-tune language models. Its main risk is amplifying biases or drifting from real-world distributions if poorly controlled.

Source: IBM · AWS · Related: Training data / dataset, Fine-tuning, RLHF (reinforcement learning from human feedback), Pretraining

# Training data / dataset

Training data is the collection of examples a model learns from during training, from which it derives the statistical patterns encoded in its parameters. For large language models this is typically a vast corpus of text, while supervised tasks use labeled input-output pairs. The quality, diversity, and representativeness of this dataset directly shape the model's capabilities and biases.

Source: Google ML Glossary · IBM · Related: Pretraining, Synthetic data, Overfitting, Fine-tuning

# Transfer learning

Transfer learning is a machine-learning approach in which knowledge gained by a model on one task is reused as the starting point for a related task, rather than training from scratch. It exploits general representations learned on large datasets to reach strong performance on new problems with limited data. The pretrain-then-fine-tune paradigm behind modern language models is a canonical example.

Source: IBM · Google ML Glossary · Related: Pretraining, Fine-tuning, PEFT (parameter-efficient fine-tuning), Training data / dataset

# Underfitting

Underfitting is a failure mode in which a model is too simple or undertrained to capture the underlying structure of the data, producing poor accuracy on both training and unseen examples. It typically stems from insufficient model capacity, over-aggressive regularization, or too few training steps. The remedy is a more expressive model, richer features, or longer training.

Source: Google ML Glossary · IBM · Related: Overfitting, Training data / dataset, Loss function, Epoch (training)

↑ Top

Generative AI & agents 28

# Agent orchestration

Agent orchestration is the coordination of one or more AI agents, their tools, memory, and sub-tasks so they work together toward a larger goal. An orchestration layer routes work, manages state and sequencing, handles errors, and can delegate steps to specialized agents. It is what turns individual model calls into a reliable, multi-step automated workflow.

Source: IBM · Related: Multi-agent system, AI agent, Agentic AI, Model Context Protocol (MCP)

# Agentic AI

Agentic AI describes systems that exhibit autonomy, planning, and goal-directed behavior, deciding sequences of actions and using tools to achieve objectives with limited human oversight. It contrasts with passive generative AI that only responds to a single prompt. The term captures the shift from chat assistants toward software that can execute multi-step workflows on a user's behalf.

Source: IBM · Related: AI agent, Autonomous agent, Agent orchestration, Multi-agent system

# AI agent

An AI agent is a system that uses a language model to pursue a goal by reasoning about steps, calling external tools, and acting on the results across multiple turns. Rather than returning a single answer, it plans, invokes functions or APIs, observes outcomes, and iterates until the task is done. Booking a trip end to end by querying flights and hotels is a typical example.

Read the full explainer →

Source: IBM · Related: Agentic AI, Tool use / tool calling, ReAct (reason + act), Autonomous agent

# Autonomous agent

An autonomous agent is an AI system that pursues a goal over many steps with minimal human intervention, setting sub-tasks, choosing actions, using tools, and adapting to results on its own. It operates in a continuous loop of planning and execution until the objective is met or a stopping condition is hit. Early examples include AutoGPT-style systems that break a high-level goal into self-directed steps.

Source: IBM · Related: AI agent, Agentic AI, ReAct (reason + act), Multi-agent system

# Computer vision

Computer vision is a field of artificial intelligence that enables machines to interpret and act on visual information from images and video. Tasks include object detection, image classification, segmentation, and facial recognition, typically using convolutional or transformer-based neural networks. It powers applications from medical imaging to autonomous vehicles and quality inspection.

Source: IBM · AWS · Related: Multimodal AI, Text-to-image, Speech recognition (ASR)

# Copilot / AI assistant

A copilot, or AI assistant, is a generative AI tool embedded in an application that helps a human complete tasks by suggesting, drafting, or automating steps while keeping the person in control. Unlike a fully autonomous agent, it augments the user, offering code completions, email drafts, or data summaries on request. GitHub Copilot and Microsoft Copilot are well-known examples.

Source: IBM · IBM · Related: AI agent, Agentic AI, Text-to-speech (TTS), Prompt engineering

# Diffusion model

A diffusion model is a generative model that learns to create data, such as images, by reversing a gradual noising process. During training it adds Gaussian noise to samples in many steps, then learns to denoise; at generation time it starts from pure noise and iteratively removes it to produce a coherent output. Diffusion powers systems like Stable Diffusion and many text-to-image tools.

Read the full explainer →

Source: arXiv (Ho et al.) · IBM · Related: Text-to-image, GAN (generative adversarial network), Multimodal AI

# Function calling

Function calling is a specific form of tool use in which a language model outputs a structured, machine-readable request, usually JSON, naming a predefined function and its arguments so that application code can execute it. It bridges natural-language intent and deterministic software, for example turning "what's the weather in Stockholm?" into a getWeather call with a city parameter. The result is then fed back to the model.

Source: IBM · Related: Tool use / tool calling, AI agent, Model Context Protocol (MCP)

# GAN (generative adversarial network)

A generative adversarial network is a generative architecture in which two neural networks train in competition: a generator creates synthetic samples while a discriminator tries to distinguish them from real data. Their adversarial contest pushes the generator to produce increasingly realistic output, such as photorealistic faces. GANs were widely used for image synthesis before diffusion models became dominant.

Source: Google · IBM · Related: Diffusion model, Text-to-image, Multimodal AI

# Generative pre-trained transformer (GPT)

A generative pre-trained transformer (GPT) is a class of large language model built on the transformer architecture and trained to predict the next token on vast amounts of text. This pretraining lets it generate coherent language, answer questions, and complete tasks, after which it is often fine-tuned or instruction-tuned. OpenAI's GPT series popularized the approach and the name.

Source: IBM · Related: In-context learning, Temperature, Top-p / top-k sampling, Multimodal AI

# Guardrails

Guardrails are the safety controls, policies, and filters placed around an AI system to keep its behavior within acceptable, safe, and compliant bounds. They can block toxic or off-topic output, enforce format and privacy rules, restrict tool actions, and detect prompt injection. Guardrails operate on inputs, outputs, or both, and are essential for deploying generative AI responsibly in production.

Source: IBM · Related: Hallucination, System prompt, AI agent, Prompt engineering

# Hallucination

A hallucination is an output from a generative AI model that is fluent and confident but factually wrong, fabricated, or unsupported by its training data or provided context. Because large language models predict statistically likely tokens rather than retrieve verified facts, they can invent citations, figures, or events. Retrieval grounding and human review are the standard mitigations.

Read the full explainer →

Source: IBM · Related: Retrieval (semantic search), Guardrails, Prompt engineering

# In-context learning

In-context learning is the ability of a large language model to adapt to a new task from examples or instructions placed directly in its prompt, without any update to its underlying weights. The model infers the pattern from the provided context and applies it to the current input during a single inference pass. It is the mechanism that makes zero-shot and few-shot prompting possible.

Source: IBM · Related: Zero-shot / few-shot learning, Prompt engineering, System prompt

# Model Context Protocol (MCP)

The Model Context Protocol (MCP) is an open standard, introduced by Anthropic, that defines a common way for AI applications to connect language models to external data sources and tools. It works like a universal adapter: an MCP server exposes resources and tools that any MCP-compatible client can use, replacing bespoke per-integration code. This standardization makes agent tool ecosystems interoperable.

Read the full explainer →

Source: Model Context Protocol · Anthropic · Related: Tool use / tool calling, Function calling, AI agent, Agent orchestration

# Multi-agent system

A multi-agent system is an architecture in which several AI agents, often with specialized roles, collaborate, delegate, or debate to solve a problem that is hard for a single agent. A coordinator may split a task among a researcher, a coder, and a reviewer agent, then combine their outputs. Dividing work this way can improve accuracy and handle broader, more complex objectives.

Source: IBM · Related: Agent orchestration, AI agent, Agentic AI, Autonomous agent

# Multimodal AI

Multimodal AI is artificial intelligence that can process and combine multiple types of data, such as text, images, audio, and video, within a single model. By aligning these modalities in a shared representation, it can answer questions about a photo, generate a caption, or transcribe and summarize a clip. Modern models like GPT-4o and Gemini are multimodal by design.

Source: IBM · Related: Text-to-image, Text-to-video, Computer vision, Speech recognition (ASR)

# Prompt engineering

Prompt engineering is the practice of designing and refining the text instructions given to a generative AI model to steer its output toward accurate, relevant, and well-formatted results. Techniques include giving clear task descriptions, worked examples, role framing, and step-by-step reasoning cues. For example, adding two solved examples before a question turns a zero-shot request into more reliable few-shot prompting.

Read the full explainer →

Source: IBM · Prompt Engineering Guide · Related: System prompt, Zero-shot / few-shot learning, In-context learning, Temperature

# ReAct (reason + act)

ReAct is an agent prompting framework that interleaves reasoning and acting, having a language model alternate between generating a thought, taking an action such as a tool call, and observing the result before continuing. This reason-act-observe loop lets the model plan, gather external information, and correct course, reducing hallucination on multi-step tasks. It was introduced in a 2022 research paper by Yao et al.

Source: arXiv (Yao et al.) · Prompt Engineering Guide · Related: AI agent, Tool use / tool calling, Agentic AI, Autonomous agent

# Retrieval (semantic search)

Retrieval, in its semantic-search form, is the process of finding relevant documents by meaning rather than exact keywords, using vector embeddings that place similar text close together in a high-dimensional space. A query is embedded and matched against stored vectors to return the most semantically related passages. It is the retrieval half of retrieval-augmented generation, supplying grounding context to a language model.

Source: IBM · IBM · Related: Hallucination, Guardrails, AI agent, Model Context Protocol (MCP)

# Speech recognition (ASR)

Speech recognition, or automatic speech recognition (ASR), is the technology that converts spoken audio into written text. It uses acoustic and language models, increasingly deep neural networks, to map sound waves to words and handle accents, background noise, and natural speech. ASR underpins voice assistants, dictation, and meeting transcription.

Source: IBM · Google Cloud · Related: Text-to-speech (TTS), Multimodal AI, Computer vision

# System prompt

A system prompt is a set of instructions given to a language model before the user's messages that establishes its role, tone, rules, and constraints for a conversation. It shapes behavior globally, such as telling the model to act as a formal financial analyst or to refuse certain requests. Unlike a single user prompt, it persists across the exchange and governs every response.

Source: IBM · Anthropic · Related: Prompt engineering, Guardrails, In-context learning

# Temperature

Temperature is a sampling parameter that controls the randomness of a language model's output by scaling the probability distribution over the next token. A low temperature near 0 makes the model deterministic and focused, favoring the highest-probability words, while a high value above 1 increases diversity and creativity at the cost of coherence. It is one of the primary knobs for tuning generation.

Source: IBM · Prompt Engineering Guide · Related: Top-p / top-k sampling, Prompt engineering, Hallucination

# Text-to-image

Text-to-image is a generative AI capability that produces original images from natural-language descriptions. A user supplies a prompt such as "a red sports car on a rainy street at night" and the model synthesizes a matching picture, typically using a diffusion process. Systems like DALL-E, Midjourney, and Stable Diffusion are leading examples.

Source: IBM · Related: Diffusion model, Text-to-video, Multimodal AI, GAN (generative adversarial network)

# Text-to-speech (TTS)

Text-to-speech is the technology that converts written text into spoken audio, synthesizing a natural-sounding human voice. Modern neural TTS systems model prosody, intonation, and timbre to produce speech that is close to indistinguishable from a real speaker. It powers screen readers, voice assistants, and audiobook narration.

Source: IBM · Google Cloud · Related: Speech recognition (ASR), Multimodal AI, Copilot / AI assistant

# Text-to-video

Text-to-video is a generative AI capability that creates short video clips from natural-language prompts. The model must generate not only realistic frames but also temporal consistency and motion across them, a harder problem than still-image generation. Systems such as OpenAI's Sora and Google's Veo demonstrate the technique.

Source: IBM · Related: Text-to-image, Diffusion model, Multimodal AI

# Tool use / tool calling

Tool use, or tool calling, is the capability that lets a language model invoke external functions, APIs, or services to gather information or take action beyond generating text. The model is given tool definitions, decides when one is needed, emits a structured call with arguments, and incorporates the returned result into its response. This lets a model fetch live data, run code, or query a database.

Source: Anthropic · Related: Function calling, AI agent, Model Context Protocol (MCP), ReAct (reason + act)

# Top-p / top-k sampling

Top-p and top-k are token-sampling strategies that restrict which candidate words a language model can choose from at each step. Top-k keeps only the k most probable tokens, while top-p (nucleus) sampling keeps the smallest set of tokens whose cumulative probability exceeds a threshold p, such as 0.9. Both trade off diversity against coherence and are often tuned alongside temperature.

Source: Prompt Engineering Guide · Google · Related: Temperature, Prompt engineering, Generative pre-trained transformer (GPT)

# Zero-shot / few-shot learning

Zero-shot learning is when a model performs a task from instructions alone, with no worked examples, while few-shot learning supplies a handful of input-output examples in the prompt to demonstrate the desired behavior. Large language models exhibit both without weight updates, relying on patterns learned during pretraining. For instance, showing three labeled reviews before asking for a fourth sentiment label is few-shot prompting.

Source: Prompt Engineering Guide · Prompt Engineering Guide · Related: In-context learning, Prompt engineering, System prompt

↑ Top

Safety, governance & business 32

# AEO (answer engine optimization)

Answer engine optimization (AEO) is the practice of optimizing content to be selected as the direct answer by AI-driven answer engines like ChatGPT, Perplexity, and Google's AI features, rather than to rank as a blue link. It emphasizes clear, structured, factual, well-sourced answers to specific questions. AEO overlaps heavily with generative engine optimization (GEO) and is a response to zero-click search.

Source: Google Search Central — AI features and your site · GEO: Generative Engine Optimization (arXiv paper) · Related: GEO (generative engine optimization), AI Overviews / AI search, Model API

# AI alignment

AI alignment is the research field focused on ensuring an AI system's goals, behavior, and outputs match the intentions and values of its human designers and users. It addresses the risk that a capable system optimizes a specified objective in ways that diverge from what was actually intended. Techniques include reinforcement learning from human feedback (RLHF), introduced by OpenAI in 2017.

Source: Stanford HAI · NIST AI Risk Management Framework · Related: AI safety, Constitutional AI, Responsible AI, Red-teaming

# AI bias / algorithmic bias

Algorithmic bias is systematic and unfair discrimination in an AI system's outputs, often arising from skewed training data, flawed design choices, or unrepresentative deployment contexts. It can produce disparate outcomes across groups defined by race, gender, age, or other attributes. NIST's Special Publication 1270 (March 2022) categorizes bias into systemic, statistical, and human sources.

Source: NIST SP 1270 — Towards a Standard for Managing Bias in AI · IBM Think — AI Bias · Related: Responsible AI, Explainable AI (XAI), AI governance, Data privacy (in AI)

# AI governance

AI governance is the set of policies, processes, roles, and controls an organization or government uses to manage AI systems responsibly across their lifecycle. It covers risk management, accountability, documentation, compliance, and oversight. Frameworks such as NIST's AI RMF (2023) and standards like ISO/IEC 42001 (2023) give organizations structured governance models.

Source: NIST AI Risk Management Framework · IBM Think — AI Governance · Related: Responsible AI, EU AI Act, NIST AI Risk Management Framework, AI transparency

# AI Overviews / AI search

AI Overviews is Google's feature that places an AI-generated summary at the top of search results, synthesizing information from multiple web sources with links. Launched broadly in the US in May 2024, it exemplifies AI search, where a generative model answers queries directly. It has reshaped SEO by reducing clicks to individual sites and elevating cited, authoritative sources.

Source: Google — AI Overviews · Google Search Central — AI features and your site · Related: GEO (generative engine optimization), AEO (answer engine optimization), Foundation model

# AI safety

AI safety is the interdisciplinary field concerned with preventing harm from AI systems, spanning near-term risks like bias and misuse and longer-term risks from highly capable models. It combines technical work (robustness, alignment, interpretability) with governance and evaluation. The UK established the world's first government AI Safety Institute in November 2023, followed by the US the same month.

Source: NIST AI Risk Management Framework · Stanford HAI · Related: AI alignment, Red-teaming, Responsible AI, AI governance

# AI transparency

AI transparency is the practice of making an AI system's capabilities, limitations, data, and decision-making sufficiently open and documented for users, regulators, and affected parties to understand and trust it. It includes disclosure that content or decisions are AI-driven, plus artifacts like model cards. The EU AI Act imposes specific transparency obligations, such as informing people when they interact with a chatbot.

Source: EU AI Act (artificialintelligenceact.eu) · NIST AI Risk Management Framework · Related: Explainable AI (XAI), Interpretability, AI governance, Watermarking (AI content)

# Artificial general intelligence (AGI)

Artificial general intelligence (AGI) is a hypothetical AI that can understand, learn, and perform any intellectual task a human can, across domains, rather than being narrow to specific tasks. There is no agreed benchmark for when AGI is achieved, and expert timelines vary widely. It contrasts with today's narrow AI and with the more speculative concept of superintelligence.

Read the full explainer →

Source: IBM Think — AGI · Stanford HAI · Related: Superintelligence (ASI), Frontier model, Foundation model, Scaling laws

# Compute governance

Compute governance is the use of AI hardware and computing resources as a lever for policy and oversight, since training frontier models requires large, trackable concentrations of specialized chips. It includes export controls, usage thresholds, and reporting requirements. The 2023 US Executive Order on AI set a training-compute reporting threshold of 10^26 floating-point operations for the most powerful models.

Source: Stanford HAI · Center for Security and Emerging Technology (CSET) · Related: FLOPs (compute), GPU / TPU, Frontier model, AI governance

# Constitutional AI

Constitutional AI is an alignment method developed by Anthropic in which a model is trained to critique and revise its own outputs against a written set of principles—a constitution—rather than relying solely on human labels for harmful content. Introduced in a December 2022 paper, it uses AI feedback (RLAIF) to make models more helpful and harmless with less human supervision.

Source: Anthropic — Constitutional AI · Anthropic — Claude's Constitution · Related: AI alignment, AI safety, Responsible AI, Foundation model

# Data privacy (in AI)

Data privacy in AI is the protection of personal and sensitive information used to train, fine-tune, or query AI systems, including preventing memorization, leakage, and unauthorized reuse. It intersects with laws like the EU's GDPR (in force since 2018). Risks include training-data extraction and models regurgitating personal data, which OWASP flags as sensitive information disclosure.

Source: OWASP Top 10 for LLM Applications · IBM Think — AI and Data Privacy · Related: AI governance, Responsible AI, Prompt injection, EU AI Act

# Deepfake

A deepfake is synthetic media—typically video, image, or audio—in which AI convincingly replaces or fabricates a person's likeness or voice to depict something that never happened. The term combines deep learning and fake, and the technique is used for fraud, disinformation, and non-consensual imagery. The EU AI Act (2024) explicitly requires deepfakes to be disclosed as artificially generated.

Source: IBM Think — Deepfakes · EU AI Act (artificialintelligenceact.eu) · Related: Watermarking (AI content), AI transparency, EU AI Act, AI safety

# EU AI Act

The EU AI Act is the European Union's comprehensive law regulating artificial intelligence, the first of its kind, adopted in 2024 and entering into force on 1 August 2024. It classifies AI systems by risk (unacceptable, high, limited, minimal) and imposes obligations accordingly, with prohibited practices applying from February 2025 and most high-risk rules phasing in through 2026 and 2027.

Source: EU AI Act (artificialintelligenceact.eu) · European Commission — AI Act · Related: AI governance, Responsible AI, Foundation model, AI transparency

# Explainable AI (XAI)

Explainable AI (XAI) is a set of methods and tools that make an AI model's outputs understandable to humans, showing which inputs or factors drove a given prediction. It is essential for high-stakes domains like credit, healthcare, and hiring, where decisions must be justified. NIST published Four Principles of Explainable AI (NISTIR 8312) in September 2021.

Source: NIST Four Principles of XAI (NISTIR 8312) · IBM Think — Explainable AI · Related: Interpretability, AI transparency, Responsible AI, AI bias / algorithmic bias

# FLOPs (compute)

FLOPs (floating-point operations) measure the total amount of computation used to train or run an AI model, and are a standard proxy for model scale. Training a frontier model can require more than 10^25 FLOPs. Regulators use compute thresholds as triggers: the EU AI Act flags general-purpose models trained above 10^25 FLOPs as potentially posing systemic risk.

Source: EU AI Act (artificialintelligenceact.eu) · Stanford HAI AI Index · Related: GPU / TPU, Compute governance, Scaling laws, Frontier model

# Foundation model

A foundation model is a large AI model trained on broad, unlabeled data at scale that can be adapted to a wide range of downstream tasks, such as language, vision, or code. The term was coined by Stanford's Center for Research on Foundation Models in 2021. Examples include GPT, Claude, Gemini, and Llama, which serve as the base for many specialized applications.

Read the full explainer →

Source: Stanford HAI — On the Opportunities and Risks of Foundation Models · IBM Think — Foundation Models · Related: Frontier model, Open-weight model, Artificial general intelligence (AGI)

# Frontier model

A frontier model is a highly capable, general-purpose foundation model at or beyond the current state of the art, whose scale and capabilities may pose novel risks. The term gained prominence with the 2023 formation of the Frontier Model Forum by Anthropic, Google, Microsoft, and OpenAI. Frontier models are a focus of AI Safety Institutes and the EU AI Act's systemic-risk provisions.

Source: Frontier Model Forum · Stanford HAI · Related: Foundation model, Artificial general intelligence (AGI), Compute governance, Model evaluation / benchmark

# GEO (generative engine optimization)

Generative engine optimization (GEO) is the practice of structuring and writing content so that AI systems—chatbots and generative search engines—cite, quote, and surface it in their answers. It adapts SEO for a world where LLMs mediate discovery. The term was introduced in a 2023 academic paper proposing measurable methods to increase a source's visibility in generative engine responses.

Source: GEO: Generative Engine Optimization (arXiv paper) · Princeton — GEO project · Related: AEO (answer engine optimization), AI Overviews / AI search, Model API

# GPU / TPU

A GPU (graphics processing unit) is a massively parallel processor that accelerates the matrix math behind AI training and inference; a TPU (tensor processing unit) is Google's custom AI accelerator built for the same purpose. GPUs, led by Nvidia, dominate the AI hardware market. These chips are the scarce resource underpinning compute governance and export-control policy.

Source: IBM Think — GPU vs CPU for AI · Google Cloud — TPU · Related: FLOPs (compute), Compute governance, Foundation model, Model API

# Interpretability

Interpretability is the degree to which humans can understand the internal mechanisms and reasoning of an AI model, as opposed to just its outputs. Mechanistic interpretability, a research subfield, reverse-engineers the specific circuits and features inside neural networks. Anthropic's 2024 work on Claude used sparse autoencoders to extract millions of interpretable features from a production model.

Source: Anthropic — Interpretability research · IBM Think — AI Interpretability · Related: Explainable AI (XAI), AI transparency, AI alignment, AI safety

# Jailbreak

A jailbreak is a prompt or technique that manipulates an AI model into bypassing its safety guardrails to produce restricted, harmful, or policy-violating output. Methods include role-play framing, obfuscation, and multi-step manipulation. OWASP lists prompt-based manipulation, which includes jailbreaks, as a top risk for large language model applications in its GenAI Top 10.

Source: OWASP Top 10 for LLM Applications · NIST AI RMF Glossary (AIRC) · Related: Prompt injection, Red-teaming, AI safety, Responsible AI

# Model API

A model API is a programming interface that lets developers send inputs to a hosted AI model and receive outputs over the internet, without running the model themselves. Providers like OpenAI, Anthropic, and Google charge per token processed. APIs are the dominant commercial delivery mechanism for closed frontier models and enable integration into apps, agents, and workflows.

Source: Anthropic API documentation · OpenAI API documentation · Related: Token cost / pricing, Open-weight model, Foundation model, Prompt injection

# Model evaluation / benchmark

Model evaluation is the process of measuring an AI model's capabilities, accuracy, safety, and limitations, often using standardized benchmarks—fixed datasets and scoring rules that allow comparison across models. Common benchmarks include MMLU for knowledge and GPQA for graduate-level reasoning. Because models can overfit to public benchmarks, evaluation increasingly combines held-out tests, human review, and red-teaming.

Source: Stanford HAI AI Index · NIST AI RMF Glossary (AIRC) · Related: Red-teaming, Scaling laws, Foundation model, Frontier model

# NIST AI Risk Management Framework

The NIST AI Risk Management Framework (AI RMF 1.0) is a voluntary, US government-published guidance for managing risks across the AI lifecycle, released on 26 January 2023. It is organized around four core functions—Govern, Map, Measure, and Manage—to help organizations build trustworthy AI. In July 2024, NIST added a Generative AI Profile (NIST-AI-600-1) as a companion resource.

Source: NIST AI Risk Management Framework · NIST AI RMF Glossary (AIRC) · Related: AI governance, Responsible AI, Model evaluation / benchmark, EU AI Act

# Open-weight model

An open-weight model is an AI model whose trained parameters (weights) are publicly released for download, letting anyone run, fine-tune, or self-host it, though the training data and code may remain closed. It differs from fully open-source models and from closed API-only models. Meta's Llama, Mistral, and DeepSeek are prominent open-weight examples.

Source: IBM Think — Open Source LLMs · Stanford HAI · Related: Foundation model, Frontier model, Model API

# Prompt injection

Prompt injection is an attack in which malicious instructions are inserted into an AI system's input—directly by a user or indirectly via external content the model reads—to override its intended behavior. Indirect prompt injection hidden in web pages or documents is especially dangerous for AI agents. OWASP ranks it LLM01, the top risk in its GenAI Top 10.

Source: OWASP Top 10 for LLM Applications · OWASP GenAI Security Project · Related: Jailbreak, Red-teaming, AI safety, Model API

# Red-teaming

AI red-teaming is a structured testing process in which people deliberately probe an AI system to find flaws, harmful outputs, and vulnerabilities such as jailbreaks or prompt injection. Named after adversarial security exercises, it surfaces failures before deployment. NIST defines it as a key evaluation method, and the EU AI Act references adversarial testing for general-purpose models with systemic risk.

Source: NIST AI RMF Glossary (AIRC) · OWASP GenAI Security Project · Related: Jailbreak, Prompt injection, Model evaluation / benchmark, AI safety

# Responsible AI

Responsible AI is the practice of designing, developing, and deploying AI systems in ways that are fair, transparent, accountable, safe, and privacy-respecting. It translates ethical principles into concrete engineering and governance controls across the model lifecycle. NIST's AI Risk Management Framework, released in January 2023, is a widely adopted voluntary standard for operationalizing these goals.

Source: NIST AI Risk Management Framework · Google Responsible AI · Related: AI governance, Explainable AI (XAI), AI bias / algorithmic bias, NIST AI Risk Management Framework

# Scaling laws

Scaling laws are empirical relationships showing that an AI model's performance improves predictably as model size, training data, and compute increase together. OpenAI documented them for language models in 2020, and DeepMind's 2022 Chinchilla paper refined the compute-optimal balance between parameters and data. These laws have driven the industry's investment in ever-larger foundation models.

Source: Scaling Laws for Neural Language Models (arXiv) · Training Compute-Optimal LLMs / Chinchilla (arXiv) · Related: Foundation model, FLOPs (compute), Frontier model, Model evaluation / benchmark

# Superintelligence (ASI)

Superintelligence (ASI) is a hypothetical AI that vastly surpasses the best human minds across virtually all domains, including scientific creativity, strategy, and social skills. Popularized by philosopher Nick Bostrom's 2014 book Superintelligence, it is a central concept in long-term AI safety debates about control and alignment. It sits beyond artificial general intelligence (AGI) on the capability spectrum.

Source: IBM Think — Superintelligence · Stanford HAI · Related: Artificial general intelligence (AGI), AI alignment, AI safety, Frontier model

# Token cost / pricing

Token cost is the price charged by an AI provider for processing text, billed per token—a chunk of text roughly three-quarters of a word in English—usually with separate rates for input and output tokens. Prices vary by model capability and have fallen sharply over time. Understanding token pricing is essential for estimating and controlling the cost of AI applications at scale.

Source: Anthropic — Pricing · OpenAI — Pricing · Related: Model API, Foundation model, Open-weight model

# Watermarking (AI content)

AI watermarking is the embedding of a hidden, detectable signal into AI-generated text, images, audio, or video so the content can later be identified as machine-produced. It supports provenance and disclosure requirements. Google DeepMind's SynthID, launched in 2023, watermarks AI images and text, and the EU AI Act requires marking of AI-generated or manipulated content.

Source: Google DeepMind SynthID · EU AI Act (artificialintelligenceact.eu) · Related: Deepfake, AI transparency, EU AI Act, Data privacy (in AI)

↑ Top