What is deep learning?

What deep learning is

Deep learning is a subfield of machine learning built on artificial neural networks that contain many layers between input and output. The word "deep" refers precisely to that layer count: a network with several or many hidden layers is "deep," whereas one with a single hidden layer is "shallow." IBM describes deep learning as a subset of machine learning that uses multilayered neural networks to simulate the complex decision-making power of the human brain.

In their influential 2015 review article "Deep learning" in Nature, Yann LeCun, Yoshua Bengio, and Geoffrey Hinton defined it as methods that "allow computational models that are composed of multiple processing layers to learn representations of data with multiple levels of abstraction." That phrase, multiple levels of abstraction, is the crux of what makes deep learning distinctive.

Every deep learning model is a neural network, but not every neural network is deep. Deep learning is best understood as the branch of machine learning that scales neural networks up in depth, and pairs them with large datasets and powerful hardware, to tackle problems that shallower models could not.

Why depth matters

The central insight of deep learning is that stacking layers lets a model build a hierarchy of features. In an image network, the first layers learn to detect simple things like edges and color gradients; middle layers combine those into textures and parts such as an eye or a wheel; and later layers combine those into whole objects like a face or a car. Each layer expresses the data in terms of the layer below it, producing progressively more abstract representations.

This matters because it replaces manual feature engineering. In older machine learning, human experts had to hand-design the input features, a slow, domain-specific, and often limiting process. Deep learning learns those features automatically from raw data. LeCun, Bengio, and Hinton emphasized this: representations are learned, not designed, which is why the same basic approach works across vision, speech, and language.

Depth also brings efficiency of representation. Theoretically and in practice, deep networks can represent certain complex functions far more compactly than shallow ones, capturing intricate structure that a wide-but-shallow network would struggle to model. That expressive power, unlocked once data and compute caught up, is why depth became the winning strategy.

The breakthroughs that proved it

Deep learning's modern era is usually dated to 2012, when a deep convolutional neural network known as AlexNet, built by Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton, won the ImageNet Large Scale Visual Recognition Challenge by a wide margin. The paper reported a top-5 error rate of 15.3%, compared with 26.2% for the second-best entry, a leap that stunned the computer-vision field and shifted it decisively toward deep learning.

Progress then accelerated with deeper architectures. In 2015, Microsoft Research's ResNet (residual networks), introduced by Kaiming He and colleagues, trained networks up to 152 layers deep by adding "skip connections" that made very deep models trainable. ResNet won the ImageNet 2015 classification task with a 3.57% top-5 error, surpassing typical human-level performance on that benchmark and confirming that depth, done right, keeps paying off.

The significance of the field was formally recognized in 2018, when Yoshua Bengio, Geoffrey Hinton, and Yann LeCun received the ACM A.M. Turing Award, often called the "Nobel Prize of computing," for their foundational work on deep neural networks.

What made deep learning possible

The core ideas behind deep learning, layered neural networks and backpropagation, existed for decades before the field took off. Three converging factors turned theory into practice in the 2010s. First, data: the internet produced enormous labeled datasets such as ImageNet's millions of images. Second, compute: graphics processing units (GPUs), originally built for video games, turned out to be ideal for the parallel math that training deep networks requires.

Third, algorithmic refinements made deep networks actually trainable. Techniques such as the ReLU activation function, better weight initialization, dropout for regularization, and architectural innovations like residual connections solved problems, most notably the "vanishing gradient" that had previously made very deep networks fail to learn.

Together, more data, more compute, and better methods, converted a niche academic idea into the most powerful tool in AI. This is why deep learning's rise is often described not as a single invention but as a scaling story.

Where deep learning is used

Deep learning is the paradigm behind nearly every headline AI capability of the past decade. Convolutional networks drive computer vision: medical imaging, autonomous-vehicle perception, quality inspection in factories, and facial recognition. Recurrent and, more recently, transformer networks drive language technology: translation, summarization, search, and conversational assistants.

The most visible example today is the large language model. Systems such as GPT and Claude are very large transformer-based deep networks trained on vast text corpora, direct descendants of the same principles that produced AlexNet. Image generators like diffusion models, speech recognizers, recommendation systems, and protein-folding tools such as AlphaFold are all deep learning at their core.

Because deep learning learns features from raw data, it excels wherever the input is rich and unstructured, pixels, audio waveforms, text, sensor streams, and wherever there is enough data to learn from. That breadth is why it has become the default approach for hard perception and generation tasks.

Limitations and the bigger picture

Deep learning's strengths come with real costs. Models are data-hungry and compute-intensive; training a state-of-the-art model can require enormous datasets, specialized hardware, and significant energy. They are also difficult to interpret, the same distributed, many-layered representations that make them powerful make it hard to explain any single decision, which is a serious issue in regulated, high-stakes domains.

Deep networks can inherit biases from their training data, behave unpredictably on inputs unlike anything they saw in training, and "hallucinate" plausible-but-wrong output in the case of generative models. They learn correlations, not causal understanding, so they can fail in ways that look surprising to humans. Responsible use pairs them with evaluation, monitoring, grounding in verified data, and human oversight.

The bigger picture is that deep learning shifted AI from hand-crafted rules to learned representations at scale, and that shift is what makes the current moment feel different from earlier AI waves. It is not a magic bullet, but it is the engine under the hood of modern artificial intelligence, and understanding it is the key to understanding where the field is headed.

Frequently asked questions

What is the difference between deep learning and machine learning?

Machine learning is the broad field of algorithms that learn patterns from data. Deep learning is a subset of machine learning that specifically uses neural networks with many layers. The key practical difference is feature engineering: traditional machine learning often needs humans to design the input features, while deep learning learns those features automatically from raw data, which is why it excels on complex inputs like images, audio, and text.

Why is it called "deep" learning?

The "deep" refers to the number of layers in the neural network. Data passes through many stacked hidden layers, each transforming it into a more abstract representation, so the network is "deep" in the sense of having depth of processing. A network with just one hidden layer is called shallow; deep learning specifically means using many layers to learn hierarchical features.

Is deep learning the same as a neural network?

Deep learning is built entirely on neural networks, but the terms are not interchangeable. A neural network can be shallow (one hidden layer) or deep (many). Deep learning specifically refers to using deep, multilayered neural networks. So every deep learning model is a neural network, but not every neural network qualifies as deep learning.

Do large language models use deep learning?

Yes. Large language models such as GPT and Claude are very large deep learning systems, specifically deep neural networks built on the transformer architecture, trained on vast amounts of text. They are among the largest and most capable products of deep learning to date, and they rely on the same core principles of layered representation learning and backpropagation that underpin the whole field.

How much data does deep learning need?

Generally a lot. Deep learning's advantage over simpler methods usually appears when large datasets are available, because the many parameters in a deep network need substantial data to be tuned well without overfitting. The exact amount depends on the task and model size, but data-hunger is one of deep learning's defining characteristics, which is why techniques like transfer learning, reusing a model pretrained on large data, are widely used.

When did deep learning become dominant?

The turning point is usually dated to 2012, when the deep convolutional network AlexNet won the ImageNet image-recognition challenge by a large margin, reporting a 15.3% top-5 error rate versus 26.2% for the runner-up. That result convinced the research community that deep networks, combined with big datasets and GPUs, were the way forward, and adoption accelerated rapidly through the mid-2010s.

Related

Explainers: Neural network, LLM, Transformer

Glossary: Neural network, Machine learning, Backpropagation, Transformer (architecture)

Sources

• LeCun, Bengio & Hinton (2015) — Deep learning (Nature)
• IBM Think — What is deep learning?
• AWS — What is deep learning?
• He et al. (2015) — Deep Residual Learning for Image Recognition (ResNet)
• Google — Machine Learning Glossary