☆ Save 8.1 Discriminative vs. Generative Models — Learning Decision Boundaries vs. Data Distributions

09/13/2026

The difference between a Discriminative Model and a Generative Model comes down to what the model is trying to learn from the data. Starting with probability notation can make the distinction seem more abstract than it needs to be, so consider a simple example involving images of cats and dogs.

A Discriminative Model learns how to tell the two classes apart. Given a new image, its job is to decide whether the image contains a cat or a dog. A Generative Model takes a different approach. By observing many cat images, it learns the probability distribution underlying cat images. Once that distribution has been modeled well enough, the model can sample from it to produce new cat images rather than simply reproducing examples from the training set.

What Do the Two Models Learn Differently?
Discriminative Model: Learning to Separate Data
Generative Model: Learning the Data Distribution
Why Generate New Data from a Probability Distribution?
Key Differences Between Discriminative and Generative Models

What Do the Two Models Learn Differently?

Suppose we have a large collection of cat and dog images. A person might distinguish them by combining cues such as ear shape, facial structure, fur patterns, and other visual features. Machine Learning models can also discover useful patterns in the data, but the learning objective determines what kind of structure they focus on.

A Discriminative Model focuses on what separates cats from dogs. It learns from examples of both classes and captures the relationships that are useful for deciding which class a new input belongs to.

A Generative Model asks a different question. Given many cat images, for example, it tries to learn what the overall distribution of cat images looks like. The goal is not to memorize individual training examples, but to capture the probabilistic structure that characterizes the data itself.

At a high level, the two approaches can be summarized as follows.

Discriminative: cat and dog images → learn how the classes differ → classify a new image
Generative: cat images → learn the probability distribution of cat images → sample from that distribution to generate a new cat image

Discriminative Model: Learning to Separate Data

Start with the Discriminative Model. Suppose the training set contains labeled images of cats and dogs. The model is built to answer a straightforward question.

“Given a new image, should it be classified as a cat or a dog?”

One way to solve this problem is to learn the probability of each class directly from the input. If x represents the input image and y represents its class, the quantity of interest can be written as follows.

$$p(y\mid x)$$

This is the probability that an input image x belongs to class y.

For example, suppose the model receives a new image and assigns a probability of 0.9 to cat and 0.1 to dog. For classification, it simply compares the two values and chooses the class with the higher probability.

Probability of cat: 0.9
Probability of dog: 0.1
Prediction: cat

More generally, the prediction can be expressed as choosing the class with the highest conditional probability among all possible classes.

$$f(x)=\arg\max_y p(y\mid x)$$

Given an input image, the class with the highest probability is selected as the final prediction.

The key objective is to separate different classes as reliably as possible. The model needs to learn where cat examples tend to lie, where dog examples tend to lie, and how to distinguish one region from the other. The boundary that separates these regions is called the Decision Boundary.

A useful way to think about a Discriminative Model·then·is as a model that learns the differences between classes and uses those differences to determine where a new input belongs. It does not need to model every detail of how the input data itself is distributed in order to make that classification decision.

Generative Model: Learning the Data Distribution

A Generative Model starts from a different objective. Instead of asking how to separate cats from dogs, set classification aside for a moment and focus only on cat images themselves.

Imagine a dataset containing 100,000 cat images. Every image contains a cat, yet no two examples are exactly alike. Some cats face the camera while others appear in profile. Their fur colors and facial features vary, and so do their positions·poses·and orientations within the image.

Despite all of this variation, cat images are not arbitrary. There is still an underlying structure that determines what can plausibly appear as a cat image. Rather than memorizing every example in the training set, a Generative Model tries to capture the probability distribution that describes how cat images can occur.

At the simplest level, we can represent this idea as follows.

$$p(x)$$

This probability distribution describes how likely different possible images x are to occur as part of the data.

The phrase “learning a probability distribution” can sound abstract at first. To make it more concrete, reduce the problem to a single feature. Real images contain huge numbers of pixels and visual attributes, but suppose for now that the only feature we care about is the direction a cat is facing.

Cat images facing roughly forward: 50%
Cat images facing left: 25%
Cat images facing right: 25%

In this simplified example, the model does not learn that a cat image has one fixed appearance. Instead, it learns that several appearances are possible and that some occur more frequently than others. A cat may face forward·left·or right, with each orientation appearing at a particular rate. This is a basic intuition for what it means to learn a probability distribution.

Real image distributions are far more complicated because many properties vary at the same time: pose·color·shape·lighting·background·position·and countless pixel-level details. The underlying idea·however·remains the same. The model tries to represent probabilistically the range of forms that plausible cat images can take.

Why Generate New Data from a Probability Distribution?

Once we understand why a Generative Model learns a data distribution, the word “Generative” becomes easier to interpret. If the model has learned the distribution well enough, it can sample from that distribution to produce new data.

Return to the simplified cat example. Suppose the model has learned how frequently cats appear facing forward·left·or right. Sampling from this distribution can then determine which orientation a newly generated cat should have.

The important point is that generation does not mean selecting an existing training image and copying it. The model is intended to learn the probabilistic structure of the data rather than a catalog of individual examples. By sampling from that learned structure, it can produce new examples that were not present as exact images in the training set.

With that distinction in mind, the overall workflow of a Generative Model can be summarized as follows.

Step 1: Observe a large collection of cat images.
Step 2: Learn a probability distribution that captures how those images can appear.
Step 3: Draw a new sample from the learned probability distribution.
Step 4: Generate a new cat image rather than reproducing an existing training example.

In other words, the process is cat images → learn the probability distribution of cat images → generate new cat images by sampling from that distribution. This is the most useful mental model to keep in mind when first learning how Generative Models work.

Key Differences Between Discriminative and Generative Models

Putting the two approaches side by side makes the distinction much clearer. For a Discriminative Model, the important question is what separates one class from another, such as cats from dogs. A Generative Model instead focuses on what a particular type of data looks like as a probability distribution and what kinds of samples are plausible under that distribution.

Discriminative Model: learns the relationships that distinguish classes such as cats and dogs.
Discriminative Model output: determines which class a new input belongs to.
Generative Model: learns the probability distribution underlying data such as a collection of cat images.
Generative Model output: can sample from the learned distribution to produce new examples, such as new cat images.

In one sentence, a Discriminative Model learns how to distinguish between data classes, while a Generative Model learns a probability distribution that describes how the data itself can occur.

The difference is easy to miss if both are viewed simply as models that take inputs and produce outputs. Their objectives are fundamentally different. A Discriminative Model asks, “Which class does this image belong to?” A Generative Model asks, “What probability distribution could produce data like this?” The first perspective naturally leads to classification and Decision Boundaries; the second leads to modeling data distributions and generating new samples.

※ This article is an independently organized and restructured summary based on lectures by Professor Sungroh Yoon at Seoul National University.

📍 Where this concept fits in the AI learning map

See where this concept sits within the full AI Universe.

📍 Current position in AI Universe

☰

Reset Show completed · Login required Loading…

🌌 AI Universe

‹

›

⭐ Concept

Select a star.

View the full AI Universe

« 8. Generative Modeling —…|8.2 Generative Modeling… »

🔖 Tags: Classification · Discriminative Models · Generative AI · Generative Models · Machine Learning · Probability Distribution