☆ Save Denoising Diffusion Probabilistic Model (DDPM) — Generating Data by Predicting Noise

09/13/2026

Denoising Diffusion Probabilistic Model (DDPM) is a probabilistic generative model built around a simple idea: instead of generating complex data from random noise in one shot, solve the problem through many small denoising steps. During training, the Forward Diffusion Process gradually corrupts real data with Gaussian noise. A neural network then learns to predict the noise present in samples at different stages of that process. Those predictions are used by the Reverse Diffusion Process to move from a noisy state toward a cleaner, more structured one. Repeating this transition eventually produces a new sample from noise.

In simple terms: rather than asking a model to turn random pixels into a complete image immediately, DDPM first defines how a clean image becomes progressively noisier. Because the noise added during training is known, the model can learn to recover it from each noisy sample. At inference time, the process runs in the opposite direction: generation starts from Gaussian noise and repeatedly applies the learned reverse transitions until recognizable data structure emerges.

The DDPM pipeline moves from forward diffusion to noise prediction and then to iterative reverse denoising

How It Works

• Forward Diffusion

  • To learn how to generate data from noise, DDPM first defines the easier direction: how data becomes noise. Starting from the original sample x0, the Forward Diffusion Process applies a Markov Chain that adds a small amount of Gaussian noise at each step.

  • $$ q(x_t \mid x_{t-1}) = \mathcal{N}\left(x_t;\sqrt{1-\beta_t}x_{t-1},\beta_t I\right) $$

    Each xt is generated from the previous state xt-1 by adding Gaussian noise according to the current noise level.

  • The parameter βt controls how much noise is introduced at timestep t. At early timesteps, most of the structure in xt-1 remains visible. As the transitions accumulate, the original signal gradually fades, and the final state approaches standard Gaussian noise.

  • $$ \alpha_t = 1-\beta_t,\qquad \bar{\alpha}_t=\prod_{s=1}^{t}\alpha_s $$

    αt describes the per-step signal retention factor, while ᾱt tracks its cumulative effect up to timestep t.

  • For example, if α is 0.8 for two consecutive steps, the cumulative factor becomes 0.8 × 0.8 = 0.64. As more steps are applied, the contribution of the original sample decreases and the contribution of Gaussian noise increases. The sequence of βt values is specified by the noise schedule.

• Reverse Diffusion Process

  • The forward process tells us exactly how noise was added, but a noisy sample xt does not directly reveal the preceding state xt-1. DDPM therefore learns a model of the conditional distribution for the previous state given the current one.

  • $$ p_\theta(x_{t-1}\mid x_t)=\mathcal{N}\left(x_{t-1};\mu_\theta(x_t,t),\sigma_t^2I\right) $$

    The Reverse Diffusion Process models a conditional probability distribution over possible previous states rather than choosing a single fixed xt-1.

  • The mean μθ is computed using the noise estimated by the neural network. The variance controls the amount of stochastic variation around that mean, and xt-1 is sampled from the resulting Gaussian distribution.

  • $$ x_{t-1}=\mu_\theta(x_t,t)+\sigma_t z $$

    The previous state xt-1 is sampled around the predicted mean with Gaussian noise z.

  • A single reverse step is not expected to turn a heavily corrupted image into a finished result. It only moves the sample to a slightly less noisy, more structured state. Repeating this transition reduces a difficult generation problem to a sequence of smaller denoising problems.

• Noise Prediction Objective

  • The reverse process depends on knowing what noise is present in xt. In the standard DDPM formulation, the neural network is therefore trained to predict the Gaussian noise used to construct the noisy sample rather than directly predicting the clean data.

  • $$ x_t=\sqrt{\bar{\alpha}_t}x_0+\sqrt{1-\bar{\alpha}_t}\epsilon $$

    x0 is the original sample, xt is the noisy sample at timestep t, and ε is the Gaussian noise used to create xt.

  • The first term carries the remaining contribution from the original data, while the second term contributes Gaussian noise. This formulation has an important practical advantage: xt can be sampled directly from x0 for any chosen timestep, so training does not need to simulate every earlier forward transition in sequence.

  • As a simple example, suppose the coefficient on the original sample is 0.8 and the coefficient on the noise is 0.6 at some timestep. Then the noisy sample can be written as:

  • $$ x_t=0.8x_0+0.6\epsilon $$

    The noisy sample xt contains contributions from both the original data and Gaussian noise.

  • Because ε is sampled explicitly when xt is created, the training target is known exactly. The model receives xt together with timestep t, predicts the noise, and compares that prediction with the actual ε.

  • $$ L=\left\|\epsilon-\epsilon_\theta(x_t,t)\right\|^2 $$

    ε is the actual noise used to construct the sample, while εθ(xt,t) is the model’s prediction. Training minimizes the difference between them.

  • The injected noise therefore serves as the learning target. As the model becomes better at predicting ε, it also becomes better at estimating what noise should be removed from xt to move toward a plausible previous state.

• Timestep and Noise Schedule

  • The same neural network must handle samples at many different noise levels, so it needs information about the current point in the diffusion process. A sample with only a small amount of corruption requires a different prediction from one that is already close to pure noise.
  • For that reason, the model receives both xt and timestep t. The timestep tells the network how noisy the current sample is and provides the context needed to make the appropriate noise prediction.
  • The noise schedule has a separate role. It defines βt at each timestep, which in turn controls how much noise is added during each forward transition. The schedule therefore determines how quickly the original signal disappears across the Forward Diffusion Process.
  • The distinction is straightforward: the timestep tells the neural network how noisy the current sample is, while the noise schedule determines how the sequence of noise levels is constructed.

• Training and Sampling

  • During training, x0 is available. A timestep t and Gaussian noise epsilon can be sampled, and the corresponding xt can be generated directly. By learning to predict ε across many timesteps, the model learns how to denoise samples across a wide range of noise levels.

  • Sampling works differently because there is no original x0. Generation starts instead from xT, drawn from standard Gaussian noise, and repeatedly applies the learned Reverse Diffusion Process.

  • $$ x_T \rightarrow x_{T-1} \rightarrow x_{T-2} \rightarrow \cdots \rightarrow x_1 \rightarrow x_0 $$

    Sampling starts from xT, which is close to Gaussian noise, and follows successive reverse steps toward a structured sample x0.

  • At each reverse step, the neural network predicts the noise in the current sample. That estimate is used to compute the mean of the reverse transition, from which the previous state is sampled. The newly sampled state then becomes the input to the next step.

  • For image generation, xT initially looks like random pixels. Structure appears gradually as the reverse transitions proceed, eventually producing a new image whose statistical properties are consistent with the distribution learned from the training data.

• Probabilistic Generation

  • DDPM is not trying to recover a particular example from the training set. Its objective is to approximate the probability distribution underlying the data and generate new samples from that distribution.
  • Denoising therefore plays a broader role than simply restoring corrupted input. In DDPM, it becomes the mechanism for moving from a simple Gaussian noise distribution toward a complex learned data distribution.
  • Different initial Gaussian noise samples produce different xT states, so the same trained model can generate different samples from different initial noise realizations.
  • DDPM is therefore a generative model that produces a distribution of possible outputs rather than a single deterministic result.

Significance and Limitations

DDPM’s main contribution is the way it turns a difficult generation problem into a sequence of learnable noise-prediction and denoising steps. The Forward Diffusion Process defines how clean data is progressively corrupted, while training teaches a neural network to estimate the noise present at arbitrary timesteps. The Reverse Diffusion Process then uses those estimates to generate new samples by gradually moving from Gaussian noise toward the learned data distribution. This formulation became the foundation for many later diffusion-based generative models.

The drawback is computational cost. Training can construct xt directly from x0 at a chosen timestep, but inference still has to begin at xT and execute many reverse steps sequentially. Generation can therefore be slow. DDPM ultimately trades the modeling power of iterative denoising for the computational overhead of repeated sampling.

Recommended prerequisite reading (3/5)

+2

• Forward Diffusion Process — Why Does a Diffusion Model Add Noise?
• Diffusion Model — How Denoising Turns Random Noise into New Data
• Generative Network — How a Generator Learns the Data Distribution

• Generative Model — Learning Data Distributions to Generate New Samples
• Text-to-Image Generation Model (Imagen) — A probabilistic generative model for synthesizing images from textual descriptions

Recommended next reading (5/15)

+5

• 8.2 Generative Modeling — Learning Data Distributions and Generating New Samples
• 6.8 After Transformers and Modern AI — Sequential Modeling Expanded into Foundation Models and Generative AI
• Implicit Generative Models — Generating Data Without Explicit Probability Density Modeling
• Deep Generative Model — How AI Learns Data Distributions and Creates New Samples
• Generator — How Generative Models Turn Latent Representations into Data

• GAN (Generative Adversarial Networks) — How Generative Models Learn Through Competition
• Deep Generative Learning — Learning the Data Distribution to Create New Samples
• Synthetic Data — Artificially Generated Training Data Used to Overcome the Limits of Real-World Data
• Discriminator — How GANs Use Adversarial Feedback to Train the Generator
• 1. Traditional Machine Learning in Practice: Learning Paradigms, Algorithm Families, and Evaluation Perspectives

• MADE — Autoregressive Density Estimation with Masked Autoencoders
• Hierarchical Generation — A Method That Builds the High-Level Structure First and Then Generates Details Step by Step
• Outline-based Generation — Why Structured Planning Produces Better Long-Form Content Than Direct Generation
• Mask-Predict — Mask-Based Iterative Refinement for Sequence Generation
• Difficulty Estimation — Why Does the Same Data Have Different Difficulty for Different Models?

Posts on the same topic (0/0)

No other posts in this section yet.

Related concepts (2/2)

• Reverse Diffusion Process — How Diffusion Models Generate Data from Noise
• Reverse Distribution — Estimating the Previous State in Diffusion Models

📍 Where this concept fits in the AI learning map

See where this concept sits within the full AI Universe.

📍 Current position in AI Universe

☰

Reset Show completed · Login required Loading…

🌌 AI Universe

‹

›

⭐ Concept

Select a star.

View the full AI Universe

« Medical Image Reconstruc…|Diffusion Model — How De… »

🔖 Tags: DDPM · diffusion-model · Forward Diffusion Process · gaussian noise · Generative Model · Noise Prediction · Reverse Diffusion Process