☆ Save 8.3 Inverse Problems — Understanding Forward and Inverse Problems

09/13/2026

Machine learning is not always about mapping an input directly to a prediction. In many applications, we observe an outcome and need to work backward to infer what could have produced it. Downsampling a sharp image, for example, is relatively straightforward. Reconstructing the original high-resolution image from the resulting low-resolution version is much harder. Fine details have already been lost, and multiple high-resolution images may be consistent with the same low-resolution observation. This distinction is central to understanding how Forward Problems differ from Inverse Problems, and why inverse problems often require probabilistic modeling rather than a single deterministic answer.

Contents

• Forward Problems and Inverse Problems
• Practical Inverse Problems and Conditional Distributions
• Why Inverse Problems Are Difficult
• Key Challenges in Generative Models

Forward Problems and Inverse Problems

A Forward Problem starts with an input and asks what output it will produce. Image classification is a familiar example: given an image of a cat, the model estimates the probability that the image belongs to the cat class. The direction of inference is straightforward—the model moves from the observed input to a predicted output.

• $$p(y|x)$$

  The conditional distribution over possible outputs given an input.

The important point is that the input is already available. If an image is given as \(x\), the model can extract its features and predict the corresponding class \(y\). In this setting, the direction from input to outcome is well defined.

An Inverse Problem reverses that perspective. We begin with an observed result and try to infer the input·signal·or parameters that could have generated it. Reconstructing a sharp high-resolution image from a blurred or low-resolution observation is a typical example: the model must reason about information that is no longer directly visible in the observation.

• $$p(x|y)$$

  The conditional distribution over possible inputs given an observed result.

This distinction is more than simply reversing the direction of a conditional probability. In the forward direction, a single high-resolution image can be downsampled into a low-resolution observation. Going backward is fundamentally different: the low-resolution image may not contain enough information to identify one unique original. Fine textures, sharp boundaries, and other details may already have been lost during downsampling.

For that reason, an inverse problem often asks not “What is the one correct solution?” but “What inputs could plausibly have produced this observation?” Generative Models are particularly useful in this setting because they can represent a distribution of plausible solutions rather than forcing the problem into a single deterministic reconstruction.

Practical Inverse Problems and Conditional Distributions

Inverse problems appear frequently in image reconstruction, where the observation contains less information than the original data. The model cannot simply reproduce what is already visible. It must infer missing structure from the available evidence while producing a result that remains consistent with the observation.

• Recovering a high-resolution image from a low-resolution observation (Super-Resolution)
• Reconstructing missing regions of an image (Image Inpainting)
• Inferring a color image from a grayscale image (Image Colorization)
• Generating a corresponding image from a stroke painting (Image-to-Image Translation)
• Reconstructing a CT slice from a sparse-view sinogram (CT Image Reconstruction)

Consider Super-Resolution. A low-resolution image may preserve the overall outline of an object while losing details such as hair texture, surface patterns, or the exact shape of small text. The model therefore has to recover a high-resolution result that is consistent with the low-resolution observation while also containing fine-scale structure that looks plausible in a real image.

• $$P(\text{high-resolution}|\text{low-resolution})$$

  The distribution of plausible high-resolution images conditioned on a low-resolution observation.

• $$P(\text{full image}|\text{mask})$$

  The distribution of complete images that could be consistent with the visible information.

• $$P(\text{color image}|\text{gray image})$$

  The distribution of plausible color images conditioned on a grayscale image.

• $$P(\text{image}|\text{stroke painting})$$

  The distribution of images that could correspond to a given stroke painting.

• $$P(\text{cross-sectional image}|\text{sparse-view sinogram})$$

  The distribution of possible cross-sectional images reconstructed from limited CT observations.

Image Colorization illustrates the same ambiguity. Suppose a car appears gray in a grayscale image. The pixel values alone cannot tell us whether the original car was red·blue·or some other color. Several realistic colorizations may be compatible with exactly the same grayscale input. It is therefore more useful to model the distribution of plausible color images consistent with the observation than to assume that there is one uniquely correct colorization.

Image Inpainting has a similar structure. When part of an image is missing, the surrounding pixels may support several different ways of completing the hidden region. More than one reconstruction can fit naturally into the visible scene. This ambiguity is precisely why these tasks are naturally expressed in terms of a Conditional Distribution.

The same perspective extends to Conditional Generation. To generate data that is consistent with an observation or other condition, a model must capture what kinds of data remain plausible under that condition. The central idea here is not any particular generation algorithm, but the underlying inverse-problem structure: starting from an observed result and reasoning backward about the possible causes that could have produced it.

Why Inverse Problems Are Difficult

The fundamental difficulty with inverse problems is that an observation does not necessarily determine a unique solution. Problems with this kind of ambiguity are often described as ill-posed. In the Colorization example above, both a red car and a blue car could be perfectly plausible explanations for the same grayscale image.

When the observation alone cannot identify a unique solution, additional assumptions are needed to narrow the search space. In practice, inverse problems often incorporate prior information or regularization to favor solutions with desirable properties. The challenge is not simply to recover missing information through computation; the model needs some basis for deciding which of many compatible solutions are more plausible.

A second challenge comes from the dimensionality of the data. An image may contain millions of pixel values, and considering every possible combination creates an enormous search space. Yet real images occupy only a tiny fraction of that space. Most arbitrary arrangements of pixel values look like noise rather than photographs of meaningful objects or scenes.

One useful way to think about this is through the idea of a High-dimensional Manifold. Meaningful images are not distributed uniformly throughout the entire high-dimensional pixel space. Instead, they can be viewed as concentrating around a much more constrained and highly structured region. Solving an inverse problem therefore requires more than finding an output that matches the observation; the solution should also lie in a region where realistic data is likely to exist.

A third challenge is Latent Variable Inference. The data we observe often reflects hidden factors that are not explicitly available as inputs. A face image, for instance, contains only pixel values, but those pixels are shaped by underlying factors such as pose and facial expression. Some inverse problems therefore require the model to reason not only about missing observations but also about the latent structure that could have generated them.

An inverse problem·then·is not simply a matter of running an input-output mapping in reverse. Information may have been lost, multiple solutions may explain the same observation, and the space of realistic data can itself be highly complex. This is why understanding the probabilistic structure of plausible solutions is central to solving inverse problems.

Key Challenges in Generative Models

Using a Generative Model to solve an inverse problem requires more than acknowledging that multiple answers may be possible. The model must represent the probability structure of complex data such as natural images, learn that structure from observed examples, and then use what it has learned to infer plausible solutions from incomplete observations. These requirements can be organized around three core problems: representation, learning, and inference.

Representation asks how a model should describe a complex data distribution. An image contains many pixels whose values are strongly related to one another, so treating every pixel as an independent variable would fail to capture the structure of natural images. What we need instead is a representation capable of modeling the Joint Distribution formed by many interacting random variables.

Once that probabilistic structure has been defined, the next challenge is learning. The true distribution that generates real-world data is not directly available to us. We only observe samples drawn from it, such as images·audio·and text. A Generative Model learns from these samples with the goal of constructing a model distribution that approximates the underlying data distribution.

• $$p_{\mathrm{data}}(x)$$

  The underlying data distribution from which the training examples are drawn.

• $$p_\theta(x)$$

  The model distribution defined by learnable parameters.

For example, we cannot directly inspect the probability distribution of all real dog photographs. What we can observe is a collection of samples drawn from that distribution. From those examples, a model can learn recurring statistical structure such as ear shapes, facial geometry, and patterns in fur, then adjust its parameters so that the model distribution captures the structure of the real data as closely as possible. A central question in learning, therefore, is how to measure and reduce the discrepancy between the true data distribution and the distribution represented by the model.

The final challenge is inference. Once a Generative Model has learned a useful description of how data is generated, we can reason in the opposite direction: starting from observed raw data and asking what higher-level structure could explain it. This perspective is closely related to Inverse Graphics, where an image is treated not merely as an array of pixels but as the observable result of an underlying generative process whose hidden structure we want to recover.

Representation·learning·and inference are not independent problems. We first need a probability model expressive enough to describe real data, then learn that model from observed samples, and finally use the learned structure to infer what may lie behind a new observation. This is the broader perspective from which inverse problems become easier to understand. Rather than mechanically reversing an output to recover an input, the goal is to reason from observations toward plausible causes while using the learned structure of real-world data to constrain the possibilities.

※ This article is an independently organized and restructured summary based on lectures by Professor Sungroh Yoon at Seoul National University.

Recommended prerequisite reading (3/5)

+2

• Conditional Generation — How Generative Models Control Their Outputs
• Boltzmann Machines — A Neural Network That Uses Probabilistic Search to Find Better States
• Joint Distribution Modeling — How AI Learns Relationships Between Variables

• Joint Probability — Representing the Simultaneous Occurrence of Events as a Single Probabilistic Structure
• Conditional Distribution — How AI Models Reason with Conditional Probabilities

Recommended next reading (5/19)

+5

• 8.9 Diffusion Models — Generating Data by Adding and Reversing Noise
• 8.8 Deep Generative Models — Major Approaches to Probability Distribution Modeling
• 8.6 Data Distribution Modeling — How Generative Models Learn Data Distributions
• 8.5 Approximate Inference — Approximating Posterior Distributions with MCMC and Variational Inference
• 8.4 Posterior Inference — Posterior Distributions and Why Exact Inference Is Hard

• Latent Variable Models — Modeling the Relationship Between Observed and Hidden Variables
• Flow-based Models — Learning Data Distributions Through Invertible Transformations
• Top-p Sampling (Nucleus Sampling) — Why the Number of Candidates Changes Dynamically
• Latent Variable Inference — Inferring Hidden Structure from Observed Data
• Likelihood-Free Modeling — Inferring Generative Processes with Simulation-Based Inference

• 8.7 Normalizing Constant — Turning Model Scores into Valid Probability Distributions
• Sampling-based Inference — Why Probabilistic Token Selection Matters in LLM Decoding
• SMLD (Score Matching with Langevin Dynamics) — Score-Based Sampling
• Sample Generation — How LLMs Select Tokens from a Probability Distribution
• NADE (Neural Autoregressive Distribution Estimator) — How Generative Models Learn Probability Distributions
• Model Distribution — How LLMs Learn Output Probabilities
• PixelCNN — How Autoregressive Image Generation Learns Pixel Probabilities
• Likelihood-Based Modeling — Learning Data Distributions with Maximum Likelihood
• Distribution Matching — How Generative Models Learn Real-World Data Distributions

Posts on the same topic (3/3)

• Conditional Density Modeling — Learning Data Distributions Under Specific Conditions
• Data Distribution vs Model Distribution — How Generative Models Approximate Real-World Data
• Density Modeling — How Probability Density Is Estimated from Image Vectors

Related concepts (8/11)

+2

• Probability Normalization — How Softmax Converts Model Outputs into a Probability Distribution
• Probability Distribution Comparison — How to Measure and Interpret Differences Between Distributions
• Unnormalized Probability Models — Why Models Use Scores Before Normalization
• Valid Probability Distributions: Why Probabilities Must Sum to 1
• Normalizing Constant — The Key to Turning Relative Scores into a Probability Distribution
• Parametric Approximation — Modeling Data Distributions with Probabilistic Models
• High-Probability Region — How Machine Learning Identifies Important Areas of a Data Distribution
• Probabilistic Model — Modeling Uncertainty with Probability Distributions

• High-Dimensional Probability Distribution — Learning Structure in High-Dimensional Data
• Hidden Variable — How Probabilistic Models Learn Unobserved Structure

• Energy-Based Model (EBM) — Learning Probability Structures Through Energy Functions

📍 Where this concept fits in the AI learning map

See where this concept sits within the full AI Universe.

📍 Current position in AI Universe

☰

Reset Show completed · Login required Loading…

🌌 AI Universe

‹

›

⭐ Concept

Select a star.

View the full AI Universe

« 8.2 Generative Modeling…|8.4 Posterior Inference… »

🔖 Tags: Conditional Distribution · Forward Problems · image reconstruction · inverse problems · Machine Learning · Probabilistic Modeling