WEBVTT

00:00:00.035 --> 00:00:01.312
A diffusion model learns

00:00:01.312 --> 00:00:05.353
to generate images by first
studying a much easier operation:

00:00:05.457 --> 00:00:06.874
destroying them.

00:00:07.326 --> 00:00:11.225
We will take one clean training
image, add a little random noise,

00:00:11.308 --> 00:00:13.315
repeat that operation many times,

00:00:13.398 --> 00:00:17.532
and then ask what a neural network would
need to learn in order to reverse it.

00:00:18.928 --> 00:00:21.332
Here is our clean data example.

00:00:21.529 --> 00:00:23.340
I am using a simple landscape

00:00:23.340 --> 00:00:27.312
so that its large shapes
and small details are easy to track.

00:00:27.624 --> 00:00:29.784
In a real training set, this position

00:00:29.784 --> 00:00:34.696
would hold an ordinary photograph
represented by a tensor of pixel values.

00:00:35.608 --> 00:00:38.279
Call that image x zero.

00:00:38.476 --> 00:00:41.182
The subscript does not name a pixel.

00:00:41.286 --> 00:00:43.757
It names a time in a corruption process,

00:00:43.840 --> 00:00:48.230
with time zero meaning that no
artificial noise has yet been added.

00:00:49.026 --> 00:00:51.163
Now take one small step.

00:00:51.360 --> 00:00:53.241
Add an independent random value

00:00:53.241 --> 00:00:58.269
to every pixel, while retaining almost
all of the image that was already there.

00:00:58.883 --> 00:01:00.948
One step would be difficult to notice,

00:01:01.031 --> 00:01:05.339
so this first comparison shows
the effect after one hundred such steps.

00:01:06.135 --> 00:01:11.184
The house, mountains, horizon,
and sun are still recognizable.

00:01:11.267 --> 00:01:14.402
But local pixel values
have begun to wander.

00:01:14.506 --> 00:01:18.547
The corruption is not a blur
that only removes high frequencies.

00:01:18.651 --> 00:01:21.728
It is random noise
entering throughout the image.

00:01:22.524 --> 00:01:24.661
Continue to step four hundred.

00:01:24.858 --> 00:01:28.188
More of the observed
tensor now comes from noise,

00:01:28.271 --> 00:01:30.722
and less comes from the original image.

00:01:30.918 --> 00:01:32.937
The broad composition survives,

00:01:33.020 --> 00:01:38.245
while windows, roof edges, and smaller
color boundaries become unreliable.

00:01:39.757 --> 00:01:41.140
Push farther.

00:01:41.336 --> 00:01:46.411
By step seven hundred, the random
component dominates most local evidence.

00:01:46.607 --> 00:01:49.647
You may still infer
that something large sits near the center,

00:01:49.730 --> 00:01:53.980
but the image itself no longer
supports confident object recognition.

00:01:54.777 --> 00:01:58.342
Near the end, even those weak
global clues disappear.

00:01:58.655 --> 00:02:02.719
The process has not replaced
the image with one fixed gray card.

00:02:03.171 --> 00:02:04.169
Every run ends

00:02:04.169 --> 00:02:09.220
in a different random tensor whose values
follow a simple Gaussian distribution.

00:02:09.924 --> 00:02:12.537
That final simplicity is deliberate.

00:02:12.641 --> 00:02:17.726
Natural images occupy a complicated
and highly structured part of pixel space.

00:02:18.039 --> 00:02:21.811
Standard Gaussian noise is a
distribution we can sample immediately,

00:02:21.894 --> 00:02:26.829
without knowing anything about
cats, houses, faces, or landscapes.

00:02:27.626 --> 00:02:31.620
So the forward process builds
a bridge between two distributions.

00:02:31.817 --> 00:02:34.883
At one end are real images
from the training data.

00:02:35.079 --> 00:02:37.852
At the other end
is nearly pure Gaussian noise,

00:02:37.935 --> 00:02:41.337
which is easy to manufacture
whenever we want a new sample.

00:02:42.478 --> 00:02:48.630
Write the bridge as a chain. We begin at x
zero, take one random transition to x one,

00:02:48.713 --> 00:02:52.545
another to x two, and continue until x T.

00:02:52.970 --> 00:02:55.432
This is called the forward process.

00:02:55.629 --> 00:02:59.496
Forward refers to the direction
from data toward noise.

00:02:59.693 --> 00:03:02.617
It is not the direction
in which images are generated,

00:03:02.700 --> 00:03:05.417
and it does not require a neural network.

00:03:06.214 --> 00:03:09.222
We choose the corruption rule ourselves.

00:03:09.418 --> 00:03:14.213
That means we know exactly how much signal
and how much random disturbance appear

00:03:14.213 --> 00:03:15.665
at every step.

00:03:15.862 --> 00:03:20.820
The entire forward chain can be simulated
on demand from any training image.

00:03:21.872 --> 00:03:23.937
At the chosen final time,

00:03:24.020 --> 00:03:27.503
x T is approximately
a standard Gaussian tensor.

00:03:27.700 --> 00:03:29.814
The approximation matters:

00:03:30.011 --> 00:03:35.326
schedules are designed so that almost no
usable information about x zero remains,

00:03:35.409 --> 00:03:38.359
while the endpoint
is numerically well behaved.

00:03:39.573 --> 00:03:42.021
One subtlety is worth fixing now.

00:03:42.104 --> 00:03:45.272
We do not repeatedly add
raw noise without restraint,

00:03:45.355 --> 00:03:48.200
because the variance
would grow without bound.

00:03:48.513 --> 00:03:52.820
Each step slightly reduces the existing
signal while adding a calibrated amount

00:03:52.820 --> 00:03:54.237
of fresh noise.

00:03:55.034 --> 00:03:57.377
That calibration gives us clean algebra,

00:03:57.460 --> 00:04:01.373
stable scales, and a known
target at every noise level.

00:04:01.570 --> 00:04:03.054
Those three facts will turn

00:04:03.137 --> 00:04:07.259
image generation into a supervised
learning problem rather than a demand

00:04:07.259 --> 00:04:11.567
that a network invent an entire
image in one unexplained leap.

00:04:12.699 --> 00:04:15.684
Now let us make one forward step precise.

00:04:15.997 --> 00:04:19.330
Choose a small positive number beta t.

00:04:19.433 --> 00:04:21.765
This is the noise schedule at step t,

00:04:21.848 --> 00:04:26.470
and it controls how much new
uncertainty enters during that transition.

00:04:27.266 --> 00:04:30.912
Define alpha t as one minus beta t.

00:04:31.016 --> 00:04:35.939
When beta is small, alpha is close
to one, which is the algebraic statement

00:04:35.939 --> 00:04:39.062
that a single step
should preserve most of its input.

00:04:40.463 --> 00:04:42.995
The next image has two ingredients.

00:04:43.307 --> 00:04:48.600
Multiply the current image by square root
alpha t, then add an independent standard

00:04:48.683 --> 00:04:53.269
Gaussian tensor multiplied
by square root one minus alpha t.

00:04:54.066 --> 00:04:56.270
Those square roots are not decorative.

00:04:56.353 --> 00:04:59.685
Variances scale with the square
of a multiplier, so this choice

00:04:59.685 --> 00:05:04.005
keeps the total variance controlled
as signal is exchanged for noise.

00:05:04.918 --> 00:05:08.506
The same statement can be written
as a conditional distribution.

00:05:08.819 --> 00:05:12.523
Given x t minus one, x t is Gaussian.

00:05:12.627 --> 00:05:15.110
Its mean is the slightly reduced image,

00:05:15.193 --> 00:05:19.536
and its covariance is one
minus alpha t times the identity.

00:05:20.332 --> 00:05:23.548
Every pixel channel receives
independent Gaussian disturbance

00:05:23.548 --> 00:05:25.674
in this basic formulation.

00:05:25.870 --> 00:05:30.212
The image structure comes from the mean,
while the covariance tells us how widely

00:05:30.212 --> 00:05:32.988
the next sample
may wander around that mean.

00:05:34.384 --> 00:05:39.284
A remarkable simplification appears when
we compose many of these Gaussian steps.

00:05:39.736 --> 00:05:42.454
Begin with the one-step
rule we just wrote.

00:05:43.250 --> 00:05:46.339
Substitute the rule for x t minus one.

00:05:46.652 --> 00:05:49.204
The surviving signal multipliers multiply,

00:05:49.287 --> 00:05:53.676
while the independent Gaussian terms
combine into another Gaussian term.

00:05:54.159 --> 00:05:56.610
Repeat all the way back to x zero.

00:05:56.807 --> 00:06:01.788
Define alpha bar t as the product
of every alpha up to time t.

00:06:02.585 --> 00:06:04.826
The result is the closed form.

00:06:05.023 --> 00:06:11.035
At any chosen time t, x t equals square
root alpha bar t times the clean image,

00:06:11.118 --> 00:06:16.680
plus square root one minus alpha bar
t times one standard Gaussian tensor.

00:06:17.058 --> 00:06:21.283
This means training never needs
to simulate all earlier corruption steps.

00:06:21.366 --> 00:06:22.827
Sample a time t,

00:06:22.910 --> 00:06:27.798
sample one epsilon, and construct x
t directly from the original image.

00:06:28.595 --> 00:06:32.554
The picture on the right is a toy
two-dimensional version of that mixture.

00:06:32.658 --> 00:06:34.769
Blue is a fixed signal vector,

00:06:34.852 --> 00:06:40.148
red is a fixed noise vector, and yellow
is the noisy observation formed from both.

00:06:40.851 --> 00:06:45.055
At a low noise fraction, yellow
stays close to the blue signal.

00:06:45.252 --> 00:06:48.991
The noise changes it,
but signal still controls the result.

00:06:49.787 --> 00:06:51.924
Increase the noise fraction.

00:06:52.121 --> 00:06:56.160
The blue contribution shrinks,
the red contribution gains weight,

00:06:56.243 --> 00:06:59.808
and the observed vector turns
away from its original direction.

00:07:01.022 --> 00:07:05.330
Near the end, almost all reliable
information comes from epsilon.

00:07:05.527 --> 00:07:08.477
The original signal
has not been abruptly deleted.

00:07:08.580 --> 00:07:12.831
Its coefficient has been driven close
to zero by the accumulated schedule.

00:07:13.627 --> 00:07:16.972
A useful summary
is the signal-to-noise ratio.

00:07:17.075 --> 00:07:19.187
Signal power is alpha bar t,

00:07:19.270 --> 00:07:24.948
noise power is one minus alpha bar t,
so their ratio falls as t increases.

00:07:25.651 --> 00:07:28.450
Keep three meanings
attached to the notation.

00:07:28.554 --> 00:07:32.142
Beta t controls the fresh
noise in one transition.

00:07:32.339 --> 00:07:36.122
Alpha bar t records the signal
surviving from the original image.

00:07:36.205 --> 00:07:41.035
And the closed form lets one epsilon
produce any desired training noise level.

00:07:41.832 --> 00:07:44.990
We chose every part
of this corruption process.

00:07:45.187 --> 00:07:47.530
Therefore, for each noisy image,

00:07:47.613 --> 00:07:52.732
we know the clean source, the time, and
the exact random tensor that was mixed in.

00:07:52.815 --> 00:07:55.973
That is the opening
a supervised learner needs.

00:07:57.074 --> 00:08:01.812
The forward process gives us noisy
images, but training still needs a target.

00:08:02.124 --> 00:08:03.899
What should the neural network predict

00:08:03.982 --> 00:08:07.768
when it receives a noisy
image x t and the time t?

00:08:08.982 --> 00:08:12.454
One tempting answer
is the clean image x zero.

00:08:12.906 --> 00:08:17.922
Some diffusion parameterizations do
predict clean data or a related quantity.

00:08:18.119 --> 00:08:22.773
But the most common first formulation asks
for something even more directly known:

00:08:22.856 --> 00:08:25.771
the noise epsilon
that we sampled ourselves.

00:08:26.823 --> 00:08:29.854
Here is the complete
learning problem as a pipeline.

00:08:29.957 --> 00:08:33.174
Begin with a clean training image x zero.

00:08:33.878 --> 00:08:37.594
Independently sample
a Gaussian noise tensor epsilon.

00:08:37.698 --> 00:08:40.703
It has exactly the same
shape as the image tensor,

00:08:40.786 --> 00:08:44.896
with one random value for every
channel and spatial location.

00:08:45.693 --> 00:08:49.209
Choose a time t, read
the schedule coefficients at that time,

00:08:49.292 --> 00:08:53.403
and combine the clean image
with epsilon to construct x t.

00:08:53.599 --> 00:08:56.839
We now possess both
the input and the label.

00:08:57.636 --> 00:09:00.679
Feed x t and t to a neural network.

00:09:00.875 --> 00:09:05.125
The time input matters because the same
visible pattern means something different

00:09:05.125 --> 00:09:07.854
at a low noise level and at a high one.

00:09:08.766 --> 00:09:12.355
The network returns a tensor
with the image's shape.

00:09:12.551 --> 00:09:14.291
Call it epsilon hat,

00:09:14.374 --> 00:09:19.076
the network's estimate of the particular
noise tensor used to create this example.

00:09:19.989 --> 00:09:24.344
Compare epsilon hat with the known
epsilon using squared error.

00:09:24.540 --> 00:09:27.011
Pixel by pixel and channel by channel,

00:09:27.094 --> 00:09:31.704
the loss penalizes disagreement between
the generated label and the prediction.

00:09:32.617 --> 00:09:33.836
Then backpropagate

00:09:33.836 --> 00:09:38.806
through the denoising network exactly
as in ordinary supervised learning.

00:09:39.258 --> 00:09:44.123
The parameters move in the direction that
reduces expected noise-prediction error.

00:09:44.827 --> 00:09:47.475
There is no human annotation step.

00:09:47.578 --> 00:09:51.375
Labels are free because the forward
process manufactured them.

00:09:51.572 --> 00:09:54.951
Yet the statistical structure
is completely supervised:

00:09:55.148 --> 00:09:58.225
each input is paired
with a definite target tensor.

00:09:59.622 --> 00:10:03.848
Let us run one training example
in the order an implementation uses.

00:10:04.045 --> 00:10:07.099
First, select a clean
image from the data set.

00:10:07.895 --> 00:10:12.308
Second, choose a time uniformly
from the allowed diffusion steps.

00:10:12.621 --> 00:10:16.717
Across training, this makes the same
network practise very light denoising,

00:10:16.800 --> 00:10:21.039
medium denoising, and recovery
from extremely noisy inputs.

00:10:21.835 --> 00:10:25.157
Third, draw a fresh Gaussian epsilon.

00:10:25.353 --> 00:10:26.896
Because it is newly sampled,

00:10:26.979 --> 00:10:31.670
the same clean image can generate
unlimited distinct supervised examples.

00:10:32.582 --> 00:10:37.936
Fourth, use the closed-form forward
equation to construct x t directly.

00:10:38.132 --> 00:10:41.581
There is no need to execute
every earlier noise step.

00:10:42.378 --> 00:10:43.354
Fifth,

00:10:43.457 --> 00:10:47.812
ask the network for epsilon
and compare it with the sampled target.

00:10:48.125 --> 00:10:50.317
Repeat with new images, times,

00:10:50.400 --> 00:10:54.964
and noise tensors until the predictor
works across the entire schedule.

00:10:55.760 --> 00:10:58.571
The equations say
the same thing compactly.

00:10:58.767 --> 00:11:01.775
Sample t uniformly from one through T.

00:11:02.316 --> 00:11:05.092
Sample epsilon from a standard Gaussian.

00:11:05.288 --> 00:11:06.914
This epsilon is randomness,

00:11:06.914 --> 00:11:10.363
but once sampled it becomes
an exact observed label.

00:11:11.159 --> 00:11:14.968
Mix x zero and epsilon
using the schedule at t.

00:11:15.280 --> 00:11:18.114
The result x t is the network input.

00:11:18.818 --> 00:11:23.300
The network receives x t
and t, and predicts epsilon hat.

00:11:23.752 --> 00:11:28.255
In image models, this predictor is often
a convolutional network or transformer

00:11:28.338 --> 00:11:30.322
with multiscale spatial features,

00:11:30.405 --> 00:11:33.610
but its architecture does not
change the learning target.

00:11:34.662 --> 00:11:37.728
Finally minimize expected squared error.

00:11:37.924 --> 00:11:42.253
Expectations mean that we average
over clean images, sampled times,

00:11:42.336 --> 00:11:44.044
and sampled noise.

00:11:44.658 --> 00:11:47.979
Standard minibatches
approximate that average.

00:11:49.032 --> 00:11:52.608
Now compare this with demanding
a finished image directly.

00:11:52.805 --> 00:11:56.997
A random seed does not come
with one uniquely correct photograph.

00:11:57.193 --> 00:11:58.980
Many outputs could be plausible,

00:11:59.063 --> 00:12:03.255
so ordinary squared error
has no single natural target.

00:12:04.052 --> 00:12:05.826
Noise prediction is different.

00:12:05.909 --> 00:12:11.912
We chose the clean image, chose t,
sampled epsilon, and computed x t.

00:12:12.109 --> 00:12:16.742
The exact epsilon is therefore
a valid label for that exact input.

00:12:17.655 --> 00:12:20.523
The network is still
learning something profound.

00:12:20.720 --> 00:12:23.214
To identify which fluctuations are noise,

00:12:23.297 --> 00:12:27.512
it must learn what natural image
structure looks like at every scale.

00:12:28.127 --> 00:12:33.016
But the optimization interface remains
an ordinary supervised regression problem.

00:12:33.812 --> 00:12:35.738
Once that predictor is accurate,

00:12:35.821 --> 00:12:40.161
we can present a noisy tensor,
estimate the disturbance hidden inside it,

00:12:40.244 --> 00:12:43.972
and take a carefully calibrated
step toward a cleaner tensor.

00:12:44.285 --> 00:12:47.501
Repeating that operation
is the generative process.

00:12:48.593 --> 00:12:52.030
Generation begins
at the endpoint of the forward process.

00:12:52.227 --> 00:12:55.861
Sample x T directly
from standard Gaussian noise.

00:12:56.174 --> 00:12:59.550
There is no hidden photograph
underneath this particular tensor,

00:12:59.634 --> 00:13:02.932
because we did not obtain it
by corrupting a data image.

00:13:04.146 --> 00:13:08.497
Ask the trained network which part
of this tensor looks like noise at time T.

00:13:08.581 --> 00:13:12.294
Then use the reverse transition
to produce x T minus one,

00:13:12.378 --> 00:13:16.117
a sample expected to contain
very slightly more image structure.

00:13:17.169 --> 00:13:20.246
One step is not supposed
to reveal a finished picture.

00:13:20.559 --> 00:13:23.749
Training taught the network
a family of modest corrections,

00:13:23.833 --> 00:13:25.866
one for every noise level.

00:13:26.062 --> 00:13:29.418
Image generation comes
from composing those corrections.

00:13:30.215 --> 00:13:34.964
After many early reverse steps,
broad statistical organization appears.

00:13:35.161 --> 00:13:38.412
Large regions begin
to behave like sky and ground.

00:13:38.516 --> 00:13:42.185
A horizon and mountain mass
can emerge even though individual

00:13:42.185 --> 00:13:44.473
pixels remain highly uncertain.

00:13:45.269 --> 00:13:47.716
Continue through medium noise levels.

00:13:47.800 --> 00:13:52.454
The process commits to object-scale
structure: a bright region in the sky,

00:13:52.538 --> 00:13:55.522
a building-sized block, and a roof shape.

00:13:55.719 --> 00:13:59.969
These are global decisions about
composition rather than finished texture.

00:14:00.766 --> 00:14:04.238
This ordering is typical,
not an absolute law.

00:14:04.435 --> 00:14:07.358
High-noise inputs preserve
little local evidence,

00:14:07.442 --> 00:14:12.690
so early reverse steps mainly settle
low-frequency, large-scale relationships.

00:14:12.887 --> 00:14:17.276
Fine edges become recoverable only
after the broader image has stabilized.

00:14:18.673 --> 00:14:21.448
At lower noise, edges sharpen.

00:14:21.760 --> 00:14:24.974
The network can now
distinguish a wall from its windows,

00:14:25.058 --> 00:14:29.493
a roof boundary from the sky,
and one contour from a nearby contour.

00:14:30.290 --> 00:14:32.926
The last steps make smaller corrections.

00:14:33.123 --> 00:14:36.556
They adjust local contrast,
texture, color consistency,

00:14:36.640 --> 00:14:41.111
and tiny geometric details while
preserving the composition chosen earlier.

00:14:41.908 --> 00:14:45.051
The completed image is not
retrieved from a training record.

00:14:45.135 --> 00:14:46.865
It is one sample reached by starting

00:14:46.865 --> 00:14:51.231
at a newly drawn noise tensor and
following the learned reverse dynamics.

00:14:52.144 --> 00:14:55.778
A second initial noise tensor would
usually produce a different landscape,

00:14:55.778 --> 00:14:58.460
arrangement, or object identity.

00:14:58.657 --> 00:15:02.477
Randomness at the start supplies
diversity, while the learned denoiser

00:15:02.477 --> 00:15:06.169
keeps the trajectory near
the distribution of natural images.

00:15:07.566 --> 00:15:09.715
Now inspect one reverse step.

00:15:09.911 --> 00:15:14.602
First, the neural network predicts
epsilon hat t from the current noisy image

00:15:14.602 --> 00:15:16.158
and the current time.

00:15:16.955 --> 00:15:18.290
The reverse mean uses

00:15:18.290 --> 00:15:22.482
that prediction with coefficients
derived from the same forward schedule.

00:15:22.678 --> 00:15:25.392
The long expression
is calibrated subtraction,

00:15:25.476 --> 00:15:28.240
not simply x t minus epsilon hat.

00:15:29.037 --> 00:15:30.837
Why the coefficients?

00:15:31.034 --> 00:15:35.760
The forward process scaled both
signal and noise at every transition.

00:15:35.956 --> 00:15:39.042
A valid reverse step
must undo that scaling

00:15:39.126 --> 00:15:43.074
while estimating which component
belongs to the clean-image direction.

00:15:43.986 --> 00:15:46.770
For a standard
stochastic diffusion sampler,

00:15:46.854 --> 00:15:50.407
add a calibrated Gaussian term sigma t z.

00:15:50.604 --> 00:15:54.053
This does not carelessly put
back the noise we removed.

00:15:54.157 --> 00:15:55.748
It represents uncertainty

00:15:55.748 --> 00:15:59.115
in which slightly cleaner image
could have produced the current one.

00:15:59.912 --> 00:16:04.219
That uncertainty can also be handled by
deterministic trajectories or by samplers

00:16:04.219 --> 00:16:07.203
using fewer, larger numerical steps.

00:16:07.516 --> 00:16:11.360
These are alternative ways
to traverse a learned denoising field.

00:16:11.673 --> 00:16:14.958
The central model is still
the time-dependent predictor trained

00:16:14.958 --> 00:16:16.515
on noisy images.

00:16:17.311 --> 00:16:21.178
The full sampling algorithm is now
short enough to read as a loop.

00:16:21.630 --> 00:16:24.812
Start by drawing x T
from a standard Gaussian.

00:16:25.608 --> 00:16:29.638
At the current time, run the network
once to predict the noise component.

00:16:30.690 --> 00:16:34.731
Use that prediction and the schedule
to sample a slightly cleaner state.

00:16:34.927 --> 00:16:38.666
The state remains noisy
unless t is already near zero.

00:16:39.370 --> 00:16:41.809
Decrease t and repeat.

00:16:42.005 --> 00:16:44.348
The same network is reused at every step,

00:16:44.432 --> 00:16:48.264
with its time input telling it
which denoising regime is required.

00:16:49.060 --> 00:16:52.811
At zero, the state
is interpreted as an image sample.

00:16:52.915 --> 00:16:55.934
Pure noise supplied
the initial randomness;

00:16:56.131 --> 00:17:00.706
the learned sequence of local reverse
decisions supplied recognizable structure.

00:17:01.317 --> 00:17:03.404
We can now generate plausible images,

00:17:03.488 --> 00:17:06.913
but we have not told the model
which plausible image we want.

00:17:07.017 --> 00:17:09.815
The final ingredient
is conditioning, where a text

00:17:09.815 --> 00:17:13.938
prompt steers the same denoising
process toward a chosen description.

00:17:15.085 --> 00:17:19.010
To request a particular
image, add a condition c.

00:17:19.323 --> 00:17:22.981
Let the prompt be:
a red sailboat at sunset.

00:17:23.084 --> 00:17:26.727
The reverse process remains
a sequence of denoising steps,

00:17:26.811 --> 00:17:30.283
but every prediction may now
depend on that description.

00:17:30.987 --> 00:17:35.794
A text encoder converts the prompt
into a sequence of learned vectors.

00:17:35.990 --> 00:17:37.755
These vectors represent tokens

00:17:37.755 --> 00:17:42.818
and their relationships, rather than a
single hand-written list of visual rules.

00:17:43.615 --> 00:17:47.760
The noisy image x t enters
the denoiser as before.

00:17:48.212 --> 00:17:52.276
Time t still tells the network
how much corruption to expect.

00:17:53.073 --> 00:17:56.545
Text features also enter
the denoising network.

00:17:56.649 --> 00:17:58.074
In common architectures,

00:17:58.158 --> 00:18:02.501
cross-attention lets spatial image
features query the token sequence.

00:18:02.698 --> 00:18:03.626
A region beginning

00:18:03.626 --> 00:18:07.772
to resemble a boat can therefore
respond strongly to the boat token.

00:18:08.569 --> 00:18:12.494
The output is still a noise
estimate, not a finished image.

00:18:12.806 --> 00:18:16.185
Conditioning changes which noise
the model predicts should be removed

00:18:16.185 --> 00:18:18.635
from a sample compatible with the prompt.

00:18:19.687 --> 00:18:23.171
This distinction keeps
the generative mechanism unified.

00:18:23.367 --> 00:18:24.944
An unconditional model

00:18:25.028 --> 00:18:29.742
and a text-conditioned model both
perform learned reverse diffusion.

00:18:29.939 --> 00:18:33.306
The condition changes
the direction of each local step.

00:18:34.703 --> 00:18:36.651
At an early, very noisy step,

00:18:36.735 --> 00:18:40.161
prompt information mainly
influences global choices.

00:18:40.473 --> 00:18:45.405
The words sailboat and sunset
can affect composition, object category,

00:18:45.489 --> 00:18:49.333
and broad color relationships
before any rope or edge exists.

00:18:50.129 --> 00:18:55.040
At later steps, the same conditioning
can influence finer details: the outline

00:18:55.040 --> 00:18:59.186
of a sail, reflections
on water, and local texture.

00:18:59.290 --> 00:19:01.899
The condition is available
throughout the trajectory,

00:19:01.983 --> 00:19:05.629
but what it can usefully control
changes with the noise level.

00:19:06.426 --> 00:19:10.165
During training, the model
sees image and text pairs.

00:19:10.362 --> 00:19:12.820
The noise target is still known exactly,

00:19:12.904 --> 00:19:16.341
so conditioning does not
destroy the supervised objective.

00:19:16.538 --> 00:19:20.428
It adds information the predictor
may use to estimate epsilon.

00:19:21.225 --> 00:19:24.217
A plain conditional
prediction may follow the text,

00:19:24.301 --> 00:19:27.785
but practitioners often
want a stronger response.

00:19:28.098 --> 00:19:30.873
Classifier-free guidance
supplies a controllable way

00:19:30.873 --> 00:19:35.506
to amplify the prompt's influence without
training a separate image classifier.

00:19:37.158 --> 00:19:40.468
The method asks the same
network for two predictions.

00:19:40.664 --> 00:19:43.637
First, run it with an empty condition.

00:19:43.834 --> 00:19:47.677
This estimates how the sample
should denoise without knowing the prompt.

00:19:48.729 --> 00:19:51.505
Second, run it with condition c.

00:19:51.702 --> 00:19:54.372
This prediction includes
both general image knowledge

00:19:54.372 --> 00:19:56.532
and the influence of the text.

00:19:57.328 --> 00:20:01.160
Subtract the unconditional
prediction from the conditional one.

00:20:01.264 --> 00:20:04.736
The difference points toward a denoising
decision that makes the current sample

00:20:04.736 --> 00:20:06.802
more compatible with the prompt.

00:20:07.599 --> 00:20:09.620
Here is the guidance formula.

00:20:09.817 --> 00:20:12.720
Start with the unconditional prediction.

00:20:12.916 --> 00:20:18.525
Then multiply the conditional direction
by a guidance strength w and add it.

00:20:19.438 --> 00:20:24.117
At w equal to zero, the conditional
direction contributes nothing.

00:20:24.314 --> 00:20:29.481
At w equal to one, the formula returns
the ordinary conditional prediction.

00:20:29.678 --> 00:20:32.616
Values above one extrapolate beyond it.

00:20:33.319 --> 00:20:37.244
The three pictures are a schematic
comparison from the same prompt.

00:20:37.441 --> 00:20:40.527
With zero guidance, the model
can produce a plausible image

00:20:40.611 --> 00:20:43.595
while largely ignoring
the requested sailboat.

00:20:44.508 --> 00:20:48.583
At one, the ordinary conditional
prediction introduces the requested boat

00:20:48.583 --> 00:20:53.263
and sunset while retaining the model's
natural range of possible compositions.

00:20:54.059 --> 00:20:58.159
Turn guidance much higher
and prompt features are pushed harder.

00:20:58.773 --> 00:21:02.892
The boat may become more
unmistakable, colors more forceful,

00:21:02.976 --> 00:21:05.066
and composition more literal.

00:21:05.863 --> 00:21:08.229
But stronger is not simply better.

00:21:08.313 --> 00:21:11.714
Extrapolation can move
the trajectory away from the distribution

00:21:11.714 --> 00:21:13.898
on which the denoiser was trained.

00:21:14.094 --> 00:21:18.248
Excessive guidance may create
harsh contrast, repeated edges,

00:21:18.332 --> 00:21:21.723
distorted geometry, or artificial texture.

00:21:22.519 --> 00:21:24.726
It also reduces diversity.

00:21:24.923 --> 00:21:29.578
If the conditional direction is amplified
on every sample, different initial noise

00:21:29.578 --> 00:21:33.956
tensors are pulled toward a narrower
set of prompt-compatible solutions.

00:21:34.752 --> 00:21:37.377
So guidance strength controls a tradeoff.

00:21:37.573 --> 00:21:41.033
Turning it up usually improves
literal prompt adherence, but tends

00:21:41.033 --> 00:21:46.815
to sacrifice variation and can eventually
sacrifice naturalness and image quality.

00:21:47.612 --> 00:21:49.993
There is no universal best number.

00:21:50.189 --> 00:21:53.461
The useful range depends
on the model, its training,

00:21:53.545 --> 00:21:59.142
the sampler, the prompt, and the desired
balance between faithfulness and variety.

00:21:59.939 --> 00:22:03.538
Now the complete diffusion
story fits into four steps.

00:22:03.642 --> 00:22:09.088
First, define a forward process that turns
real images into known Gaussian noise.

00:22:09.885 --> 00:22:13.667
Second, create supervised
examples by sampling a clean image,

00:22:13.751 --> 00:22:15.911
a time, and epsilon.

00:22:16.108 --> 00:22:17.362
Train a neural network

00:22:17.362 --> 00:22:21.542
to predict that exact epsilon
from the noisy image and time.

00:22:22.339 --> 00:22:25.819
Third, generate by starting
from fresh Gaussian noise

00:22:25.903 --> 00:22:28.400
and applying learned reverse steps.

00:22:28.713 --> 00:22:30.663
Large-scale structure usually settles

00:22:30.663 --> 00:22:35.772
before fine detail because the available
information changes across noise levels.

00:22:36.569 --> 00:22:39.762
Fourth, condition the denoiser on text.

00:22:39.959 --> 00:22:43.532
Classifier-free guidance
amplifies the conditional direction,

00:22:43.616 --> 00:22:48.609
trading diversity and eventually
naturalness for stronger prompt adherence.

00:22:49.405 --> 00:22:53.420
The central trick is therefore not
that one network learns to paint an image

00:22:53.504 --> 00:22:55.025
in a single leap.

00:22:55.129 --> 00:22:58.458
It learns a supervised family
of noise-removal decisions,

00:22:58.542 --> 00:22:59.855
and generation emerges

00:22:59.855 --> 00:23:04.558
when those modest decisions are chained
from randomness all the way to an image.