WEBVTT

00:00:00.058 --> 00:00:03.853
Suppose two hospitals are comparing
a treatment with ordinary care.

00:00:03.936 --> 00:00:09.022
The treatment will perform better among
mild cases, and better among severe cases.

00:00:09.125 --> 00:00:13.190
Then we will pool everyone,
and the apparent winner will reverse.

00:00:13.642 --> 00:00:16.347
That reversal is Simpson's paradox.

00:00:17.744 --> 00:00:20.450
Here is the question in its sharpest form.

00:00:20.646 --> 00:00:23.919
Can every clinically relevant
subgroup favor the treatment

00:00:24.002 --> 00:00:26.382
while the total favors no treatment?

00:00:26.579 --> 00:00:27.613
Yes.

00:00:28.065 --> 00:00:31.140
The important task is not
merely to be surprised.

00:00:31.223 --> 00:00:34.289
It is to see exactly
how the reversal is made.

00:00:35.085 --> 00:00:39.927
We will use one thousand treated patients
and one thousand untreated patients.

00:00:40.240 --> 00:00:43.340
The entries are successes
over total cases, followed

00:00:43.340 --> 00:00:45.581
by the corresponding success rate.

00:00:46.494 --> 00:00:48.492
Start with mild cases.

00:00:48.688 --> 00:00:52.683
Among the treated patients,
ninety of one hundred succeed.

00:00:52.996 --> 00:00:55.539
That is a ninety percent success rate.

00:00:56.452 --> 00:01:01.015
Without treatment, seven hundred twenty
of nine hundred mild cases succeed.

00:01:01.212 --> 00:01:03.012
That is eighty percent.

00:01:03.464 --> 00:01:07.667
Within mild cases, treatment
is ahead by ten percentage points.

00:01:08.464 --> 00:01:10.473
Now take severe cases.

00:01:10.577 --> 00:01:14.453
Among treated patients, two hundred
seventy of nine hundred succeed,

00:01:14.536 --> 00:01:16.185
giving thirty percent.

00:01:16.982 --> 00:01:20.475
Without treatment, twenty
of one hundred severe cases succeed,

00:01:20.558 --> 00:01:22.207
giving twenty percent.

00:01:22.404 --> 00:01:25.749
Treatment is again ahead
by ten percentage points.

00:01:26.545 --> 00:01:28.705
Read the mild row once more.

00:01:28.902 --> 00:01:31.921
Ninety percent with treatment
is ten points above eighty percent

00:01:31.921 --> 00:01:33.233
without treatment.

00:01:34.146 --> 00:01:35.853
Read the severe row.

00:01:36.050 --> 00:01:38.894
Thirty percent with treatment
is ten points above twenty percent

00:01:38.894 --> 00:01:40.230
without treatment.

00:01:40.427 --> 00:01:44.595
There is no subgroup here where ordinary
care has the higher success rate.

00:01:45.392 --> 00:01:48.493
Before revealing the total,
make a prediction.

00:01:48.596 --> 00:01:51.732
Treatment wins
by the same margin in both rows.

00:01:51.929 --> 00:01:55.807
It is natural to expect treatment
to win when the rows are added.

00:01:56.604 --> 00:01:58.834
Now add the actual counts.

00:01:59.030 --> 00:02:03.406
Treatment has three hundred sixty
successes among one thousand patients,

00:02:03.489 --> 00:02:05.393
for thirty-six percent.

00:02:05.706 --> 00:02:09.571
No treatment has seven hundred
forty successes among one thousand,

00:02:09.654 --> 00:02:11.407
for seventy-four percent.

00:02:12.204 --> 00:02:15.223
The pooled comparison
therefore says the opposite.

00:02:15.420 --> 00:02:19.484
Thirty-six minus seventy-four is negative
thirty-eight percentage points.

00:02:20.281 --> 00:02:23.614
Within mild cases,
treatment wins by ten points.

00:02:23.810 --> 00:02:27.213
Within severe cases,
treatment wins by ten points.

00:02:27.409 --> 00:02:30.533
After pooling, treatment
loses by thirty-eight points.

00:02:30.730 --> 00:02:35.108
Every fraction is correct, and the
reversal is not an arithmetic error.

00:02:35.904 --> 00:02:38.842
What changed was the comparison itself.

00:02:39.039 --> 00:02:42.778
The subgroup rows compare
patients with similar severity.

00:02:42.882 --> 00:02:46.446
The total compares two populations
with radically different mixtures

00:02:46.446 --> 00:02:48.792
of mild and severe cases.

00:02:48.988 --> 00:02:51.706
That mixture is where the paradox lives.

00:02:52.780 --> 00:02:55.973
The pooled rates become
understandable as soon as we draw

00:02:55.973 --> 00:02:58.029
who entered each comparison.

00:02:58.225 --> 00:03:02.603
Blue means mild cases
and red means severe cases.

00:03:02.707 --> 00:03:05.691
Each strip contains one thousand patients.

00:03:06.395 --> 00:03:09.995
The treatment strip
is almost entirely severe cases.

00:03:10.191 --> 00:03:13.849
Nine hundred are severe,
and only one hundred are mild.

00:03:14.646 --> 00:03:18.408
The no-treatment strip
has exactly the opposite composition.

00:03:18.605 --> 00:03:22.634
Only one hundred cases are severe,
while nine hundred are mild.

00:03:23.431 --> 00:03:26.079
These labels name the two populations.

00:03:26.531 --> 00:03:29.690
Treatment is being judged
mostly on severe patients.

00:03:29.794 --> 00:03:32.975
No treatment is being judged
mostly on mild patients.

00:03:33.772 --> 00:03:37.616
Severity matters enormously even
before considering treatment.

00:03:37.928 --> 00:03:41.711
Without treatment, mild cases
succeed eighty percent of the time,

00:03:41.794 --> 00:03:45.127
while severe cases
succeed only twenty percent.

00:03:45.924 --> 00:03:49.001
That is a sixty-point baseline-risk gap.

00:03:49.198 --> 00:03:49.836
It is much

00:03:49.836 --> 00:03:54.086
larger than the treatment's ten-point
advantage inside either severity group.

00:03:54.883 --> 00:03:57.426
A pooled rate is a weighted average.

00:03:57.739 --> 00:04:00.930
For treatment, mild cases
have weight one tenth,

00:04:01.013 --> 00:04:03.742
while severe cases
have weight nine tenths.

00:04:04.353 --> 00:04:07.080
Those weights place ninety
percent beside a small group

00:04:07.163 --> 00:04:09.703
and thirty percent beside a large group.

00:04:09.786 --> 00:04:12.399
The weighted average
is thirty-six percent.

00:04:13.103 --> 00:04:15.542
For no treatment, the weights switch.

00:04:15.645 --> 00:04:21.230
Mild cases now receive weight nine tenths,
and severe cases receive only one tenth.

00:04:21.841 --> 00:04:24.594
That weighted average
is seventy-four percent.

00:04:24.906 --> 00:04:28.459
It is high because the no-treatment
population contains many more

00:04:28.459 --> 00:04:32.001
of the patients who had a high
baseline chance of success.

00:04:33.053 --> 00:04:36.860
Pooling has not averaged the same
ingredients in the same proportions.

00:04:36.943 --> 00:04:38.812
It has compared one average dominated

00:04:38.812 --> 00:04:43.085
by severe cases with another
average dominated by mild cases.

00:04:44.389 --> 00:04:47.420
We can separate
the reversal into two forces.

00:04:47.617 --> 00:04:51.008
First comes the treatment
effect visible within either group:

00:04:51.111 --> 00:04:53.422
a positive ten percentage points.

00:04:54.219 --> 00:04:56.716
Second comes the composition effect.

00:04:57.029 --> 00:05:01.465
The treatment population has eighty
percentage points fewer mild cases.

00:05:01.661 --> 00:05:05.551
Mild and severe baseline success
rates are sixty points apart.

00:05:06.464 --> 00:05:09.971
Multiplying those differences
gives negative forty-eight points.

00:05:10.168 --> 00:05:14.580
The unfavorable case mix costs forty-eight
points in the pooled comparison.

00:05:15.377 --> 00:05:19.406
Combine a positive ten-point treatment
effect with a negative forty-eight point

00:05:19.406 --> 00:05:20.951
composition effect.

00:05:21.147 --> 00:05:23.351
The result is negative
thirty-eight points,

00:05:23.434 --> 00:05:26.314
exactly the aggregate
reversal we observed.

00:05:27.529 --> 00:05:29.968
The total answers a real question:

00:05:30.164 --> 00:05:33.636
what fraction succeeded
in each observed population?

00:05:33.833 --> 00:05:36.702
It does not answer
the like-for-like question:

00:05:36.898 --> 00:05:37.560
what would happen

00:05:37.560 --> 00:05:42.205
if treatment and no treatment were applied
to populations with the same severity mix?

00:05:43.257 --> 00:05:45.905
Now turn the arithmetic into geometry.

00:05:46.357 --> 00:05:50.444
The horizontal axis records
the fraction of cases that are mild.

00:05:50.641 --> 00:05:54.066
The vertical axis records
the resulting success rate.

00:05:54.863 --> 00:05:58.463
At the far left,
the population is entirely severe.

00:05:58.660 --> 00:06:00.608
Treatment succeeds at thirty percent,

00:06:00.692 --> 00:06:03.537
while no treatment
succeeds at twenty percent.

00:06:04.333 --> 00:06:08.096
At the far right,
the population is entirely mild.

00:06:08.292 --> 00:06:10.391
Treatment succeeds at ninety percent,

00:06:10.475 --> 00:06:13.448
while no treatment
succeeds at eighty percent.

00:06:14.361 --> 00:06:17.844
Every mixture between
those endpoints is a weighted average.

00:06:18.459 --> 00:06:23.858
Moving right gives more weight to mild
cases, so both overall success rates rise.

00:06:24.655 --> 00:06:28.811
Most importantly, the red treatment
line stays ten percentage points above

00:06:28.811 --> 00:06:31.923
the green control line
at every common mixture.

00:06:32.720 --> 00:06:35.809
Start with a population
that is ten percent mild.

00:06:35.913 --> 00:06:37.257
At this same mixture,

00:06:37.341 --> 00:06:41.428
treatment is thirty-six percent
and no treatment is twenty-six percent.

00:06:42.225 --> 00:06:45.395
Sweep the common mixture
toward ninety percent mild.

00:06:45.592 --> 00:06:47.250
The two points move together,

00:06:47.334 --> 00:06:50.806
and the yellow gap remains ten
points from one end to the other.

00:06:51.602 --> 00:06:54.448
The same-mix formula confirms the picture.

00:06:54.644 --> 00:06:58.322
Both lines gain sixty points
for a full change in mild share,

00:06:58.406 --> 00:07:02.388
so those matching terms cancel
and leave a ten-point treatment advantage.

00:07:03.185 --> 00:07:05.516
If both groups had the same case mix,

00:07:05.600 --> 00:07:08.503
every vertical comparison
would favor treatment.

00:07:08.816 --> 00:07:12.462
That is the geometric form
of the two subgroup results.

00:07:13.166 --> 00:07:17.311
But the study did not observe both
groups at one horizontal position.

00:07:17.508 --> 00:07:19.412
Remove the common-mixture comparison

00:07:19.412 --> 00:07:22.745
and mark the two populations
that were actually observed.

00:07:23.541 --> 00:07:25.500
The treated population sits here,

00:07:25.584 --> 00:07:29.521
at ten percent mild
and thirty-six percent successful.

00:07:30.318 --> 00:07:32.024
The untreated population sits far

00:07:32.024 --> 00:07:36.646
to the right, at ninety percent mild
and seventy-four percent successful.

00:07:37.443 --> 00:07:40.098
Those two points
are not vertically aligned.

00:07:40.182 --> 00:07:43.349
The control point benefits
from a much easier case mix,

00:07:43.433 --> 00:07:48.264
so it can sit higher even though its
entire green line lies below the red line.

00:07:49.061 --> 00:07:52.684
The observed comparison reads
diagonally across the picture:

00:07:52.880 --> 00:07:55.458
left-hand red against right-hand green.

00:07:55.655 --> 00:07:58.439
The like-for-like
comparison reads vertically:

00:07:58.523 --> 00:08:01.902
red against green at one
shared mild-case fraction.

00:08:02.606 --> 00:08:05.068
That is the reversal made visible.

00:08:05.381 --> 00:08:08.888
Weighted averages are points
between subgroup endpoints.

00:08:09.084 --> 00:08:13.171
Different weights select different
horizontal positions, and a large enough

00:08:13.171 --> 00:08:17.003
horizontal shift can overwhelm
the vertical treatment advantage.

00:08:18.146 --> 00:08:21.337
The practical question is not
whether totals are forbidden.

00:08:21.420 --> 00:08:23.450
Totals are useful summaries.

00:08:23.533 --> 00:08:26.062
The question is whether
the pooled total answers

00:08:26.145 --> 00:08:28.479
the decision you believe it answers.

00:08:29.880 --> 00:08:31.808
Begin with severity.

00:08:32.004 --> 00:08:35.348
In our example, severity
strongly predicts the outcome.

00:08:35.800 --> 00:08:40.503
Mild cases have a much higher baseline
chance of success than severe cases.

00:08:41.300 --> 00:08:45.225
Severity also predicts
which comparison group a patient enters.

00:08:45.421 --> 00:08:48.160
The treatment group
received mostly severe cases,

00:08:48.243 --> 00:08:51.622
while the untreated group
received mostly mild cases.

00:08:52.419 --> 00:08:55.229
Treatment choice
may itself affect the outcome.

00:08:55.426 --> 00:08:58.836
But severity points toward both
treatment choice and outcome,

00:08:58.920 --> 00:09:03.124
creating a backdoor route that mixes
treatment effects with case selection.

00:09:03.921 --> 00:09:06.336
This is why the first question matters.

00:09:06.951 --> 00:09:10.783
Does the subgroup variable predict
the outcome even without treatment?

00:09:11.397 --> 00:09:15.682
If it does, changing the subgroup
mix can change an aggregate rate.

00:09:16.479 --> 00:09:17.988
Second, are treatment

00:09:17.988 --> 00:09:21.774
and comparison cases distributed
differently across those subgroups?

00:09:21.970 --> 00:09:24.964
A risk factor cannot create
this particular distortion

00:09:25.047 --> 00:09:27.474
if both groups carry the same mix.

00:09:28.271 --> 00:09:31.813
Third, which population
is the decision about?

00:09:32.009 --> 00:09:36.190
A hospital may need the expected
result for its actual patient mix.

00:09:36.386 --> 00:09:41.356
A causal comparison may instead need
both options evaluated on one common mix.

00:09:42.153 --> 00:09:43.508
Pooling is informative

00:09:43.592 --> 00:09:47.215
when treatment assignment is balanced
across the important risk groups.

00:09:47.528 --> 00:09:52.278
Then each overall rate combines comparable
ingredients in comparable proportions.

00:09:52.981 --> 00:09:57.359
Pooling can also be appropriate when the
target mixture is deliberately specified.

00:09:57.556 --> 00:10:02.128
If a health system expects seventy percent
mild and thirty percent severe cases,

00:10:02.211 --> 00:10:05.451
it can standardize both
options to that same mix.

00:10:06.248 --> 00:10:10.486
In that setting, the pooled answer
is not pretending to be universal.

00:10:10.590 --> 00:10:13.118
It is an answer for a named population,

00:10:13.202 --> 00:10:16.349
using weights chosen
to represent that population.

00:10:17.262 --> 00:10:21.778
Pooling becomes dangerous when baseline
risks differ greatly, treatment exposure

00:10:21.778 --> 00:10:23.622
is uneven across the risk groups,

00:10:23.705 --> 00:10:28.048
and the crude total is read as though
it were a like-for-like treatment effect.

00:10:28.845 --> 00:10:32.178
Our example satisfies all
three warning conditions.

00:10:32.282 --> 00:10:35.696
Severity changes baseline
success by sixty points.

00:10:35.799 --> 00:10:39.179
The mild share changes
from ten percent to ninety percent.

00:10:39.491 --> 00:10:42.987
The pooled rates then
compare unlike populations.

00:10:43.783 --> 00:10:46.638
A responsible analysis
would report the raw total

00:10:46.721 --> 00:10:51.699
and the severity-specific results,
then use stratification, standardization,

00:10:51.782 --> 00:10:57.019
regression, matching, or a suitable
study design to compare like with like.

00:10:57.723 --> 00:11:02.111
One final caution. Finding a Simpson
reversal does not automatically prove

00:11:02.111 --> 00:11:04.457
that the subgroup result is causal.

00:11:04.654 --> 00:11:08.843
Severity may be measured imperfectly,
other confounders may remain,

00:11:08.926 --> 00:11:13.629
and the chosen subgroups may themselves
be consequences of earlier decisions.

00:11:14.542 --> 00:11:17.178
The reversal is a diagnostic signal.

00:11:17.282 --> 00:11:20.124
It tells us that aggregation
and composition matter,

00:11:20.208 --> 00:11:21.403
and that we should investigate

00:11:21.403 --> 00:11:24.945
how patients entered each group
before drawing a treatment conclusion.

00:11:25.742 --> 00:11:28.471
So ask what the total literally measures.

00:11:28.923 --> 00:11:33.208
In this study, thirty-six and seventy-four
percent describe outcomes in two observed

00:11:33.208 --> 00:11:37.272
populations. They are valid
summaries of those populations.

00:11:38.068 --> 00:11:41.041
Then ask what comparison
the decision requires.

00:11:41.238 --> 00:11:43.001
If the goal is a treatment effect,

00:11:43.084 --> 00:11:46.498
compare treatment and no
treatment at the same severity mix.

00:11:46.694 --> 00:11:49.118
If the goal is operational forecasting,

00:11:49.202 --> 00:11:53.324
use the mix expected in the population
where the decision will be applied.

00:11:53.935 --> 00:11:58.034
The lesson of Simpson's paradox
is not simply to distrust averages.

00:11:58.138 --> 00:12:01.584
It is to identify their weights,
understand what creates those weights,

00:12:01.668 --> 00:12:02.899
and make sure both sides

00:12:02.899 --> 00:12:06.812
of a comparison answer the same
question for the same target population.