{"version":1,"lectureId":"01M14V03BNSC4HNC1D114614TX","attempt":0,"publication":{"slug":"bayes-through-the-medical-test-paradox","title":"Why a 99% Accurate Test Can Still Be Wrong","subject":"statistics","summary":"A visual introduction to base rates and Bayes' theorem for viewers without statistics training. Starting with 10,000 people and a rare disease, the lecture separates true positives from false positives, reads the probability directly from those populations, and only then introduces Bayes' formula. It shows how prevalence and symmetric test accuracy change the meaning of a positive result, then follows the same population through a second conditionally independent test.","metaDescription":"See how a 99% sensitive and 99% specific test can be wrong for a rare disease, then watch prevalence, accuracy, and retesting change the odds.","transcript":"Suppose a medical test is described as ninety-nine percent accurate, and it comes back positive. That sounds almost conclusive. But if the disease is rare, the positive result can still be more likely wrong than right. We are going to see why by counting people before writing any probability formula. Here is the question in its most personal form. A positive result says you have a rare disease. Does ninety-nine percent accurate mean there is a ninety-nine percent chance you have it? No. That number describes how the test behaves inside known groups. It does not yet answer what group a positive person probably came from. Take ten thousand people. The large gray field represents the people in this cohort who do not have the disease. I have magnified the affected people so we can actually see them. Let the disease affect one person in a thousand. That is a prevalence of zero point one percent. In ten thousand people, only ten actually have the disease. The remaining nine thousand nine hundred ninety are healthy. Now we must say exactly what ninety-nine percent accurate means. For this lecture, it means two things. Sensitivity is ninety-nine percent, so among people who truly have the disease, the test is positive ninety-nine percent of the time. Specificity is also ninety-nine percent, so among healthy people, the test is negative ninety-nine percent of the time. Apply sensitivity to the ten affected people. Ninety-nine percent of ten is nine point nine. So across many cohorts like this one, we expect about nine point nine true positive results. Now turn to the healthy majority. Ninety-nine percent specificity leaves a one percent false-positive rate. One percent sounds tiny, but it acts on nine thousand nine hundred ninety people. One percent of that enormous healthy group is ninety-nine point nine false positives. The decimal counts are expected counts, averages over many equally sized cohorts. In one real cohort we would see whole people, very close to these values. The test has now run on all ten thousand people. But after a positive result, most of that original cohort is no longer relevant. We need a new reference group: everyone whose result was positive. The green pile contains the positive results from people who truly have the disease. Its expected size is nine point nine. These are the true positives. The yellow pile contains positive results from healthy people. Its expected size is ninety-nine point nine. These are false positives, contributed by the enormous healthy majority. Pause on the picture. The test is excellent inside either group. Yet the false-positive pile is about ten times taller, because the healthy group supplying it began nine hundred ninety-nine times larger than the disease group. Now gather the two piles. Write nine point nine true positives above ninety-nine point nine false positives. Rule beneath them and add. The positive-test group contains about one hundred nine point eight people. To answer our question, ask what fraction of that positive group came from the green pile. The numerator is nine point nine true positives. The denominator is every positive result, one hundred nine point eight. That fraction is about nine percent. So after one positive test, the chance of actually having the disease is only about nine percent under our assumptions. The complementary probability is about ninety-one percent. In other words, this positive result is probably wrong, even though the test has ninety-nine percent sensitivity and ninety-nine percent specificity. Nothing paradoxical happened. The test made errors on only one percent of healthy people. There were simply so many healthy people that their small error rate produced far more positive results than the rare disease did. Now that the populations are visible, we can compress the same reasoning into Bayes' theorem. Start with the rule we already used: true positives divided by all positive results. The vertical bar means given. P of D given positive asks: among people known to have a positive result, what fraction have disease? The phrase after the bar names the reference group. Before building the formula, name its ingredients. P of D is prevalence, the fraction who have the disease before testing. Here it is zero point zero zero one. P of positive given D is sensitivity, the positive rate inside the disease group. Here it is zero point nine nine. P of D complement is the healthy share, zero point nine nine nine. And P of positive given D complement is the false-positive rate, zero point zero one. Now replace each pile by the probability that creates it. Sensitivity times prevalence creates the true-positive share. False-positive rate times healthy share creates the false-positive share. The numerator keeps the true-positive route. The denominator adds both routes into the positive group. This is exactly what the two visible piles did, with the common population size canceled out. Substitute our values. The true-positive route is zero point nine nine times zero point zero zero one. The false-positive route is zero point zero one times zero point nine nine nine. The result is about zero point zero nine zero, or nine percent. Bayes' theorem has not introduced a new argument. It has merely named the count in a form that works for any cohort size. The common mistake is to reverse the condition. Ninety-nine percent sensitivity describes positive results among people already known to have disease. We wanted disease among people already known to have a positive result. Those are different questions. Bayes' formula lets us change one ingredient at a time. First keep the test fixed at ninety-nine percent sensitivity and specificity, and vary only the disease prevalence. The horizontal coordinate is prevalence as a percentage. The vertical coordinate is the chance of disease after a positive result. At zero point one percent prevalence, our yellow point reads about nine percent. Raise prevalence to one percent. Now one person in a hundred has the disease before testing. The point climbs to fifty percent, because the expected true-positive and false-positive piles are equal. Raise prevalence to five percent. The test has not improved at all, but the positive result now means about eighty-three point nine percent. At ten percent prevalence, the posterior reaches about ninety-one point seven percent. The same test result means something very different in a high-risk population than in a low-risk population. Prevalence is the starting information, sometimes called the prior probability. A positive test updates that starting point. It does not erase it. Now restore the very rare prevalence of zero point one percent and change the test itself. To keep the phrase accuracy unambiguous, a will mean both sensitivity and specificity. At ninety-nine percent accuracy, the point again sits near nine percent. Its label gives accuracy first and the posterior probability second. Drop accuracy to ninety-five percent. The posterior falls below two percent. A five percent false-positive rate applied to nearly ten thousand healthy people overwhelms the true-positive pile. Return to ninety-nine percent, and we recover about nine percent. Now push the accuracy to ninety-nine point nine percent. The false-positive rate falls from one percent to one tenth of one percent. That extra nine in the accuracy raises the posterior to about fifty percent. For an extremely rare disease, tiny changes in the false-positive rate can matter enormously because that rate acts on the healthy majority. So the phrase ninety-nine percent accurate is incomplete on its own. We need sensitivity, specificity, and prevalence. Change any one of them and the meaning of a positive result can swing dramatically. Suppose the same person is tested again and the second result is also positive. Begin with the group that survived the first test: about nine point nine true positives and ninety-nine point nine false positives. Assume the second test is conditionally independent of the first. Among the people who truly have disease, it again detects ninety-nine percent. Ninety-nine percent of nine point nine is nine point eight zero one. Among the healthy people who produced the first false positive, only one percent produce another false positive independently. One percent of ninety-nine point nine is zero point nine nine nine. Now read the two surviving piles. About nine point eight people are true positives twice, while about one person is falsely positive twice. The green pile is finally much larger than the yellow pile. The probability of disease after two positive results is the green count divided by the two surviving counts together. That is about ninety point eight percent. One independent repeat test has moved the answer from about nine percent to about ninety-one percent by filtering both piles again. There is a compact way to understand that jump. Start with disease odds of ten to nine thousand nine hundred ninety, which reduce to one to nine hundred ninety-nine. A positive result is ninety-nine times more likely when disease is present than when it is absent. That factor, ninety-nine, is called the positive likelihood ratio. The first positive result multiplies the prior odds by ninety-nine. That produces the same roughly nine percent probability we found from the first two piles. Under conditional independence, the second positive multiplies by the same factor again. Two positives contribute ninety-nine times ninety-nine, changing the odds by a factor of nine thousand eight hundred one. Convert those final odds back to a probability and we recover ninety point eight percent. The count method and the odds method are two views of the same update. The independence assumption matters. If both tests use the same sample, the same instrument, or the same biological signal, their errors may be correlated. A repeated error can then be more likely than this calculation assumes, so the second positive may add less evidence. The lesson is not to distrust accurate tests. It is to ask the complete question. How rare is the disease? What are the sensitivity and specificity? And is new evidence genuinely independent? With those facts, a surprising positive result becomes a count we can understand.","watch":{"version":1,"scenes":[{"title":"Ten Thousand People","start":0,"end":131.74064583333333,"objects":{"assumption":"a Text [text] that says \"Assumption: 99% sensitivity and 99% specificity.\"","card":"a Title that says \"Probability for Everyday Decisions — Why a 99% Accurate Test Can Still Be Wrong\"","crowd":"a Figure (x_range=(0.0, 10.0), y_range=(0.0, 6.0), aspect=(5.0, 3.0))","disease_dots":"a Point [red] drawn in crowd (location=(0.75, 4.9))","disease_dots_10":"a Point [red] drawn in crowd (location=(2.4299999999999997, 5.32))","disease_dots_2":"a Point [red] drawn in crowd (location=(1.17, 4.9))","disease_dots_3":"a Point [red] drawn in crowd (location=(1.5899999999999999, 4.9))","disease_dots_4":"a Point [red] drawn in crowd (location=(2.01, 4.9))","disease_dots_5":"a Point [red] drawn in crowd (location=(2.4299999999999997, 4.9))","disease_dots_6":"a Point [red] drawn in crowd (location=(0.75, 5.32))","disease_dots_7":"a Point [red] drawn in crowd (location=(1.17, 5.32))","disease_dots_8":"a Point [red] drawn in crowd (location=(1.5899999999999999, 5.32))","disease_dots_9":"a Point [red] drawn in crowd (location=(2.01, 5.32))","disease_label":"a Math [red] that says \"$10 thin upright(\"people with disease\")$\" drawn in crowd","false_positive_count":"a Math [text] that says \"$upright(\"false positives\") = 0.01 dot.op 9990 = 99.9$\"","false_positive_rate":"a Math [text] that says \"$upright(\"false-positive rate\") = 100% - 99% = 1%$\"","healthy_block":"a Polygon [gray] drawn in crowd (vertices=((0.4, 0.4), (9.6, 0.4), (9.6, 4.4), (0.4, 4.4)), fill_opacity=0.18)","healthy_count":"a Math [text] that says \"$upright(\"without disease\") = 9990$\"","healthy_label":"a Math [gray] that says \"$9990 thin upright(\"people without disease\")$\" drawn in crowd","population":"a Math [text] that says \"$upright(\"population\") = 10000$\"","prevalence":"a Math [text] that says \"$P(D) = 0.1% = frac(10, 10000)$\"","question":"a Panel that says \"A test described as 99% accurate says you have a rare disease. How likely is it that the positive result is correct?\"","true_positive_count":"a Math [text] that says \"$upright(\"true positives\") = 0.99 dot.op 10 = 9.9$\""},"beats":[{"start":0,"say":"Suppose a medical test is described as ninety-nine percent accurate, and it comes back positive. That sounds almost conclusive. But if the disease is rare, the positive result can still be more likely wrong than right. We are going to see why by counting people before writing any probability formula.","live":[],"does":[[0,"card is shown on the screen, written out."],[1.5,"card: enter:write-left-to-right."],[17.9605,"card is hidden from the screen — left the board."]]},{"start":19.1605,"say":"Here is the question in its most personal form. A positive result says you have a rare disease. Does ninety-nine percent accurate mean there is a ninety-nine percent chance you have it? No. That number describes how the test behaves inside known groups. It does not yet answer what group a positive person probably came from.","live":null,"does":[[19.1605,"question is shown on the screen, written out."],[38.792500000000004,"question moves to a new place on the board."]]},{"start":39.3925,"say":"Take ten thousand people. The large gray field represents the people in this cohort who do not have the disease. I have magnified the affected people so we can actually see them.","live":["question"],"does":[[39.3925,"crowd is shown on the screen, written out."],[40.066,"population is shown on the screen, written out."],[42.364999999999995,"healthy_block is shown on the screen, written out."],[45.024,"healthy_label is shown on the screen, written out."]]},{"start":51.1035,"say":"Let the disease affect one person in a thousand. That is a prevalence of zero point one percent. In ten thousand people, only ten actually have the disease. The remaining nine thousand nine hundred ninety are healthy.","live":["question","population","crowd","healthy_block","healthy_label"],"does":[[55.132000000000005,"prevalence is shown on the screen, written out."],[58.080999999999996,"disease_dots is shown on the screen, written out."],[58.080999999999996,"disease_dots_2 is shown on the screen, written out."],[58.080999999999996,"disease_dots_3 is shown on the screen, written out."],[58.080999999999996,"disease_dots_4 is shown on the screen, written out."],[58.080999999999996,"disease_dots_5 is shown on the screen, written out."],[58.080999999999996,"disease_dots_6 is shown on the screen, written out."],[58.080999999999996,"disease_dots_7 is shown on the screen, written out."],[58.080999999999996,"disease_dots_8 is shown on the screen, written out."],[58.080999999999996,"disease_dots_9 is shown on the screen, written out."],[58.080999999999996,"disease_dots_10 is shown on the screen, written out."],[58.080999999999996,"disease_label is shown on the screen, written out."],[62.296,"healthy_count is shown on the screen, written out."],[64.374,"healthy_block is indicated — a transient flash."],[65.233,"crowd moves to a new place on the board."],[65.233,"healthy_count is hidden from the screen — left the board."],[65.233,"population is hidden from the screen — left the board."],[65.233,"prevalence is hidden from the screen — left the board."]]},{"start":65.833,"say":"Now we must say exactly what ninety-nine percent accurate means. For this lecture, it means two things. Sensitivity is ninety-nine percent, so among people who truly have the disease, the test is positive ninety-nine percent of the time. Specificity is also ninety-nine percent, so among healthy people, the test is negative ninety-nine percent of the time.","live":["question","crowd","healthy_block","healthy_label","disease_dots","disease_dots_2","disease_dots_3","disease_dots_4","disease_dots_5","disease_dots_6","disease_dots_7","disease_dots_8","disease_dots_9","disease_dots_10","disease_label"],"does":[[65.833,"assumption is shown on the screen, written out."],[73.032,"disease_label is indicated — a transient flash."],[81.03099999999999,"healthy_label is indicated — a transient flash."]]},{"start":88.875,"say":"Apply sensitivity to the ten affected people. Ninety-nine percent of ten is nine point nine. So across many cohorts like this one, we expect about nine point nine true positive results.","live":["question","crowd","healthy_block","healthy_label","disease_dots","disease_dots_2","disease_dots_3","disease_dots_4","disease_dots_5","disease_dots_6","disease_dots_7","disease_dots_8","disease_dots_9","disease_dots_10","disease_label","assumption"],"does":[[92.70599999999999,"true_positive_count is shown on the screen, written out."],[98.54599999999998,"true_positive_count (the \"9.9\" part) is emphasized."],[101.4485,"true_positive_count (the \"9.9\" part) is no longer emphasized."]]},{"start":102.0485,"say":"Now turn to the healthy majority. Ninety-nine percent specificity leaves a one percent false-positive rate. One percent sounds tiny, but it acts on nine thousand nine hundred ninety people.","live":["question","crowd","healthy_block","healthy_label","disease_dots","disease_dots_2","disease_dots_3","disease_dots_4","disease_dots_5","disease_dots_6","disease_dots_7","disease_dots_8","disease_dots_9","disease_dots_10","disease_label","assumption","true_positive_count"],"does":[[103.46499999999999,"healthy_block is indicated — a transient flash."],[107.204,"false_positive_rate is shown on the screen, written out."]]},{"start":115.01350000000001,"say":"One percent of that enormous healthy group is ninety-nine point nine false positives. The decimal counts are expected counts, averages over many equally sized cohorts. In one real cohort we would see whole people, very close to these values.","live":["question","crowd","healthy_block","healthy_label","disease_dots","disease_dots_2","disease_dots_3","disease_dots_4","disease_dots_5","disease_dots_6","disease_dots_7","disease_dots_8","disease_dots_9","disease_dots_10","disease_label","assumption","true_positive_count","false_positive_rate"],"does":[[115.42,"false_positive_count is shown on the screen, written out."],[118.17099999999999,"false_positive_count (the \"99.9\" part) is emphasized."],[130.44897916666667,"false_positive_count (the \"99.9\" part) is no longer emphasized."],[130.69897916666667,"assumption is hidden from the screen — left the board."],[130.69897916666667,"crowd is hidden from the screen — left the board."],[130.69897916666667,"healthy_block is hidden from the screen — crowd left the board."],[130.69897916666667,"healthy_label is hidden from the screen — crowd left the board."],[130.69897916666667,"disease_dots is hidden from the screen — crowd left the board."],[130.69897916666667,"disease_dots_2 is hidden from the screen — crowd left the board."],[130.69897916666667,"disease_dots_3 is hidden from the screen — crowd left the board."],[130.69897916666667,"disease_dots_4 is hidden from the screen — crowd left the board."],[130.69897916666667,"disease_dots_5 is hidden from the screen — crowd left the board."],[130.69897916666667,"disease_dots_6 is hidden from the screen — crowd left the board."],[130.69897916666667,"disease_dots_7 is hidden from the screen — crowd left the board."],[130.69897916666667,"disease_dots_8 is hidden from the screen — crowd left the board."],[130.69897916666667,"disease_dots_9 is hidden from the screen — crowd left the board."],[130.69897916666667,"disease_dots_10 is hidden from the screen — crowd left the board."],[130.69897916666667,"disease_label is hidden from the screen — crowd left the board."],[130.69897916666667,"false_positive_count is hidden from the screen — left the board."],[130.69897916666667,"false_positive_rate is hidden from the screen — left the board."],[130.69897916666667,"question is hidden from the screen — left the board."],[130.69897916666667,"true_positive_count is hidden from the screen — left the board."]]}]},{"title":"Read the Positive Pile","start":131.74064583333333,"end":251.75231250000002,"objects":{"baseline":"a Line [gray] drawn in piles (start=(0.25, 0.0), end=(4.75, 0.0))","false_bar":"a Polygon [yellow] drawn in piles (vertices=((3.15, 0.0), (4.45, 0.0), (4.45, 99.9), (3.15, 99.9)), fill_opacity=0.55)","false_name":"a Math [yellow] that says \"$upright(\"false positives\")$\" drawn in piles","false_value":"a Math [yellow] that says \"$99.9$\" drawn in piles","piles":"a Figure (x_range=(0.0, 5.0), y_range=(-18.0, 112.0))","positive_sum":"an Arithmetic [text] that says \"$9.9 99.9 109.8$\" (operator='+', operands=('9.9', '99.9'), result='109.8')","posterior":"a Math [text] that says \"$P(D | +) = frac(9.9, 109.8)$\"","prompt":"a Tex [text] that says \"Gather everyone whose test is positive.\"","true_bar":"a Polygon [green] drawn in piles (vertices=((0.55, 0.0), (1.85, 0.0), (1.85, 9.9), (0.55, 9.9)), fill_opacity=0.75)","true_name":"a Math [green] that says \"$upright(\"true positives\")$\" drawn in piles","true_value":"a Math [green] that says \"$9.9$\" drawn in piles","wrong":"a Math [text] that says \"$P(D^c | +) approx 91.0%$\""},"beats":[{"start":131.74064583333333,"say":"The test has now run on all ten thousand people. But after a positive result, most of that original cohort is no longer relevant. We need a new reference group: everyone whose result was positive.","live":[],"does":[[131.74064583333333,"piles is shown on the screen, written out."],[131.74064583333333,"baseline is shown on the screen, written out."],[135.61864583333335,"piles moves to a new place on the board."],[135.61864583333335,"prompt is shown on the screen, written out."]]},{"start":145.07664583333334,"say":"The green pile contains the positive results from people who truly have the disease. Its expected size is nine point nine. These are the true positives.","live":["piles","prompt","baseline"],"does":[[145.58764583333334,"true_bar is shown on the screen, written out."],[151.86864583333335,"true_value is shown on the screen, written out."],[153.64464583333333,"true_name is shown on the screen, written out."]]},{"start":155.53364583333334,"say":"The yellow pile contains positive results from healthy people. Its expected size is ninety-nine point nine. These are false positives, contributed by the enormous healthy majority.","live":["piles","prompt","baseline","true_bar","true_value","true_name"],"does":[[156.11464583333333,"false_bar is shown on the screen, written out."],[160.85064583333332,"false_value is shown on the screen, written out."],[162.99964583333332,"false_name is shown on the screen, written out."]]},{"start":167.61614583333335,"say":"Pause on the picture. The test is excellent inside either group. Yet the false-positive pile is about ten times taller, because the healthy group supplying it began nine hundred ninety-nine times larger than the disease group.","live":["piles","prompt","baseline","true_bar","true_value","true_name","false_bar","false_value","false_name"],"does":[[170.37964583333334,"true_bar is indicated — a transient flash."],[174.50064583333335,"false_bar is indicated — a transient flash."],[176.11464583333333,"false_name is indicated — a transient flash."]]},{"start":182.04414583333335,"say":"Now gather the two piles. Write nine point nine true positives above ninety-nine point nine false positives. Rule beneath them and add. The positive-test group contains about one hundred nine point eight people.","live":null,"does":[[185.13264583333333,"positive_sum is shown on the screen, written out."],[187.43164583333333,"positive_sum is shown on the screen, written out."],[190.33364583333332,"positive_sum is shown on the screen, drawn."],[191.13364583333333,"positive_sum is shown on the screen, drawn."],[193.10864583333333,"positive_sum is indicated — a transient flash."],[195.12864583333334,"positive_sum is shown on the screen, written out."]]},{"start":197.87664583333333,"say":"To answer our question, ask what fraction of that positive group came from the green pile. The numerator is nine point nine true positives. The denominator is every positive result, one hundred nine point eight.","live":null,"does":[[200.36064583333334,"posterior is shown on the screen, written out."],[204.04164583333332,"posterior (the \"9.9\" part) is emphasized."],[207.39664583333334,"posterior (the \"109.8\" part) is emphasized."],[207.39664583333334,"posterior (the \"9.9\" part) is no longer emphasized."],[211.54164583333332,"posterior (the \"109.8\" part) is no longer emphasized."]]},{"start":212.1416458333333,"say":"That fraction is about nine percent. So after one positive test, the chance of actually having the disease is only about nine percent under our assumptions.","live":["posterior","piles","prompt","baseline","true_bar","true_value","true_name","false_bar","false_value","false_name"],"does":[[213.80164583333334,"posterior becomes \"$P(D | +) = frac(9.9, 109.8) approx 9.0%$\"."],[222.38164583333332,"A box is drawn around posterior."]]},{"start":222.98164583333335,"say":"The complementary probability is about ninety-one percent. In other words, this positive result is probably wrong, even though the test has ninety-nine percent sensitivity and ninety-nine percent specificity.","live":null,"does":[[225.5936458333333,"wrong is shown on the screen, written out."],[225.5936458333333,"wrong (the \"91.0%\" part) is emphasized."],[236.42514583333332,"wrong (the \"91.0%\" part) is no longer emphasized."]]},{"start":237.02514583333334,"say":"Nothing paradoxical happened. The test made errors on only one percent of healthy people. There were simply so many healthy people that their small error rate produced far more positive results than the rare disease did.","live":["posterior","wrong","piles","prompt","baseline","true_bar","true_value","true_name","false_bar","false_value","false_name"],"does":[[244.38664583333332,"false_bar is indicated — a transient flash."],[250.71064583333333,"piles is hidden from the screen — left the board."],[250.71064583333333,"baseline is hidden from the screen — piles left the board."],[250.71064583333333,"true_bar is hidden from the screen — piles left the board."],[250.71064583333333,"true_value is hidden from the screen — piles left the board."],[250.71064583333333,"true_name is hidden from the screen — piles left the board."],[250.71064583333333,"false_bar is hidden from the screen — piles left the board."],[250.71064583333333,"false_value is hidden from the screen — piles left the board."],[250.71064583333333,"false_name is hidden from the screen — piles left the board."],[250.71064583333333,"positive_sum is hidden from the screen — left the board."],[250.71064583333333,"posterior is hidden from the screen — left the board."],[250.71064583333333,"prompt is hidden from the screen — left the board."],[250.71064583333333,"wrong is hidden from the screen — left the board."]]}]},{"title":"Bayes Names the Count","start":251.75231250000002,"end":385.5296875,"objects":{"bayes_work":"a Derivation [text] that says \"$P(D | +) &= frac(upright(\"true positives\"), upright(\"true positives\") + upright(\"false positives\")) \\ &= frac(P(+ | D) P(D), P(+ | D) P(D) + P(+ | D^c) P(D^c)) \\ &= frac(0.99 dot.op 0.001, 0.99 dot.op 0.001 + 0.01 dot.op 0.999) \\ &approx 0…$\"","distinction":"a Text [text] that says \"$P(+|D)$ asks about test results among diseased people. $P(D|+)$ asks about disease among positive results.\"","heading":"a Heading that says \"The Same Count, Written Generally\"","terms":"a Table [text] that says \"Term Meaning Here $P(D)$ prevalence $0.001$ $P(+|D)$ sensitivity $0.99$ $P(D^c)$ healthy share $0.999$ $P(+|D^c)$ false-positive rate $0.01$\" (rows=(('Term', 'Meaning', 'Here'), ('$P(D)$', 'prevalence', '$0.001$…, header=True)"},"beats":[{"start":251.75231250000002,"say":"Now that the populations are visible, we can compress the same reasoning into Bayes' theorem. Start with the rule we already used: true positives divided by all positive results.","live":[],"does":[[251.75231250000002,"heading is shown on the screen, written out."],[260.01831250000004,"bayes_work is shown on the screen, written out."]]},{"start":264.0438125,"say":"The vertical bar means given. P of D given positive asks: among people known to have a positive result, what fraction have disease? The phrase after the bar names the reference group.","live":["heading"],"does":[[265.72731250000004,"bayes_work (the \"D | +\" part) is emphasized."],[276.4313125,"bayes_work (the \"D | +\" part) is no longer emphasized."]]},{"start":278.1223125,"say":"Before building the formula, name its ingredients. P of D is prevalence, the fraction who have the disease before testing. Here it is zero point zero zero one.","live":null,"does":[[280.81631250000004,"terms is shown on the screen, written out."],[282.8943125,"terms is shown on the screen, written out."],[282.8943125,"terms (the \"row=2\" part) is emphasized."],[290.13881250000003,"terms (the \"row=2\" part) is no longer emphasized."]]},{"start":290.7388125,"say":"P of positive given D is sensitivity, the positive rate inside the disease group. Here it is zero point nine nine.","live":null,"does":[[293.2693125,"terms is shown on the screen, written out."],[293.2693125,"terms (the \"row=3\" part) is emphasized."],[300.1423125,"terms (the \"row=3\" part) is no longer emphasized."]]},{"start":300.7423125,"say":"P of D complement is the healthy share, zero point nine nine nine. And P of positive given D complement is the false-positive rate, zero point zero one.","live":null,"does":[[302.60031250000003,"terms is shown on the screen, written out."],[302.60031250000003,"terms (the \"row=4\" part) is emphasized."],[309.2643125,"terms is shown on the screen, written out."],[309.2643125,"terms (the \"row=4\" part) is no longer emphasized."],[309.2643125,"terms (the \"row=5\" part) is emphasized."],[312.5968125,"terms (the \"row=5\" part) is no longer emphasized."]]},{"start":313.1968125,"say":"Now replace each pile by the probability that creates it. Sensitivity times prevalence creates the true-positive share. False-positive rate times healthy share creates the false-positive share.","live":null,"does":[[313.90531250000004,"bayes_work is shown on the screen, written out."],[317.5853125,"bayes_work (the \"P(+ | D) P(D)\" part) is emphasized."],[321.6953125,"bayes_work (the \"P(+ | D) P(D)\" part) is no longer emphasized."],[321.6953125,"bayes_work (the \"P(+ | D^c) P(D^c)\" part) is emphasized."],[327.1518125,"bayes_work (the \"P(+ | D^c) P(D^c)\" part) is no longer emphasized."]]},{"start":327.7518125,"say":"The numerator keeps the true-positive route. The denominator adds both routes into the positive group. This is exactly what the two visible piles did, with the common population size canceled out.","live":null,"does":[[328.1353125,"bayes_work (the \"P(+ | D) P(D)\" part) is emphasized."],[330.7943125,"bayes_work (the \"P(+ | D) P(D)\" part) is no longer emphasized."],[330.7943125,"bayes_work (the \"P(+ | D) P(D) + P(+ | D^c) P(D^c)\" part) is emphasized."],[340.3608125,"bayes_work (the \"P(+ | D) P(D) + P(+ | D^c) P(D^c)\" part) is no longer emphasized."]]},{"start":340.9608125,"say":"Substitute our values. The true-positive route is zero point nine nine times zero point zero zero one. The false-positive route is zero point zero one times zero point nine nine nine.","live":null,"does":[[341.3673125,"bayes_work is shown on the screen, written out."],[342.3773125,"terms (the \"column=3\" part) is emphasized."],[354.7773125,"terms (the \"column=3\" part) is no longer emphasized."]]},{"start":355.3773125,"say":"The result is about zero point zero nine zero, or nine percent. Bayes' theorem has not introduced a new argument. It has merely named the count in a form that works for any cohort size.","live":null,"does":[[355.9693125,"bayes_work is shown on the screen, written out."],[359.1733125,"A box is drawn around bayes_work."]]},{"start":368.4348125,"say":"The common mistake is to reverse the condition. Ninety-nine percent sensitivity describes positive results among people already known to have disease. We wanted disease among people already known to have a positive result. Those are different questions.","live":null,"does":[[369.9213125,"distinction is shown on the screen, written out."],[372.7883125,"distinction (the \"$P(+|D)$\" part) is emphasized."],[378.7443125,"distinction (the \"$P(D|+)$\" part) is emphasized."],[384.23802083333334,"distinction (the \"$P(+|D)$\" part) is no longer emphasized."],[384.23802083333334,"distinction (the \"$P(D|+)$\" part) is no longer emphasized."],[384.48802083333334,"bayes_work is hidden from the screen — left the board."],[384.48802083333334,"distinction is hidden from the screen — left the board."],[384.48802083333334,"heading is hidden from the screen — left the board."],[384.48802083333334,"terms is hidden from the screen — left the board."]]}]},{"title":"Watch the Answer Swing","start":385.5296875,"end":535.6572708333333,"objects":{"accuracy":"a VariableNumber (initial_value=99.0, format_spec='.1f')","accuracy_axes":"an Axes (x_range=(90.0, 99.95), y_range=(0.0, 70.0), x_ticks_every=2.0)","accuracy_curve":"a FunctionPlot [blue] drawn in accuracy_axes (function=<function>, x_range=(90.0, 99.95))","accuracy_formula":"a Math [text] that says \"$P(D | +) = frac(a p, a p + (1-a)(1-p))$\"","accuracy_heading":"a Heading that says \"Change the Accuracy\"","accuracy_note":"a Text [text] that says \"Here a is both sensitivity and specificity, while prevalence stays at 0.1%.\"","accuracy_point":"a PlotPoint [yellow] labelled \"(99.0, 9.0)\" drawn in accuracy_axes (target='accuracy_curve', x=<VariableNumber accuracy = 99.9>)","p":"a VariableNumber (initial_value=0.1, format_spec='.1f')","point":"a Point [yellow] drawn in prevalence_axes (location=(1.0, 50.0))","posterior_a":"a VariableNumber (initial_value=9.0, format_spec='.1f')","posterior_p":"a VariableNumber (initial_value=9.0, format_spec='.1f')","prevalence_axes":"an Axes (x_range=(0.0, 10.0), y_range=(0.0, 100.0), x_ticks_every=1.0)","prevalence_curve":"a FunctionPlot [blue] drawn in prevalence_axes (function=<function>, x_range=(0.05, 10.0))","prevalence_formula":"a Math [text] that says \"$P(D | +) = frac(0.99 p, 0.99 p + 0.01 (1-p))$\"","prevalence_heading":"a Heading that says \"Change the Prevalence\"","prevalence_note":"a Text [text] that says \"The test stays fixed. Only the fraction of people with disease changes.\"","prevalence_point":"a PlotPoint [yellow] labelled \"(0.1, 9.0)\" drawn in prevalence_axes (target='prevalence_curve', x=<VariableNumber p = 10.0>)"},"beats":[{"start":385.5296875,"say":"Bayes' formula lets us change one ingredient at a time. First keep the test fixed at ninety-nine percent sensitivity and specificity, and vary only the disease prevalence.","live":[],"does":[[385.5296875,"prevalence_heading is shown on the screen, written out."],[385.94768750000003,"prevalence_formula is shown on the screen, written out."],[390.4056875,"prevalence_note is shown on the screen, written out."],[394.5856875,"prevalence_axes is shown on the screen, written out."],[394.5856875,"prevalence_curve is shown on the screen, drawn."]]},{"start":397.5306875,"say":"The horizontal coordinate is prevalence as a percentage. The vertical coordinate is the chance of disease after a positive result. At zero point one percent prevalence, our yellow point reads about nine percent.","live":["prevalence_formula","prevalence_note","prevalence_axes","prevalence_heading","prevalence_curve"],"does":[[408.5946875,"prevalence_point is shown on the screen, written out."],[409.6866875,"prevalence_point is indicated — a transient flash."]]},{"start":411.43568750000003,"say":"Raise prevalence to one percent. Now one person in a hundred has the disease before testing. The point climbs to fifty percent, because the expected true-positive and false-positive piles are equal.","live":["prevalence_formula","prevalence_note","prevalence_axes","prevalence_heading","prevalence_curve","prevalence_point"],"does":[[412.8756875,"prevalence_point is redrawn as the numbers it depends on change."],[412.8756875,"p ticks to 1.0."],[412.8756875,"posterior_p ticks to 50.0."],[418.94668750000005,"point is shown on the screen, grown."],[420.94668750000005,"point is hidden from the screen."]]},{"start":424.6436875,"say":"Raise prevalence to five percent. The test has not improved at all, but the positive result now means about eighty-three point nine percent.","live":null,"does":[[426.0836875,"prevalence_point is redrawn as the numbers it depends on change."],[426.0836875,"p ticks to 5.0."],[426.0836875,"posterior_p ticks to 83.9."]]},{"start":433.8696875,"say":"At ten percent prevalence, the posterior reaches about ninety-one point seven percent. The same test result means something very different in a high-risk population than in a low-risk population.","live":null,"does":[[434.4266875,"prevalence_point is redrawn as the numbers it depends on change."],[434.4266875,"p ticks to 10.0."],[434.4266875,"posterior_p ticks to 91.7."],[441.5666875,"prevalence_point is indicated — a transient flash."]]},{"start":446.28868750000004,"say":"Prevalence is the starting information, sometimes called the prior probability. A positive test updates that starting point. It does not erase it.","live":null,"does":[[446.6366875,"prevalence_formula (the \"p\" part) is emphasized."],[456.30818750000003,"prevalence_axes is hidden from the screen — left the board."],[456.30818750000003,"prevalence_curve is hidden from the screen — prevalence_axes left the board."],[456.30818750000003,"prevalence_point is hidden from the screen — prevalence_axes left the board."],[456.30818750000003,"prevalence_formula is hidden from the screen — left the board."],[456.30818750000003,"prevalence_heading is hidden from the screen — left the board."],[456.30818750000003,"prevalence_note is hidden from the screen — left the board."],[456.30818750000003,"prevalence_formula (the \"p\" part) is no longer emphasized."]]},{"start":457.5081875,"say":"Now restore the very rare prevalence of zero point one percent and change the test itself. To keep the phrase accuracy unambiguous, a will mean both sensitivity and specificity.","live":[],"does":[[457.5081875,"accuracy_heading is shown on the screen, written out."],[461.89668750000004,"accuracy_axes is shown on the screen, written out."],[461.89668750000004,"accuracy_curve is shown on the screen, drawn."],[466.74968750000005,"accuracy_axes moves to a new place on the board."],[466.74968750000005,"accuracy_formula is shown on the screen, written out."],[467.3186875,"accuracy_note is shown on the screen, written out."]]},{"start":470.26418750000005,"say":"At ninety-nine percent accuracy, the point again sits near nine percent. Its label gives accuracy first and the posterior probability second.","live":["accuracy_formula","accuracy_note","accuracy_axes","accuracy_heading","accuracy_curve"],"does":[[472.8996875,"accuracy_point is shown on the screen, written out."],[474.0836875,"accuracy_point is indicated — a transient flash."]]},{"start":480.68568750000003,"say":"Drop accuracy to ninety-five percent. The posterior falls below two percent. A five percent false-positive rate applied to nearly ten thousand healthy people overwhelms the true-positive pile.","live":["accuracy_formula","accuracy_note","accuracy_axes","accuracy_heading","accuracy_curve","accuracy_point"],"does":[[482.1606875,"accuracy_point is redrawn as the numbers it depends on change."],[482.1606875,"accuracy ticks to 95.0."],[482.1606875,"posterior_a ticks to 1.9."]]},{"start":493.9176875,"say":"Return to ninety-nine percent, and we recover about nine percent. Now push the accuracy to ninety-nine point nine percent. The false-positive rate falls from one percent to one tenth of one percent.","live":null,"does":[[494.8116875,"accuracy_point is redrawn as the numbers it depends on change."],[494.8116875,"accuracy ticks to 99.0."],[494.8116875,"posterior_a ticks to 9.0."],[499.81568749999997,"accuracy_point is redrawn as the numbers it depends on change."],[499.81568749999997,"accuracy ticks to 99.9."],[499.81568749999997,"posterior_a ticks to 50.0."]]},{"start":507.3351875,"say":"That extra nine in the accuracy raises the posterior to about fifty percent. For an extremely rare disease, tiny changes in the false-positive rate can matter enormously because that rate acts on the healthy majority.","live":null,"does":[[511.0976875,"accuracy_point is indicated — a transient flash."],[519.5956874999999,"accuracy_formula (the \"(1-a)(1-p)\" part) is emphasized."],[521.1051875,"accuracy_formula (the \"(1-a)(1-p)\" part) is no longer emphasized."]]},{"start":521.7051875,"say":"So the phrase ninety-nine percent accurate is incomplete on its own. We need sensitivity, specificity, and prevalence. Change any one of them and the meaning of a positive result can swing dramatically.","live":null,"does":[[534.6156041666667,"accuracy_axes is hidden from the screen — left the board."],[534.6156041666667,"accuracy_curve is hidden from the screen — accuracy_axes left the board."],[534.6156041666667,"accuracy_point is hidden from the screen — accuracy_axes left the board."],[534.6156041666667,"accuracy_formula is hidden from the screen — left the board."],[534.6156041666667,"accuracy_heading is hidden from the screen — left the board."],[534.6156041666667,"accuracy_note is hidden from the screen — left the board."]]}]},{"title":"Test Again","start":535.6572708333333,"end":695.2673125,"objects":{"after_false":"a Polygon [yellow] drawn in filter_picture (vertices=((6.75, 0.0), (7.85, 0.0), (7.85, 0.999), (6.75, 0.999)), fill_opacity=0.65)","after_false_value":"a Math [yellow] that says \"$0.999$\" drawn in filter_picture","after_label":"a Math [text] that says \"$upright(\"positive twice\")$\" drawn in filter_picture","after_true":"a Polygon [green] drawn in filter_picture (vertices=((5.15, 0.0), (6.25, 0.0), (6.25, 9.801), (5.15, 9.801)), fill_opacity=0.78)","after_true_value":"a Math [green] that says \"$9.801$\" drawn in filter_picture","baseline":"a Line [gray] drawn in filter_picture (start=(0.25, 0.0), end=(8.75, 0.0))","before_false":"a Polygon [yellow] drawn in filter_picture (vertices=((2.15, 0.0), (3.25, 0.0), (3.25, 99.9), (2.15, 99.9)), fill_opacity=0.24)","before_false_value":"a Math [yellow] that says \"$99.9$\" drawn in filter_picture","before_label":"a Math [gray] that says \"$upright(\"after one positive\")$\" drawn in filter_picture","before_line":"a Math [text] that says \"$upright(\"one positive\"): thin 9.9 thin upright(\"true\"), thin 99.9 thin upright(\"false\")$\"","before_true":"a Polygon [green] drawn in filter_picture (vertices=((0.55, 0.0), (1.65, 0.0), (1.65, 9.9), (0.55, 9.9)), fill_opacity=0.28)","before_true_value":"a Math [green] that says \"$9.9$\" drawn in filter_picture","filter_picture":"a Figure (x_range=(0.0, 9.0), y_range=(-18.0, 112.0), aspect=(9.0, 4.5))","heading":"a Heading that says \"A Second Independent Positive Test\"","independence":"a Text [text] that says \"Assumption: given disease status, the two test errors are independent.\"","odds_heading":"a Heading that says \"Why Two Positives Are So Much Stronger\"","odds_result":"a Math [text] that says \"$frac(9801, 9801 + 999) approx 90.8%$\"","odds_work":"a Derivation [text] that says \"$upright(\"prior odds\") &= frac(10,9990) = frac(1,999) \\ upright(\"positive likelihood ratio\") &= frac(0.99,0.01) = 99 \\ upright(\"after one positive\") &= frac(1,999) dot.op 99 \\ upright(\"after two positives\") &= frac(1,999) dot.op 99 dot.op 99$\"","second_false":"a Math [text] that says \"$99.9 dot.op 0.01 = 0.999$\"","second_fraction":"a Math [text] that says \"$P(D | +,+) = frac(9.801, 9.801 + 0.999)$\"","second_posterior":"a Math [text] that says \"$P(D | +,+) approx 90.8%$\"","second_true":"a Math [text] that says \"$9.9 dot.op 0.99 = 9.801$\""},"beats":[{"start":535.6572708333333,"say":"Suppose the same person is tested again and the second result is also positive. Begin with the group that survived the first test: about nine point nine true positives and ninety-nine point nine false positives.","live":[],"does":[[535.6572708333333,"heading is shown on the screen, written out."],[535.6572708333333,"filter_picture is shown on the screen, written out."],[535.6572708333333,"baseline is shown on the screen, written out."],[540.5912708333333,"filter_picture moves to a new place on the board."],[540.5912708333333,"before_line is shown on the screen, written out."],[542.2752708333334,"before_label is shown on the screen, written out."],[543.7142708333333,"before_true is shown on the screen, written out."],[543.7142708333333,"before_true_value is shown on the screen, written out."],[545.8392708333333,"before_false is shown on the screen, written out."],[545.8392708333333,"before_false_value is shown on the screen, written out."]]},{"start":548.7787708333333,"say":"Assume the second test is conditionally independent of the first. Among the people who truly have disease, it again detects ninety-nine percent. Ninety-nine percent of nine point nine is nine point eight zero one.","live":["before_line","filter_picture","heading","baseline","before_true","before_true_value","before_false","before_false_value","before_label"],"does":[[549.4232708333333,"after_label is shown on the screen, written out."],[555.6342708333333,"second_true is shown on the screen, written out."],[559.6282708333333,"after_true is shown on the screen, written out."],[559.6282708333333,"after_true_value is shown on the screen, written out."]]},{"start":562.0622708333333,"say":"Among the healthy people who produced the first false positive, only one percent produce another false positive independently. One percent of ninety-nine point nine is zero point nine nine nine.","live":["before_line","second_true","filter_picture","heading","baseline","before_true","before_true_value","before_false","before_false_value","before_label","after_true","after_true_value","after_label"],"does":[[565.8472708333334,"second_false is shown on the screen, written out."],[570.9672708333333,"after_false is shown on the screen, written out."],[570.9672708333333,"after_false_value is shown on the screen, written out."]]},{"start":573.2852708333334,"say":"Now read the two surviving piles. About nine point eight people are true positives twice, while about one person is falsely positive twice. The green pile is finally much larger than the yellow pile.","live":["before_line","second_true","second_false","filter_picture","heading","baseline","before_true","before_true_value","before_false","before_false_value","before_label","after_true","after_true_value","after_label","after_false","after_false_value"],"does":[[582.6892708333334,"after_true is indicated — a transient flash."],[584.5692708333333,"after_false is indicated — a transient flash."]]},{"start":586.2607708333333,"say":"The probability of disease after two positive results is the green count divided by the two surviving counts together.","live":null,"does":[[586.8302708333333,"second_fraction is shown on the screen, written out."],[589.9642708333333,"second_fraction (the \"9.801\" part) is emphasized."],[592.1122708333334,"second_fraction (the \"9.801\" part) is no longer emphasized."],[592.1122708333334,"second_fraction (the \"9.801 + 0.999\" part) is emphasized."],[592.9482708333333,"second_fraction (the \"9.801 + 0.999\" part) is no longer emphasized."]]},{"start":593.5482708333333,"say":"That is about ninety point eight percent. One independent repeat test has moved the answer from about nine percent to about ninety-one percent by filtering both piles again.","live":["before_line","second_true","second_false","second_fraction","filter_picture","heading","baseline","before_true","before_true_value","before_false","before_false_value","before_label","after_true","after_true_value","after_label","after_false","after_false_value"],"does":[[594.4662708333333,"second_posterior is shown on the screen, written out."],[602.4532708333334,"after_true is indicated — a transient flash."],[602.7202708333333,"after_false is indicated — a transient flash."],[603.9047708333333,"before_line is hidden from the screen — left the board."],[603.9047708333333,"filter_picture is hidden from the screen — left the board."],[603.9047708333333,"baseline is hidden from the screen — filter_picture left the board."],[603.9047708333333,"before_true is hidden from the screen — filter_picture left the board."],[603.9047708333333,"before_true_value is hidden from the screen — filter_picture left the board."],[603.9047708333333,"before_false is hidden from the screen — filter_picture left the board."],[603.9047708333333,"before_false_value is hidden from the screen — filter_picture left the board."],[603.9047708333333,"before_label is hidden from the screen — filter_picture left the board."],[603.9047708333333,"after_true is hidden from the screen — filter_picture left the board."],[603.9047708333333,"after_true_value is hidden from the screen — filter_picture left the board."],[603.9047708333333,"after_label is hidden from the screen — filter_picture left the board."],[603.9047708333333,"after_false is hidden from the screen — filter_picture left the board."],[603.9047708333333,"after_false_value is hidden from the screen — filter_picture left the board."],[603.9047708333333,"heading is hidden from the screen — left the board."],[603.9047708333333,"second_false is hidden from the screen — left the board."],[603.9047708333333,"second_fraction is hidden from the screen — left the board."],[603.9047708333333,"second_posterior is hidden from the screen — left the board."],[603.9047708333333,"second_true is hidden from the screen — left the board."],[603.9047708333333,"A box is drawn around second_posterior."]]},{"start":604.5047708333333,"say":"There is a compact way to understand that jump. Start with disease odds of ten to nine thousand nine hundred ninety, which reduce to one to nine hundred ninety-nine.","live":[],"does":[[604.5047708333333,"odds_heading is shown on the screen, written out."],[606.8262708333333,"independence is shown on the screen, written out."],[608.7072708333333,"odds_work is shown on the screen, written out."]]},{"start":614.8682708333333,"say":"A positive result is ninety-nine times more likely when disease is present than when it is absent. That factor, ninety-nine, is called the positive likelihood ratio.","live":["independence","odds_heading"],"does":[[616.4592708333333,"odds_work is shown on the screen, written out."],[621.1372708333333,"odds_work (the \"99\" part) is emphasized."],[625.3982708333333,"odds_work (the \"99\" part) is no longer emphasized."]]},{"start":625.9982708333333,"say":"The first positive result multiplies the prior odds by ninety-nine. That produces the same roughly nine percent probability we found from the first two piles.","live":null,"does":[[627.6582708333333,"odds_work is shown on the screen, written out."],[629.2262708333333,"odds_work (the \"99\" part) is emphasized."],[635.2862708333333,"odds_work (the \"99\" part) is no longer emphasized."]]},{"start":635.8862708333334,"say":"Under conditional independence, the second positive multiplies by the same factor again. Two positives contribute ninety-nine times ninety-nine, changing the odds by a factor of nine thousand eight hundred one.","live":null,"does":[[637.7672708333333,"odds_work is shown on the screen, written out."],[642.5732708333333,"odds_work (the \"99 dot.op 99\" part) is emphasized."],[647.7742708333333,"odds_work (the \"99 dot.op 99\" part) is no longer emphasized."]]},{"start":648.3742708333333,"say":"Convert those final odds back to a probability and we recover ninety point eight percent. The count method and the odds method are two views of the same update.","live":null,"does":[[650.4762708333333,"odds_result is shown on the screen, written out."],[652.1012708333333,"A box is drawn around odds_result."]]},{"start":659.0287708333333,"say":"The independence assumption matters. If both tests use the same sample, the same instrument, or the same biological signal, their errors may be correlated. A repeated error can then be more likely than this calculation assumes, so the second positive may add less evidence.","live":["independence","odds_result","odds_heading"],"does":[[659.5632708333333,"independence (the \"independent\" part) is emphasized."],[675.7942708333333,"independence (the \"independent\" part) is no longer emphasized."]]},{"start":676.3942708333333,"say":"The lesson is not to distrust accurate tests. It is to ask the complete question. How rare is the disease? What are the sensitivity and specificity? And is new evidence genuinely independent? With those facts, a surprising positive result becomes a count we can understand.","live":null,"does":[[680.3532708333332,"odds_result is indicated — a transient flash."],[694.2256458333334,"independence is hidden from the screen — left the board."],[694.2256458333334,"odds_heading is hidden from the screen — left the board."],[694.2256458333334,"odds_result is hidden from the screen — left the board."],[694.2256458333334,"odds_work is hidden from the screen — left the board."]]}]}]},"durationSeconds":695,"chapters":[{"title":"Ten Thousand People","startSeconds":0,"narration":"Suppose a medical test is described as ninety-nine percent accurate, and it comes back positive. That sounds almost conclusive. But if the disease is rare, the positive result can still be more likely wrong than right. We are going to see why by counting people before writing any probability formula. Here is the question in its most personal form. A positive result says you have a rare disease. Does ninety-nine percent accurate mean there is a ninety-nine percent chance you have it? No. That number describes how the test behaves inside known groups. It does not yet answer what group a positive person probably came from. Take ten thousand people. The large gray field represents the people in this cohort who do not have the disease. I have magnified the affected people so we can actually see them. Let the disease affect one person in a thousand. That is a prevalence of zero point one percent. In ten thousand people, only ten actually have the disease. The remaining nine thousand nine hundred ninety are healthy. Now we must say exactly what ninety-nine percent accurate means. For this lecture, it means two things. Sensitivity is ninety-nine percent, so among people who truly have the disease, the test is positive ninety-nine percent of the time. Specificity is also ninety-nine percent, so among healthy people, the test is negative ninety-nine percent of the time. Apply sensitivity to the ten affected people. Ninety-nine percent of ten is nine point nine. So across many cohorts like this one, we expect about nine point nine true positive results. Now turn to the healthy majority. Ninety-nine percent specificity leaves a one percent false-positive rate. One percent sounds tiny, but it acts on nine thousand nine hundred ninety people. One percent of that enormous healthy group is ninety-nine point nine false positives. The decimal counts are expected counts, averages over many equally sized cohorts. In one real cohort we would see whole people, very close to these values."},{"title":"Read the Positive Pile","startSeconds":131.74064583333333,"narration":"The test has now run on all ten thousand people. But after a positive result, most of that original cohort is no longer relevant. We need a new reference group: everyone whose result was positive. The green pile contains the positive results from people who truly have the disease. Its expected size is nine point nine. These are the true positives. The yellow pile contains positive results from healthy people. Its expected size is ninety-nine point nine. These are false positives, contributed by the enormous healthy majority. Pause on the picture. The test is excellent inside either group. Yet the false-positive pile is about ten times taller, because the healthy group supplying it began nine hundred ninety-nine times larger than the disease group. Now gather the two piles. Write nine point nine true positives above ninety-nine point nine false positives. Rule beneath them and add. The positive-test group contains about one hundred nine point eight people. To answer our question, ask what fraction of that positive group came from the green pile. The numerator is nine point nine true positives. The denominator is every positive result, one hundred nine point eight. That fraction is about nine percent. So after one positive test, the chance of actually having the disease is only about nine percent under our assumptions. The complementary probability is about ninety-one percent. In other words, this positive result is probably wrong, even though the test has ninety-nine percent sensitivity and ninety-nine percent specificity. Nothing paradoxical happened. The test made errors on only one percent of healthy people. There were simply so many healthy people that their small error rate produced far more positive results than the rare disease did."},{"title":"Bayes Names the Count","startSeconds":251.75231250000002,"narration":"Now that the populations are visible, we can compress the same reasoning into Bayes' theorem. Start with the rule we already used: true positives divided by all positive results. The vertical bar means given. P of D given positive asks: among people known to have a positive result, what fraction have disease? The phrase after the bar names the reference group. Before building the formula, name its ingredients. P of D is prevalence, the fraction who have the disease before testing. Here it is zero point zero zero one. P of positive given D is sensitivity, the positive rate inside the disease group. Here it is zero point nine nine. P of D complement is the healthy share, zero point nine nine nine. And P of positive given D complement is the false-positive rate, zero point zero one. Now replace each pile by the probability that creates it. Sensitivity times prevalence creates the true-positive share. False-positive rate times healthy share creates the false-positive share. The numerator keeps the true-positive route. The denominator adds both routes into the positive group. This is exactly what the two visible piles did, with the common population size canceled out. Substitute our values. The true-positive route is zero point nine nine times zero point zero zero one. The false-positive route is zero point zero one times zero point nine nine nine. The result is about zero point zero nine zero, or nine percent. Bayes' theorem has not introduced a new argument. It has merely named the count in a form that works for any cohort size. The common mistake is to reverse the condition. Ninety-nine percent sensitivity describes positive results among people already known to have disease. We wanted disease among people already known to have a positive result. Those are different questions."},{"title":"Watch the Answer Swing","startSeconds":385.5296875,"narration":"Bayes' formula lets us change one ingredient at a time. First keep the test fixed at ninety-nine percent sensitivity and specificity, and vary only the disease prevalence. The horizontal coordinate is prevalence as a percentage. The vertical coordinate is the chance of disease after a positive result. At zero point one percent prevalence, our yellow point reads about nine percent. Raise prevalence to one percent. Now one person in a hundred has the disease before testing. The point climbs to fifty percent, because the expected true-positive and false-positive piles are equal. Raise prevalence to five percent. The test has not improved at all, but the positive result now means about eighty-three point nine percent. At ten percent prevalence, the posterior reaches about ninety-one point seven percent. The same test result means something very different in a high-risk population than in a low-risk population. Prevalence is the starting information, sometimes called the prior probability. A positive test updates that starting point. It does not erase it. Now restore the very rare prevalence of zero point one percent and change the test itself. To keep the phrase accuracy unambiguous, a will mean both sensitivity and specificity. At ninety-nine percent accuracy, the point again sits near nine percent. Its label gives accuracy first and the posterior probability second. Drop accuracy to ninety-five percent. The posterior falls below two percent. A five percent false-positive rate applied to nearly ten thousand healthy people overwhelms the true-positive pile. Return to ninety-nine percent, and we recover about nine percent. Now push the accuracy to ninety-nine point nine percent. The false-positive rate falls from one percent to one tenth of one percent. That extra nine in the accuracy raises the posterior to about fifty percent. For an extremely rare disease, tiny changes in the false-positive rate can matter enormously because that rate acts on the healthy majority. So the phrase ninety-nine percent accurate is incomplete on its own. We need sensitivity, specificity, and prevalence. Change any one of them and the meaning of a positive result can swing dramatically."},{"title":"Test Again","startSeconds":535.6572708333333,"narration":"Suppose the same person is tested again and the second result is also positive. Begin with the group that survived the first test: about nine point nine true positives and ninety-nine point nine false positives. Assume the second test is conditionally independent of the first. Among the people who truly have disease, it again detects ninety-nine percent. Ninety-nine percent of nine point nine is nine point eight zero one. Among the healthy people who produced the first false positive, only one percent produce another false positive independently. One percent of ninety-nine point nine is zero point nine nine nine. Now read the two surviving piles. About nine point eight people are true positives twice, while about one person is falsely positive twice. The green pile is finally much larger than the yellow pile. The probability of disease after two positive results is the green count divided by the two surviving counts together. That is about ninety point eight percent. One independent repeat test has moved the answer from about nine percent to about ninety-one percent by filtering both piles again. There is a compact way to understand that jump. Start with disease odds of ten to nine thousand nine hundred ninety, which reduce to one to nine hundred ninety-nine. A positive result is ninety-nine times more likely when disease is present than when it is absent. That factor, ninety-nine, is called the positive likelihood ratio. The first positive result multiplies the prior odds by ninety-nine. That produces the same roughly nine percent probability we found from the first two piles. Under conditional independence, the second positive multiplies by the same factor again. Two positives contribute ninety-nine times ninety-nine, changing the odds by a factor of nine thousand eight hundred one. Convert those final odds back to a probability and we recover ninety point eight percent. The count method and the odds method are two views of the same update. The independence assumption matters. If both tests use the same sample, the same instrument, or the same biological signal, their errors may be correlated. A repeated error can then be more likely than this calculation assumes, so the second positive may add less evidence. The lesson is not to distrust accurate tests. It is to ask the complete question. How rare is the disease? What are the sensitivity and specificity? And is new evidence genuinely independent? With those facts, a surprising positive result becomes a count we can understand."}]}}