{"version":1,"lectureId":"01M14TYJMMAQ9SQDTPSVK4XZGD","attempt":0,"publication":{"slug":"bias-variance-and-the-shape-of-overfitting","title":"Overfitting Made Visible: Bias, Variance, and the Role of Data","subject":"machine-learning","summary":"A visual introduction to overfitting built from one controlled experiment. The same twenty noisy observations are fitted with polynomials from a straight line through an exact degree-nineteen interpolant. Training and test error make both failure modes measurable, repeated fresh samples turn bias and variance into visible behavior, and a final comparison shows how additional data stabilizes flexible models and moves the balance toward greater complexity.","metaDescription":"See underfitting, overfitting, training and test error, bias, variance, and the stabilizing role of additional data in one visual experiment.","transcript":"Overfitting is often used as a warning word: this model overfits, that model generalizes. I want to make the warning visible and measurable. We will give several polynomial models the same twenty noisy observations, then watch each model decide what story those observations tell. Here are the twenty observations. Each yellow dot contains two things: a curved underlying relationship and a small disturbance that pushed the measurement above or below that relationship. Suppose we know only the dots. Our first candidate is a polynomial of degree one, which is simply a straight line. It can tilt and it can move up or down, but it cannot bend. The line catches the broad upward trend, but look at the ends and the middle. The dots curve away from it in an organized pattern. Those are not isolated misses. The model lacks the shape needed to follow the relationship. Allow degree three. The line gives way to a curve, and the fit can now follow the large bend. It still refuses to chase every individual dot. That refusal is useful. A noisy observation is not the relationship itself. Some of the vertical displacement belongs to chance, so a model that leaves small residuals may be behaving more sensibly than one that erases them. Now allow degree seven. The curve gains more freedom. It bends toward several individual observations and begins to reproduce their alternating rises and falls. Finally allow degree nineteen. Twenty coefficients are enough to force a polynomial through twenty distinct points. The red curve touches every yellow dot exactly. On the training observations this looks perfect. Between the observations, however, the curve changes direction again and again. It has treated each upward or downward disturbance as if it were a real feature of the world. Because this is a controlled example, we may reveal the relationship that generated the observations. The blue curve is the signal. The red curve agrees with all twenty recorded values, yet it disagrees repeatedly with the relationship between them. So failure happens in two directions. The straight line is too rigid to represent the signal. The degree-nineteen polynomial is flexible enough to represent both the signal and the accidental noise. One underfits; the other overfits. The central question is therefore not whether a model can fit these observations. It is whether the fitted pattern continues to work at places the fitting process did not get to see. Start with the rigid failure. The blue curve is the underlying relationship, the yellow dots are our original twenty observations, and the green line is the best straight model available to us. Now add many more observations from the same relationship. Eighty gray points make the bend much easier to see, but they do not give a straight line the ability to bend. More data can reveal that this model is wrong, but it cannot repair the model family. These red gaps are systematic. On one stretch the line is too high; on another it is too low. This persistent miss is underfitting. Now look at the flexible failure. The right-hand graph uses the original twenty observations again. The red degree-nineteen polynomial passes through every one of them. If we measure only at those yellow locations, every vertical residual is zero. Its mean squared training error is therefore zero. But now place green observations at new locations between the training points. They follow the same blue relationship and were not available while the polynomial was fitted. The magenta gaps measure predictions at those unseen locations. Several are large because the red curve swings away between neighboring training points. Perfect recall of the training set did not produce reliable prediction. That gives us an operational definition. Overfitting is not simply a complicated-looking curve. It is a measurable gap: very small error on the data used for fitting, paired with substantially larger error on new data. The straight line and the degree-nineteen curve fail for opposite reasons. The line cannot express enough structure. The interpolating curve reacts to structure that was never really present. To compare models with one number, use mean squared error. For each observation, subtract prediction from observation, square that residual, add the squares, and divide by the number of observations. Training mean squared error uses the observations that selected the fitted curve. Test mean squared error repeats the same calculation on observations held aside from that selection. The formulas look nearly identical. Their difference is experimental: one asks how well the model remembers its lesson, and the other asks how well it handles a new question. Now repeat the fitting experiment for degree one, degree two, degree three, and so on up to degree nineteen. For every fitted model, record both errors. The blue curve is training error. It falls as degree increases because each larger polynomial family contains more ways to accommodate the observations. That decrease is built into the fitting process. A more flexible model can imitate a simpler one if that is best, or use its extra freedom to reduce the residuals further. The red curve is test error. At first it also falls. Moving beyond a straight line lets the model capture the real curvature, so predictions improve. Near degree five, test error reaches its minimum. In this experiment that is the useful region: enough flexibility to represent the signal, without enough freedom to chase every disturbance. Past that point, training error keeps falling but test error turns upward. The two measurements separate because extra complexity is helping on observations already seen while hurting predictions elsewhere. At degree nineteen the training point is almost on the horizontal axis. The test point is high. Their vertical separation is the observable signature of the interpolation failure we just watched. Low complexity and high complexity therefore have different diagnoses. On the left, both errors are large because the model misses the signal. On the right, only training error is tiny, because the model has learned details that do not transfer. Selecting the degree with the smallest training error would always reward the most flexible candidate. That is exactly the decision rule that invites overfitting. Instead, reserve data for validation or testing and look for low error there. The marked point is not a law that degree five is always right. It is the answer for this relationship, this noise level, and this amount of data. Change the noise, the sampling locations, or the amount of data, and the red curve can change. That dependence is the doorway to bias and variance. Imagine repeating the entire study. Draw twenty fresh noisy observations from the same relationship, fit the model, discard that sample, and do it again. On the left, each gray line is a straight model fitted to a different fresh sample. Here is the first fit, then the second, the third, the fourth, and the fifth. The fitted lines move a little because the observations change, but they remain a tight family. Their small spread means low variance. Now average the five fitted predictions at every x. The red line is that average. It still misses the blue curve in the same organized way, especially near the ends. That persistent difference is bias. Even if we repeated the study many more times, averaging straight lines would still produce a straight line. Repetition cannot create curvature that the model family forbids. Now repeat the same experiment with a highly flexible polynomial. The blue curve on the right is still the same relationship. Each colored curve is a refit on a new sample of twenty observations. One sample pulls the curve upward here, another pulls it downward, and another creates a different sequence of bends. The model family is capable of following the underlying curve, so its average can be close to the truth. But the individual fits spread widely. Which data set happened to arrive has a large effect on the prediction. That sensitivity is variance. High variance means that if we repeated the data collection, our fitted answer could change substantially even though the data-generating process did not. Bias and variance are therefore not labels attached permanently to one curve. They describe behavior across repeated samples. Bias asks where the average fit lands. Variance asks how widely the individual fits scatter around that average. Bias is the difference between the average fitted prediction and the true relationship. A rigid straight model has substantial bias here because its average prediction cannot reproduce the bend. Variance is the spread of fitted predictions at the same input across fresh samples. A flexible model can have low bias and still have high variance if each sample leads it toward a different detailed curve. Underfitting is the visible face of excessive bias. Overfitting is the visible face of excessive sensitivity to the sample. The useful model must control both. Return to repeated fitting, but change the amount of evidence. On the left, each colored flexible curve was fitted from only twenty observations. The fits disagree because a handful of noisy points can pull a flexible polynomial in noticeably different directions. The blue relationship is fixed; the fitted curves are not. On the right, fit the same degree of polynomial after collecting one hundred observations. Each individual observation now has less power to redirect the entire curve. The colored curves still differ, because noise has not vanished. But their spread is much narrower. More data has reduced variance without changing the basic model family. Notice what more data did not do. It did not make every complex model safe, and it would not make a straight line bend. It made the fitted details less dependent on a few particular observations. We can summarize that change with two error contributions. Blue is error from bias. It falls as complexity gives the model more shapes to choose from. Red is error from variance. With only twenty observations, it rises as flexible models become increasingly sensitive to the sample. The two contributions cross near this yellow point. To the left, missing structure dominates. To the right, sample sensitivity dominates. The crossing is a useful picture of balance, not a universal decision rule. Now keep the blue bias curve and increase the data to one hundred observations. The right-hand graph shows the same declining bias contribution. The red variance contribution is lower because repeated fits are more stable. Its crossing with bias moves to the right, toward a more flexible polynomial. This does not say that complexity is good by itself. It says complexity becomes easier to support when the data constrain it from many directions. The final choice still belongs to validation or test performance. The point of the whole experiment is now visible. Underfitting is a systematic miss that survives more observations. Overfitting is a gap between fitting performance and new-data performance, driven by excessive sensitivity to the sample. First, training error asks about observations the model already used. Second, test error asks about genuinely separate observations. The difference between those questions is what makes overfitting measurable. Third, bias is the persistent miss seen across fresh samples. Fourth, variance is the spread of the refitted predictions, and adding data can narrow that spread. A useful model has enough freedom to represent the relationship and enough evidence to keep that freedom under control. The goal is not a curve that remembers every point. It is a curve whose accuracy survives the arrival of the next point.","watch":{"version":1,"scenes":[{"title":"One Data Set, Many Models","start":0,"end":167.81166666666664,"objects":{"axes":"an Axes (x_range=(-3.0, 3.0), y_range=(-3.6, 1.8), x_ticks_every=1.0)","caption":"a Math [text] that says \"$upright(\"polynomial degree\") = 1$\"","card":"a Title that says \"Practical Machine Learning — Overfitting Made Visible: Bias, Variance, and the Role of Data\"","cubic_fit":"a FunctionPlot [green] labelled \"upright(\"degree 3\")\" drawn in axes (function=<function>, x_range=(-3.0, 3.0))","data_marks":"a Point [yellow] drawn in axes (location=(3.0, 0.9300000000000002))","data_marks_10":"a Point [yellow] drawn in axes (location=(0.24773803641699718, 0.025208575785553577))","data_marks_11":"a Point [yellow] drawn in axes (location=(-0.24773803641699682, 0.7526967357268568))","data_marks_12":"a Point [yellow] drawn in axes (location=(-0.7364564614223967, -0.6026773153050619))","data_marks_13":"a Point [yellow] drawn in axes (location=(-1.2050862739589083, -0.02419937766054925))","data_marks_14":"a Point [yellow] drawn in axes (location=(-1.640844474367281, -1.4870911669330995))","data_marks_15":"a Point [yellow] drawn in axes (location=(-2.0318447148772227, -0.9606253233498832))","data_marks_16":"a Point [yellow] drawn in axes (location=(-2.3674215281891806, -2.4109250850880968))","data_marks_17":"a Point [yellow] drawn in axes (location=(-2.638421253619467, -1.8041596975698724))","data_marks_18":"a Point [yellow] drawn in axes (location=(-2.8374517251019045, -3.1098022614171272))","data_marks_19":"a Point [yellow] drawn in axes (location=(-2.959083910208167, -2.303608116392006))","data_marks_2":"a Point [yellow] drawn in axes (location=(2.959083910208167, -0.048615815163021825))","data_marks_20":"a Point [yellow] drawn in axes (location=(-3.0, -3.37))","data_marks_3":"a Point [yellow] drawn in axes (location=(2.837451725101904, 1.0113946361949688))","data_marks_4":"a Point [yellow] drawn in axes (location=(2.6384212536194673, 0.09810368141154147))","data_marks_5":"a Point [yellow] drawn in axes (location=(2.3674215281891806, 1.1932385959200025))","data_marks_6":"a Point [yellow] drawn in axes (location=(2.031844714877223, 0.2744038630150619))","data_marks_7":"a Point [yellow] drawn in axes (location=(1.6408444743672808, 1.31783775487091))","data_marks_8":"a Point [yellow] drawn in axes (location=(1.2050862739589085, 0.3013955236942498))","data_marks_9":"a Point [yellow] drawn in axes (location=(0.7364564614223977, 1.2074247922595749))","heading":"a Heading that says \"The Same Twenty Observations\"","line_fit":"a FunctionPlot [green] labelled \"upright(\"degree 1\")\" drawn in axes (function=<function>, x_range=(-3.0, 3.0))","nineteen_fit":"a FunctionPlot [red] labelled \"upright(\"degree 19\")\" drawn in axes (function=<function>, x_range=(-3.0, 3.0))","point":"a Point [yellow] drawn in axes (location=(-2.3, -1.8171999999999997))","point_2":"a Point [yellow] drawn in axes (location=(1.3, 0.8108000000000002))","point_3":"a Point [yellow] drawn in axes (location=(-0.8, -0.6124887971186718))","point_4":"a Point [yellow] drawn in axes (location=(0.8, 1.1820887971186707))","seven_fit":"a FunctionPlot [green] labelled \"upright(\"degree 7\")\" drawn in axes (function=<function>, x_range=(-3.0, 3.0))","true_curve":"a FunctionPlot [blue] labelled \"upright(\"true relationship\")\" drawn in axes (function=<function>, x_range=(-3.0, 3.0))"},"beats":[{"start":0,"say":"Overfitting is often used as a warning word: this model overfits, that model generalizes. I want to make the warning visible and measurable. We will give several polynomial models the same twenty noisy observations, then watch each model decide what story those observations tell.","live":[],"does":[[0,"card is shown on the screen, written out."],[1.5,"card: enter:write-left-to-right."],[18.367,"card is hidden from the screen — left the board."]]},{"start":19.567,"say":"Here are the twenty observations. Each yellow dot contains two things: a curved underlying relationship and a small disturbance that pushed the measurement above or below that relationship.","live":null,"does":[[19.567,"heading is shown on the screen, written out."],[19.567,"axes is shown on the screen, written out."],[20.531,"data_marks is shown on the screen, written out."],[20.561,"data_marks_2 is shown on the screen, written out."],[20.590999999999998,"data_marks_3 is shown on the screen, written out."],[20.621,"data_marks_4 is shown on the screen, written out."],[20.651,"data_marks_5 is shown on the screen, written out."],[20.680999999999997,"data_marks_6 is shown on the screen, written out."],[20.711,"data_marks_7 is shown on the screen, written out."],[20.741,"data_marks_8 is shown on the screen, written out."],[20.770999999999997,"data_marks_9 is shown on the screen, written out."],[20.801,"data_marks_10 is shown on the screen, written out."],[20.831,"data_marks_11 is shown on the screen, written out."],[20.860999999999997,"data_marks_12 is shown on the screen, written out."],[20.891,"data_marks_13 is shown on the screen, written out."],[20.921,"data_marks_14 is shown on the screen, written out."],[20.951,"data_marks_15 is shown on the screen, written out."],[20.980999999999998,"data_marks_16 is shown on the screen, written out."],[21.011,"data_marks_17 is shown on the screen, written out."],[21.041,"data_marks_18 is shown on the screen, written out."],[21.070999999999998,"data_marks_19 is shown on the screen, written out."],[21.101,"data_marks_20 is shown on the screen, written out."]]},{"start":32.996,"say":"Suppose we know only the dots. Our first candidate is a polynomial of degree one, which is simply a straight line. It can tilt and it can move up or down, but it cannot bend.","live":["axes","heading","data_marks","data_marks_2","data_marks_3","data_marks_4","data_marks_5","data_marks_6","data_marks_7","data_marks_8","data_marks_9","data_marks_10","data_marks_11","data_marks_12","data_marks_13","data_marks_14","data_marks_15","data_marks_16","data_marks_17","data_marks_18","data_marks_19","data_marks_20"],"does":[[37.988,"caption is shown on the screen, written out."],[39.916,"line_fit is shown on the screen, drawn."]]},{"start":45.508,"say":"The line catches the broad upward trend, but look at the ends and the middle. The dots curve away from it in an organized pattern. Those are not isolated misses. The model lacks the shape needed to follow the relationship.","live":["axes","caption","heading","data_marks","data_marks_2","data_marks_3","data_marks_4","data_marks_5","data_marks_6","data_marks_7","data_marks_8","data_marks_9","data_marks_10","data_marks_11","data_marks_12","data_marks_13","data_marks_14","data_marks_15","data_marks_16","data_marks_17","data_marks_18","data_marks_19","data_marks_20","line_fit"],"does":[[48.94500000000001,"point is shown on the screen, grown."],[49.757000000000005,"point_2 is shown on the screen, grown."],[50.94500000000001,"point is hidden from the screen."],[51.757000000000005,"point_2 is hidden from the screen."]]},{"start":60.040000000000006,"say":"Allow degree three. The line gives way to a curve, and the fit can now follow the large bend. It still refuses to chase every individual dot.","live":null,"does":[[61.178000000000004,"caption becomes \"$upright(\"polynomial degree\") = 3$\"."],[61.178000000000004,"line_fit is hidden from the screen."],[61.178000000000004,"cubic_fit is shown on the screen, written out."]]},{"start":70.5895,"say":"That refusal is useful. A noisy observation is not the relationship itself. Some of the vertical displacement belongs to chance, so a model that leaves small residuals may be behaving more sensibly than one that erases them.","live":["axes","caption","heading","data_marks","data_marks_2","data_marks_3","data_marks_4","data_marks_5","data_marks_6","data_marks_7","data_marks_8","data_marks_9","data_marks_10","data_marks_11","data_marks_12","data_marks_13","data_marks_14","data_marks_15","data_marks_16","data_marks_17","data_marks_18","data_marks_19","data_marks_20","cubic_fit"],"does":[[73.701,"data_marks_5 is indicated — a transient flash."],[81.201,"data_marks_12 is indicated — a transient flash."]]},{"start":85.818,"say":"Now allow degree seven. The curve gains more freedom. It bends toward several individual observations and begins to reproduce their alternating rises and falls.","live":null,"does":[[87.13,"caption becomes \"$upright(\"polynomial degree\") = 7$\"."],[87.13,"cubic_fit is hidden from the screen."],[87.13,"seven_fit is shown on the screen, written out."]]},{"start":96.71549999999999,"say":"Finally allow degree nineteen. Twenty coefficients are enough to force a polynomial through twenty distinct points. The red curve touches every yellow dot exactly.","live":["axes","caption","heading","data_marks","data_marks_2","data_marks_3","data_marks_4","data_marks_5","data_marks_6","data_marks_7","data_marks_8","data_marks_9","data_marks_10","data_marks_11","data_marks_12","data_marks_13","data_marks_14","data_marks_15","data_marks_16","data_marks_17","data_marks_18","data_marks_19","data_marks_20","seven_fit"],"does":[[98.47999999999999,"caption becomes \"$upright(\"polynomial degree\") = 19$\"."],[98.47999999999999,"seven_fit is hidden from the screen."],[98.47999999999999,"nineteen_fit is shown on the screen, written out."]]},{"start":109.065,"say":"On the training observations this looks perfect. Between the observations, however, the curve changes direction again and again. It has treated each upward or downward disturbance as if it were a real feature of the world.","live":["axes","caption","heading","data_marks","data_marks_2","data_marks_3","data_marks_4","data_marks_5","data_marks_6","data_marks_7","data_marks_8","data_marks_9","data_marks_10","data_marks_11","data_marks_12","data_marks_13","data_marks_14","data_marks_15","data_marks_16","data_marks_17","data_marks_18","data_marks_19","data_marks_20","nineteen_fit"],"does":[[112.29299999999998,"point_3 is shown on the screen, grown."],[114.26699999999998,"nineteen_fit is indicated — a transient flash."],[114.29299999999998,"point_3 is hidden from the screen."],[115.58999999999997,"point_4 is shown on the screen, grown."],[117.58999999999997,"point_4 is hidden from the screen."]]},{"start":123.39999999999999,"say":"Because this is a controlled example, we may reveal the relationship that generated the observations. The blue curve is the signal. The red curve agrees with all twenty recorded values, yet it disagrees repeatedly with the relationship between them.","live":null,"does":[[130.029,"true_curve is shown on the screen, drawn."],[131.15499999999997,"true_curve is indicated — a transient flash."],[132.50199999999998,"nineteen_fit is indicated — a transient flash."]]},{"start":139.6495,"say":"So failure happens in two directions. The straight line is too rigid to represent the signal. The degree-nineteen polynomial is flexible enough to represent both the signal and the accidental noise. One underfits; the other overfits.","live":["axes","caption","heading","data_marks","data_marks_2","data_marks_3","data_marks_4","data_marks_5","data_marks_6","data_marks_7","data_marks_8","data_marks_9","data_marks_10","data_marks_11","data_marks_12","data_marks_13","data_marks_14","data_marks_15","data_marks_16","data_marks_17","data_marks_18","data_marks_19","data_marks_20","nineteen_fit","true_curve"],"does":[[145.99999999999997,"caption is indicated — a transient flash."]]},{"start":155.838,"say":"The central question is therefore not whether a model can fit these observations. It is whether the fitted pattern continues to work at places the fitting process did not get to see.","live":null,"does":[[166.76999999999998,"axes is hidden from the screen — left the board."],[166.76999999999998,"data_marks is hidden from the screen — axes left the board."],[166.76999999999998,"data_marks_2 is hidden from the screen — axes left the board."],[166.76999999999998,"data_marks_3 is hidden from the screen — axes left the board."],[166.76999999999998,"data_marks_4 is hidden from the screen — axes left the board."],[166.76999999999998,"data_marks_5 is hidden from the screen — axes left the board."],[166.76999999999998,"data_marks_6 is hidden from the screen — axes left the board."],[166.76999999999998,"data_marks_7 is hidden from the screen — axes left the board."],[166.76999999999998,"data_marks_8 is hidden from the screen — axes left the board."],[166.76999999999998,"data_marks_9 is hidden from the screen — axes left the board."],[166.76999999999998,"data_marks_10 is hidden from the screen — axes left the board."],[166.76999999999998,"data_marks_11 is hidden from the screen — axes left the board."],[166.76999999999998,"data_marks_12 is hidden from the screen — axes left the board."],[166.76999999999998,"data_marks_13 is hidden from the screen — axes left the board."],[166.76999999999998,"data_marks_14 is hidden from the screen — axes left the board."],[166.76999999999998,"data_marks_15 is hidden from the screen — axes left the board."],[166.76999999999998,"data_marks_16 is hidden from the screen — axes left the board."],[166.76999999999998,"data_marks_17 is hidden from the screen — axes left the board."],[166.76999999999998,"data_marks_18 is hidden from the screen — axes left the board."],[166.76999999999998,"data_marks_19 is hidden from the screen — axes left the board."],[166.76999999999998,"data_marks_20 is hidden from the screen — axes left the board."],[166.76999999999998,"nineteen_fit is hidden from the screen — axes left the board."],[166.76999999999998,"true_curve is hidden from the screen — axes left the board."],[166.76999999999998,"caption is hidden from the screen — left the board."],[166.76999999999998,"heading is hidden from the screen — left the board."]]}]},{"title":"Two Kinds of Failure","start":167.81166666666664,"end":327.3694583333333,"objects":{"definitions":"a Panel that says \"Training error measures predictions on observations used to fit the model. Test error measures predictions on separate observations that played no part in fitting it.\"","heading":"a Heading that says \"Failure in Both Directions\"","heading_2":"a Heading that says \"Training and Test Error\"","left_label":"a Tex [text] that says \"Too rigid\"","mse_test":"a Math [text] that says \"$upright(\"MSE\")_(upright(\"test\")) = frac(1, m) sum_j (y_j - hat(y)_j)^2$\"","mse_train":"a Math [text] that says \"$upright(\"MSE\")_(upright(\"train\")) = frac(1, n) sum_i (y_i - hat(y)_i)^2$\"","over_axes":"an Axes (x_range=(-3.0, 3.0), y_range=(-3.6, 1.8), x_ticks_every=1.0)","over_fit":"a FunctionPlot [red] drawn in over_axes (function=<function>, x_range=(-3.0, 3.0))","over_truth":"a FunctionPlot [blue] drawn in over_axes (function=<function>, x_range=(-3.0, 3.0))","over_twenty":"a Point [yellow] drawn in over_axes (location=(3.0, 0.9300000000000002))","over_twenty_10":"a Point [yellow] drawn in over_axes (location=(0.24773803641699718, 0.025208575785553577))","over_twenty_11":"a Point [yellow] drawn in over_axes (location=(-0.24773803641699682, 0.7526967357268568))","over_twenty_12":"a Point [yellow] drawn in over_axes (location=(-0.7364564614223967, -0.6026773153050619))","over_twenty_13":"a Point [yellow] drawn in over_axes (location=(-1.2050862739589083, -0.02419937766054925))","over_twenty_14":"a Point [yellow] drawn in over_axes (location=(-1.640844474367281, -1.4870911669330995))","over_twenty_15":"a Point [yellow] drawn in over_axes (location=(-2.0318447148772227, -0.9606253233498832))","over_twenty_16":"a Point [yellow] drawn in over_axes (location=(-2.3674215281891806, -2.4109250850880968))","over_twenty_17":"a Point [yellow] drawn in over_axes (location=(-2.638421253619467, -1.8041596975698724))","over_twenty_18":"a Point [yellow] drawn in over_axes (location=(-2.8374517251019045, -3.1098022614171272))","over_twenty_19":"a Point [yellow] drawn in over_axes (location=(-2.959083910208167, -2.303608116392006))","over_twenty_2":"a Point [yellow] drawn in over_axes (location=(2.959083910208167, -0.048615815163021825))","over_twenty_20":"a Point [yellow] drawn in over_axes (location=(-3.0, -3.37))","over_twenty_3":"a Point [yellow] drawn in over_axes (location=(2.837451725101904, 1.0113946361949688))","over_twenty_4":"a Point [yellow] drawn in over_axes (location=(2.6384212536194673, 0.09810368141154147))","over_twenty_5":"a Point [yellow] drawn in over_axes (location=(2.3674215281891806, 1.1932385959200025))","over_twenty_6":"a Point [yellow] drawn in over_axes (location=(2.031844714877223, 0.2744038630150619))","over_twenty_7":"a Point [yellow] drawn in over_axes (location=(1.6408444743672808, 1.31783775487091))","over_twenty_8":"a Point [yellow] drawn in over_axes (location=(1.2050862739589085, 0.3013955236942498))","over_twenty_9":"a Point [yellow] drawn in over_axes (location=(0.7364564614223977, 1.2074247922595749))","right_label":"a Tex [text] that says \"Too responsive\"","test_marks":"a Point [green] drawn in over_axes (location=(-2.65, -2.3215500000000002))","test_marks_10":"a Point [green] drawn in over_axes (location=(2.65, 0.59345))","test_marks_2":"a Point [green] drawn in over_axes (location=(-2.05, -1.4839499999999997))","test_marks_3":"a Point [green] drawn in over_axes (location=(-1.45, -0.7759499999999999))","test_marks_4":"a Point [green] drawn in over_axes (location=(-0.85, -0.19755))","test_marks_5":"a Point [green] drawn in over_axes (location=(-0.25, 0.25125000000000003))","test_marks_6":"a Point [green] drawn in over_axes (location=(0.35, 0.57045))","test_marks_7":"a Point [green] drawn in over_axes (location=(0.95, 0.76005))","test_marks_8":"a Point [green] drawn in over_axes (location=(1.55, 0.8200500000000002))","test_marks_9":"a Point [green] drawn in over_axes (location=(2.15, 0.7504500000000002))","test_readout":"a Math [green] that says \"$upright(\"test MSE\") > 0$\" drawn in over_axes","test_residuals":"a Line [magenta] drawn in over_axes (start=(-2.65, -2.3215500000000002), end=(-2.65, -1.82756352737211), dashed=True)","test_residuals_10":"a Line [magenta] drawn in over_axes (start=(2.65, 0.59345), end=(2.65, 0.09946352737211012), dashed=True)","test_residuals_2":"a Line [magenta] drawn in over_axes (start=(-2.05, -1.4839499999999997), end=(-2.05, -0.9900905529924808), dashed=True)","test_residuals_3":"a Line [magenta] drawn in over_axes (start=(-1.45, -0.7759499999999999), end=(-1.45, -0.8555749140628064), dashed=True)","test_residuals_4":"a Line [magenta] drawn in over_axes (start=(-0.85, -0.19755), end=(-0.85, -0.5648549122571341), dashed=True)","test_residuals_5":"a Line [magenta] drawn in over_axes (start=(-0.25, 0.25125000000000003), end=(-0.25, 0.7511983384419643), dashed=True)","test_residuals_6":"a Line [magenta] drawn in over_axes (start=(0.35, 0.57045), end=(0.35, 0.1726896666190087), dashed=True)","test_residuals_7":"a Line [magenta] drawn in over_axes (start=(0.95, 0.76005), end=(0.95, 0.8402687600971048), dashed=True)","test_residuals_8":"a Line [magenta] drawn in over_axes (start=(1.55, 0.8200500000000002), end=(1.55, 1.2090045306849817), dashed=True)","test_residuals_9":"a Line [magenta] drawn in over_axes (start=(2.15, 0.7504500000000002), end=(2.15, 0.4990845185639889), dashed=True)","train_readout":"a Math [yellow] that says \"$upright(\"training MSE\") = 0.000$\" drawn in over_axes","under_axes":"an Axes (x_range=(-3.0, 3.0), y_range=(-3.6, 1.8), x_ticks_every=1.0)","under_line":"a FunctionPlot [green] drawn in under_axes (function=<function>, x_range=(-3.0, 3.0))","under_more":"a Point [gray] drawn in under_axes (location=(-3.0, -2.87), marker_radius=0.035)","under_more_10":"a Point [gray] drawn in under_axes (location=(-2.3164556962025316, -1.9730848368698612), marker_radius=0.035)","under_more_11":"a Point [gray] drawn in under_axes (location=(-2.240506329113924, -1.563707163843767), marker_radius=0.035)","under_more_12":"a Point [gray] drawn in under_axes (location=(-2.1645569620253164, -1.8119692356331485), marker_radius=0.035)","under_more_13":"a Point [gray] drawn in under_axes (location=(-2.088607594936709, -1.3840481732727037), marker_radius=0.035)","under_more_14":"a Point [gray] drawn in under_axes (location=(-2.0126582278481013, -1.5290319905132814), marker_radius=0.035)","under_more_15":"a Point [gray] drawn in under_axes (location=(-1.9367088607594938, -1.3222122140010508), marker_radius=0.035)","under_more_16":"a Point [gray] drawn in under_axes (location=(-1.860759493670886, -1.1865071704778114), marker_radius=0.035)","under_more_17":"a Point [gray] drawn in under_axes (location=(-1.7848101265822784, -1.2819279781859345), marker_radius=0.035)","under_more_18":"a Point [gray] drawn in under_axes (location=(-1.7088607594936709, -0.8962331215517699), marker_radius=0.035)","under_more_19":"a Point [gray] drawn in under_axes (location=(-1.6329113924050633, -1.1572469075692413), marker_radius=0.035)","under_more_2":"a Point [gray] drawn in under_axes (location=(-2.9240506329113924, -2.6703124483484895), marker_radius=0.035)","under_more_20":"a Point [gray] drawn in under_axes (location=(-1.5569620253164558, -0.7379417777261894), marker_radius=0.035)","under_more_21":"a Point [gray] drawn in under_axes (location=(-1.481012658227848, -0.9099507197082083), marker_radius=0.035)","under_more_22":"a Point [gray] drawn in under_axes (location=(-1.4050632911392404, -0.7010053739545179), marker_radius=0.035)","under_more_23":"a Point [gray] drawn in under_axes (location=(-1.3291139240506329, -0.5974742296665826), marker_radius=0.035)","under_more_24":"a Point [gray] drawn in under_axes (location=(-1.2531645569620253, -0.692201242721242), marker_radius=0.035)","under_more_25":"a Point [gray] drawn in under_axes (location=(-1.1772151898734178, -0.33094265579120674), marker_radius=0.035)","under_more_26":"a Point [gray] drawn in under_axes (location=(-1.1012658227848102, -0.6038216231759829), marker_radius=0.035)","under_more_27":"a Point [gray] drawn in under_axes (location=(-1.0253164556962024, -0.19398195186296024), marker_radius=0.035)","under_more_28":"a Point [gray] drawn in under_axes (location=(-0.9493670886075951, -0.39236648584332007), marker_radius=0.035)","under_more_29":"a Point [gray] drawn in under_axes (location=(-0.8734177215189876, -0.18162049375226394), marker_radius=0.035)","under_more_3":"a Point [gray] drawn in under_axes (location=(-2.848101265822785, -2.765655851631252), marker_radius=0.035)","under_more_30":"a Point [gray] drawn in under_axes (location=(-0.7974683544303796, -0.11032527395094173), marker_radius=0.035)","under_more_31":"a Point [gray] drawn in under_axes (location=(-0.721518987341772, -0.2039214188444216), marker_radius=0.035)","under_more_32":"a Point [gray] drawn in under_axes (location=(-0.6455696202531644, 0.13217263166811496), marker_radius=0.035)","under_more_33":"a Point [gray] drawn in under_axes (location=(-0.5696202531645569, -0.15169177929664995), marker_radius=0.035)","under_more_34":"a Point [gray] drawn in under_axes (location=(-0.49367088607594933, 0.24782000734812762), marker_radius=0.035)","under_more_35":"a Point [gray] drawn in under_axes (location=(-0.4177215189873418, 0.023739534474326557), marker_radius=0.035)","under_more_36":"a Point [gray] drawn in under_axes (location=(-0.3417721518987342, 0.2359196881642116), marker_radius=0.035)","under_more_37":"a Point [gray] drawn in under_axes (location=(-0.26582278481012667, 0.27496198765919205), marker_radius=0.035)","under_more_38":"a Point [gray] drawn in under_axes (location=(-0.1898734177215191, 0.18289392655972697), marker_radius=0.035)","under_more_39":"a Point [gray] drawn in under_axes (location=(-0.11392405063291156, 0.49312221329845274), marker_radius=0.035)","under_more_4":"a Point [gray] drawn in under_axes (location=(-2.7721518987341773, -2.3333720852896733), marker_radius=0.035)","under_more_40":"a Point [gray] drawn in under_axes (location=(-0.037974683544304, 0.19914306335698384), marker_radius=0.035)","under_more_41":"a Point [gray] drawn in under_axes (location=(0.037974683544304, 0.5874538331383163), marker_radius=0.035)","under_more_42":"a Point [gray] drawn in under_axes (location=(0.11392405063291156, 0.3383854657557492), marker_radius=0.035)","under_more_43":"a Point [gray] drawn in under_axes (location=(0.1898734177215191, 0.5515926668326753), marker_radius=0.035)","under_more_44":"a Point [gray] drawn in under_axes (location=(0.26582278481012667, 0.558410122871631), marker_radius=0.035)","under_more_45":"a Point [gray] drawn in under_axes (location=(0.3417721518987342, 0.46822649278154854), marker_radius=0.035)","under_more_46":"a Point [gray] drawn in under_axes (location=(0.4177215189873418, 0.7519166117155476), marker_radius=0.035)","under_more_47":"a Point [gray] drawn in under_axes (location=(0.49367088607594933, 0.4486821790870714), marker_radius=0.035)","under_more_48":"a Point [gray] drawn in under_axes (location=(0.5696202531645569, 0.8249103150582828), marker_radius=0.035)","under_more_49":"a Point [gray] drawn in under_axes (location=(0.6455696202531644, 0.5515886875200892), marker_radius=0.035)","under_more_5":"a Point [gray] drawn in under_axes (location=(-2.6962025316455698, -2.567991370314388), marker_radius=0.035)","under_more_50":"a Point [gray] drawn in under_axes (location=(0.721518987341772, 0.7653762280242439), marker_radius=0.035)","under_more_51":"a Point [gray] drawn in under_axes (location=(0.7974683544303796, 0.7400419186982076), marker_radius=0.035)","under_more_52":"a Point [gray] drawn in under_axes (location=(0.8734177215189876, 0.6520572918447377), marker_radius=0.035)","under_more_53":"a Point [gray] drawn in under_axes (location=(0.9493670886075951, 0.908567372908395), marker_radius=0.035)","under_more_54":"a Point [gray] drawn in under_axes (location=(1.0253164556962027, 0.5969236790559147), marker_radius=0.035)","under_more_55":"a Point [gray] drawn in under_axes (location=(1.1012658227848098, 0.9601813224101202), marker_radius=0.035)","under_more_56":"a Point [gray] drawn in under_axes (location=(1.1772151898734178, 0.6633657900799991), marker_radius=0.035)","under_more_57":"a Point [gray] drawn in under_axes (location=(1.253164556962025, 0.877248504757848), marker_radius=0.035)","under_more_58":"a Point [gray] drawn in under_axes (location=(1.3291139240506329, 0.8198803234831035), marker_radius=0.035)","under_more_59":"a Point [gray] drawn in under_axes (location=(1.4050632911392409, 0.7343666968129939), marker_radius=0.035)","under_more_6":"a Point [gray] drawn in under_axes (location=(-2.620253164556962, -2.132288840073776), marker_radius=0.035)","under_more_60":"a Point [gray] drawn in under_axes (location=(1.481012658227848, 0.9630870368654897), marker_radius=0.035)","under_more_61":"a Point [gray] drawn in under_axes (location=(1.556962025316456, 0.6438645160912888), marker_radius=0.035)","under_more_62":"a Point [gray] drawn in under_axes (location=(1.6329113924050631, 0.9932598249323683), marker_radius=0.035)","under_more_63":"a Point [gray] drawn in under_axes (location=(1.708860759493671, 0.6737325323347665), marker_radius=0.035)","under_more_64":"a Point [gray] drawn in under_axes (location=(1.7848101265822782, 0.887188032931261), marker_radius=0.035)","under_more_65":"a Point [gray] drawn in under_axes (location=(1.8607594936708862, 0.7979483885207774), marker_radius=0.035)","under_more_66":"a Point [gray] drawn in under_axes (location=(1.9367088607594933, 0.7151344917222289), marker_radius=0.035)","under_more_67":"a Point [gray] drawn in under_axes (location=(2.0126582278481013, 0.9154891056722809), marker_radius=0.035)","under_more_68":"a Point [gray] drawn in under_axes (location=(2.0886075949367084, 0.5895004924386543), marker_radius=0.035)","under_more_69":"a Point [gray] drawn in under_axes (location=(2.1645569620253164, 0.9241399106854595), marker_radius=0.035)","under_more_7":"a Point [gray] drawn in under_axes (location=(-2.5443037974683547, -2.24962977264546), marker_radius=0.035)","under_more_70":"a Point [gray] drawn in under_axes (location=(2.2405063291139244, 0.5827038016786734), marker_radius=0.035)","under_more_71":"a Point [gray] drawn in under_axes (location=(2.3164556962025316, 0.7951738059271779), marker_radius=0.035)","under_more_72":"a Point [gray] drawn in under_axes (location=(2.3924050632911396, 0.6742692094119506), marker_radius=0.035)","under_more_73":"a Point [gray] drawn in under_axes (location=(2.4683544303797467, 0.5943399230113228), marker_radius=0.035)","under_more_74":"a Point [gray] drawn in under_axes (location=(2.5443037974683547, 0.7657880091609773), marker_radius=0.035)","under_more_75":"a Point [gray] drawn in under_axes (location=(2.620253164556962, 0.43382627044051236), marker_radius=0.035)","under_more_76":"a Point [gray] drawn in under_axes (location=(2.6962025316455698, 0.75281680109208), marker_radius=0.035)","under_more_77":"a Point [gray] drawn in under_axes (location=(2.772151898734177, 0.3902935761264736), marker_radius=0.035)","under_more_78":"a Point [gray] drawn in under_axes (location=(2.848101265822785, 0.6011853280554427), marker_radius=0.035)","under_more_79":"a Point [gray] drawn in under_axes (location=(2.924050632911392, 0.44886586730688843), marker_radius=0.035)","under_more_8":"a Point [gray] drawn in under_axes (location=(-2.4683544303797467, -2.0452180998191922), marker_radius=0.035)","under_more_80":"a Point [gray] drawn in under_axes (location=(3.0, 0.37196175232051404), marker_radius=0.035)","under_more_9":"a Point [gray] drawn in under_axes (location=(-2.392405063291139, -1.8774460539472317), marker_radius=0.035)","under_residuals":"a Line [red] drawn in under_axes (start=(-2.4, -1.9567999999999999), end=(-2.4, -1.46), dashed=True)","under_residuals_2":"a Line [red] drawn in under_axes (start=(-1.2, -0.5192), end=(-1.2, -0.8), dashed=True)","under_residuals_3":"a Line [red] drawn in under_axes (start=(0.0, 0.4), end=(0.0, -0.14), dashed=True)","under_residuals_4":"a Line [red] drawn in under_axes (start=(1.2, 0.8008000000000001), end=(1.2, 0.52), dashed=True)","under_residuals_5":"a Line [red] drawn in under_axes (start=(2.4, 0.6832000000000003), end=(2.4, 1.1800000000000002), dashed=True)","under_truth":"a FunctionPlot [blue] drawn in under_axes (function=<function>, x_range=(-3.0, 3.0))","under_twenty":"a Point [yellow] drawn in under_axes (location=(3.0, 0.9300000000000002))","under_twenty_10":"a Point [yellow] drawn in under_axes (location=(0.24773803641699718, 0.025208575785553577))","under_twenty_11":"a Point [yellow] drawn in under_axes (location=(-0.24773803641699682, 0.7526967357268568))","under_twenty_12":"a Point [yellow] drawn in under_axes (location=(-0.7364564614223967, -0.6026773153050619))","under_twenty_13":"a Point [yellow] drawn in under_axes (location=(-1.2050862739589083, -0.02419937766054925))","under_twenty_14":"a Point [yellow] drawn in under_axes (location=(-1.640844474367281, -1.4870911669330995))","under_twenty_15":"a Point [yellow] drawn in under_axes (location=(-2.0318447148772227, -0.9606253233498832))","under_twenty_16":"a Point [yellow] drawn in under_axes (location=(-2.3674215281891806, -2.4109250850880968))","under_twenty_17":"a Point [yellow] drawn in under_axes (location=(-2.638421253619467, -1.8041596975698724))","under_twenty_18":"a Point [yellow] drawn in under_axes (location=(-2.8374517251019045, -3.1098022614171272))","under_twenty_19":"a Point [yellow] drawn in under_axes (location=(-2.959083910208167, -2.303608116392006))","under_twenty_2":"a Point [yellow] drawn in under_axes (location=(2.959083910208167, -0.048615815163021825))","under_twenty_20":"a Point [yellow] drawn in under_axes (location=(-3.0, -3.37))","under_twenty_3":"a Point [yellow] drawn in under_axes (location=(2.837451725101904, 1.0113946361949688))","under_twenty_4":"a Point [yellow] drawn in under_axes (location=(2.6384212536194673, 0.09810368141154147))","under_twenty_5":"a Point [yellow] drawn in under_axes (location=(2.3674215281891806, 1.1932385959200025))","under_twenty_6":"a Point [yellow] drawn in under_axes (location=(2.031844714877223, 0.2744038630150619))","under_twenty_7":"a Point [yellow] drawn in under_axes (location=(1.6408444743672808, 1.31783775487091))","under_twenty_8":"a Point [yellow] drawn in under_axes (location=(1.2050862739589085, 0.3013955236942498))","under_twenty_9":"a Point [yellow] drawn in under_axes (location=(0.7364564614223977, 1.2074247922595749))"},"beats":[{"start":167.81166666666664,"say":"Start with the rigid failure. The blue curve is the underlying relationship, the yellow dots are our original twenty observations, and the green line is the best straight model available to us.","live":[],"does":[[167.81166666666664,"heading is shown on the screen, written out."],[167.81166666666664,"left_label is shown on the screen, written out."],[167.81166666666664,"under_axes is shown on the screen, written out."],[170.45866666666663,"under_truth is shown on the screen, written out."],[174.92866666666663,"under_twenty is shown on the screen, written out."],[174.94866666666664,"under_twenty_2 is shown on the screen, written out."],[174.96866666666665,"under_twenty_3 is shown on the screen, written out."],[174.98866666666663,"under_twenty_4 is shown on the screen, written out."],[175.00866666666664,"under_twenty_5 is shown on the screen, written out."],[175.02866666666665,"under_twenty_6 is shown on the screen, written out."],[175.04866666666663,"under_twenty_7 is shown on the screen, written out."],[175.06866666666664,"under_twenty_8 is shown on the screen, written out."],[175.08866666666663,"under_twenty_9 is shown on the screen, written out."],[175.10866666666664,"under_twenty_10 is shown on the screen, written out."],[175.12866666666665,"under_twenty_11 is shown on the screen, written out."],[175.14866666666663,"under_twenty_12 is shown on the screen, written out."],[175.16866666666664,"under_twenty_13 is shown on the screen, written out."],[175.18866666666665,"under_twenty_14 is shown on the screen, written out."],[175.20866666666663,"under_twenty_15 is shown on the screen, written out."],[175.22866666666664,"under_twenty_16 is shown on the screen, written out."],[175.24866666666665,"under_twenty_17 is shown on the screen, written out."],[175.26866666666663,"under_twenty_18 is shown on the screen, written out."],[175.28866666666664,"under_twenty_19 is shown on the screen, written out."],[175.30866666666662,"under_twenty_20 is shown on the screen, written out."],[176.60066666666663,"under_line is shown on the screen, written out."]]},{"start":180.23066666666665,"say":"Now add many more observations from the same relationship. Eighty gray points make the bend much easier to see, but they do not give a straight line the ability to bend.","live":["left_label","under_axes","heading","under_truth","under_twenty","under_twenty_2","under_twenty_3","under_twenty_4","under_twenty_5","under_twenty_6","under_twenty_7","under_twenty_8","under_twenty_9","under_twenty_10","under_twenty_11","under_twenty_12","under_twenty_13","under_twenty_14","under_twenty_15","under_twenty_16","under_twenty_17","under_twenty_18","under_twenty_19","under_twenty_20","under_line"],"does":[[184.47966666666665,"under_more is shown on the screen, written out."],[184.49466666666663,"under_more_2 is shown on the screen, written out."],[184.50966666666665,"under_more_3 is shown on the screen, written out."],[184.52466666666663,"under_more_4 is shown on the screen, written out."],[184.53966666666665,"under_more_5 is shown on the screen, written out."],[184.55466666666663,"under_more_6 is shown on the screen, written out."],[184.56966666666665,"under_more_7 is shown on the screen, written out."],[184.58466666666664,"under_more_8 is shown on the screen, written out."],[184.59966666666665,"under_more_9 is shown on the screen, written out."],[184.61466666666664,"under_more_10 is shown on the screen, written out."],[184.62966666666665,"under_more_11 is shown on the screen, written out."],[184.64466666666664,"under_more_12 is shown on the screen, written out."],[184.65966666666662,"under_more_13 is shown on the screen, written out."],[184.67466666666664,"under_more_14 is shown on the screen, written out."],[184.68966666666665,"under_more_15 is shown on the screen, written out."],[184.70466666666664,"under_more_16 is shown on the screen, written out."],[184.71966666666663,"under_more_17 is shown on the screen, written out."],[184.73466666666664,"under_more_18 is shown on the screen, written out."],[184.74966666666663,"under_more_19 is shown on the screen, written out."],[184.76466666666664,"under_more_20 is shown on the screen, written out."],[184.77966666666663,"under_more_21 is shown on the screen, written out."],[184.79466666666664,"under_more_22 is shown on the screen, written out."],[184.80966666666663,"under_more_23 is shown on the screen, written out."],[184.82466666666664,"under_more_24 is shown on the screen, written out."],[184.83966666666663,"under_more_25 is shown on the screen, written out."],[184.85466666666665,"under_more_26 is shown on the screen, written out."],[184.86966666666663,"under_more_27 is shown on the screen, written out."],[184.88466666666665,"under_more_28 is shown on the screen, written out."],[184.89966666666663,"under_more_29 is shown on the screen, written out."],[184.91466666666665,"under_more_30 is shown on the screen, written out."],[184.92966666666663,"under_more_31 is shown on the screen, written out."],[184.94466666666665,"under_more_32 is shown on the screen, written out."],[184.95966666666664,"under_more_33 is shown on the screen, written out."],[184.97466666666665,"under_more_34 is shown on the screen, written out."],[184.98966666666664,"under_more_35 is shown on the screen, written out."],[185.00466666666665,"under_more_36 is shown on the screen, written out."],[185.01966666666664,"under_more_37 is shown on the screen, written out."],[185.03466666666662,"under_more_38 is shown on the screen, written out."],[185.04966666666664,"under_more_39 is shown on the screen, written out."],[185.06466666666665,"under_more_40 is shown on the screen, written out."],[185.07966666666664,"under_more_41 is shown on the screen, written out."],[185.09466666666663,"under_more_42 is shown on the screen, written out."],[185.10966666666664,"under_more_43 is shown on the screen, written out."],[185.12466666666663,"under_more_44 is shown on the screen, written out."],[185.13966666666664,"under_more_45 is shown on the screen, written out."],[185.15466666666663,"under_more_46 is shown on the screen, written out."],[185.16966666666664,"under_more_47 is shown on the screen, written out."],[185.18466666666663,"under_more_48 is shown on the screen, written out."],[185.19966666666664,"under_more_49 is shown on the screen, written out."],[185.21466666666663,"under_more_50 is shown on the screen, written out."],[185.22966666666665,"under_more_51 is shown on the screen, written out."],[185.24466666666663,"under_more_52 is shown on the screen, written out."],[185.25966666666665,"under_more_53 is shown on the screen, written out."],[185.27466666666663,"under_more_54 is shown on the screen, written out."],[185.28966666666665,"under_more_55 is shown on the screen, written out."],[185.30466666666663,"under_more_56 is shown on the screen, written out."],[185.31966666666665,"under_more_57 is shown on the screen, written out."],[185.33466666666664,"under_more_58 is shown on the screen, written out."],[185.34966666666665,"under_more_59 is shown on the screen, written out."],[185.36466666666664,"under_more_60 is shown on the screen, written out."],[185.37966666666665,"under_more_61 is shown on the screen, written out."],[185.39466666666664,"under_more_62 is shown on the screen, written out."],[185.40966666666662,"under_more_63 is shown on the screen, written out."],[185.42466666666664,"under_more_64 is shown on the screen, written out."],[185.43966666666665,"under_more_65 is shown on the screen, written out."],[185.45466666666664,"under_more_66 is shown on the screen, written out."],[185.46966666666663,"under_more_67 is shown on the screen, written out."],[185.48466666666664,"under_more_68 is shown on the screen, written out."],[185.49966666666663,"under_more_69 is shown on the screen, written out."],[185.51466666666664,"under_more_70 is shown on the screen, written out."],[185.52966666666663,"under_more_71 is shown on the screen, written out."],[185.54466666666664,"under_more_72 is shown on the screen, written out."],[185.55966666666663,"under_more_73 is shown on the screen, written out."],[185.57466666666664,"under_more_74 is shown on the screen, written out."],[185.58966666666663,"under_more_75 is shown on the screen, written out."],[185.60466666666665,"under_more_76 is shown on the screen, written out."],[185.61966666666663,"under_more_77 is shown on the screen, written out."],[185.63466666666665,"under_more_78 is shown on the screen, written out."],[185.64966666666663,"under_more_79 is shown on the screen, written out."],[185.66466666666665,"under_more_80 is shown on the screen, written out."]]},{"start":191.50016666666664,"say":"More data can reveal that this model is wrong, but it cannot repair the model family. These red gaps are systematic. On one stretch the line is too high; on another it is too low. This persistent miss is underfitting.","live":["left_label","under_axes","heading","under_truth","under_twenty","under_twenty_2","under_twenty_3","under_twenty_4","under_twenty_5","under_twenty_6","under_twenty_7","under_twenty_8","under_twenty_9","under_twenty_10","under_twenty_11","under_twenty_12","under_twenty_13","under_twenty_14","under_twenty_15","under_twenty_16","under_twenty_17","under_twenty_18","under_twenty_19","under_twenty_20","under_line","under_more","under_more_2","under_more_3","under_more_4","under_more_5","under_more_6","under_more_7","under_more_8","under_more_9","under_more_10","under_more_11","under_more_12","under_more_13","under_more_14","under_more_15","under_more_16","under_more_17","under_more_18","under_more_19","under_more_20","under_more_21","under_more_22","under_more_23","under_more_24","under_more_25","under_more_26","under_more_27","under_more_28","under_more_29","under_more_30","under_more_31","under_more_32","under_more_33","under_more_34","under_more_35","under_more_36","under_more_37","under_more_38","under_more_39","under_more_40","under_more_41","under_more_42","under_more_43","under_more_44","under_more_45","under_more_46","under_more_47","under_more_48","under_more_49","under_more_50","under_more_51","under_more_52","under_more_53","under_more_54","under_more_55","under_more_56","under_more_57","under_more_58","under_more_59","under_more_60","under_more_61","under_more_62","under_more_63","under_more_64","under_more_65","under_more_66","under_more_67","under_more_68","under_more_69","under_more_70","under_more_71","under_more_72","under_more_73","under_more_74","under_more_75","under_more_76","under_more_77","under_more_78","under_more_79","under_more_80"],"does":[[197.78066666666663,"under_residuals is shown on the screen, written out."],[197.86066666666665,"under_residuals_2 is shown on the screen, written out."],[197.94066666666663,"under_residuals_3 is shown on the screen, written out."],[198.02066666666664,"under_residuals_4 is shown on the screen, written out."],[198.10066666666665,"under_residuals_5 is shown on the screen, written out."],[206.60466666666665,"under_line is indicated — a transient flash."]]},{"start":208.30716666666663,"say":"Now look at the flexible failure. The right-hand graph uses the original twenty observations again. The red degree-nineteen polynomial passes through every one of them.","live":["left_label","under_axes","heading","under_truth","under_twenty","under_twenty_2","under_twenty_3","under_twenty_4","under_twenty_5","under_twenty_6","under_twenty_7","under_twenty_8","under_twenty_9","under_twenty_10","under_twenty_11","under_twenty_12","under_twenty_13","under_twenty_14","under_twenty_15","under_twenty_16","under_twenty_17","under_twenty_18","under_twenty_19","under_twenty_20","under_line","under_more","under_more_2","under_more_3","under_more_4","under_more_5","under_more_6","under_more_7","under_more_8","under_more_9","under_more_10","under_more_11","under_more_12","under_more_13","under_more_14","under_more_15","under_more_16","under_more_17","under_more_18","under_more_19","under_more_20","under_more_21","under_more_22","under_more_23","under_more_24","under_more_25","under_more_26","under_more_27","under_more_28","under_more_29","under_more_30","under_more_31","under_more_32","under_more_33","under_more_34","under_more_35","under_more_36","under_more_37","under_more_38","under_more_39","under_more_40","under_more_41","under_more_42","under_more_43","under_more_44","under_more_45","under_more_46","under_more_47","under_more_48","under_more_49","under_more_50","under_more_51","under_more_52","under_more_53","under_more_54","under_more_55","under_more_56","under_more_57","under_more_58","under_more_59","under_more_60","under_more_61","under_more_62","under_more_63","under_more_64","under_more_65","under_more_66","under_more_67","under_more_68","under_more_69","under_more_70","under_more_71","under_more_72","under_more_73","under_more_74","under_more_75","under_more_76","under_more_77","under_more_78","under_more_79","under_more_80","under_residuals","under_residuals_2","under_residuals_3","under_residuals_4","under_residuals_5"],"does":[[209.32866666666663,"right_label is shown on the screen, written out."],[211.31466666666665,"over_axes is shown on the screen, written out."],[212.56766666666664,"over_truth is shown on the screen, written out."],[213.00966666666665,"over_twenty is shown on the screen, written out."],[213.02966666666663,"over_twenty_2 is shown on the screen, written out."],[213.04966666666664,"over_twenty_3 is shown on the screen, written out."],[213.06966666666665,"over_twenty_4 is shown on the screen, written out."],[213.08966666666663,"over_twenty_5 is shown on the screen, written out."],[213.10966666666664,"over_twenty_6 is shown on the screen, written out."],[213.12966666666662,"over_twenty_7 is shown on the screen, written out."],[213.14966666666663,"over_twenty_8 is shown on the screen, written out."],[213.16966666666661,"over_twenty_9 is shown on the screen, written out."],[213.18966666666662,"over_twenty_10 is shown on the screen, written out."],[213.20966666666664,"over_twenty_11 is shown on the screen, written out."],[213.22966666666662,"over_twenty_12 is shown on the screen, written out."],[213.24966666666663,"over_twenty_13 is shown on the screen, written out."],[213.26966666666664,"over_twenty_14 is shown on the screen, written out."],[213.28966666666662,"over_twenty_15 is shown on the screen, written out."],[213.30966666666663,"over_twenty_16 is shown on the screen, written out."],[213.32966666666664,"over_twenty_17 is shown on the screen, written out."],[213.34966666666662,"over_twenty_18 is shown on the screen, written out."],[213.36966666666663,"over_twenty_19 is shown on the screen, written out."],[213.38966666666664,"over_twenty_20 is shown on the screen, written out."],[215.59866666666665,"over_fit is shown on the screen, written out."]]},{"start":220.12216666666663,"say":"If we measure only at those yellow locations, every vertical residual is zero. Its mean squared training error is therefore zero.","live":["left_label","under_axes","right_label","over_axes","heading","under_truth","under_twenty","under_twenty_2","under_twenty_3","under_twenty_4","under_twenty_5","under_twenty_6","under_twenty_7","under_twenty_8","under_twenty_9","under_twenty_10","under_twenty_11","under_twenty_12","under_twenty_13","under_twenty_14","under_twenty_15","under_twenty_16","under_twenty_17","under_twenty_18","under_twenty_19","under_twenty_20","under_line","under_more","under_more_2","under_more_3","under_more_4","under_more_5","under_more_6","under_more_7","under_more_8","under_more_9","under_more_10","under_more_11","under_more_12","under_more_13","under_more_14","under_more_15","under_more_16","under_more_17","under_more_18","under_more_19","under_more_20","under_more_21","under_more_22","under_more_23","under_more_24","under_more_25","under_more_26","under_more_27","under_more_28","under_more_29","under_more_30","under_more_31","under_more_32","under_more_33","under_more_34","under_more_35","under_more_36","under_more_37","under_more_38","under_more_39","under_more_40","under_more_41","under_more_42","under_more_43","under_more_44","under_more_45","under_more_46","under_more_47","under_more_48","under_more_49","under_more_50","under_more_51","under_more_52","under_more_53","under_more_54","under_more_55","under_more_56","under_more_57","under_more_58","under_more_59","under_more_60","under_more_61","under_more_62","under_more_63","under_more_64","under_more_65","under_more_66","under_more_67","under_more_68","under_more_69","under_more_70","under_more_71","under_more_72","under_more_73","under_more_74","under_more_75","under_more_76","under_more_77","under_more_78","under_more_79","under_more_80","under_residuals","under_residuals_2","under_residuals_3","under_residuals_4","under_residuals_5","over_truth","over_twenty","over_twenty_2","over_twenty_3","over_twenty_4","over_twenty_5","over_twenty_6","over_twenty_7","over_twenty_8","over_twenty_9","over_twenty_10","over_twenty_11","over_twenty_12","over_twenty_13","over_twenty_14","over_twenty_15","over_twenty_16","over_twenty_17","over_twenty_18","over_twenty_19","over_twenty_20","over_fit"],"does":[[221.95666666666665,"over_twenty is indicated — a transient flash."],[221.95666666666665,"over_twenty_5 is indicated — a transient flash."],[221.95666666666665,"over_twenty_9 is indicated — a transient flash."],[221.95666666666665,"over_twenty_13 is indicated — a transient flash."],[221.95666666666665,"over_twenty_17 is indicated — a transient flash."],[224.89466666666664,"train_readout is shown on the screen, written out."]]},{"start":230.04516666666663,"say":"But now place green observations at new locations between the training points. They follow the same blue relationship and were not available while the polynomial was fitted.","live":["left_label","under_axes","right_label","over_axes","heading","under_truth","under_twenty","under_twenty_2","under_twenty_3","under_twenty_4","under_twenty_5","under_twenty_6","under_twenty_7","under_twenty_8","under_twenty_9","under_twenty_10","under_twenty_11","under_twenty_12","under_twenty_13","under_twenty_14","under_twenty_15","under_twenty_16","under_twenty_17","under_twenty_18","under_twenty_19","under_twenty_20","under_line","under_more","under_more_2","under_more_3","under_more_4","under_more_5","under_more_6","under_more_7","under_more_8","under_more_9","under_more_10","under_more_11","under_more_12","under_more_13","under_more_14","under_more_15","under_more_16","under_more_17","under_more_18","under_more_19","under_more_20","under_more_21","under_more_22","under_more_23","under_more_24","under_more_25","under_more_26","under_more_27","under_more_28","under_more_29","under_more_30","under_more_31","under_more_32","under_more_33","under_more_34","under_more_35","under_more_36","under_more_37","under_more_38","under_more_39","under_more_40","under_more_41","under_more_42","under_more_43","under_more_44","under_more_45","under_more_46","under_more_47","under_more_48","under_more_49","under_more_50","under_more_51","under_more_52","under_more_53","under_more_54","under_more_55","under_more_56","under_more_57","under_more_58","under_more_59","under_more_60","under_more_61","under_more_62","under_more_63","under_more_64","under_more_65","under_more_66","under_more_67","under_more_68","under_more_69","under_more_70","under_more_71","under_more_72","under_more_73","under_more_74","under_more_75","under_more_76","under_more_77","under_more_78","under_more_79","under_more_80","under_residuals","under_residuals_2","under_residuals_3","under_residuals_4","under_residuals_5","over_truth","over_twenty","over_twenty_2","over_twenty_3","over_twenty_4","over_twenty_5","over_twenty_6","over_twenty_7","over_twenty_8","over_twenty_9","over_twenty_10","over_twenty_11","over_twenty_12","over_twenty_13","over_twenty_14","over_twenty_15","over_twenty_16","over_twenty_17","over_twenty_18","over_twenty_19","over_twenty_20","over_fit","train_readout"],"does":[[231.36866666666663,"test_marks is shown on the screen, written out."],[231.41866666666664,"test_marks_2 is shown on the screen, written out."],[231.46866666666665,"test_marks_3 is shown on the screen, written out."],[231.51866666666663,"test_marks_4 is shown on the screen, written out."],[231.56866666666664,"test_marks_5 is shown on the screen, written out."],[231.61866666666663,"test_marks_6 is shown on the screen, written out."],[231.66866666666664,"test_marks_7 is shown on the screen, written out."],[231.71866666666665,"test_marks_8 is shown on the screen, written out."],[231.76866666666663,"test_marks_9 is shown on the screen, written out."],[231.81866666666664,"test_marks_10 is shown on the screen, written out."]]},{"start":241.29116666666664,"say":"The magenta gaps measure predictions at those unseen locations. Several are large because the red curve swings away between neighboring training points. Perfect recall of the training set did not produce reliable prediction.","live":["left_label","under_axes","right_label","over_axes","heading","under_truth","under_twenty","under_twenty_2","under_twenty_3","under_twenty_4","under_twenty_5","under_twenty_6","under_twenty_7","under_twenty_8","under_twenty_9","under_twenty_10","under_twenty_11","under_twenty_12","under_twenty_13","under_twenty_14","under_twenty_15","under_twenty_16","under_twenty_17","under_twenty_18","under_twenty_19","under_twenty_20","under_line","under_more","under_more_2","under_more_3","under_more_4","under_more_5","under_more_6","under_more_7","under_more_8","under_more_9","under_more_10","under_more_11","under_more_12","under_more_13","under_more_14","under_more_15","under_more_16","under_more_17","under_more_18","under_more_19","under_more_20","under_more_21","under_more_22","under_more_23","under_more_24","under_more_25","under_more_26","under_more_27","under_more_28","under_more_29","under_more_30","under_more_31","under_more_32","under_more_33","under_more_34","under_more_35","under_more_36","under_more_37","under_more_38","under_more_39","under_more_40","under_more_41","under_more_42","under_more_43","under_more_44","under_more_45","under_more_46","under_more_47","under_more_48","under_more_49","under_more_50","under_more_51","under_more_52","under_more_53","under_more_54","under_more_55","under_more_56","under_more_57","under_more_58","under_more_59","under_more_60","under_more_61","under_more_62","under_more_63","under_more_64","under_more_65","under_more_66","under_more_67","under_more_68","under_more_69","under_more_70","under_more_71","under_more_72","under_more_73","under_more_74","under_more_75","under_more_76","under_more_77","under_more_78","under_more_79","under_more_80","under_residuals","under_residuals_2","under_residuals_3","under_residuals_4","under_residuals_5","over_truth","over_twenty","over_twenty_2","over_twenty_3","over_twenty_4","over_twenty_5","over_twenty_6","over_twenty_7","over_twenty_8","over_twenty_9","over_twenty_10","over_twenty_11","over_twenty_12","over_twenty_13","over_twenty_14","over_twenty_15","over_twenty_16","over_twenty_17","over_twenty_18","over_twenty_19","over_twenty_20","over_fit","train_readout","test_marks","test_marks_2","test_marks_3","test_marks_4","test_marks_5","test_marks_6","test_marks_7","test_marks_8","test_marks_9","test_marks_10"],"does":[[241.76766666666663,"test_residuals is shown on the screen, written out."],[241.82766666666663,"test_residuals_2 is shown on the screen, written out."],[241.88766666666663,"test_residuals_3 is shown on the screen, written out."],[241.94766666666663,"test_residuals_4 is shown on the screen, written out."],[242.00766666666664,"test_residuals_5 is shown on the screen, written out."],[242.06766666666664,"test_residuals_6 is shown on the screen, written out."],[242.12766666666664,"test_residuals_7 is shown on the screen, written out."],[242.18766666666664,"test_residuals_8 is shown on the screen, written out."],[242.24766666666665,"test_residuals_9 is shown on the screen, written out."],[242.30766666666665,"test_residuals_10 is shown on the screen, written out."],[246.59666666666664,"test_readout is shown on the screen, written out."]]},{"start":256.35716666666667,"say":"That gives us an operational definition. Overfitting is not simply a complicated-looking curve. It is a measurable gap: very small error on the data used for fitting, paired with substantially larger error on new data.","live":["left_label","under_axes","right_label","over_axes","heading","under_truth","under_twenty","under_twenty_2","under_twenty_3","under_twenty_4","under_twenty_5","under_twenty_6","under_twenty_7","under_twenty_8","under_twenty_9","under_twenty_10","under_twenty_11","under_twenty_12","under_twenty_13","under_twenty_14","under_twenty_15","under_twenty_16","under_twenty_17","under_twenty_18","under_twenty_19","under_twenty_20","under_line","under_more","under_more_2","under_more_3","under_more_4","under_more_5","under_more_6","under_more_7","under_more_8","under_more_9","under_more_10","under_more_11","under_more_12","under_more_13","under_more_14","under_more_15","under_more_16","under_more_17","under_more_18","under_more_19","under_more_20","under_more_21","under_more_22","under_more_23","under_more_24","under_more_25","under_more_26","under_more_27","under_more_28","under_more_29","under_more_30","under_more_31","under_more_32","under_more_33","under_more_34","under_more_35","under_more_36","under_more_37","under_more_38","under_more_39","under_more_40","under_more_41","under_more_42","under_more_43","under_more_44","under_more_45","under_more_46","under_more_47","under_more_48","under_more_49","under_more_50","under_more_51","under_more_52","under_more_53","under_more_54","under_more_55","under_more_56","under_more_57","under_more_58","under_more_59","under_more_60","under_more_61","under_more_62","under_more_63","under_more_64","under_more_65","under_more_66","under_more_67","under_more_68","under_more_69","under_more_70","under_more_71","under_more_72","under_more_73","under_more_74","under_more_75","under_more_76","under_more_77","under_more_78","under_more_79","under_more_80","under_residuals","under_residuals_2","under_residuals_3","under_residuals_4","under_residuals_5","over_truth","over_twenty","over_twenty_2","over_twenty_3","over_twenty_4","over_twenty_5","over_twenty_6","over_twenty_7","over_twenty_8","over_twenty_9","over_twenty_10","over_twenty_11","over_twenty_12","over_twenty_13","over_twenty_14","over_twenty_15","over_twenty_16","over_twenty_17","over_twenty_18","over_twenty_19","over_twenty_20","over_fit","train_readout","test_marks","test_marks_2","test_marks_3","test_marks_4","test_marks_5","test_marks_6","test_marks_7","test_marks_8","test_marks_9","test_marks_10","test_residuals","test_residuals_2","test_residuals_3","test_residuals_4","test_residuals_5","test_residuals_6","test_residuals_7","test_residuals_8","test_residuals_9","test_residuals_10","test_readout"],"does":[[265.88966666666664,"train_readout is indicated — a transient flash."],[269.92966666666666,"test_readout is indicated — a transient flash."]]},{"start":272.6306666666666,"say":"The straight line and the degree-nineteen curve fail for opposite reasons. The line cannot express enough structure. The interpolating curve reacts to structure that was never really present.","live":null,"does":[[285.3786666666666,"heading is hidden from the screen — left the board."],[285.3786666666666,"left_label is hidden from the screen — left the board."],[285.3786666666666,"over_axes is hidden from the screen — left the board."],[285.3786666666666,"over_truth is hidden from the screen — over_axes left the board."],[285.3786666666666,"over_twenty is hidden from the screen — over_axes left the board."],[285.3786666666666,"over_twenty_2 is hidden from the screen — over_axes left the board."],[285.3786666666666,"over_twenty_3 is hidden from the screen — over_axes left the board."],[285.3786666666666,"over_twenty_4 is hidden from the screen — over_axes left the board."],[285.3786666666666,"over_twenty_5 is hidden from the screen — over_axes left the board."],[285.3786666666666,"over_twenty_6 is hidden from the screen — over_axes left the board."],[285.3786666666666,"over_twenty_7 is hidden from the screen — over_axes left the board."],[285.3786666666666,"over_twenty_8 is hidden from the screen — over_axes left the board."],[285.3786666666666,"over_twenty_9 is hidden from the screen — over_axes left the board."],[285.3786666666666,"over_twenty_10 is hidden from the screen — over_axes left the board."],[285.3786666666666,"over_twenty_11 is hidden from the screen — over_axes left the board."],[285.3786666666666,"over_twenty_12 is hidden from the screen — over_axes left the board."],[285.3786666666666,"over_twenty_13 is hidden from the screen — over_axes left the board."],[285.3786666666666,"over_twenty_14 is hidden from the screen — over_axes left the board."],[285.3786666666666,"over_twenty_15 is hidden from the screen — over_axes left the board."],[285.3786666666666,"over_twenty_16 is hidden from the screen — over_axes left the board."],[285.3786666666666,"over_twenty_17 is hidden from the screen — over_axes left the board."],[285.3786666666666,"over_twenty_18 is hidden from the screen — over_axes left the board."],[285.3786666666666,"over_twenty_19 is hidden from the screen — over_axes left the board."],[285.3786666666666,"over_twenty_20 is hidden from the screen — over_axes left the board."],[285.3786666666666,"over_fit is hidden from the screen — over_axes left the board."],[285.3786666666666,"train_readout is hidden from the screen — over_axes left the board."],[285.3786666666666,"test_marks is hidden from the screen — over_axes left the board."],[285.3786666666666,"test_marks_2 is hidden from the screen — over_axes left the board."],[285.3786666666666,"test_marks_3 is hidden from the screen — over_axes left the board."],[285.3786666666666,"test_marks_4 is hidden from the screen — over_axes left the board."],[285.3786666666666,"test_marks_5 is hidden from the screen — over_axes left the board."],[285.3786666666666,"test_marks_6 is hidden from the screen — over_axes left the board."],[285.3786666666666,"test_marks_7 is hidden from the screen — over_axes left the board."],[285.3786666666666,"test_marks_8 is hidden from the screen — over_axes left the board."],[285.3786666666666,"test_marks_9 is hidden from the screen — over_axes left the board."],[285.3786666666666,"test_marks_10 is hidden from the screen — over_axes left the board."],[285.3786666666666,"test_residuals is hidden from the screen — over_axes left the board."],[285.3786666666666,"test_residuals_2 is hidden from the screen — over_axes left the board."],[285.3786666666666,"test_residuals_3 is hidden from the screen — over_axes left the board."],[285.3786666666666,"test_residuals_4 is hidden from the screen — over_axes left the board."],[285.3786666666666,"test_residuals_5 is hidden from the screen — over_axes left the board."],[285.3786666666666,"test_residuals_6 is hidden from the screen — over_axes left the board."],[285.3786666666666,"test_residuals_7 is hidden from the screen — over_axes left the board."],[285.3786666666666,"test_residuals_8 is hidden from the screen — over_axes left the board."],[285.3786666666666,"test_residuals_9 is hidden from the screen — over_axes left the board."],[285.3786666666666,"test_residuals_10 is hidden from the screen — over_axes left the board."],[285.3786666666666,"test_readout is hidden from the screen — over_axes left the board."],[285.3786666666666,"right_label is hidden from the screen — left the board."],[285.3786666666666,"under_axes is hidden from the screen — left the board."],[285.3786666666666,"under_truth is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_twenty is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_twenty_2 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_twenty_3 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_twenty_4 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_twenty_5 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_twenty_6 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_twenty_7 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_twenty_8 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_twenty_9 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_twenty_10 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_twenty_11 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_twenty_12 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_twenty_13 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_twenty_14 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_twenty_15 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_twenty_16 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_twenty_17 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_twenty_18 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_twenty_19 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_twenty_20 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_line is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_more is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_more_2 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_more_3 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_more_4 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_more_5 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_more_6 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_more_7 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_more_8 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_more_9 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_more_10 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_more_11 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_more_12 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_more_13 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_more_14 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_more_15 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_more_16 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_more_17 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_more_18 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_more_19 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_more_20 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_more_21 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_more_22 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_more_23 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_more_24 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_more_25 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_more_26 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_more_27 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_more_28 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_more_29 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_more_30 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_more_31 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_more_32 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_more_33 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_more_34 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_more_35 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_more_36 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_more_37 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_more_38 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_more_39 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_more_40 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_more_41 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_more_42 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_more_43 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_more_44 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_more_45 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_more_46 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_more_47 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_more_48 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_more_49 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_more_50 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_more_51 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_more_52 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_more_53 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_more_54 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_more_55 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_more_56 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_more_57 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_more_58 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_more_59 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_more_60 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_more_61 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_more_62 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_more_63 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_more_64 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_more_65 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_more_66 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_more_67 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_more_68 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_more_69 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_more_70 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_more_71 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_more_72 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_more_73 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_more_74 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_more_75 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_more_76 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_more_77 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_more_78 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_more_79 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_more_80 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_residuals is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_residuals_2 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_residuals_3 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_residuals_4 is hidden from the screen — under_axes left the board."],[285.3786666666666,"under_residuals_5 is hidden from the screen — under_axes left the board."]]},{"start":286.57866666666666,"say":"To compare models with one number, use mean squared error. For each observation, subtract prediction from observation, square that residual, add the squares, and divide by the number of observations.","live":[],"does":[[289.5856666666666,"mse_train is shown on the screen, written out."],[295.19266666666664,"mse_train (the \"(y_i - hat(y)_i)^2\" part) is emphasized."],[298.2116666666666,"mse_train (the \"(y_i - hat(y)_i)^2\" part) is no longer emphasized."]]},{"start":301.5516666666666,"say":"Training mean squared error uses the observations that selected the fitted curve. Test mean squared error repeats the same calculation on observations held aside from that selection.","live":["mse_train"],"does":[[307.39166666666665,"mse_test is shown on the screen, written out."],[311.9186666666666,"definitions is shown on the screen, written out."]]},{"start":314.99216666666666,"say":"The formulas look nearly identical. Their difference is experimental: one asks how well the model remembers its lesson, and the other asks how well it handles a new question.","live":["mse_train","mse_test","definitions"],"does":[[320.06566666666663,"mse_train (the \"upright(\"train\")\" part) is emphasized."],[323.3746666666666,"mse_test (the \"upright(\"test\")\" part) is emphasized."],[323.3746666666666,"mse_train (the \"upright(\"train\")\" part) is no longer emphasized."],[326.0777916666666,"mse_test (the \"upright(\"test\")\" part) is no longer emphasized."],[326.3277916666666,"definitions is hidden from the screen — left the board."],[326.3277916666666,"mse_test is hidden from the screen — left the board."],[326.3277916666666,"mse_train is hidden from the screen — left the board."]]}]},{"title":"Error Against Complexity","start":327.3694583333333,"end":461.30274999999995,"objects":{"axes":"an Axes (x_range=(1.0, 19.0), y_range=(0.0, 4.0), x_ticks_every=2.0)","choice":"a Math [text] that says \"$d^star = arg min_d E_(upright(\"test\"))(d)$\"","degree":"a VariableNumber (initial_value=1.0, format_spec='.0f')","heading":"a Heading that says \"Two Errors, Two Different Curves\"","minimum":"a PlotPoint [green] labelled \"upright(\"lowest test error\")\" drawn in axes (target='test_curve', x=5.0)","point":"a Point [yellow] drawn in axes (location=(1.0, 1.17))","point_2":"a Point [yellow] drawn in axes (location=(19.0, 3.5320000000000005))","test_curve":"a FunctionPlot [red] labelled \"E_(upright(\"test\"))\" drawn in axes (function=<function>, x_range=(1.0, 19.0))","test_tracker":"a PlotPoint [red] drawn in axes (target='test_curve', x=<VariableNumber degree = 19.0>)","train_curve":"a FunctionPlot [blue] labelled \"E_(upright(\"train\"))\" drawn in axes (function=<function>, x_range=(1.0, 19.0))","train_tracker":"a PlotPoint [blue] drawn in axes (target='train_curve', x=<VariableNumber degree = 19.0>)"},"beats":[{"start":327.3694583333333,"say":"Now repeat the fitting experiment for degree one, degree two, degree three, and so on up to degree nineteen. For every fitted model, record both errors.","live":[],"does":[[327.3694583333333,"heading is shown on the screen, written out."],[327.3694583333333,"axes is shown on the screen, written out."]]},{"start":338.6619583333333,"say":"The blue curve is training error. It falls as degree increases because each larger polynomial family contains more ways to accommodate the observations.","live":["axes","heading"],"does":[[339.17345833333326,"train_curve is shown on the screen, drawn."],[341.1354583333333,"train_curve is indicated — a transient flash."]]},{"start":348.55045833333327,"say":"That decrease is built into the fitting process. A more flexible model can imitate a simpler one if that is best, or use its extra freedom to reduce the residuals further.","live":["axes","heading","train_curve"],"does":[[350.3154583333333,"train_curve is indicated — a transient flash."]]},{"start":359.8434583333333,"say":"The red curve is test error. At first it also falls. Moving beyond a straight line lets the model capture the real curvature, so predictions improve.","live":null,"does":[[360.5284583333333,"test_curve is shown on the screen, drawn."],[362.9194583333333,"train_tracker is shown on the screen, written out."],[362.9194583333333,"test_tracker is shown on the screen, written out."],[369.5954583333333,"train_tracker is redrawn as the numbers it depends on change."],[369.5954583333333,"test_tracker is redrawn as the numbers it depends on change."],[369.5954583333333,"degree ticks to 5.0."]]},{"start":371.22845833333326,"say":"Near degree five, test error reaches its minimum. In this experiment that is the useful region: enough flexibility to represent the signal, without enough freedom to chase every disturbance.","live":["axes","heading","train_curve","test_curve","train_tracker","test_tracker"],"does":[[374.1774583333333,"minimum is shown on the screen, written out."],[376.96445833333325,"choice is shown on the screen, written out."]]},{"start":383.87995833333326,"say":"Past that point, training error keeps falling but test error turns upward. The two measurements separate because extra complexity is helping on observations already seen while hurting predictions elsewhere.","live":["axes","choice","heading","train_curve","test_curve","train_tracker","test_tracker","minimum"],"does":[[384.22845833333326,"train_tracker is redrawn as the numbers it depends on change."],[384.22845833333326,"test_tracker is redrawn as the numbers it depends on change."],[384.22845833333326,"degree ticks to 12.0."],[385.3074583333333,"train_tracker is indicated — a transient flash."],[386.8864583333333,"test_tracker is indicated — a transient flash."]]},{"start":397.18095833333325,"say":"At degree nineteen the training point is almost on the horizontal axis. The test point is high. Their vertical separation is the observable signature of the interpolation failure we just watched.","live":null,"does":[[398.1214583333333,"train_tracker is redrawn as the numbers it depends on change."],[398.1214583333333,"test_tracker is redrawn as the numbers it depends on change."],[398.1214583333333,"degree ticks to 19.0."],[398.66645833333325,"train_tracker is indicated — a transient flash."],[402.02245833333325,"test_tracker is indicated — a transient flash."]]},{"start":409.4369583333333,"say":"Low complexity and high complexity therefore have different diagnoses. On the left, both errors are large because the model misses the signal. On the right, only training error is tiny, because the model has learned details that do not transfer.","live":null,"does":[[414.4294583333333,"point is shown on the screen, grown."],[416.4294583333333,"point is hidden from the screen."],[419.00345833333324,"point_2 is shown on the screen, grown."],[421.00345833333324,"point_2 is hidden from the screen."]]},{"start":425.7224583333333,"say":"Selecting the degree with the smallest training error would always reward the most flexible candidate. That is exactly the decision rule that invites overfitting.","live":null,"does":[[427.8464583333333,"train_curve is indicated — a transient flash."]]},{"start":436.08645833333327,"say":"Instead, reserve data for validation or testing and look for low error there. The marked point is not a law that degree five is always right. It is the answer for this relationship, this noise level, and this amount of data.","live":null,"does":[[440.3464583333333,"A box is drawn around choice."],[442.1924583333333,"minimum is indicated — a transient flash."]]},{"start":450.81495833333327,"say":"Change the noise, the sampling locations, or the amount of data, and the red curve can change. That dependence is the doorway to bias and variance.","live":null,"does":[[460.2610833333333,"axes is hidden from the screen — left the board."],[460.2610833333333,"train_curve is hidden from the screen — axes left the board."],[460.2610833333333,"test_curve is hidden from the screen — axes left the board."],[460.2610833333333,"train_tracker is hidden from the screen — axes left the board."],[460.2610833333333,"test_tracker is hidden from the screen — axes left the board."],[460.2610833333333,"minimum is hidden from the screen — axes left the board."],[460.2610833333333,"choice is hidden from the screen — left the board."],[460.2610833333333,"heading is hidden from the screen — left the board."]]}]},{"title":"Bias and Variance","start":461.30274999999995,"end":622.6099166666665,"objects":{"bias_equation":"a Math [text] that says \"$upright(\"bias\")(x) = upright(\"average fitted prediction\")(x) - f(x)$\"","definition":"a Panel that says \"Bias is systematic error that remains after averaging many refits. Variance is sensitivity to which particular sample happened to be observed.\"","heading":"a Heading that says \"Refit on Fresh Samples\"","heading_2":"a Heading that says \"Two Questions About Repetition\"","high_average":"a FunctionPlot [cyan] labelled \"upright(\"average fit\")\" drawn in high_axes (function=<function>, x_range=(-3.0, 3.0))","high_axes":"an Axes (x_range=(-3.0, 3.0), y_range=(-3.6, 1.8), x_ticks_every=1.0)","high_fits":"a FunctionPlot [red] drawn in high_axes (function=<function>, x_range=(-3.0, 3.0))","high_fits_2":"a FunctionPlot [green] drawn in high_axes (function=<function>, x_range=(-3.0, 3.0))","high_fits_3":"a FunctionPlot [yellow] drawn in high_axes (function=<function>, x_range=(-3.0, 3.0))","high_fits_4":"a FunctionPlot [magenta] drawn in high_axes (function=<function>, x_range=(-3.0, 3.0))","high_fits_5":"a FunctionPlot [gray] drawn in high_axes (function=<function>, x_range=(-3.0, 3.0))","high_truth":"a FunctionPlot [blue] drawn in high_axes (function=<function>, x_range=(-3.0, 3.0))","left_label":"a Tex [text] that says \"Low flexibility\"","low_average":"a FunctionPlot [red] labelled \"upright(\"average fit\")\" drawn in low_axes (function=<function>, x_range=(-3.0, 3.0))","low_axes":"an Axes (x_range=(-3.0, 3.0), y_range=(-3.6, 1.8), x_ticks_every=1.0)","low_fits":"a FunctionPlot [gray] drawn in low_axes (function=<function>, x_range=(-3.0, 3.0))","low_fits_2":"a FunctionPlot [gray] drawn in low_axes (function=<function>, x_range=(-3.0, 3.0))","low_fits_3":"a FunctionPlot [gray] drawn in low_axes (function=<function>, x_range=(-3.0, 3.0))","low_fits_4":"a FunctionPlot [gray] drawn in low_axes (function=<function>, x_range=(-3.0, 3.0))","low_fits_5":"a FunctionPlot [gray] drawn in low_axes (function=<function>, x_range=(-3.0, 3.0))","low_truth":"a FunctionPlot [blue] drawn in low_axes (function=<function>, x_range=(-3.0, 3.0))","point":"a Point [yellow] drawn in high_axes (location=(1.6, 1.1765380974851398))","point_2":"a Point [yellow] drawn in high_axes (location=(1.6, 0.4618619025148607))","right_label":"a Tex [text] that says \"High flexibility\"","variance_equation":"a Math [text] that says \"$upright(\"variance\")(x) = upright(\"spread of fitted predictions at\") thin x$\""},"beats":[{"start":461.30274999999995,"say":"Imagine repeating the entire study. Draw twenty fresh noisy observations from the same relationship, fit the model, discard that sample, and do it again.","live":[],"does":[[461.30274999999995,"heading is shown on the screen, written out."],[461.30274999999995,"left_label is shown on the screen, written out."],[461.30274999999995,"low_axes is shown on the screen, written out."],[466.6667499999999,"low_truth is shown on the screen, written out."]]},{"start":471.62024999999994,"say":"On the left, each gray line is a straight model fitted to a different fresh sample. Here is the first fit, then the second, the third, the fourth, and the fifth.","live":["left_label","low_axes","heading","low_truth"],"does":[[477.86674999999997,"low_fits is shown on the screen, written out."],[479.1667499999999,"low_fits_2 is shown on the screen, written out."],[479.8637499999999,"low_fits_3 is shown on the screen, written out."],[480.67674999999997,"low_fits_4 is shown on the screen, written out."],[481.54774999999995,"low_fits_5 is shown on the screen, written out."]]},{"start":483.01824999999997,"say":"The fitted lines move a little because the observations change, but they remain a tight family. Their small spread means low variance.","live":["left_label","low_axes","heading","low_truth","low_fits","low_fits_2","low_fits_3","low_fits_4","low_fits_5"],"does":[[487.49974999999995,"low_fits is indicated — a transient flash."],[487.49974999999995,"low_fits_2 is indicated — a transient flash."],[487.49974999999995,"low_fits_3 is indicated — a transient flash."],[487.49974999999995,"low_fits_4 is indicated — a transient flash."],[487.49974999999995,"low_fits_5 is indicated — a transient flash."]]},{"start":492.45324999999997,"say":"Now average the five fitted predictions at every x. The red line is that average. It still misses the blue curve in the same organized way, especially near the ends.","live":null,"does":[[496.5057499999999,"low_average is shown on the screen, drawn."],[497.3757499999999,"low_average is indicated — a transient flash."],[499.04774999999995,"low_truth is indicated — a transient flash."]]},{"start":503.57224999999994,"say":"That persistent difference is bias. Even if we repeated the study many more times, averaging straight lines would still produce a straight line. Repetition cannot create curvature that the model family forbids.","live":["left_label","low_axes","heading","low_truth","low_fits","low_fits_2","low_fits_3","low_fits_4","low_fits_5","low_average"],"does":[[504.82574999999997,"The segment (-2.4, -1.46) to (-2.4, -1.9567999999999999) in low_axes is lit up."],[517.4927499999999,"low_axes: retire a lit segment (unemphasize_line)."]]},{"start":518.0927499999999,"say":"Now repeat the same experiment with a highly flexible polynomial. The blue curve on the right is still the same relationship.","live":null,"does":[[518.0927499999999,"high_axes is shown on the screen, written out."],[520.8327499999999,"right_label is shown on the screen, written out."],[522.9337499999999,"high_truth is shown on the screen, written out."]]},{"start":526.65725,"say":"Each colored curve is a refit on a new sample of twenty observations. One sample pulls the curve upward here, another pulls it downward, and another creates a different sequence of bends.","live":["left_label","low_axes","right_label","high_axes","heading","low_truth","low_fits","low_fits_2","low_fits_3","low_fits_4","low_fits_5","low_average","high_truth"],"does":[[527.27275,"high_fits is shown on the screen, written out."],[534.0407499999999,"high_fits_2 is shown on the screen, written out."],[535.8177499999999,"high_fits_3 is shown on the screen, written out."],[536.73475,"high_fits_4 is shown on the screen, written out."],[537.6747499999999,"high_fits_5 is shown on the screen, written out."]]},{"start":539.3087499999999,"say":"The model family is capable of following the underlying curve, so its average can be close to the truth. But the individual fits spread widely. Which data set happened to arrive has a large effect on the prediction.","live":["left_label","low_axes","right_label","high_axes","heading","low_truth","low_fits","low_fits_2","low_fits_3","low_fits_4","low_fits_5","low_average","high_truth","high_fits","high_fits_2","high_fits_3","high_fits_4","high_fits_5"],"does":[[543.66175,"high_average is shown on the screen, drawn."],[544.7647499999999,"high_truth is indicated — a transient flash."],[547.2027499999999,"high_fits is indicated — a transient flash."],[547.2027499999999,"high_fits_2 is indicated — a transient flash."],[547.2027499999999,"high_fits_3 is indicated — a transient flash."],[547.2027499999999,"high_fits_4 is indicated — a transient flash."]]},{"start":553.87475,"say":"That sensitivity is variance. High variance means that if we repeated the data collection, our fitted answer could change substantially even though the data-generating process did not.","live":["left_label","low_axes","right_label","high_axes","heading","low_truth","low_fits","low_fits_2","low_fits_3","low_fits_4","low_fits_5","low_average","high_truth","high_fits","high_fits_2","high_fits_3","high_fits_4","high_fits_5","high_average"],"does":[[561.1077499999999,"point is shown on the screen, grown."],[561.42175,"point_2 is shown on the screen, grown."],[563.1077499999999,"point is hidden from the screen."],[563.42175,"point_2 is hidden from the screen."]]},{"start":565.78275,"say":"Bias and variance are therefore not labels attached permanently to one curve. They describe behavior across repeated samples. Bias asks where the average fit lands. Variance asks how widely the individual fits scatter around that average.","live":null,"does":[[582.9192499999999,"heading is hidden from the screen — left the board."],[582.9192499999999,"high_axes is hidden from the screen — left the board."],[582.9192499999999,"high_truth is hidden from the screen — high_axes left the board."],[582.9192499999999,"high_fits is hidden from the screen — high_axes left the board."],[582.9192499999999,"high_fits_2 is hidden from the screen — high_axes left the board."],[582.9192499999999,"high_fits_3 is hidden from the screen — high_axes left the board."],[582.9192499999999,"high_fits_4 is hidden from the screen — high_axes left the board."],[582.9192499999999,"high_fits_5 is hidden from the screen — high_axes left the board."],[582.9192499999999,"high_average is hidden from the screen — high_axes left the board."],[582.9192499999999,"left_label is hidden from the screen — left the board."],[582.9192499999999,"low_axes is hidden from the screen — left the board."],[582.9192499999999,"low_truth is hidden from the screen — low_axes left the board."],[582.9192499999999,"low_fits is hidden from the screen — low_axes left the board."],[582.9192499999999,"low_fits_2 is hidden from the screen — low_axes left the board."],[582.9192499999999,"low_fits_3 is hidden from the screen — low_axes left the board."],[582.9192499999999,"low_fits_4 is hidden from the screen — low_axes left the board."],[582.9192499999999,"low_fits_5 is hidden from the screen — low_axes left the board."],[582.9192499999999,"low_average is hidden from the screen — low_axes left the board."],[582.9192499999999,"right_label is hidden from the screen — left the board."]]},{"start":584.11925,"say":"Bias is the difference between the average fitted prediction and the true relationship. A rigid straight model has substantial bias here because its average prediction cannot reproduce the bend.","live":[],"does":[[584.67675,"bias_equation is shown on the screen, written out."],[586.5107499999999,"bias_equation (the \"upright(\"average fitted prediction\")(x)\" part) is emphasized."],[588.26375,"bias_equation (the \"f(x)\" part) is emphasized."],[588.26375,"bias_equation (the \"upright(\"average fitted prediction\")(x)\" part) is no longer emphasized."],[596.7742499999999,"bias_equation (the \"f(x)\" part) is no longer emphasized."]]},{"start":597.37425,"say":"Variance is the spread of fitted predictions at the same input across fresh samples. A flexible model can have low bias and still have high variance if each sample leads it toward a different detailed curve.","live":["bias_equation"],"does":[[597.7227499999999,"variance_equation is shown on the screen, written out."],[598.54675,"variance_equation (the \"upright(\"spread of fitted predictions at\") thin x\" part) is emphasized."],[601.46075,"definition is shown on the screen, written out."],[610.09875,"variance_equation (the \"upright(\"spread of fitted predictions at\") thin x\" part) is no longer emphasized."]]},{"start":610.69875,"say":"Underfitting is the visible face of excessive bias. Overfitting is the visible face of excessive sensitivity to the sample. The useful model must control both.","live":["bias_equation","variance_equation","definition"],"does":[[613.3577499999999,"definition (the \"Bias\" part) is indicated — a transient flash."],[617.0727499999999,"definition (the \"Variance\" part) is indicated — a transient flash."],[621.5682499999999,"bias_equation is hidden from the screen — left the board."],[621.5682499999999,"definition is hidden from the screen — left the board."],[621.5682499999999,"variance_equation is hidden from the screen — left the board."]]}]},{"title":"More Data Moves the Balance","start":622.6099166666665,"end":813.0411666666665,"objects":{"balance_heading":"a Heading that says \"The Balance Point Moves\"","closing":"a Math [text] that says \"$upright(\"useful complexity\") = upright(\"enough shape, controlled sensitivity\")$\"","fits_heading":"a Heading that says \"Refits Become More Stable\"","heading":"a Heading that says \"What to Carry Away\"","hundred_label":"a Tex [text] that says \"100 observations\"","large_axes":"an Axes (x_range=(1.0, 19.0), y_range=(0.0, 1.3), x_ticks_every=2.0)","large_bias":"a FunctionPlot [blue] labelled \"upright(\"bias error\")\" drawn in large_axes (function=<function>, x_range=(1.0, 19.0))","large_cross":"a Point [yellow] labelled \"upright(\"new balance\")\" drawn in large_axes (location=(16.0, 0.09425980088148803))","large_label":"a Tex [text] that says \"100 observations\"","large_variance":"a FunctionPlot [red] labelled \"upright(\"variance error\")\" drawn in large_axes (function=<function>, x_range=(1.0, 19.0))","narrow_axes":"an Axes (x_range=(-3.0, 3.0), y_range=(-3.6, 1.8), x_ticks_every=1.0)","narrow_fits":"a FunctionPlot [red] drawn in narrow_axes (function=<function>, x_range=(-3.0, 3.0))","narrow_fits_2":"a FunctionPlot [green] drawn in narrow_axes (function=<function>, x_range=(-3.0, 3.0))","narrow_fits_3":"a FunctionPlot [yellow] drawn in narrow_axes (function=<function>, x_range=(-3.0, 3.0))","narrow_fits_4":"a FunctionPlot [magenta] drawn in narrow_axes (function=<function>, x_range=(-3.0, 3.0))","narrow_truth":"a FunctionPlot [blue] drawn in narrow_axes (function=<function>, x_range=(-3.0, 3.0))","small_axes":"an Axes (x_range=(1.0, 19.0), y_range=(0.0, 1.3), x_ticks_every=2.0)","small_bias":"a FunctionPlot [blue] labelled \"upright(\"bias error\")\" drawn in small_axes (function=<function>, x_range=(1.0, 19.0))","small_cross":"a Point [yellow] labelled \"upright(\"balance\")\" drawn in small_axes (location=(11.5, 0.16911350187157478))","small_label":"a Tex [text] that says \"20 observations\"","small_variance":"a FunctionPlot [red] labelled \"upright(\"variance error\")\" drawn in small_axes (function=<function>, x_range=(1.0, 19.0))","summary":"a Block [text] that says \"Training error measures fit to observations the model already used. Test error measures prediction on observations held outside fitting. Bias is the systematic miss that remains across repeated samples. Variance is the spread of refitted p…\"","twenty_label":"a Tex [text] that says \"20 observations\"","wide_axes":"an Axes (x_range=(-3.0, 3.0), y_range=(-3.6, 1.8), x_ticks_every=1.0)","wide_fits":"a FunctionPlot [red] drawn in wide_axes (function=<function>, x_range=(-3.0, 3.0))","wide_fits_2":"a FunctionPlot [green] drawn in wide_axes (function=<function>, x_range=(-3.0, 3.0))","wide_fits_3":"a FunctionPlot [yellow] drawn in wide_axes (function=<function>, x_range=(-3.0, 3.0))","wide_fits_4":"a FunctionPlot [magenta] drawn in wide_axes (function=<function>, x_range=(-3.0, 3.0))","wide_truth":"a FunctionPlot [blue] drawn in wide_axes (function=<function>, x_range=(-3.0, 3.0))"},"beats":[{"start":622.6099166666665,"say":"Return to repeated fitting, but change the amount of evidence. On the left, each colored flexible curve was fitted from only twenty observations.","live":[],"does":[[622.6099166666665,"fits_heading is shown on the screen, written out."],[622.6099166666665,"twenty_label is shown on the screen, written out."],[622.6099166666665,"wide_axes is shown on the screen, written out."],[627.1379166666666,"wide_truth is shown on the screen, written out."],[628.1479166666666,"wide_fits is shown on the screen, written out."],[628.2279166666666,"wide_fits_2 is shown on the screen, written out."],[628.3079166666665,"wide_fits_3 is shown on the screen, written out."],[628.3879166666666,"wide_fits_4 is shown on the screen, written out."]]},{"start":632.6839166666665,"say":"The fits disagree because a handful of noisy points can pull a flexible polynomial in noticeably different directions. The blue relationship is fixed; the fitted curves are not.","live":["twenty_label","wide_axes","fits_heading","wide_truth","wide_fits","wide_fits_2","wide_fits_3","wide_fits_4"],"does":[[633.4379166666665,"wide_fits is indicated — a transient flash."],[633.4379166666665,"wide_fits_2 is indicated — a transient flash."],[633.4379166666665,"wide_fits_3 is indicated — a transient flash."],[633.4379166666665,"wide_fits_4 is indicated — a transient flash."],[640.7299166666666,"wide_truth is indicated — a transient flash."]]},{"start":644.8939166666665,"say":"On the right, fit the same degree of polynomial after collecting one hundred observations. Each individual observation now has less power to redirect the entire curve.","live":null,"does":[[645.5679166666665,"narrow_axes is shown on the screen, written out."],[646.5889166666666,"narrow_truth is shown on the screen, written out."],[647.3319166666665,"narrow_fits is shown on the screen, written out."],[647.4119166666666,"narrow_fits_2 is shown on the screen, written out."],[647.4919166666665,"narrow_fits_3 is shown on the screen, written out."],[647.5719166666665,"narrow_fits_4 is shown on the screen, written out."],[649.0269166666666,"hundred_label is shown on the screen, written out."]]},{"start":656.4889166666666,"say":"The colored curves still differ, because noise has not vanished. But their spread is much narrower. More data has reduced variance without changing the basic model family.","live":["twenty_label","wide_axes","hundred_label","narrow_axes","fits_heading","wide_truth","wide_fits","wide_fits_2","wide_fits_3","wide_fits_4","narrow_truth","narrow_fits","narrow_fits_2","narrow_fits_3","narrow_fits_4"],"does":[[662.5839166666666,"narrow_fits is indicated — a transient flash."],[662.5839166666666,"narrow_fits_2 is indicated — a transient flash."],[662.5839166666666,"narrow_fits_3 is indicated — a transient flash."],[662.5839166666666,"narrow_fits_4 is indicated — a transient flash."]]},{"start":668.6174166666665,"say":"Notice what more data did not do. It did not make every complex model safe, and it would not make a straight line bend. It made the fitted details less dependent on a few particular observations.","live":null,"does":[[681.3419166666665,"fits_heading is hidden from the screen — left the board."],[681.3419166666665,"hundred_label is hidden from the screen — left the board."],[681.3419166666665,"narrow_axes is hidden from the screen — left the board."],[681.3419166666665,"narrow_truth is hidden from the screen — narrow_axes left the board."],[681.3419166666665,"narrow_fits is hidden from the screen — narrow_axes left the board."],[681.3419166666665,"narrow_fits_2 is hidden from the screen — narrow_axes left the board."],[681.3419166666665,"narrow_fits_3 is hidden from the screen — narrow_axes left the board."],[681.3419166666665,"narrow_fits_4 is hidden from the screen — narrow_axes left the board."],[681.3419166666665,"twenty_label is hidden from the screen — left the board."],[681.3419166666665,"wide_axes is hidden from the screen — left the board."],[681.3419166666665,"wide_truth is hidden from the screen — wide_axes left the board."],[681.3419166666665,"wide_fits is hidden from the screen — wide_axes left the board."],[681.3419166666665,"wide_fits_2 is hidden from the screen — wide_axes left the board."],[681.3419166666665,"wide_fits_3 is hidden from the screen — wide_axes left the board."],[681.3419166666665,"wide_fits_4 is hidden from the screen — wide_axes left the board."]]},{"start":682.5419166666666,"say":"We can summarize that change with two error contributions. Blue is error from bias. It falls as complexity gives the model more shapes to choose from.","live":[],"does":[[682.5419166666666,"balance_heading is shown on the screen, written out."],[682.5419166666666,"small_label is shown on the screen, written out."],[682.5419166666666,"small_axes is shown on the screen, written out."],[686.7209166666665,"small_bias is shown on the screen, drawn."]]},{"start":693.7179166666665,"say":"Red is error from variance. With only twenty observations, it rises as flexible models become increasingly sensitive to the sample.","live":["small_label","small_axes","balance_heading","small_bias"],"does":[[694.1939166666665,"small_variance is shown on the screen, drawn."]]},{"start":703.8964166666665,"say":"The two contributions cross near this yellow point. To the left, missing structure dominates. To the right, sample sensitivity dominates. The crossing is a useful picture of balance, not a universal decision rule.","live":["small_label","small_axes","balance_heading","small_bias","small_variance"],"does":[[706.3579166666666,"small_cross is shown on the screen, written out."],[708.3549166666666,"small_bias is indicated — a transient flash."],[711.4659166666665,"small_variance is indicated — a transient flash."]]},{"start":720.1349166666665,"say":"Now keep the blue bias curve and increase the data to one hundred observations. The right-hand graph shows the same declining bias contribution.","live":["small_label","small_axes","balance_heading","small_bias","small_variance","small_cross"],"does":[[720.1349166666665,"large_axes is shown on the screen, written out."],[721.2719166666666,"large_bias is shown on the screen, drawn."],[723.7689166666665,"large_label is shown on the screen, written out."]]},{"start":730.0224166666666,"say":"The red variance contribution is lower because repeated fits are more stable. Its crossing with bias moves to the right, toward a more flexible polynomial.","live":["small_label","small_axes","large_label","large_axes","balance_heading","small_bias","small_variance","small_cross","large_bias"],"does":[[730.5569166666666,"large_variance is shown on the screen, drawn."],[735.5029166666666,"large_cross is shown on the screen, written out."],[737.0349166666665,"large_cross is indicated — a transient flash."]]},{"start":740.5029166666666,"say":"This does not say that complexity is good by itself. It says complexity becomes easier to support when the data constrain it from many directions. The final choice still belongs to validation or test performance.","live":["small_label","small_axes","large_label","large_axes","balance_heading","small_bias","small_variance","small_cross","large_bias","large_variance","large_cross"],"does":[[747.8409166666665,"large_variance is indicated — a transient flash."],[750.9059166666666,"large_cross is indicated — a transient flash."]]},{"start":754.5589166666666,"say":"The point of the whole experiment is now visible. Underfitting is a systematic miss that survives more observations. Overfitting is a gap between fitting performance and new-data performance, driven by excessive sensitivity to the sample.","live":null,"does":[[769.4894166666666,"balance_heading is hidden from the screen — left the board."],[769.4894166666666,"large_axes is hidden from the screen — left the board."],[769.4894166666666,"large_bias is hidden from the screen — large_axes left the board."],[769.4894166666666,"large_variance is hidden from the screen — large_axes left the board."],[769.4894166666666,"large_cross is hidden from the screen — large_axes left the board."],[769.4894166666666,"large_label is hidden from the screen — left the board."],[769.4894166666666,"small_axes is hidden from the screen — left the board."],[769.4894166666666,"small_bias is hidden from the screen — small_axes left the board."],[769.4894166666666,"small_variance is hidden from the screen — small_axes left the board."],[769.4894166666666,"small_cross is hidden from the screen — small_axes left the board."],[769.4894166666666,"small_label is hidden from the screen — left the board."]]},{"start":770.6894166666666,"say":"First, training error asks about observations the model already used. Second, test error asks about genuinely separate observations. The difference between those questions is what makes overfitting measurable.","live":[],"does":[[771.0379166666665,"summary is shown on the screen, written out."],[771.8389166666666,"summary (the \"Training error\" part) is emphasized."],[776.9589166666666,"summary (the \"Test error\" part) is emphasized."],[776.9589166666666,"summary (the \"Training error\" part) is no longer emphasized."]]},{"start":785.2789166666665,"say":"Third, bias is the persistent miss seen across fresh samples. Fourth, variance is the spread of the refitted predictions, and adding data can narrow that spread.","live":["summary"],"does":[[786.6959166666666,"summary (the \"Bias\" part) is emphasized."],[786.6959166666666,"summary (the \"Test error\" part) is no longer emphasized."],[791.6649166666666,"summary (the \"Bias\" part) is no longer emphasized."],[791.6649166666666,"summary (the \"Variance\" part) is emphasized."],[797.2959166666666,"summary (the \"Variance\" part) is no longer emphasized."]]},{"start":797.8959166666666,"say":"A useful model has enough freedom to represent the relationship and enough evidence to keep that freedom under control. The goal is not a curve that remembers every point. It is a curve whose accuracy survives the arrival of the next point.","live":null,"does":[[799.5099166666665,"closing is shown on the screen, written out."],[809.5289166666665,"A box is drawn around closing."],[811.9994999999999,"closing is hidden from the screen — left the board."],[811.9994999999999,"summary is hidden from the screen — left the board."]]}]}]},"durationSeconds":813,"chapters":[{"title":"One Data Set, Many Models","startSeconds":0,"narration":"Overfitting is often used as a warning word: this model overfits, that model generalizes. I want to make the warning visible and measurable. We will give several polynomial models the same twenty noisy observations, then watch each model decide what story those observations tell. Here are the twenty observations. Each yellow dot contains two things: a curved underlying relationship and a small disturbance that pushed the measurement above or below that relationship. Suppose we know only the dots. Our first candidate is a polynomial of degree one, which is simply a straight line. It can tilt and it can move up or down, but it cannot bend. The line catches the broad upward trend, but look at the ends and the middle. The dots curve away from it in an organized pattern. Those are not isolated misses. The model lacks the shape needed to follow the relationship. Allow degree three. The line gives way to a curve, and the fit can now follow the large bend. It still refuses to chase every individual dot. That refusal is useful. A noisy observation is not the relationship itself. Some of the vertical displacement belongs to chance, so a model that leaves small residuals may be behaving more sensibly than one that erases them. Now allow degree seven. The curve gains more freedom. It bends toward several individual observations and begins to reproduce their alternating rises and falls. Finally allow degree nineteen. Twenty coefficients are enough to force a polynomial through twenty distinct points. The red curve touches every yellow dot exactly. On the training observations this looks perfect. Between the observations, however, the curve changes direction again and again. It has treated each upward or downward disturbance as if it were a real feature of the world. Because this is a controlled example, we may reveal the relationship that generated the observations. The blue curve is the signal. The red curve agrees with all twenty recorded values, yet it disagrees repeatedly with the relationship between them. So failure happens in two directions. The straight line is too rigid to represent the signal. The degree-nineteen polynomial is flexible enough to represent both the signal and the accidental noise. One underfits; the other overfits. The central question is therefore not whether a model can fit these observations. It is whether the fitted pattern continues to work at places the fitting process did not get to see."},{"title":"Two Kinds of Failure","startSeconds":167.81166666666664,"narration":"Start with the rigid failure. The blue curve is the underlying relationship, the yellow dots are our original twenty observations, and the green line is the best straight model available to us. Now add many more observations from the same relationship. Eighty gray points make the bend much easier to see, but they do not give a straight line the ability to bend. More data can reveal that this model is wrong, but it cannot repair the model family. These red gaps are systematic. On one stretch the line is too high; on another it is too low. This persistent miss is underfitting. Now look at the flexible failure. The right-hand graph uses the original twenty observations again. The red degree-nineteen polynomial passes through every one of them. If we measure only at those yellow locations, every vertical residual is zero. Its mean squared training error is therefore zero. But now place green observations at new locations between the training points. They follow the same blue relationship and were not available while the polynomial was fitted. The magenta gaps measure predictions at those unseen locations. Several are large because the red curve swings away between neighboring training points. Perfect recall of the training set did not produce reliable prediction. That gives us an operational definition. Overfitting is not simply a complicated-looking curve. It is a measurable gap: very small error on the data used for fitting, paired with substantially larger error on new data. The straight line and the degree-nineteen curve fail for opposite reasons. The line cannot express enough structure. The interpolating curve reacts to structure that was never really present. To compare models with one number, use mean squared error. For each observation, subtract prediction from observation, square that residual, add the squares, and divide by the number of observations. Training mean squared error uses the observations that selected the fitted curve. Test mean squared error repeats the same calculation on observations held aside from that selection. The formulas look nearly identical. Their difference is experimental: one asks how well the model remembers its lesson, and the other asks how well it handles a new question."},{"title":"Error Against Complexity","startSeconds":327.3694583333333,"narration":"Now repeat the fitting experiment for degree one, degree two, degree three, and so on up to degree nineteen. For every fitted model, record both errors. The blue curve is training error. It falls as degree increases because each larger polynomial family contains more ways to accommodate the observations. That decrease is built into the fitting process. A more flexible model can imitate a simpler one if that is best, or use its extra freedom to reduce the residuals further. The red curve is test error. At first it also falls. Moving beyond a straight line lets the model capture the real curvature, so predictions improve. Near degree five, test error reaches its minimum. In this experiment that is the useful region: enough flexibility to represent the signal, without enough freedom to chase every disturbance. Past that point, training error keeps falling but test error turns upward. The two measurements separate because extra complexity is helping on observations already seen while hurting predictions elsewhere. At degree nineteen the training point is almost on the horizontal axis. The test point is high. Their vertical separation is the observable signature of the interpolation failure we just watched. Low complexity and high complexity therefore have different diagnoses. On the left, both errors are large because the model misses the signal. On the right, only training error is tiny, because the model has learned details that do not transfer. Selecting the degree with the smallest training error would always reward the most flexible candidate. That is exactly the decision rule that invites overfitting. Instead, reserve data for validation or testing and look for low error there. The marked point is not a law that degree five is always right. It is the answer for this relationship, this noise level, and this amount of data. Change the noise, the sampling locations, or the amount of data, and the red curve can change. That dependence is the doorway to bias and variance."},{"title":"Bias and Variance","startSeconds":461.30274999999995,"narration":"Imagine repeating the entire study. Draw twenty fresh noisy observations from the same relationship, fit the model, discard that sample, and do it again. On the left, each gray line is a straight model fitted to a different fresh sample. Here is the first fit, then the second, the third, the fourth, and the fifth. The fitted lines move a little because the observations change, but they remain a tight family. Their small spread means low variance. Now average the five fitted predictions at every x. The red line is that average. It still misses the blue curve in the same organized way, especially near the ends. That persistent difference is bias. Even if we repeated the study many more times, averaging straight lines would still produce a straight line. Repetition cannot create curvature that the model family forbids. Now repeat the same experiment with a highly flexible polynomial. The blue curve on the right is still the same relationship. Each colored curve is a refit on a new sample of twenty observations. One sample pulls the curve upward here, another pulls it downward, and another creates a different sequence of bends. The model family is capable of following the underlying curve, so its average can be close to the truth. But the individual fits spread widely. Which data set happened to arrive has a large effect on the prediction. That sensitivity is variance. High variance means that if we repeated the data collection, our fitted answer could change substantially even though the data-generating process did not. Bias and variance are therefore not labels attached permanently to one curve. They describe behavior across repeated samples. Bias asks where the average fit lands. Variance asks how widely the individual fits scatter around that average. Bias is the difference between the average fitted prediction and the true relationship. A rigid straight model has substantial bias here because its average prediction cannot reproduce the bend. Variance is the spread of fitted predictions at the same input across fresh samples. A flexible model can have low bias and still have high variance if each sample leads it toward a different detailed curve. Underfitting is the visible face of excessive bias. Overfitting is the visible face of excessive sensitivity to the sample. The useful model must control both."},{"title":"More Data Moves the Balance","startSeconds":622.6099166666665,"narration":"Return to repeated fitting, but change the amount of evidence. On the left, each colored flexible curve was fitted from only twenty observations. The fits disagree because a handful of noisy points can pull a flexible polynomial in noticeably different directions. The blue relationship is fixed; the fitted curves are not. On the right, fit the same degree of polynomial after collecting one hundred observations. Each individual observation now has less power to redirect the entire curve. The colored curves still differ, because noise has not vanished. But their spread is much narrower. More data has reduced variance without changing the basic model family. Notice what more data did not do. It did not make every complex model safe, and it would not make a straight line bend. It made the fitted details less dependent on a few particular observations. We can summarize that change with two error contributions. Blue is error from bias. It falls as complexity gives the model more shapes to choose from. Red is error from variance. With only twenty observations, it rises as flexible models become increasingly sensitive to the sample. The two contributions cross near this yellow point. To the left, missing structure dominates. To the right, sample sensitivity dominates. The crossing is a useful picture of balance, not a universal decision rule. Now keep the blue bias curve and increase the data to one hundred observations. The right-hand graph shows the same declining bias contribution. The red variance contribution is lower because repeated fits are more stable. Its crossing with bias moves to the right, toward a more flexible polynomial. This does not say that complexity is good by itself. It says complexity becomes easier to support when the data constrain it from many directions. The final choice still belongs to validation or test performance. The point of the whole experiment is now visible. Underfitting is a systematic miss that survives more observations. Overfitting is a gap between fitting performance and new-data performance, driven by excessive sensitivity to the sample. First, training error asks about observations the model already used. Second, test error asks about genuinely separate observations. The difference between those questions is what makes overfitting measurable. Third, bias is the persistent miss seen across fresh samples. Fourth, variance is the spread of the refitted predictions, and adding data can narrow that spread. A useful model has enough freedom to represent the relationship and enough evidence to keep that freedom under control. The goal is not a curve that remembers every point. It is a curve whose accuracy survives the arrival of the next point."}]}}