# Overfitting Made Visible: Bias, Variance, and the Role of Data > A visual introduction to overfitting built from one controlled experiment. The same twenty noisy observations are fitted with polynomials from a straight line through an exact degree-nineteen interpolant. Training and test error make both failure modes measurable, repeated fresh samples turn bias and variance into visible behavior, and a final comparison shows how additional data stabilizes flexible models and moves the balance toward greater complexity. - Canonical watch page: [Overfitting Made Visible: Bias, Variance, and the Role of Data](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting) - Publisher: [Academa, Inc.](https://academa.ai) - Subject: Machine Learning - Published: 2026-08-28T22:51:22.000Z - Updated: 2026-08-28T22:51:22.000Z - Duration: PT813S (13 minutes 33 seconds) - Chapters: 5 - Views: 1 - Language: en-US - Access: Free - Video stream: [HLS content](https://academa.ai/media/l/01M14TYJMMAQ9SQDTPSVK4XZGD/0/dark/master.m3u8) - Audiovisual record: [Semantic JSON](https://academa.ai/media/l/01M14TYJMMAQ9SQDTPSVK4XZGD/0/semantic.json) - Thumbnail: [Image](https://academa.ai/media/l/01M14TYJMMAQ9SQDTPSVK4XZGD/0/dark/poster.jpg) ## Description See underfitting, overfitting, training and test error, bias, variance, and the stabilizing role of additional data in one visual experiment. ## Chapters - [00:00–02:47.812 · One Data Set, Many Models](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=0) - [02:47.812–05:27.369 · Two Kinds of Failure](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=167.81166666666664) - [05:27.369–07:41.303 · Error Against Complexity](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=327.3694583333333) - [07:41.303–10:22.61 · Bias and Variance](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=461.30274999999995) - [10:22.61–13:33 · More Data Moves the Balance](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=622.6099166666665) ## Transcript ### [00:00 · One Data Set, Many Models](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=0) Overfitting is often used as a warning word: this model overfits, that model generalizes. I want to make the warning visible and measurable. We will give several polynomial models the same twenty noisy observations, then watch each model decide what story those observations tell. Here are the twenty observations. Each yellow dot contains two things: a curved underlying relationship and a small disturbance that pushed the measurement above or below that relationship. Suppose we know only the dots. Our first candidate is a polynomial of degree one, which is simply a straight line. It can tilt and it can move up or down, but it cannot bend. The line catches the broad upward trend, but look at the ends and the middle. The dots curve away from it in an organized pattern. Those are not isolated misses. The model lacks the shape needed to follow the relationship. Allow degree three. The line gives way to a curve, and the fit can now follow the large bend. It still refuses to chase every individual dot. That refusal is useful. A noisy observation is not the relationship itself. Some of the vertical displacement belongs to chance, so a model that leaves small residuals may be behaving more sensibly than one that erases them. Now allow degree seven. The curve gains more freedom. It bends toward several individual observations and begins to reproduce their alternating rises and falls. Finally allow degree nineteen. Twenty coefficients are enough to force a polynomial through twenty distinct points. The red curve touches every yellow dot exactly. On the training observations this looks perfect. Between the observations, however, the curve changes direction again and again. It has treated each upward or downward disturbance as if it were a real feature of the world. Because this is a controlled example, we may reveal the relationship that generated the observations. The blue curve is the signal. The red curve agrees with all twenty recorded values, yet it disagrees repeatedly with the relationship between them. So failure happens in two directions. The straight line is too rigid to represent the signal. The degree-nineteen polynomial is flexible enough to represent both the signal and the accidental noise. One underfits; the other overfits. The central question is therefore not whether a model can fit these observations. It is whether the fitted pattern continues to work at places the fitting process did not get to see. ### [02:47.812 · Two Kinds of Failure](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=167.81166666666664) Start with the rigid failure. The blue curve is the underlying relationship, the yellow dots are our original twenty observations, and the green line is the best straight model available to us. Now add many more observations from the same relationship. Eighty gray points make the bend much easier to see, but they do not give a straight line the ability to bend. More data can reveal that this model is wrong, but it cannot repair the model family. These red gaps are systematic. On one stretch the line is too high; on another it is too low. This persistent miss is underfitting. Now look at the flexible failure. The right-hand graph uses the original twenty observations again. The red degree-nineteen polynomial passes through every one of them. If we measure only at those yellow locations, every vertical residual is zero. Its mean squared training error is therefore zero. But now place green observations at new locations between the training points. They follow the same blue relationship and were not available while the polynomial was fitted. The magenta gaps measure predictions at those unseen locations. Several are large because the red curve swings away between neighboring training points. Perfect recall of the training set did not produce reliable prediction. That gives us an operational definition. Overfitting is not simply a complicated-looking curve. It is a measurable gap: very small error on the data used for fitting, paired with substantially larger error on new data. The straight line and the degree-nineteen curve fail for opposite reasons. The line cannot express enough structure. The interpolating curve reacts to structure that was never really present. To compare models with one number, use mean squared error. For each observation, subtract prediction from observation, square that residual, add the squares, and divide by the number of observations. Training mean squared error uses the observations that selected the fitted curve. Test mean squared error repeats the same calculation on observations held aside from that selection. The formulas look nearly identical. Their difference is experimental: one asks how well the model remembers its lesson, and the other asks how well it handles a new question. ### [05:27.369 · Error Against Complexity](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=327.3694583333333) Now repeat the fitting experiment for degree one, degree two, degree three, and so on up to degree nineteen. For every fitted model, record both errors. The blue curve is training error. It falls as degree increases because each larger polynomial family contains more ways to accommodate the observations. That decrease is built into the fitting process. A more flexible model can imitate a simpler one if that is best, or use its extra freedom to reduce the residuals further. The red curve is test error. At first it also falls. Moving beyond a straight line lets the model capture the real curvature, so predictions improve. Near degree five, test error reaches its minimum. In this experiment that is the useful region: enough flexibility to represent the signal, without enough freedom to chase every disturbance. Past that point, training error keeps falling but test error turns upward. The two measurements separate because extra complexity is helping on observations already seen while hurting predictions elsewhere. At degree nineteen the training point is almost on the horizontal axis. The test point is high. Their vertical separation is the observable signature of the interpolation failure we just watched. Low complexity and high complexity therefore have different diagnoses. On the left, both errors are large because the model misses the signal. On the right, only training error is tiny, because the model has learned details that do not transfer. Selecting the degree with the smallest training error would always reward the most flexible candidate. That is exactly the decision rule that invites overfitting. Instead, reserve data for validation or testing and look for low error there. The marked point is not a law that degree five is always right. It is the answer for this relationship, this noise level, and this amount of data. Change the noise, the sampling locations, or the amount of data, and the red curve can change. That dependence is the doorway to bias and variance. ### [07:41.303 · Bias and Variance](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=461.30274999999995) Imagine repeating the entire study. Draw twenty fresh noisy observations from the same relationship, fit the model, discard that sample, and do it again. On the left, each gray line is a straight model fitted to a different fresh sample. Here is the first fit, then the second, the third, the fourth, and the fifth. The fitted lines move a little because the observations change, but they remain a tight family. Their small spread means low variance. Now average the five fitted predictions at every x. The red line is that average. It still misses the blue curve in the same organized way, especially near the ends. That persistent difference is bias. Even if we repeated the study many more times, averaging straight lines would still produce a straight line. Repetition cannot create curvature that the model family forbids. Now repeat the same experiment with a highly flexible polynomial. The blue curve on the right is still the same relationship. Each colored curve is a refit on a new sample of twenty observations. One sample pulls the curve upward here, another pulls it downward, and another creates a different sequence of bends. The model family is capable of following the underlying curve, so its average can be close to the truth. But the individual fits spread widely. Which data set happened to arrive has a large effect on the prediction. That sensitivity is variance. High variance means that if we repeated the data collection, our fitted answer could change substantially even though the data-generating process did not. Bias and variance are therefore not labels attached permanently to one curve. They describe behavior across repeated samples. Bias asks where the average fit lands. Variance asks how widely the individual fits scatter around that average. Bias is the difference between the average fitted prediction and the true relationship. A rigid straight model has substantial bias here because its average prediction cannot reproduce the bend. Variance is the spread of fitted predictions at the same input across fresh samples. A flexible model can have low bias and still have high variance if each sample leads it toward a different detailed curve. Underfitting is the visible face of excessive bias. Overfitting is the visible face of excessive sensitivity to the sample. The useful model must control both. ### [10:22.61 · More Data Moves the Balance](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=622.6099166666665) Return to repeated fitting, but change the amount of evidence. On the left, each colored flexible curve was fitted from only twenty observations. The fits disagree because a handful of noisy points can pull a flexible polynomial in noticeably different directions. The blue relationship is fixed; the fitted curves are not. On the right, fit the same degree of polynomial after collecting one hundred observations. Each individual observation now has less power to redirect the entire curve. The colored curves still differ, because noise has not vanished. But their spread is much narrower. More data has reduced variance without changing the basic model family. Notice what more data did not do. It did not make every complex model safe, and it would not make a straight line bend. It made the fitted details less dependent on a few particular observations. We can summarize that change with two error contributions. Blue is error from bias. It falls as complexity gives the model more shapes to choose from. Red is error from variance. With only twenty observations, it rises as flexible models become increasingly sensitive to the sample. The two contributions cross near this yellow point. To the left, missing structure dominates. To the right, sample sensitivity dominates. The crossing is a useful picture of balance, not a universal decision rule. Now keep the blue bias curve and increase the data to one hundred observations. The right-hand graph shows the same declining bias contribution. The red variance contribution is lower because repeated fits are more stable. Its crossing with bias moves to the right, toward a more flexible polynomial. This does not say that complexity is good by itself. It says complexity becomes easier to support when the data constrain it from many directions. The final choice still belongs to validation or test performance. The point of the whole experiment is now visible. Underfitting is a systematic miss that survives more observations. Overfitting is a gap between fitting performance and new-data performance, driven by excessive sensitivity to the sample. First, training error asks about observations the model already used. Second, test error asks about genuinely separate observations. The difference between those questions is what makes overfitting measurable. Third, bias is the persistent miss seen across fresh samples. Fourth, variance is the spread of the refitted predictions, and adding data can narrow that spread. A useful model has enough freedom to represent the relationship and enough evidence to keep that freedom under control. The goal is not a curve that remembers every point. It is a curve whose accuracy survives the arrival of the next point. ## About Academa, Inc. Academa makes technical knowledge easier to understand through visual lectures and lets learners request new lecture videos on the topics they need. ## Complete audiovisual record Immutable source: [semantic.json](https://academa.ai/media/l/01M14TYJMMAQ9SQDTPSVK4XZGD/0/semantic.json) Record version: 1. Render attempt: 0. ### How to read this timeline Each scene owns its object identifiers. A beat's board is the complete board when listed, empty when marked empty, and unchanged from the nearest earlier listed board in the same scene when marked unchanged. Action times are absolute positions in the published video. ### Scene 1: [One Data Set, Many Models](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=0) Span: 00:00–02:47.812 (0s–167.81166666666664s). #### Objects - axes: an Axes (x\_range=(-3.0, 3.0), y\_range=(-3.6, 1.8), x\_ticks\_every=1.0) - caption: a Math \[text\] that says "$upright("polynomial degree") = 1$" - card: a Title that says "Practical Machine Learning — Overfitting Made Visible: Bias, Variance, and the Role of Data" - cubic\_fit: a FunctionPlot \[green\] labelled "upright("degree 3")" drawn in axes (function=\, x\_range=(-3.0, 3.0)) - data\_marks: a Point \[yellow\] drawn in axes (location=(3.0, 0.9300000000000002)) - data\_marks\_10: a Point \[yellow\] drawn in axes (location=(0.24773803641699718, 0.025208575785553577)) - data\_marks\_11: a Point \[yellow\] drawn in axes (location=(-0.24773803641699682, 0.7526967357268568)) - data\_marks\_12: a Point \[yellow\] drawn in axes (location=(-0.7364564614223967, -0.6026773153050619)) - data\_marks\_13: a Point \[yellow\] drawn in axes (location=(-1.2050862739589083, -0.02419937766054925)) - data\_marks\_14: a Point \[yellow\] drawn in axes (location=(-1.640844474367281, -1.4870911669330995)) - data\_marks\_15: a Point \[yellow\] drawn in axes (location=(-2.0318447148772227, -0.9606253233498832)) - data\_marks\_16: a Point \[yellow\] drawn in axes (location=(-2.3674215281891806, -2.4109250850880968)) - data\_marks\_17: a Point \[yellow\] drawn in axes (location=(-2.638421253619467, -1.8041596975698724)) - data\_marks\_18: a Point \[yellow\] drawn in axes (location=(-2.8374517251019045, -3.1098022614171272)) - data\_marks\_19: a Point \[yellow\] drawn in axes (location=(-2.959083910208167, -2.303608116392006)) - data\_marks\_2: a Point \[yellow\] drawn in axes (location=(2.959083910208167, -0.048615815163021825)) - data\_marks\_20: a Point \[yellow\] drawn in axes (location=(-3.0, -3.37)) - data\_marks\_3: a Point \[yellow\] drawn in axes (location=(2.837451725101904, 1.0113946361949688)) - data\_marks\_4: a Point \[yellow\] drawn in axes (location=(2.6384212536194673, 0.09810368141154147)) - data\_marks\_5: a Point \[yellow\] drawn in axes (location=(2.3674215281891806, 1.1932385959200025)) - data\_marks\_6: a Point \[yellow\] drawn in axes (location=(2.031844714877223, 0.2744038630150619)) - data\_marks\_7: a Point \[yellow\] drawn in axes (location=(1.6408444743672808, 1.31783775487091)) - data\_marks\_8: a Point \[yellow\] drawn in axes (location=(1.2050862739589085, 0.3013955236942498)) - data\_marks\_9: a Point \[yellow\] drawn in axes (location=(0.7364564614223977, 1.2074247922595749)) - heading: a Heading that says "The Same Twenty Observations" - line\_fit: a FunctionPlot \[green\] labelled "upright("degree 1")" drawn in axes (function=\, x\_range=(-3.0, 3.0)) - nineteen\_fit: a FunctionPlot \[red\] labelled "upright("degree 19")" drawn in axes (function=\, x\_range=(-3.0, 3.0)) - point: a Point \[yellow\] drawn in axes (location=(-2.3, -1.8171999999999997)) - point\_2: a Point \[yellow\] drawn in axes (location=(1.3, 0.8108000000000002)) - point\_3: a Point \[yellow\] drawn in axes (location=(-0.8, -0.6124887971186718)) - point\_4: a Point \[yellow\] drawn in axes (location=(0.8, 1.1820887971186707)) - seven\_fit: a FunctionPlot \[green\] labelled "upright("degree 7")" drawn in axes (function=\, x\_range=(-3.0, 3.0)) - true\_curve: a FunctionPlot \[blue\] labelled "upright("true relationship")" drawn in axes (function=\, x\_range=(-3.0, 3.0)) #### Beats ##### [00:00](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=0) Narration: Overfitting is often used as a warning word: this model overfits, that model generalizes. I want to make the warning visible and measurable. We will give several polynomial models the same twenty noisy observations, then watch each model decide what story those observations tell. Board: Empty. Actions: - [00:00](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=0): card is shown on the screen, written out. - [00:1.5](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=1.5): card: enter:write-left-to-right. - [00:18.367](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=18.367): card is hidden from the screen — left the board. ##### [00:19.567](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=19.567) Narration: Here are the twenty observations. Each yellow dot contains two things: a curved underlying relationship and a small disturbance that pushed the measurement above or below that relationship. Board: Unchanged from the preceding beat in this scene. Actions: - [00:19.567](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=19.567): heading is shown on the screen, written out. - [00:19.567](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=19.567): axes is shown on the screen, written out. - [00:20.531](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=20.531): data\_marks is shown on the screen, written out. - [00:20.561](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=20.561): data\_marks\_2 is shown on the screen, written out. - [00:20.591](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=20.590999999999998): data\_marks\_3 is shown on the screen, written out. - [00:20.621](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=20.621): data\_marks\_4 is shown on the screen, written out. - [00:20.651](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=20.651): data\_marks\_5 is shown on the screen, written out. - [00:20.681](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=20.680999999999997): data\_marks\_6 is shown on the screen, written out. - [00:20.711](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=20.711): data\_marks\_7 is shown on the screen, written out. - [00:20.741](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=20.741): data\_marks\_8 is shown on the screen, written out. - [00:20.771](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=20.770999999999997): data\_marks\_9 is shown on the screen, written out. - [00:20.801](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=20.801): data\_marks\_10 is shown on the screen, written out. - [00:20.831](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=20.831): data\_marks\_11 is shown on the screen, written out. - [00:20.861](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=20.860999999999997): data\_marks\_12 is shown on the screen, written out. - [00:20.891](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=20.891): data\_marks\_13 is shown on the screen, written out. - [00:20.921](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=20.921): data\_marks\_14 is shown on the screen, written out. - [00:20.951](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=20.951): data\_marks\_15 is shown on the screen, written out. - [00:20.981](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=20.980999999999998): data\_marks\_16 is shown on the screen, written out. - [00:21.011](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=21.011): data\_marks\_17 is shown on the screen, written out. - [00:21.041](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=21.041): data\_marks\_18 is shown on the screen, written out. - [00:21.071](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=21.070999999999998): data\_marks\_19 is shown on the screen, written out. - [00:21.101](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=21.101): data\_marks\_20 is shown on the screen, written out. ##### [00:32.996](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=32.996) Narration: Suppose we know only the dots. Our first candidate is a polynomial of degree one, which is simply a straight line. It can tilt and it can move up or down, but it cannot bend. Board: axes — an Axes (x\_range=(-3.0, 3.0), y\_range=(-3.6, 1.8), x\_ticks\_every=1.0); heading — a Heading that says "The Same Twenty Observations"; data\_marks — a Point \[yellow\] drawn in axes (location=(3.0, 0.9300000000000002)); data\_marks\_2 — a Point \[yellow\] drawn in axes (location=(2.959083910208167, -0.048615815163021825)); data\_marks\_3 — a Point \[yellow\] drawn in axes (location=(2.837451725101904, 1.0113946361949688)); data\_marks\_4 — a Point \[yellow\] drawn in axes (location=(2.6384212536194673, 0.09810368141154147)); data\_marks\_5 — a Point \[yellow\] drawn in axes (location=(2.3674215281891806, 1.1932385959200025)); data\_marks\_6 — a Point \[yellow\] drawn in axes (location=(2.031844714877223, 0.2744038630150619)); data\_marks\_7 — a Point \[yellow\] drawn in axes (location=(1.6408444743672808, 1.31783775487091)); data\_marks\_8 — a Point \[yellow\] drawn in axes (location=(1.2050862739589085, 0.3013955236942498)); data\_marks\_9 — a Point \[yellow\] drawn in axes (location=(0.7364564614223977, 1.2074247922595749)); data\_marks\_10 — a Point \[yellow\] drawn in axes (location=(0.24773803641699718, 0.025208575785553577)); data\_marks\_11 — a Point \[yellow\] drawn in axes (location=(-0.24773803641699682, 0.7526967357268568)); data\_marks\_12 — a Point \[yellow\] drawn in axes (location=(-0.7364564614223967, -0.6026773153050619)); data\_marks\_13 — a Point \[yellow\] drawn in axes (location=(-1.2050862739589083, -0.02419937766054925)); data\_marks\_14 — a Point \[yellow\] drawn in axes (location=(-1.640844474367281, -1.4870911669330995)); data\_marks\_15 — a Point \[yellow\] drawn in axes (location=(-2.0318447148772227, -0.9606253233498832)); data\_marks\_16 — a Point \[yellow\] drawn in axes (location=(-2.3674215281891806, -2.4109250850880968)); data\_marks\_17 — a Point \[yellow\] drawn in axes (location=(-2.638421253619467, -1.8041596975698724)); data\_marks\_18 — a Point \[yellow\] drawn in axes (location=(-2.8374517251019045, -3.1098022614171272)); data\_marks\_19 — a Point \[yellow\] drawn in axes (location=(-2.959083910208167, -2.303608116392006)); data\_marks\_20 — a Point \[yellow\] drawn in axes (location=(-3.0, -3.37)) Actions: - [00:37.988](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=37.988): caption is shown on the screen, written out. - [00:39.916](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=39.916): line\_fit is shown on the screen, drawn. ##### [00:45.508](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=45.508) Narration: The line catches the broad upward trend, but look at the ends and the middle. The dots curve away from it in an organized pattern. Those are not isolated misses. The model lacks the shape needed to follow the relationship. Board: axes — an Axes (x\_range=(-3.0, 3.0), y\_range=(-3.6, 1.8), x\_ticks\_every=1.0); caption — a Math \[text\] that says "$upright("polynomial degree") = 1$"; heading — a Heading that says "The Same Twenty Observations"; data\_marks — a Point \[yellow\] drawn in axes (location=(3.0, 0.9300000000000002)); data\_marks\_2 — a Point \[yellow\] drawn in axes (location=(2.959083910208167, -0.048615815163021825)); data\_marks\_3 — a Point \[yellow\] drawn in axes (location=(2.837451725101904, 1.0113946361949688)); data\_marks\_4 — a Point \[yellow\] drawn in axes (location=(2.6384212536194673, 0.09810368141154147)); data\_marks\_5 — a Point \[yellow\] drawn in axes (location=(2.3674215281891806, 1.1932385959200025)); data\_marks\_6 — a Point \[yellow\] drawn in axes (location=(2.031844714877223, 0.2744038630150619)); data\_marks\_7 — a Point \[yellow\] drawn in axes (location=(1.6408444743672808, 1.31783775487091)); data\_marks\_8 — a Point \[yellow\] drawn in axes (location=(1.2050862739589085, 0.3013955236942498)); data\_marks\_9 — a Point \[yellow\] drawn in axes (location=(0.7364564614223977, 1.2074247922595749)); data\_marks\_10 — a Point \[yellow\] drawn in axes (location=(0.24773803641699718, 0.025208575785553577)); data\_marks\_11 — a Point \[yellow\] drawn in axes (location=(-0.24773803641699682, 0.7526967357268568)); data\_marks\_12 — a Point \[yellow\] drawn in axes (location=(-0.7364564614223967, -0.6026773153050619)); data\_marks\_13 — a Point \[yellow\] drawn in axes (location=(-1.2050862739589083, -0.02419937766054925)); data\_marks\_14 — a Point \[yellow\] drawn in axes (location=(-1.640844474367281, -1.4870911669330995)); data\_marks\_15 — a Point \[yellow\] drawn in axes (location=(-2.0318447148772227, -0.9606253233498832)); data\_marks\_16 — a Point \[yellow\] drawn in axes (location=(-2.3674215281891806, -2.4109250850880968)); data\_marks\_17 — a Point \[yellow\] drawn in axes (location=(-2.638421253619467, -1.8041596975698724)); data\_marks\_18 — a Point \[yellow\] drawn in axes (location=(-2.8374517251019045, -3.1098022614171272)); data\_marks\_19 — a Point \[yellow\] drawn in axes (location=(-2.959083910208167, -2.303608116392006)); data\_marks\_20 — a Point \[yellow\] drawn in axes (location=(-3.0, -3.37)); line\_fit — a FunctionPlot \[green\] labelled "upright("degree 1")" drawn in axes (function=\, x\_range=(-3.0, 3.0)) Actions: - [00:48.945](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=48.94500000000001): point is shown on the screen, grown. - [00:49.757](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=49.757000000000005): point\_2 is shown on the screen, grown. - [00:50.945](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=50.94500000000001): point is hidden from the screen. - [00:51.757](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=51.757000000000005): point\_2 is hidden from the screen. ##### [01:0.04](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=60.040000000000006) Narration: Allow degree three. The line gives way to a curve, and the fit can now follow the large bend. It still refuses to chase every individual dot. Board: Unchanged from the preceding beat in this scene. Actions: - [01:1.178](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=61.178000000000004): caption becomes "$upright("polynomial degree") = 3$". - [01:1.178](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=61.178000000000004): line\_fit is hidden from the screen. - [01:1.178](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=61.178000000000004): cubic\_fit is shown on the screen, written out. ##### [01:10.59](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=70.5895) Narration: That refusal is useful. A noisy observation is not the relationship itself. Some of the vertical displacement belongs to chance, so a model that leaves small residuals may be behaving more sensibly than one that erases them. Board: axes — an Axes (x\_range=(-3.0, 3.0), y\_range=(-3.6, 1.8), x\_ticks\_every=1.0); caption — a Math \[text\] that says "$upright("polynomial degree") = 1$"; heading — a Heading that says "The Same Twenty Observations"; data\_marks — a Point \[yellow\] drawn in axes (location=(3.0, 0.9300000000000002)); data\_marks\_2 — a Point \[yellow\] drawn in axes (location=(2.959083910208167, -0.048615815163021825)); data\_marks\_3 — a Point \[yellow\] drawn in axes (location=(2.837451725101904, 1.0113946361949688)); data\_marks\_4 — a Point \[yellow\] drawn in axes (location=(2.6384212536194673, 0.09810368141154147)); data\_marks\_5 — a Point \[yellow\] drawn in axes (location=(2.3674215281891806, 1.1932385959200025)); data\_marks\_6 — a Point \[yellow\] drawn in axes (location=(2.031844714877223, 0.2744038630150619)); data\_marks\_7 — a Point \[yellow\] drawn in axes (location=(1.6408444743672808, 1.31783775487091)); data\_marks\_8 — a Point \[yellow\] drawn in axes (location=(1.2050862739589085, 0.3013955236942498)); data\_marks\_9 — a Point \[yellow\] drawn in axes (location=(0.7364564614223977, 1.2074247922595749)); data\_marks\_10 — a Point \[yellow\] drawn in axes (location=(0.24773803641699718, 0.025208575785553577)); data\_marks\_11 — a Point \[yellow\] drawn in axes (location=(-0.24773803641699682, 0.7526967357268568)); data\_marks\_12 — a Point \[yellow\] drawn in axes (location=(-0.7364564614223967, -0.6026773153050619)); data\_marks\_13 — a Point \[yellow\] drawn in axes (location=(-1.2050862739589083, -0.02419937766054925)); data\_marks\_14 — a Point \[yellow\] drawn in axes (location=(-1.640844474367281, -1.4870911669330995)); data\_marks\_15 — a Point \[yellow\] drawn in axes (location=(-2.0318447148772227, -0.9606253233498832)); data\_marks\_16 — a Point \[yellow\] drawn in axes (location=(-2.3674215281891806, -2.4109250850880968)); data\_marks\_17 — a Point \[yellow\] drawn in axes (location=(-2.638421253619467, -1.8041596975698724)); data\_marks\_18 — a Point \[yellow\] drawn in axes (location=(-2.8374517251019045, -3.1098022614171272)); data\_marks\_19 — a Point \[yellow\] drawn in axes (location=(-2.959083910208167, -2.303608116392006)); data\_marks\_20 — a Point \[yellow\] drawn in axes (location=(-3.0, -3.37)); cubic\_fit — a FunctionPlot \[green\] labelled "upright("degree 3")" drawn in axes (function=\, x\_range=(-3.0, 3.0)) Actions: - [01:13.701](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=73.701): data\_marks\_5 is indicated — a transient flash. - [01:21.201](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=81.201): data\_marks\_12 is indicated — a transient flash. ##### [01:25.818](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=85.818) Narration: Now allow degree seven. The curve gains more freedom. It bends toward several individual observations and begins to reproduce their alternating rises and falls. Board: Unchanged from the preceding beat in this scene. Actions: - [01:27.13](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=87.13): caption becomes "$upright("polynomial degree") = 7$". - [01:27.13](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=87.13): cubic\_fit is hidden from the screen. - [01:27.13](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=87.13): seven\_fit is shown on the screen, written out. ##### [01:36.715](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=96.71549999999999) Narration: Finally allow degree nineteen. Twenty coefficients are enough to force a polynomial through twenty distinct points. The red curve touches every yellow dot exactly. Board: axes — an Axes (x\_range=(-3.0, 3.0), y\_range=(-3.6, 1.8), x\_ticks\_every=1.0); caption — a Math \[text\] that says "$upright("polynomial degree") = 1$"; heading — a Heading that says "The Same Twenty Observations"; data\_marks — a Point \[yellow\] drawn in axes (location=(3.0, 0.9300000000000002)); data\_marks\_2 — a Point \[yellow\] drawn in axes (location=(2.959083910208167, -0.048615815163021825)); data\_marks\_3 — a Point \[yellow\] drawn in axes (location=(2.837451725101904, 1.0113946361949688)); data\_marks\_4 — a Point \[yellow\] drawn in axes (location=(2.6384212536194673, 0.09810368141154147)); data\_marks\_5 — a Point \[yellow\] drawn in axes (location=(2.3674215281891806, 1.1932385959200025)); data\_marks\_6 — a Point \[yellow\] drawn in axes (location=(2.031844714877223, 0.2744038630150619)); data\_marks\_7 — a Point \[yellow\] drawn in axes (location=(1.6408444743672808, 1.31783775487091)); data\_marks\_8 — a Point \[yellow\] drawn in axes (location=(1.2050862739589085, 0.3013955236942498)); data\_marks\_9 — a Point \[yellow\] drawn in axes (location=(0.7364564614223977, 1.2074247922595749)); data\_marks\_10 — a Point \[yellow\] drawn in axes (location=(0.24773803641699718, 0.025208575785553577)); data\_marks\_11 — a Point \[yellow\] drawn in axes (location=(-0.24773803641699682, 0.7526967357268568)); data\_marks\_12 — a Point \[yellow\] drawn in axes (location=(-0.7364564614223967, -0.6026773153050619)); data\_marks\_13 — a Point \[yellow\] drawn in axes (location=(-1.2050862739589083, -0.02419937766054925)); data\_marks\_14 — a Point \[yellow\] drawn in axes (location=(-1.640844474367281, -1.4870911669330995)); data\_marks\_15 — a Point \[yellow\] drawn in axes (location=(-2.0318447148772227, -0.9606253233498832)); data\_marks\_16 — a Point \[yellow\] drawn in axes (location=(-2.3674215281891806, -2.4109250850880968)); data\_marks\_17 — a Point \[yellow\] drawn in axes (location=(-2.638421253619467, -1.8041596975698724)); data\_marks\_18 — a Point \[yellow\] drawn in axes (location=(-2.8374517251019045, -3.1098022614171272)); data\_marks\_19 — a Point \[yellow\] drawn in axes (location=(-2.959083910208167, -2.303608116392006)); data\_marks\_20 — a Point \[yellow\] drawn in axes (location=(-3.0, -3.37)); seven\_fit — a FunctionPlot \[green\] labelled "upright("degree 7")" drawn in axes (function=\, x\_range=(-3.0, 3.0)) Actions: - [01:38.48](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=98.47999999999999): caption becomes "$upright("polynomial degree") = 19$". - [01:38.48](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=98.47999999999999): seven\_fit is hidden from the screen. - [01:38.48](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=98.47999999999999): nineteen\_fit is shown on the screen, written out. ##### [01:49.065](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=109.065) Narration: On the training observations this looks perfect. Between the observations, however, the curve changes direction again and again. It has treated each upward or downward disturbance as if it were a real feature of the world. Board: axes — an Axes (x\_range=(-3.0, 3.0), y\_range=(-3.6, 1.8), x\_ticks\_every=1.0); caption — a Math \[text\] that says "$upright("polynomial degree") = 1$"; heading — a Heading that says "The Same Twenty Observations"; data\_marks — a Point \[yellow\] drawn in axes (location=(3.0, 0.9300000000000002)); data\_marks\_2 — a Point \[yellow\] drawn in axes (location=(2.959083910208167, -0.048615815163021825)); data\_marks\_3 — a Point \[yellow\] drawn in axes (location=(2.837451725101904, 1.0113946361949688)); data\_marks\_4 — a Point \[yellow\] drawn in axes (location=(2.6384212536194673, 0.09810368141154147)); data\_marks\_5 — a Point \[yellow\] drawn in axes (location=(2.3674215281891806, 1.1932385959200025)); data\_marks\_6 — a Point \[yellow\] drawn in axes (location=(2.031844714877223, 0.2744038630150619)); data\_marks\_7 — a Point \[yellow\] drawn in axes (location=(1.6408444743672808, 1.31783775487091)); data\_marks\_8 — a Point \[yellow\] drawn in axes (location=(1.2050862739589085, 0.3013955236942498)); data\_marks\_9 — a Point \[yellow\] drawn in axes (location=(0.7364564614223977, 1.2074247922595749)); data\_marks\_10 — a Point \[yellow\] drawn in axes (location=(0.24773803641699718, 0.025208575785553577)); data\_marks\_11 — a Point \[yellow\] drawn in axes (location=(-0.24773803641699682, 0.7526967357268568)); data\_marks\_12 — a Point \[yellow\] drawn in axes (location=(-0.7364564614223967, -0.6026773153050619)); data\_marks\_13 — a Point \[yellow\] drawn in axes (location=(-1.2050862739589083, -0.02419937766054925)); data\_marks\_14 — a Point \[yellow\] drawn in axes (location=(-1.640844474367281, -1.4870911669330995)); data\_marks\_15 — a Point \[yellow\] drawn in axes (location=(-2.0318447148772227, -0.9606253233498832)); data\_marks\_16 — a Point \[yellow\] drawn in axes (location=(-2.3674215281891806, -2.4109250850880968)); data\_marks\_17 — a Point \[yellow\] drawn in axes (location=(-2.638421253619467, -1.8041596975698724)); data\_marks\_18 — a Point \[yellow\] drawn in axes (location=(-2.8374517251019045, -3.1098022614171272)); data\_marks\_19 — a Point \[yellow\] drawn in axes (location=(-2.959083910208167, -2.303608116392006)); data\_marks\_20 — a Point \[yellow\] drawn in axes (location=(-3.0, -3.37)); nineteen\_fit — a FunctionPlot \[red\] labelled "upright("degree 19")" drawn in axes (function=\, x\_range=(-3.0, 3.0)) Actions: - [01:52.293](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=112.29299999999998): point\_3 is shown on the screen, grown. - [01:54.267](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=114.26699999999998): nineteen\_fit is indicated — a transient flash. - [01:54.293](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=114.29299999999998): point\_3 is hidden from the screen. - [01:55.59](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=115.58999999999997): point\_4 is shown on the screen, grown. - [01:57.59](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=117.58999999999997): point\_4 is hidden from the screen. ##### [02:3.4](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=123.39999999999999) Narration: Because this is a controlled example, we may reveal the relationship that generated the observations. The blue curve is the signal. The red curve agrees with all twenty recorded values, yet it disagrees repeatedly with the relationship between them. Board: Unchanged from the preceding beat in this scene. Actions: - [02:10.029](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=130.029): true\_curve is shown on the screen, drawn. - [02:11.155](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=131.15499999999997): true\_curve is indicated — a transient flash. - [02:12.502](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=132.50199999999998): nineteen\_fit is indicated — a transient flash. ##### [02:19.649](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=139.6495) Narration: So failure happens in two directions. The straight line is too rigid to represent the signal. The degree-nineteen polynomial is flexible enough to represent both the signal and the accidental noise. One underfits; the other overfits. Board: axes — an Axes (x\_range=(-3.0, 3.0), y\_range=(-3.6, 1.8), x\_ticks\_every=1.0); caption — a Math \[text\] that says "$upright("polynomial degree") = 1$"; heading — a Heading that says "The Same Twenty Observations"; data\_marks — a Point \[yellow\] drawn in axes (location=(3.0, 0.9300000000000002)); data\_marks\_2 — a Point \[yellow\] drawn in axes (location=(2.959083910208167, -0.048615815163021825)); data\_marks\_3 — a Point \[yellow\] drawn in axes (location=(2.837451725101904, 1.0113946361949688)); data\_marks\_4 — a Point \[yellow\] drawn in axes (location=(2.6384212536194673, 0.09810368141154147)); data\_marks\_5 — a Point \[yellow\] drawn in axes (location=(2.3674215281891806, 1.1932385959200025)); data\_marks\_6 — a Point \[yellow\] drawn in axes (location=(2.031844714877223, 0.2744038630150619)); data\_marks\_7 — a Point \[yellow\] drawn in axes (location=(1.6408444743672808, 1.31783775487091)); data\_marks\_8 — a Point \[yellow\] drawn in axes (location=(1.2050862739589085, 0.3013955236942498)); data\_marks\_9 — a Point \[yellow\] drawn in axes (location=(0.7364564614223977, 1.2074247922595749)); data\_marks\_10 — a Point \[yellow\] drawn in axes (location=(0.24773803641699718, 0.025208575785553577)); data\_marks\_11 — a Point \[yellow\] drawn in axes (location=(-0.24773803641699682, 0.7526967357268568)); data\_marks\_12 — a Point \[yellow\] drawn in axes (location=(-0.7364564614223967, -0.6026773153050619)); data\_marks\_13 — a Point \[yellow\] drawn in axes (location=(-1.2050862739589083, -0.02419937766054925)); data\_marks\_14 — a Point \[yellow\] drawn in axes (location=(-1.640844474367281, -1.4870911669330995)); data\_marks\_15 — a Point \[yellow\] drawn in axes (location=(-2.0318447148772227, -0.9606253233498832)); data\_marks\_16 — a Point \[yellow\] drawn in axes (location=(-2.3674215281891806, -2.4109250850880968)); data\_marks\_17 — a Point \[yellow\] drawn in axes (location=(-2.638421253619467, -1.8041596975698724)); data\_marks\_18 — a Point \[yellow\] drawn in axes (location=(-2.8374517251019045, -3.1098022614171272)); data\_marks\_19 — a Point \[yellow\] drawn in axes (location=(-2.959083910208167, -2.303608116392006)); data\_marks\_20 — a Point \[yellow\] drawn in axes (location=(-3.0, -3.37)); nineteen\_fit — a FunctionPlot \[red\] labelled "upright("degree 19")" drawn in axes (function=\, x\_range=(-3.0, 3.0)); true\_curve — a FunctionPlot \[blue\] labelled "upright("true relationship")" drawn in axes (function=\, x\_range=(-3.0, 3.0)) Actions: - [02:26](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=145.99999999999997): caption is indicated — a transient flash. ##### [02:35.838](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=155.838) Narration: The central question is therefore not whether a model can fit these observations. It is whether the fitted pattern continues to work at places the fitting process did not get to see. Board: Unchanged from the preceding beat in this scene. Actions: - [02:46.77](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=166.76999999999998): axes is hidden from the screen — left the board. - [02:46.77](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=166.76999999999998): data\_marks is hidden from the screen — axes left the board. - [02:46.77](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=166.76999999999998): data\_marks\_2 is hidden from the screen — axes left the board. - [02:46.77](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=166.76999999999998): data\_marks\_3 is hidden from the screen — axes left the board. - [02:46.77](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=166.76999999999998): data\_marks\_4 is hidden from the screen — axes left the board. - [02:46.77](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=166.76999999999998): data\_marks\_5 is hidden from the screen — axes left the board. - [02:46.77](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=166.76999999999998): data\_marks\_6 is hidden from the screen — axes left the board. - [02:46.77](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=166.76999999999998): data\_marks\_7 is hidden from the screen — axes left the board. - [02:46.77](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=166.76999999999998): data\_marks\_8 is hidden from the screen — axes left the board. - [02:46.77](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=166.76999999999998): data\_marks\_9 is hidden from the screen — axes left the board. - [02:46.77](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=166.76999999999998): data\_marks\_10 is hidden from the screen — axes left the board. - [02:46.77](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=166.76999999999998): data\_marks\_11 is hidden from the screen — axes left the board. - [02:46.77](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=166.76999999999998): data\_marks\_12 is hidden from the screen — axes left the board. - [02:46.77](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=166.76999999999998): data\_marks\_13 is hidden from the screen — axes left the board. - [02:46.77](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=166.76999999999998): data\_marks\_14 is hidden from the screen — axes left the board. - [02:46.77](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=166.76999999999998): data\_marks\_15 is hidden from the screen — axes left the board. - [02:46.77](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=166.76999999999998): data\_marks\_16 is hidden from the screen — axes left the board. - [02:46.77](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=166.76999999999998): data\_marks\_17 is hidden from the screen — axes left the board. - [02:46.77](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=166.76999999999998): data\_marks\_18 is hidden from the screen — axes left the board. - [02:46.77](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=166.76999999999998): data\_marks\_19 is hidden from the screen — axes left the board. - [02:46.77](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=166.76999999999998): data\_marks\_20 is hidden from the screen — axes left the board. - [02:46.77](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=166.76999999999998): nineteen\_fit is hidden from the screen — axes left the board. - [02:46.77](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=166.76999999999998): true\_curve is hidden from the screen — axes left the board. - [02:46.77](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=166.76999999999998): caption is hidden from the screen — left the board. - [02:46.77](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=166.76999999999998): heading is hidden from the screen — left the board. ### Scene 2: [Two Kinds of Failure](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=167.81166666666664) Span: 02:47.812–05:27.369 (167.81166666666664s–327.3694583333333s). #### Objects - definitions: a Panel that says "Training error measures predictions on observations used to fit the model. Test error measures predictions on separate observations that played no part in fitting it." - heading: a Heading that says "Failure in Both Directions" - heading\_2: a Heading that says "Training and Test Error" - left\_label: a Tex \[text\] that says "Too rigid" - mse\_test: a Math \[text\] that says "$upright("MSE")\_(upright("test")) = frac(1, m) sum\_j (y\_j - hat(y)\_j)^2$" - mse\_train: a Math \[text\] that says "$upright("MSE")\_(upright("train")) = frac(1, n) sum\_i (y\_i - hat(y)\_i)^2$" - over\_axes: an Axes (x\_range=(-3.0, 3.0), y\_range=(-3.6, 1.8), x\_ticks\_every=1.0) - over\_fit: a FunctionPlot \[red\] drawn in over\_axes (function=\, x\_range=(-3.0, 3.0)) - over\_truth: a FunctionPlot \[blue\] drawn in over\_axes (function=\, x\_range=(-3.0, 3.0)) - over\_twenty: a Point \[yellow\] drawn in over\_axes (location=(3.0, 0.9300000000000002)) - over\_twenty\_10: a Point \[yellow\] drawn in over\_axes (location=(0.24773803641699718, 0.025208575785553577)) - over\_twenty\_11: a Point \[yellow\] drawn in over\_axes (location=(-0.24773803641699682, 0.7526967357268568)) - over\_twenty\_12: a Point \[yellow\] drawn in over\_axes (location=(-0.7364564614223967, -0.6026773153050619)) - over\_twenty\_13: a Point \[yellow\] drawn in over\_axes (location=(-1.2050862739589083, -0.02419937766054925)) - over\_twenty\_14: a Point \[yellow\] drawn in over\_axes (location=(-1.640844474367281, -1.4870911669330995)) - over\_twenty\_15: a Point \[yellow\] drawn in over\_axes (location=(-2.0318447148772227, -0.9606253233498832)) - over\_twenty\_16: a Point \[yellow\] drawn in over\_axes (location=(-2.3674215281891806, -2.4109250850880968)) - over\_twenty\_17: a Point \[yellow\] drawn in over\_axes (location=(-2.638421253619467, -1.8041596975698724)) - over\_twenty\_18: a Point \[yellow\] drawn in over\_axes (location=(-2.8374517251019045, -3.1098022614171272)) - over\_twenty\_19: a Point \[yellow\] drawn in over\_axes (location=(-2.959083910208167, -2.303608116392006)) - over\_twenty\_2: a Point \[yellow\] drawn in over\_axes (location=(2.959083910208167, -0.048615815163021825)) - over\_twenty\_20: a Point \[yellow\] drawn in over\_axes (location=(-3.0, -3.37)) - over\_twenty\_3: a Point \[yellow\] drawn in over\_axes (location=(2.837451725101904, 1.0113946361949688)) - over\_twenty\_4: a Point \[yellow\] drawn in over\_axes (location=(2.6384212536194673, 0.09810368141154147)) - over\_twenty\_5: a Point \[yellow\] drawn in over\_axes (location=(2.3674215281891806, 1.1932385959200025)) - over\_twenty\_6: a Point \[yellow\] drawn in over\_axes (location=(2.031844714877223, 0.2744038630150619)) - over\_twenty\_7: a Point \[yellow\] drawn in over\_axes (location=(1.6408444743672808, 1.31783775487091)) - over\_twenty\_8: a Point \[yellow\] drawn in over\_axes (location=(1.2050862739589085, 0.3013955236942498)) - over\_twenty\_9: a Point \[yellow\] drawn in over\_axes (location=(0.7364564614223977, 1.2074247922595749)) - right\_label: a Tex \[text\] that says "Too responsive" - test\_marks: a Point \[green\] drawn in over\_axes (location=(-2.65, -2.3215500000000002)) - test\_marks\_10: a Point \[green\] drawn in over\_axes (location=(2.65, 0.59345)) - test\_marks\_2: a Point \[green\] drawn in over\_axes (location=(-2.05, -1.4839499999999997)) - test\_marks\_3: a Point \[green\] drawn in over\_axes (location=(-1.45, -0.7759499999999999)) - test\_marks\_4: a Point \[green\] drawn in over\_axes (location=(-0.85, -0.19755)) - test\_marks\_5: a Point \[green\] drawn in over\_axes (location=(-0.25, 0.25125000000000003)) - test\_marks\_6: a Point \[green\] drawn in over\_axes (location=(0.35, 0.57045)) - test\_marks\_7: a Point \[green\] drawn in over\_axes (location=(0.95, 0.76005)) - test\_marks\_8: a Point \[green\] drawn in over\_axes (location=(1.55, 0.8200500000000002)) - test\_marks\_9: a Point \[green\] drawn in over\_axes (location=(2.15, 0.7504500000000002)) - test\_readout: a Math \[green\] that says "$upright("test MSE") \> 0$" drawn in over\_axes - test\_residuals: a Line \[magenta\] drawn in over\_axes (start=(-2.65, -2.3215500000000002), end=(-2.65, -1.82756352737211), dashed=True) - test\_residuals\_10: a Line \[magenta\] drawn in over\_axes (start=(2.65, 0.59345), end=(2.65, 0.09946352737211012), dashed=True) - test\_residuals\_2: a Line \[magenta\] drawn in over\_axes (start=(-2.05, -1.4839499999999997), end=(-2.05, -0.9900905529924808), dashed=True) - test\_residuals\_3: a Line \[magenta\] drawn in over\_axes (start=(-1.45, -0.7759499999999999), end=(-1.45, -0.8555749140628064), dashed=True) - test\_residuals\_4: a Line \[magenta\] drawn in over\_axes (start=(-0.85, -0.19755), end=(-0.85, -0.5648549122571341), dashed=True) - test\_residuals\_5: a Line \[magenta\] drawn in over\_axes (start=(-0.25, 0.25125000000000003), end=(-0.25, 0.7511983384419643), dashed=True) - test\_residuals\_6: a Line \[magenta\] drawn in over\_axes (start=(0.35, 0.57045), end=(0.35, 0.1726896666190087), dashed=True) - test\_residuals\_7: a Line \[magenta\] drawn in over\_axes (start=(0.95, 0.76005), end=(0.95, 0.8402687600971048), dashed=True) - test\_residuals\_8: a Line \[magenta\] drawn in over\_axes (start=(1.55, 0.8200500000000002), end=(1.55, 1.2090045306849817), dashed=True) - test\_residuals\_9: a Line \[magenta\] drawn in over\_axes (start=(2.15, 0.7504500000000002), end=(2.15, 0.4990845185639889), dashed=True) - train\_readout: a Math \[yellow\] that says "$upright("training MSE") = 0.000$" drawn in over\_axes - under\_axes: an Axes (x\_range=(-3.0, 3.0), y\_range=(-3.6, 1.8), x\_ticks\_every=1.0) - under\_line: a FunctionPlot \[green\] drawn in under\_axes (function=\, x\_range=(-3.0, 3.0)) - under\_more: a Point \[gray\] drawn in under\_axes (location=(-3.0, -2.87), marker\_radius=0.035) - under\_more\_10: a Point \[gray\] drawn in under\_axes (location=(-2.3164556962025316, -1.9730848368698612), marker\_radius=0.035) - under\_more\_11: a Point \[gray\] drawn in under\_axes (location=(-2.240506329113924, -1.563707163843767), marker\_radius=0.035) - under\_more\_12: a Point \[gray\] drawn in under\_axes (location=(-2.1645569620253164, -1.8119692356331485), marker\_radius=0.035) - under\_more\_13: a Point \[gray\] drawn in under\_axes (location=(-2.088607594936709, -1.3840481732727037), marker\_radius=0.035) - under\_more\_14: a Point \[gray\] drawn in under\_axes (location=(-2.0126582278481013, -1.5290319905132814), marker\_radius=0.035) - under\_more\_15: a Point \[gray\] drawn in under\_axes (location=(-1.9367088607594938, -1.3222122140010508), marker\_radius=0.035) - under\_more\_16: a Point \[gray\] drawn in under\_axes (location=(-1.860759493670886, -1.1865071704778114), marker\_radius=0.035) - under\_more\_17: a Point \[gray\] drawn in under\_axes (location=(-1.7848101265822784, -1.2819279781859345), marker\_radius=0.035) - under\_more\_18: a Point \[gray\] drawn in under\_axes (location=(-1.7088607594936709, -0.8962331215517699), marker\_radius=0.035) - under\_more\_19: a Point \[gray\] drawn in under\_axes (location=(-1.6329113924050633, -1.1572469075692413), marker\_radius=0.035) - under\_more\_2: a Point \[gray\] drawn in under\_axes (location=(-2.9240506329113924, -2.6703124483484895), marker\_radius=0.035) - under\_more\_20: a Point \[gray\] drawn in under\_axes (location=(-1.5569620253164558, -0.7379417777261894), marker\_radius=0.035) - under\_more\_21: a Point \[gray\] drawn in under\_axes (location=(-1.481012658227848, -0.9099507197082083), marker\_radius=0.035) - under\_more\_22: a Point \[gray\] drawn in under\_axes (location=(-1.4050632911392404, -0.7010053739545179), marker\_radius=0.035) - under\_more\_23: a Point \[gray\] drawn in under\_axes (location=(-1.3291139240506329, -0.5974742296665826), marker\_radius=0.035) - under\_more\_24: a Point \[gray\] drawn in under\_axes (location=(-1.2531645569620253, -0.692201242721242), marker\_radius=0.035) - under\_more\_25: a Point \[gray\] drawn in under\_axes (location=(-1.1772151898734178, -0.33094265579120674), marker\_radius=0.035) - under\_more\_26: a Point \[gray\] drawn in under\_axes (location=(-1.1012658227848102, -0.6038216231759829), marker\_radius=0.035) - under\_more\_27: a Point \[gray\] drawn in under\_axes (location=(-1.0253164556962024, -0.19398195186296024), marker\_radius=0.035) - under\_more\_28: a Point \[gray\] drawn in under\_axes (location=(-0.9493670886075951, -0.39236648584332007), marker\_radius=0.035) - under\_more\_29: a Point \[gray\] drawn in under\_axes (location=(-0.8734177215189876, -0.18162049375226394), marker\_radius=0.035) - under\_more\_3: a Point \[gray\] drawn in under\_axes (location=(-2.848101265822785, -2.765655851631252), marker\_radius=0.035) - under\_more\_30: a Point \[gray\] drawn in under\_axes (location=(-0.7974683544303796, -0.11032527395094173), marker\_radius=0.035) - under\_more\_31: a Point \[gray\] drawn in under\_axes (location=(-0.721518987341772, -0.2039214188444216), marker\_radius=0.035) - under\_more\_32: a Point \[gray\] drawn in under\_axes (location=(-0.6455696202531644, 0.13217263166811496), marker\_radius=0.035) - under\_more\_33: a Point \[gray\] drawn in under\_axes (location=(-0.5696202531645569, -0.15169177929664995), marker\_radius=0.035) - under\_more\_34: a Point \[gray\] drawn in under\_axes (location=(-0.49367088607594933, 0.24782000734812762), marker\_radius=0.035) - under\_more\_35: a Point \[gray\] drawn in under\_axes (location=(-0.4177215189873418, 0.023739534474326557), marker\_radius=0.035) - under\_more\_36: a Point \[gray\] drawn in under\_axes (location=(-0.3417721518987342, 0.2359196881642116), marker\_radius=0.035) - under\_more\_37: a Point \[gray\] drawn in under\_axes (location=(-0.26582278481012667, 0.27496198765919205), marker\_radius=0.035) - under\_more\_38: a Point \[gray\] drawn in under\_axes (location=(-0.1898734177215191, 0.18289392655972697), marker\_radius=0.035) - under\_more\_39: a Point \[gray\] drawn in under\_axes (location=(-0.11392405063291156, 0.49312221329845274), marker\_radius=0.035) - under\_more\_4: a Point \[gray\] drawn in under\_axes (location=(-2.7721518987341773, -2.3333720852896733), marker\_radius=0.035) - under\_more\_40: a Point \[gray\] drawn in under\_axes (location=(-0.037974683544304, 0.19914306335698384), marker\_radius=0.035) - under\_more\_41: a Point \[gray\] drawn in under\_axes (location=(0.037974683544304, 0.5874538331383163), marker\_radius=0.035) - under\_more\_42: a Point \[gray\] drawn in under\_axes (location=(0.11392405063291156, 0.3383854657557492), marker\_radius=0.035) - under\_more\_43: a Point \[gray\] drawn in under\_axes (location=(0.1898734177215191, 0.5515926668326753), marker\_radius=0.035) - under\_more\_44: a Point \[gray\] drawn in under\_axes (location=(0.26582278481012667, 0.558410122871631), marker\_radius=0.035) - under\_more\_45: a Point \[gray\] drawn in under\_axes (location=(0.3417721518987342, 0.46822649278154854), marker\_radius=0.035) - under\_more\_46: a Point \[gray\] drawn in under\_axes (location=(0.4177215189873418, 0.7519166117155476), marker\_radius=0.035) - under\_more\_47: a Point \[gray\] drawn in under\_axes (location=(0.49367088607594933, 0.4486821790870714), marker\_radius=0.035) - under\_more\_48: a Point \[gray\] drawn in under\_axes (location=(0.5696202531645569, 0.8249103150582828), marker\_radius=0.035) - under\_more\_49: a Point \[gray\] drawn in under\_axes (location=(0.6455696202531644, 0.5515886875200892), marker\_radius=0.035) - under\_more\_5: a Point \[gray\] drawn in under\_axes (location=(-2.6962025316455698, -2.567991370314388), marker\_radius=0.035) - under\_more\_50: a Point \[gray\] drawn in under\_axes (location=(0.721518987341772, 0.7653762280242439), marker\_radius=0.035) - under\_more\_51: a Point \[gray\] drawn in under\_axes (location=(0.7974683544303796, 0.7400419186982076), marker\_radius=0.035) - under\_more\_52: a Point \[gray\] drawn in under\_axes (location=(0.8734177215189876, 0.6520572918447377), marker\_radius=0.035) - under\_more\_53: a Point \[gray\] drawn in under\_axes (location=(0.9493670886075951, 0.908567372908395), marker\_radius=0.035) - under\_more\_54: a Point \[gray\] drawn in under\_axes (location=(1.0253164556962027, 0.5969236790559147), marker\_radius=0.035) - under\_more\_55: a Point \[gray\] drawn in under\_axes (location=(1.1012658227848098, 0.9601813224101202), marker\_radius=0.035) - under\_more\_56: a Point \[gray\] drawn in under\_axes (location=(1.1772151898734178, 0.6633657900799991), marker\_radius=0.035) - under\_more\_57: a Point \[gray\] drawn in under\_axes (location=(1.253164556962025, 0.877248504757848), marker\_radius=0.035) - under\_more\_58: a Point \[gray\] drawn in under\_axes (location=(1.3291139240506329, 0.8198803234831035), marker\_radius=0.035) - under\_more\_59: a Point \[gray\] drawn in under\_axes (location=(1.4050632911392409, 0.7343666968129939), marker\_radius=0.035) - under\_more\_6: a Point \[gray\] drawn in under\_axes (location=(-2.620253164556962, -2.132288840073776), marker\_radius=0.035) - under\_more\_60: a Point \[gray\] drawn in under\_axes (location=(1.481012658227848, 0.9630870368654897), marker\_radius=0.035) - under\_more\_61: a Point \[gray\] drawn in under\_axes (location=(1.556962025316456, 0.6438645160912888), marker\_radius=0.035) - under\_more\_62: a Point \[gray\] drawn in under\_axes (location=(1.6329113924050631, 0.9932598249323683), marker\_radius=0.035) - under\_more\_63: a Point \[gray\] drawn in under\_axes (location=(1.708860759493671, 0.6737325323347665), marker\_radius=0.035) - under\_more\_64: a Point \[gray\] drawn in under\_axes (location=(1.7848101265822782, 0.887188032931261), marker\_radius=0.035) - under\_more\_65: a Point \[gray\] drawn in under\_axes (location=(1.8607594936708862, 0.7979483885207774), marker\_radius=0.035) - under\_more\_66: a Point \[gray\] drawn in under\_axes (location=(1.9367088607594933, 0.7151344917222289), marker\_radius=0.035) - under\_more\_67: a Point \[gray\] drawn in under\_axes (location=(2.0126582278481013, 0.9154891056722809), marker\_radius=0.035) - under\_more\_68: a Point \[gray\] drawn in under\_axes (location=(2.0886075949367084, 0.5895004924386543), marker\_radius=0.035) - under\_more\_69: a Point \[gray\] drawn in under\_axes (location=(2.1645569620253164, 0.9241399106854595), marker\_radius=0.035) - under\_more\_7: a Point \[gray\] drawn in under\_axes (location=(-2.5443037974683547, -2.24962977264546), marker\_radius=0.035) - under\_more\_70: a Point \[gray\] drawn in under\_axes (location=(2.2405063291139244, 0.5827038016786734), marker\_radius=0.035) - under\_more\_71: a Point \[gray\] drawn in under\_axes (location=(2.3164556962025316, 0.7951738059271779), marker\_radius=0.035) - under\_more\_72: a Point \[gray\] drawn in under\_axes (location=(2.3924050632911396, 0.6742692094119506), marker\_radius=0.035) - under\_more\_73: a Point \[gray\] drawn in under\_axes (location=(2.4683544303797467, 0.5943399230113228), marker\_radius=0.035) - under\_more\_74: a Point \[gray\] drawn in under\_axes (location=(2.5443037974683547, 0.7657880091609773), marker\_radius=0.035) - under\_more\_75: a Point \[gray\] drawn in under\_axes (location=(2.620253164556962, 0.43382627044051236), marker\_radius=0.035) - under\_more\_76: a Point \[gray\] drawn in under\_axes (location=(2.6962025316455698, 0.75281680109208), marker\_radius=0.035) - under\_more\_77: a Point \[gray\] drawn in under\_axes (location=(2.772151898734177, 0.3902935761264736), marker\_radius=0.035) - under\_more\_78: a Point \[gray\] drawn in under\_axes (location=(2.848101265822785, 0.6011853280554427), marker\_radius=0.035) - under\_more\_79: a Point \[gray\] drawn in under\_axes (location=(2.924050632911392, 0.44886586730688843), marker\_radius=0.035) - under\_more\_8: a Point \[gray\] drawn in under\_axes (location=(-2.4683544303797467, -2.0452180998191922), marker\_radius=0.035) - under\_more\_80: a Point \[gray\] drawn in under\_axes (location=(3.0, 0.37196175232051404), marker\_radius=0.035) - under\_more\_9: a Point \[gray\] drawn in under\_axes (location=(-2.392405063291139, -1.8774460539472317), marker\_radius=0.035) - under\_residuals: a Line \[red\] drawn in under\_axes (start=(-2.4, -1.9567999999999999), end=(-2.4, -1.46), dashed=True) - under\_residuals\_2: a Line \[red\] drawn in under\_axes (start=(-1.2, -0.5192), end=(-1.2, -0.8), dashed=True) - under\_residuals\_3: a Line \[red\] drawn in under\_axes (start=(0.0, 0.4), end=(0.0, -0.14), dashed=True) - under\_residuals\_4: a Line \[red\] drawn in under\_axes (start=(1.2, 0.8008000000000001), end=(1.2, 0.52), dashed=True) - under\_residuals\_5: a Line \[red\] drawn in under\_axes (start=(2.4, 0.6832000000000003), end=(2.4, 1.1800000000000002), dashed=True) - under\_truth: a FunctionPlot \[blue\] drawn in under\_axes (function=\, x\_range=(-3.0, 3.0)) - under\_twenty: a Point \[yellow\] drawn in under\_axes (location=(3.0, 0.9300000000000002)) - under\_twenty\_10: a Point \[yellow\] drawn in under\_axes (location=(0.24773803641699718, 0.025208575785553577)) - under\_twenty\_11: a Point \[yellow\] drawn in under\_axes (location=(-0.24773803641699682, 0.7526967357268568)) - under\_twenty\_12: a Point \[yellow\] drawn in under\_axes (location=(-0.7364564614223967, -0.6026773153050619)) - under\_twenty\_13: a Point \[yellow\] drawn in under\_axes (location=(-1.2050862739589083, -0.02419937766054925)) - under\_twenty\_14: a Point \[yellow\] drawn in under\_axes (location=(-1.640844474367281, -1.4870911669330995)) - under\_twenty\_15: a Point \[yellow\] drawn in under\_axes (location=(-2.0318447148772227, -0.9606253233498832)) - under\_twenty\_16: a Point \[yellow\] drawn in under\_axes (location=(-2.3674215281891806, -2.4109250850880968)) - under\_twenty\_17: a Point \[yellow\] drawn in under\_axes (location=(-2.638421253619467, -1.8041596975698724)) - under\_twenty\_18: a Point \[yellow\] drawn in under\_axes (location=(-2.8374517251019045, -3.1098022614171272)) - under\_twenty\_19: a Point \[yellow\] drawn in under\_axes (location=(-2.959083910208167, -2.303608116392006)) - under\_twenty\_2: a Point \[yellow\] drawn in under\_axes (location=(2.959083910208167, -0.048615815163021825)) - under\_twenty\_20: a Point \[yellow\] drawn in under\_axes (location=(-3.0, -3.37)) - under\_twenty\_3: a Point \[yellow\] drawn in under\_axes (location=(2.837451725101904, 1.0113946361949688)) - under\_twenty\_4: a Point \[yellow\] drawn in under\_axes (location=(2.6384212536194673, 0.09810368141154147)) - under\_twenty\_5: a Point \[yellow\] drawn in under\_axes (location=(2.3674215281891806, 1.1932385959200025)) - under\_twenty\_6: a Point \[yellow\] drawn in under\_axes (location=(2.031844714877223, 0.2744038630150619)) - under\_twenty\_7: a Point \[yellow\] drawn in under\_axes (location=(1.6408444743672808, 1.31783775487091)) - under\_twenty\_8: a Point \[yellow\] drawn in under\_axes (location=(1.2050862739589085, 0.3013955236942498)) - under\_twenty\_9: a Point \[yellow\] drawn in under\_axes (location=(0.7364564614223977, 1.2074247922595749)) #### Beats ##### [02:47.812](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=167.81166666666664) Narration: Start with the rigid failure. The blue curve is the underlying relationship, the yellow dots are our original twenty observations, and the green line is the best straight model available to us. Board: Empty. Actions: - [02:47.812](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=167.81166666666664): heading is shown on the screen, written out. - [02:47.812](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=167.81166666666664): left\_label is shown on the screen, written out. - [02:47.812](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=167.81166666666664): under\_axes is shown on the screen, written out. - [02:50.459](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=170.45866666666663): under\_truth is shown on the screen, written out. - [02:54.929](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=174.92866666666663): under\_twenty is shown on the screen, written out. - [02:54.949](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=174.94866666666664): under\_twenty\_2 is shown on the screen, written out. - [02:54.969](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=174.96866666666665): under\_twenty\_3 is shown on the screen, written out. - [02:54.989](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=174.98866666666663): under\_twenty\_4 is shown on the screen, written out. - [02:55.009](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=175.00866666666664): under\_twenty\_5 is shown on the screen, written out. - [02:55.029](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=175.02866666666665): under\_twenty\_6 is shown on the screen, written out. - [02:55.049](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=175.04866666666663): under\_twenty\_7 is shown on the screen, written out. - [02:55.069](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=175.06866666666664): under\_twenty\_8 is shown on the screen, written out. - [02:55.089](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=175.08866666666663): under\_twenty\_9 is shown on the screen, written out. - [02:55.109](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=175.10866666666664): under\_twenty\_10 is shown on the screen, written out. - [02:55.129](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=175.12866666666665): under\_twenty\_11 is shown on the screen, written out. - [02:55.149](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=175.14866666666663): under\_twenty\_12 is shown on the screen, written out. - [02:55.169](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=175.16866666666664): under\_twenty\_13 is shown on the screen, written out. - [02:55.189](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=175.18866666666665): under\_twenty\_14 is shown on the screen, written out. - [02:55.209](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=175.20866666666663): under\_twenty\_15 is shown on the screen, written out. - [02:55.229](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=175.22866666666664): under\_twenty\_16 is shown on the screen, written out. - [02:55.249](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=175.24866666666665): under\_twenty\_17 is shown on the screen, written out. - [02:55.269](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=175.26866666666663): under\_twenty\_18 is shown on the screen, written out. - [02:55.289](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=175.28866666666664): under\_twenty\_19 is shown on the screen, written out. - [02:55.309](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=175.30866666666662): under\_twenty\_20 is shown on the screen, written out. - [02:56.601](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=176.60066666666663): under\_line is shown on the screen, written out. ##### [03:0.231](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=180.23066666666665) Narration: Now add many more observations from the same relationship. Eighty gray points make the bend much easier to see, but they do not give a straight line the ability to bend. Board: left\_label — a Tex \[text\] that says "Too rigid"; under\_axes — an Axes (x\_range=(-3.0, 3.0), y\_range=(-3.6, 1.8), x\_ticks\_every=1.0); heading — a Heading that says "Failure in Both Directions"; under\_truth — a FunctionPlot \[blue\] drawn in under\_axes (function=\, x\_range=(-3.0, 3.0)); under\_twenty — a Point \[yellow\] drawn in under\_axes (location=(3.0, 0.9300000000000002)); under\_twenty\_2 — a Point \[yellow\] drawn in under\_axes (location=(2.959083910208167, -0.048615815163021825)); under\_twenty\_3 — a Point \[yellow\] drawn in under\_axes (location=(2.837451725101904, 1.0113946361949688)); under\_twenty\_4 — a Point \[yellow\] drawn in under\_axes (location=(2.6384212536194673, 0.09810368141154147)); under\_twenty\_5 — a Point \[yellow\] drawn in under\_axes (location=(2.3674215281891806, 1.1932385959200025)); under\_twenty\_6 — a Point \[yellow\] drawn in under\_axes (location=(2.031844714877223, 0.2744038630150619)); under\_twenty\_7 — a Point \[yellow\] drawn in under\_axes (location=(1.6408444743672808, 1.31783775487091)); under\_twenty\_8 — a Point \[yellow\] drawn in under\_axes (location=(1.2050862739589085, 0.3013955236942498)); under\_twenty\_9 — a Point \[yellow\] drawn in under\_axes (location=(0.7364564614223977, 1.2074247922595749)); under\_twenty\_10 — a Point \[yellow\] drawn in under\_axes (location=(0.24773803641699718, 0.025208575785553577)); under\_twenty\_11 — a Point \[yellow\] drawn in under\_axes (location=(-0.24773803641699682, 0.7526967357268568)); under\_twenty\_12 — a Point \[yellow\] drawn in under\_axes (location=(-0.7364564614223967, -0.6026773153050619)); under\_twenty\_13 — a Point \[yellow\] drawn in under\_axes (location=(-1.2050862739589083, -0.02419937766054925)); under\_twenty\_14 — a Point \[yellow\] drawn in under\_axes (location=(-1.640844474367281, -1.4870911669330995)); under\_twenty\_15 — a Point \[yellow\] drawn in under\_axes (location=(-2.0318447148772227, -0.9606253233498832)); under\_twenty\_16 — a Point \[yellow\] drawn in under\_axes (location=(-2.3674215281891806, -2.4109250850880968)); under\_twenty\_17 — a Point \[yellow\] drawn in under\_axes (location=(-2.638421253619467, -1.8041596975698724)); under\_twenty\_18 — a Point \[yellow\] drawn in under\_axes (location=(-2.8374517251019045, -3.1098022614171272)); under\_twenty\_19 — a Point \[yellow\] drawn in under\_axes (location=(-2.959083910208167, -2.303608116392006)); under\_twenty\_20 — a Point \[yellow\] drawn in under\_axes (location=(-3.0, -3.37)); under\_line — a FunctionPlot \[green\] drawn in under\_axes (function=\, x\_range=(-3.0, 3.0)) Actions: - [03:4.48](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=184.47966666666665): under\_more is shown on the screen, written out. - [03:4.495](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=184.49466666666663): under\_more\_2 is shown on the screen, written out. - [03:4.51](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=184.50966666666665): under\_more\_3 is shown on the screen, written out. - [03:4.525](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=184.52466666666663): under\_more\_4 is shown on the screen, written out. - [03:4.54](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=184.53966666666665): under\_more\_5 is shown on the screen, written out. - [03:4.555](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=184.55466666666663): under\_more\_6 is shown on the screen, written out. - [03:4.57](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=184.56966666666665): under\_more\_7 is shown on the screen, written out. - [03:4.585](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=184.58466666666664): under\_more\_8 is shown on the screen, written out. - [03:4.6](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=184.59966666666665): under\_more\_9 is shown on the screen, written out. - [03:4.615](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=184.61466666666664): under\_more\_10 is shown on the screen, written out. - [03:4.63](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=184.62966666666665): under\_more\_11 is shown on the screen, written out. - [03:4.645](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=184.64466666666664): under\_more\_12 is shown on the screen, written out. - [03:4.66](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=184.65966666666662): under\_more\_13 is shown on the screen, written out. - [03:4.675](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=184.67466666666664): under\_more\_14 is shown on the screen, written out. - [03:4.69](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=184.68966666666665): under\_more\_15 is shown on the screen, written out. - [03:4.705](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=184.70466666666664): under\_more\_16 is shown on the screen, written out. - [03:4.72](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=184.71966666666663): under\_more\_17 is shown on the screen, written out. - [03:4.735](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=184.73466666666664): under\_more\_18 is shown on the screen, written out. - [03:4.75](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=184.74966666666663): under\_more\_19 is shown on the screen, written out. - [03:4.765](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=184.76466666666664): under\_more\_20 is shown on the screen, written out. - [03:4.78](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=184.77966666666663): under\_more\_21 is shown on the screen, written out. - [03:4.795](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=184.79466666666664): under\_more\_22 is shown on the screen, written out. - [03:4.81](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=184.80966666666663): under\_more\_23 is shown on the screen, written out. - [03:4.825](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=184.82466666666664): under\_more\_24 is shown on the screen, written out. - [03:4.84](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=184.83966666666663): under\_more\_25 is shown on the screen, written out. - [03:4.855](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=184.85466666666665): under\_more\_26 is shown on the screen, written out. - [03:4.87](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=184.86966666666663): under\_more\_27 is shown on the screen, written out. - [03:4.885](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=184.88466666666665): under\_more\_28 is shown on the screen, written out. - [03:4.9](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=184.89966666666663): under\_more\_29 is shown on the screen, written out. - [03:4.915](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=184.91466666666665): under\_more\_30 is shown on the screen, written out. - [03:4.93](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=184.92966666666663): under\_more\_31 is shown on the screen, written out. - [03:4.945](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=184.94466666666665): under\_more\_32 is shown on the screen, written out. - [03:4.96](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=184.95966666666664): under\_more\_33 is shown on the screen, written out. - [03:4.975](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=184.97466666666665): under\_more\_34 is shown on the screen, written out. - [03:4.99](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=184.98966666666664): under\_more\_35 is shown on the screen, written out. - [03:5.005](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=185.00466666666665): under\_more\_36 is shown on the screen, written out. - [03:5.02](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=185.01966666666664): under\_more\_37 is shown on the screen, written out. - [03:5.035](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=185.03466666666662): under\_more\_38 is shown on the screen, written out. - [03:5.05](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=185.04966666666664): under\_more\_39 is shown on the screen, written out. - [03:5.065](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=185.06466666666665): under\_more\_40 is shown on the screen, written out. - [03:5.08](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=185.07966666666664): under\_more\_41 is shown on the screen, written out. - [03:5.095](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=185.09466666666663): under\_more\_42 is shown on the screen, written out. - [03:5.11](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=185.10966666666664): under\_more\_43 is shown on the screen, written out. - [03:5.125](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=185.12466666666663): under\_more\_44 is shown on the screen, written out. - [03:5.14](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=185.13966666666664): under\_more\_45 is shown on the screen, written out. - [03:5.155](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=185.15466666666663): under\_more\_46 is shown on the screen, written out. - [03:5.17](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=185.16966666666664): under\_more\_47 is shown on the screen, written out. - [03:5.185](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=185.18466666666663): under\_more\_48 is shown on the screen, written out. - [03:5.2](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=185.19966666666664): under\_more\_49 is shown on the screen, written out. - [03:5.215](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=185.21466666666663): under\_more\_50 is shown on the screen, written out. - [03:5.23](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=185.22966666666665): under\_more\_51 is shown on the screen, written out. - [03:5.245](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=185.24466666666663): under\_more\_52 is shown on the screen, written out. - [03:5.26](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=185.25966666666665): under\_more\_53 is shown on the screen, written out. - [03:5.275](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=185.27466666666663): under\_more\_54 is shown on the screen, written out. - [03:5.29](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=185.28966666666665): under\_more\_55 is shown on the screen, written out. - [03:5.305](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=185.30466666666663): under\_more\_56 is shown on the screen, written out. - [03:5.32](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=185.31966666666665): under\_more\_57 is shown on the screen, written out. - [03:5.335](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=185.33466666666664): under\_more\_58 is shown on the screen, written out. - [03:5.35](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=185.34966666666665): under\_more\_59 is shown on the screen, written out. - [03:5.365](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=185.36466666666664): under\_more\_60 is shown on the screen, written out. - [03:5.38](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=185.37966666666665): under\_more\_61 is shown on the screen, written out. - [03:5.395](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=185.39466666666664): under\_more\_62 is shown on the screen, written out. - [03:5.41](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=185.40966666666662): under\_more\_63 is shown on the screen, written out. - [03:5.425](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=185.42466666666664): under\_more\_64 is shown on the screen, written out. - [03:5.44](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=185.43966666666665): under\_more\_65 is shown on the screen, written out. - [03:5.455](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=185.45466666666664): under\_more\_66 is shown on the screen, written out. - [03:5.47](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=185.46966666666663): under\_more\_67 is shown on the screen, written out. - [03:5.485](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=185.48466666666664): under\_more\_68 is shown on the screen, written out. - [03:5.5](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=185.49966666666663): under\_more\_69 is shown on the screen, written out. - [03:5.515](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=185.51466666666664): under\_more\_70 is shown on the screen, written out. - [03:5.53](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=185.52966666666663): under\_more\_71 is shown on the screen, written out. - [03:5.545](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=185.54466666666664): under\_more\_72 is shown on the screen, written out. - [03:5.56](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=185.55966666666663): under\_more\_73 is shown on the screen, written out. - [03:5.575](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=185.57466666666664): under\_more\_74 is shown on the screen, written out. - [03:5.59](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=185.58966666666663): under\_more\_75 is shown on the screen, written out. - [03:5.605](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=185.60466666666665): under\_more\_76 is shown on the screen, written out. - [03:5.62](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=185.61966666666663): under\_more\_77 is shown on the screen, written out. - [03:5.635](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=185.63466666666665): under\_more\_78 is shown on the screen, written out. - [03:5.65](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=185.64966666666663): under\_more\_79 is shown on the screen, written out. - [03:5.665](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=185.66466666666665): under\_more\_80 is shown on the screen, written out. ##### [03:11.5](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=191.50016666666664) Narration: More data can reveal that this model is wrong, but it cannot repair the model family. These red gaps are systematic. On one stretch the line is too high; on another it is too low. This persistent miss is underfitting. Board: left\_label — a Tex \[text\] that says "Too rigid"; under\_axes — an Axes (x\_range=(-3.0, 3.0), y\_range=(-3.6, 1.8), x\_ticks\_every=1.0); heading — a Heading that says "Failure in Both Directions"; under\_truth — a FunctionPlot \[blue\] drawn in under\_axes (function=\, x\_range=(-3.0, 3.0)); under\_twenty — a Point \[yellow\] drawn in under\_axes (location=(3.0, 0.9300000000000002)); under\_twenty\_2 — a Point \[yellow\] drawn in under\_axes (location=(2.959083910208167, -0.048615815163021825)); under\_twenty\_3 — a Point \[yellow\] drawn in under\_axes (location=(2.837451725101904, 1.0113946361949688)); under\_twenty\_4 — a Point \[yellow\] drawn in under\_axes (location=(2.6384212536194673, 0.09810368141154147)); under\_twenty\_5 — a Point \[yellow\] drawn in under\_axes (location=(2.3674215281891806, 1.1932385959200025)); under\_twenty\_6 — a Point \[yellow\] drawn in under\_axes (location=(2.031844714877223, 0.2744038630150619)); under\_twenty\_7 — a Point \[yellow\] drawn in under\_axes (location=(1.6408444743672808, 1.31783775487091)); under\_twenty\_8 — a Point \[yellow\] drawn in under\_axes (location=(1.2050862739589085, 0.3013955236942498)); under\_twenty\_9 — a Point \[yellow\] drawn in under\_axes (location=(0.7364564614223977, 1.2074247922595749)); under\_twenty\_10 — a Point \[yellow\] drawn in under\_axes (location=(0.24773803641699718, 0.025208575785553577)); under\_twenty\_11 — a Point \[yellow\] drawn in under\_axes (location=(-0.24773803641699682, 0.7526967357268568)); under\_twenty\_12 — a Point \[yellow\] drawn in under\_axes (location=(-0.7364564614223967, -0.6026773153050619)); under\_twenty\_13 — a Point \[yellow\] drawn in under\_axes (location=(-1.2050862739589083, -0.02419937766054925)); under\_twenty\_14 — a Point \[yellow\] drawn in under\_axes (location=(-1.640844474367281, -1.4870911669330995)); under\_twenty\_15 — a Point \[yellow\] drawn in under\_axes (location=(-2.0318447148772227, -0.9606253233498832)); under\_twenty\_16 — a Point \[yellow\] drawn in under\_axes (location=(-2.3674215281891806, -2.4109250850880968)); under\_twenty\_17 — a Point \[yellow\] drawn in under\_axes (location=(-2.638421253619467, -1.8041596975698724)); under\_twenty\_18 — a Point \[yellow\] drawn in under\_axes (location=(-2.8374517251019045, -3.1098022614171272)); under\_twenty\_19 — a Point \[yellow\] drawn in under\_axes (location=(-2.959083910208167, -2.303608116392006)); under\_twenty\_20 — a Point \[yellow\] drawn in under\_axes (location=(-3.0, -3.37)); under\_line — a FunctionPlot \[green\] drawn in under\_axes (function=\, x\_range=(-3.0, 3.0)); under\_more — a Point \[gray\] drawn in under\_axes (location=(-3.0, -2.87), marker\_radius=0.035); under\_more\_2 — a Point \[gray\] drawn in under\_axes (location=(-2.9240506329113924, -2.6703124483484895), marker\_radius=0.035); under\_more\_3 — a Point \[gray\] drawn in under\_axes (location=(-2.848101265822785, -2.765655851631252), marker\_radius=0.035); under\_more\_4 — a Point \[gray\] drawn in under\_axes (location=(-2.7721518987341773, -2.3333720852896733), marker\_radius=0.035); under\_more\_5 — a Point \[gray\] drawn in under\_axes (location=(-2.6962025316455698, -2.567991370314388), marker\_radius=0.035); under\_more\_6 — a Point \[gray\] drawn in under\_axes (location=(-2.620253164556962, -2.132288840073776), marker\_radius=0.035); under\_more\_7 — a Point \[gray\] drawn in under\_axes (location=(-2.5443037974683547, -2.24962977264546), marker\_radius=0.035); under\_more\_8 — a Point \[gray\] drawn in under\_axes (location=(-2.4683544303797467, -2.0452180998191922), marker\_radius=0.035); under\_more\_9 — a Point \[gray\] drawn in under\_axes (location=(-2.392405063291139, -1.8774460539472317), marker\_radius=0.035); under\_more\_10 — a Point \[gray\] drawn in under\_axes (location=(-2.3164556962025316, -1.9730848368698612), marker\_radius=0.035); under\_more\_11 — a Point \[gray\] drawn in under\_axes (location=(-2.240506329113924, -1.563707163843767), marker\_radius=0.035); under\_more\_12 — a Point \[gray\] drawn in under\_axes (location=(-2.1645569620253164, -1.8119692356331485), marker\_radius=0.035); under\_more\_13 — a Point \[gray\] drawn in under\_axes (location=(-2.088607594936709, -1.3840481732727037), marker\_radius=0.035); under\_more\_14 — a Point \[gray\] drawn in under\_axes (location=(-2.0126582278481013, -1.5290319905132814), marker\_radius=0.035); under\_more\_15 — a Point \[gray\] drawn in under\_axes (location=(-1.9367088607594938, -1.3222122140010508), marker\_radius=0.035); under\_more\_16 — a Point \[gray\] drawn in under\_axes (location=(-1.860759493670886, -1.1865071704778114), marker\_radius=0.035); under\_more\_17 — a Point \[gray\] drawn in under\_axes (location=(-1.7848101265822784, -1.2819279781859345), marker\_radius=0.035); under\_more\_18 — a Point \[gray\] drawn in under\_axes (location=(-1.7088607594936709, -0.8962331215517699), marker\_radius=0.035); under\_more\_19 — a Point \[gray\] drawn in under\_axes (location=(-1.6329113924050633, -1.1572469075692413), marker\_radius=0.035); under\_more\_20 — a Point \[gray\] drawn in under\_axes (location=(-1.5569620253164558, -0.7379417777261894), marker\_radius=0.035); under\_more\_21 — a Point \[gray\] drawn in under\_axes (location=(-1.481012658227848, -0.9099507197082083), marker\_radius=0.035); under\_more\_22 — a Point \[gray\] drawn in under\_axes (location=(-1.4050632911392404, -0.7010053739545179), marker\_radius=0.035); under\_more\_23 — a Point \[gray\] drawn in under\_axes (location=(-1.3291139240506329, -0.5974742296665826), marker\_radius=0.035); under\_more\_24 — a Point \[gray\] drawn in under\_axes (location=(-1.2531645569620253, -0.692201242721242), marker\_radius=0.035); under\_more\_25 — a Point \[gray\] drawn in under\_axes (location=(-1.1772151898734178, -0.33094265579120674), marker\_radius=0.035); under\_more\_26 — a Point \[gray\] drawn in under\_axes (location=(-1.1012658227848102, -0.6038216231759829), marker\_radius=0.035); under\_more\_27 — a Point \[gray\] drawn in under\_axes (location=(-1.0253164556962024, -0.19398195186296024), marker\_radius=0.035); under\_more\_28 — a Point \[gray\] drawn in under\_axes (location=(-0.9493670886075951, -0.39236648584332007), marker\_radius=0.035); under\_more\_29 — a Point \[gray\] drawn in under\_axes (location=(-0.8734177215189876, -0.18162049375226394), marker\_radius=0.035); under\_more\_30 — a Point \[gray\] drawn in under\_axes (location=(-0.7974683544303796, -0.11032527395094173), marker\_radius=0.035); under\_more\_31 — a Point \[gray\] drawn in under\_axes (location=(-0.721518987341772, -0.2039214188444216), marker\_radius=0.035); under\_more\_32 — a Point \[gray\] drawn in under\_axes (location=(-0.6455696202531644, 0.13217263166811496), marker\_radius=0.035); under\_more\_33 — a Point \[gray\] drawn in under\_axes (location=(-0.5696202531645569, -0.15169177929664995), marker\_radius=0.035); under\_more\_34 — a Point \[gray\] drawn in under\_axes (location=(-0.49367088607594933, 0.24782000734812762), marker\_radius=0.035); under\_more\_35 — a Point \[gray\] drawn in under\_axes (location=(-0.4177215189873418, 0.023739534474326557), marker\_radius=0.035); under\_more\_36 — a Point \[gray\] drawn in under\_axes (location=(-0.3417721518987342, 0.2359196881642116), marker\_radius=0.035); under\_more\_37 — a Point \[gray\] drawn in under\_axes (location=(-0.26582278481012667, 0.27496198765919205), marker\_radius=0.035); under\_more\_38 — a Point \[gray\] drawn in under\_axes (location=(-0.1898734177215191, 0.18289392655972697), marker\_radius=0.035); under\_more\_39 — a Point \[gray\] drawn in under\_axes (location=(-0.11392405063291156, 0.49312221329845274), marker\_radius=0.035); under\_more\_40 — a Point \[gray\] drawn in under\_axes (location=(-0.037974683544304, 0.19914306335698384), marker\_radius=0.035); under\_more\_41 — a Point \[gray\] drawn in under\_axes (location=(0.037974683544304, 0.5874538331383163), marker\_radius=0.035); under\_more\_42 — a Point \[gray\] drawn in under\_axes (location=(0.11392405063291156, 0.3383854657557492), marker\_radius=0.035); under\_more\_43 — a Point \[gray\] drawn in under\_axes (location=(0.1898734177215191, 0.5515926668326753), marker\_radius=0.035); under\_more\_44 — a Point \[gray\] drawn in under\_axes (location=(0.26582278481012667, 0.558410122871631), marker\_radius=0.035); under\_more\_45 — a Point \[gray\] drawn in under\_axes (location=(0.3417721518987342, 0.46822649278154854), marker\_radius=0.035); under\_more\_46 — a Point \[gray\] drawn in under\_axes (location=(0.4177215189873418, 0.7519166117155476), marker\_radius=0.035); under\_more\_47 — a Point \[gray\] drawn in under\_axes (location=(0.49367088607594933, 0.4486821790870714), marker\_radius=0.035); under\_more\_48 — a Point \[gray\] drawn in under\_axes (location=(0.5696202531645569, 0.8249103150582828), marker\_radius=0.035); under\_more\_49 — a Point \[gray\] drawn in under\_axes (location=(0.6455696202531644, 0.5515886875200892), marker\_radius=0.035); under\_more\_50 — a Point \[gray\] drawn in under\_axes (location=(0.721518987341772, 0.7653762280242439), marker\_radius=0.035); under\_more\_51 — a Point \[gray\] drawn in under\_axes (location=(0.7974683544303796, 0.7400419186982076), marker\_radius=0.035); under\_more\_52 — a Point \[gray\] drawn in under\_axes (location=(0.8734177215189876, 0.6520572918447377), marker\_radius=0.035); under\_more\_53 — a Point \[gray\] drawn in under\_axes (location=(0.9493670886075951, 0.908567372908395), marker\_radius=0.035); under\_more\_54 — a Point \[gray\] drawn in under\_axes (location=(1.0253164556962027, 0.5969236790559147), marker\_radius=0.035); under\_more\_55 — a Point \[gray\] drawn in under\_axes (location=(1.1012658227848098, 0.9601813224101202), marker\_radius=0.035); under\_more\_56 — a Point \[gray\] drawn in under\_axes (location=(1.1772151898734178, 0.6633657900799991), marker\_radius=0.035); under\_more\_57 — a Point \[gray\] drawn in under\_axes (location=(1.253164556962025, 0.877248504757848), marker\_radius=0.035); under\_more\_58 — a Point \[gray\] drawn in under\_axes (location=(1.3291139240506329, 0.8198803234831035), marker\_radius=0.035); under\_more\_59 — a Point \[gray\] drawn in under\_axes (location=(1.4050632911392409, 0.7343666968129939), marker\_radius=0.035); under\_more\_60 — a Point \[gray\] drawn in under\_axes (location=(1.481012658227848, 0.9630870368654897), marker\_radius=0.035); under\_more\_61 — a Point \[gray\] drawn in under\_axes (location=(1.556962025316456, 0.6438645160912888), marker\_radius=0.035); under\_more\_62 — a Point \[gray\] drawn in under\_axes (location=(1.6329113924050631, 0.9932598249323683), marker\_radius=0.035); under\_more\_63 — a Point \[gray\] drawn in under\_axes (location=(1.708860759493671, 0.6737325323347665), marker\_radius=0.035); under\_more\_64 — a Point \[gray\] drawn in under\_axes (location=(1.7848101265822782, 0.887188032931261), marker\_radius=0.035); under\_more\_65 — a Point \[gray\] drawn in under\_axes (location=(1.8607594936708862, 0.7979483885207774), marker\_radius=0.035); under\_more\_66 — a Point \[gray\] drawn in under\_axes (location=(1.9367088607594933, 0.7151344917222289), marker\_radius=0.035); under\_more\_67 — a Point \[gray\] drawn in under\_axes (location=(2.0126582278481013, 0.9154891056722809), marker\_radius=0.035); under\_more\_68 — a Point \[gray\] drawn in under\_axes (location=(2.0886075949367084, 0.5895004924386543), marker\_radius=0.035); under\_more\_69 — a Point \[gray\] drawn in under\_axes (location=(2.1645569620253164, 0.9241399106854595), marker\_radius=0.035); under\_more\_70 — a Point \[gray\] drawn in under\_axes (location=(2.2405063291139244, 0.5827038016786734), marker\_radius=0.035); under\_more\_71 — a Point \[gray\] drawn in under\_axes (location=(2.3164556962025316, 0.7951738059271779), marker\_radius=0.035); under\_more\_72 — a Point \[gray\] drawn in under\_axes (location=(2.3924050632911396, 0.6742692094119506), marker\_radius=0.035); under\_more\_73 — a Point \[gray\] drawn in under\_axes (location=(2.4683544303797467, 0.5943399230113228), marker\_radius=0.035); under\_more\_74 — a Point \[gray\] drawn in under\_axes (location=(2.5443037974683547, 0.7657880091609773), marker\_radius=0.035); under\_more\_75 — a Point \[gray\] drawn in under\_axes (location=(2.620253164556962, 0.43382627044051236), marker\_radius=0.035); under\_more\_76 — a Point \[gray\] drawn in under\_axes (location=(2.6962025316455698, 0.75281680109208), marker\_radius=0.035); under\_more\_77 — a Point \[gray\] drawn in under\_axes (location=(2.772151898734177, 0.3902935761264736), marker\_radius=0.035); under\_more\_78 — a Point \[gray\] drawn in under\_axes (location=(2.848101265822785, 0.6011853280554427), marker\_radius=0.035); under\_more\_79 — a Point \[gray\] drawn in under\_axes (location=(2.924050632911392, 0.44886586730688843), marker\_radius=0.035); under\_more\_80 — a Point \[gray\] drawn in under\_axes (location=(3.0, 0.37196175232051404), marker\_radius=0.035) Actions: - [03:17.781](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=197.78066666666663): under\_residuals is shown on the screen, written out. - [03:17.861](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=197.86066666666665): under\_residuals\_2 is shown on the screen, written out. - [03:17.941](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=197.94066666666663): under\_residuals\_3 is shown on the screen, written out. - [03:18.021](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=198.02066666666664): under\_residuals\_4 is shown on the screen, written out. - [03:18.101](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=198.10066666666665): under\_residuals\_5 is shown on the screen, written out. - [03:26.605](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=206.60466666666665): under\_line is indicated — a transient flash. ##### [03:28.307](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=208.30716666666663) Narration: Now look at the flexible failure. The right-hand graph uses the original twenty observations again. The red degree-nineteen polynomial passes through every one of them. Board: left\_label — a Tex \[text\] that says "Too rigid"; under\_axes — an Axes (x\_range=(-3.0, 3.0), y\_range=(-3.6, 1.8), x\_ticks\_every=1.0); heading — a Heading that says "Failure in Both Directions"; under\_truth — a FunctionPlot \[blue\] drawn in under\_axes (function=\, x\_range=(-3.0, 3.0)); under\_twenty — a Point \[yellow\] drawn in under\_axes (location=(3.0, 0.9300000000000002)); under\_twenty\_2 — a Point \[yellow\] drawn in under\_axes (location=(2.959083910208167, -0.048615815163021825)); under\_twenty\_3 — a Point \[yellow\] drawn in under\_axes (location=(2.837451725101904, 1.0113946361949688)); under\_twenty\_4 — a Point \[yellow\] drawn in under\_axes (location=(2.6384212536194673, 0.09810368141154147)); under\_twenty\_5 — a Point \[yellow\] drawn in under\_axes (location=(2.3674215281891806, 1.1932385959200025)); under\_twenty\_6 — a Point \[yellow\] drawn in under\_axes (location=(2.031844714877223, 0.2744038630150619)); under\_twenty\_7 — a Point \[yellow\] drawn in under\_axes (location=(1.6408444743672808, 1.31783775487091)); under\_twenty\_8 — a Point \[yellow\] drawn in under\_axes (location=(1.2050862739589085, 0.3013955236942498)); under\_twenty\_9 — a Point \[yellow\] drawn in under\_axes (location=(0.7364564614223977, 1.2074247922595749)); under\_twenty\_10 — a Point \[yellow\] drawn in under\_axes (location=(0.24773803641699718, 0.025208575785553577)); under\_twenty\_11 — a Point \[yellow\] drawn in under\_axes (location=(-0.24773803641699682, 0.7526967357268568)); under\_twenty\_12 — a Point \[yellow\] drawn in under\_axes (location=(-0.7364564614223967, -0.6026773153050619)); under\_twenty\_13 — a Point \[yellow\] drawn in under\_axes (location=(-1.2050862739589083, -0.02419937766054925)); under\_twenty\_14 — a Point \[yellow\] drawn in under\_axes (location=(-1.640844474367281, -1.4870911669330995)); under\_twenty\_15 — a Point \[yellow\] drawn in under\_axes (location=(-2.0318447148772227, -0.9606253233498832)); under\_twenty\_16 — a Point \[yellow\] drawn in under\_axes (location=(-2.3674215281891806, -2.4109250850880968)); under\_twenty\_17 — a Point \[yellow\] drawn in under\_axes (location=(-2.638421253619467, -1.8041596975698724)); under\_twenty\_18 — a Point \[yellow\] drawn in under\_axes (location=(-2.8374517251019045, -3.1098022614171272)); under\_twenty\_19 — a Point \[yellow\] drawn in under\_axes (location=(-2.959083910208167, -2.303608116392006)); under\_twenty\_20 — a Point \[yellow\] drawn in under\_axes (location=(-3.0, -3.37)); under\_line — a FunctionPlot \[green\] drawn in under\_axes (function=\, x\_range=(-3.0, 3.0)); under\_more — a Point \[gray\] drawn in under\_axes (location=(-3.0, -2.87), marker\_radius=0.035); under\_more\_2 — a Point \[gray\] drawn in under\_axes (location=(-2.9240506329113924, -2.6703124483484895), marker\_radius=0.035); under\_more\_3 — a Point \[gray\] drawn in under\_axes (location=(-2.848101265822785, -2.765655851631252), marker\_radius=0.035); under\_more\_4 — a Point \[gray\] drawn in under\_axes (location=(-2.7721518987341773, -2.3333720852896733), marker\_radius=0.035); under\_more\_5 — a Point \[gray\] drawn in under\_axes (location=(-2.6962025316455698, -2.567991370314388), marker\_radius=0.035); under\_more\_6 — a Point \[gray\] drawn in under\_axes (location=(-2.620253164556962, -2.132288840073776), marker\_radius=0.035); under\_more\_7 — a Point \[gray\] drawn in under\_axes (location=(-2.5443037974683547, -2.24962977264546), marker\_radius=0.035); under\_more\_8 — a Point \[gray\] drawn in under\_axes (location=(-2.4683544303797467, -2.0452180998191922), marker\_radius=0.035); under\_more\_9 — a Point \[gray\] drawn in under\_axes (location=(-2.392405063291139, -1.8774460539472317), marker\_radius=0.035); under\_more\_10 — a Point \[gray\] drawn in under\_axes (location=(-2.3164556962025316, -1.9730848368698612), marker\_radius=0.035); under\_more\_11 — a Point \[gray\] drawn in under\_axes (location=(-2.240506329113924, -1.563707163843767), marker\_radius=0.035); under\_more\_12 — a Point \[gray\] drawn in under\_axes (location=(-2.1645569620253164, -1.8119692356331485), marker\_radius=0.035); under\_more\_13 — a Point \[gray\] drawn in under\_axes (location=(-2.088607594936709, -1.3840481732727037), marker\_radius=0.035); under\_more\_14 — a Point \[gray\] drawn in under\_axes (location=(-2.0126582278481013, -1.5290319905132814), marker\_radius=0.035); under\_more\_15 — a Point \[gray\] drawn in under\_axes (location=(-1.9367088607594938, -1.3222122140010508), marker\_radius=0.035); under\_more\_16 — a Point \[gray\] drawn in under\_axes (location=(-1.860759493670886, -1.1865071704778114), marker\_radius=0.035); under\_more\_17 — a Point \[gray\] drawn in under\_axes (location=(-1.7848101265822784, -1.2819279781859345), marker\_radius=0.035); under\_more\_18 — a Point \[gray\] drawn in under\_axes (location=(-1.7088607594936709, -0.8962331215517699), marker\_radius=0.035); under\_more\_19 — a Point \[gray\] drawn in under\_axes (location=(-1.6329113924050633, -1.1572469075692413), marker\_radius=0.035); under\_more\_20 — a Point \[gray\] drawn in under\_axes (location=(-1.5569620253164558, -0.7379417777261894), marker\_radius=0.035); under\_more\_21 — a Point \[gray\] drawn in under\_axes (location=(-1.481012658227848, -0.9099507197082083), marker\_radius=0.035); under\_more\_22 — a Point \[gray\] drawn in under\_axes (location=(-1.4050632911392404, -0.7010053739545179), marker\_radius=0.035); under\_more\_23 — a Point \[gray\] drawn in under\_axes (location=(-1.3291139240506329, -0.5974742296665826), marker\_radius=0.035); under\_more\_24 — a Point \[gray\] drawn in under\_axes (location=(-1.2531645569620253, -0.692201242721242), marker\_radius=0.035); under\_more\_25 — a Point \[gray\] drawn in under\_axes (location=(-1.1772151898734178, -0.33094265579120674), marker\_radius=0.035); under\_more\_26 — a Point \[gray\] drawn in under\_axes (location=(-1.1012658227848102, -0.6038216231759829), marker\_radius=0.035); under\_more\_27 — a Point \[gray\] drawn in under\_axes (location=(-1.0253164556962024, -0.19398195186296024), marker\_radius=0.035); under\_more\_28 — a Point \[gray\] drawn in under\_axes (location=(-0.9493670886075951, -0.39236648584332007), marker\_radius=0.035); under\_more\_29 — a Point \[gray\] drawn in under\_axes (location=(-0.8734177215189876, -0.18162049375226394), marker\_radius=0.035); under\_more\_30 — a Point \[gray\] drawn in under\_axes (location=(-0.7974683544303796, -0.11032527395094173), marker\_radius=0.035); under\_more\_31 — a Point \[gray\] drawn in under\_axes (location=(-0.721518987341772, -0.2039214188444216), marker\_radius=0.035); under\_more\_32 — a Point \[gray\] drawn in under\_axes (location=(-0.6455696202531644, 0.13217263166811496), marker\_radius=0.035); under\_more\_33 — a Point \[gray\] drawn in under\_axes (location=(-0.5696202531645569, -0.15169177929664995), marker\_radius=0.035); under\_more\_34 — a Point \[gray\] drawn in under\_axes (location=(-0.49367088607594933, 0.24782000734812762), marker\_radius=0.035); under\_more\_35 — a Point \[gray\] drawn in under\_axes (location=(-0.4177215189873418, 0.023739534474326557), marker\_radius=0.035); under\_more\_36 — a Point \[gray\] drawn in under\_axes (location=(-0.3417721518987342, 0.2359196881642116), marker\_radius=0.035); under\_more\_37 — a Point \[gray\] drawn in under\_axes (location=(-0.26582278481012667, 0.27496198765919205), marker\_radius=0.035); under\_more\_38 — a Point \[gray\] drawn in under\_axes (location=(-0.1898734177215191, 0.18289392655972697), marker\_radius=0.035); under\_more\_39 — a Point \[gray\] drawn in under\_axes (location=(-0.11392405063291156, 0.49312221329845274), marker\_radius=0.035); under\_more\_40 — a Point \[gray\] drawn in under\_axes (location=(-0.037974683544304, 0.19914306335698384), marker\_radius=0.035); under\_more\_41 — a Point \[gray\] drawn in under\_axes (location=(0.037974683544304, 0.5874538331383163), marker\_radius=0.035); under\_more\_42 — a Point \[gray\] drawn in under\_axes (location=(0.11392405063291156, 0.3383854657557492), marker\_radius=0.035); under\_more\_43 — a Point \[gray\] drawn in under\_axes (location=(0.1898734177215191, 0.5515926668326753), marker\_radius=0.035); under\_more\_44 — a Point \[gray\] drawn in under\_axes (location=(0.26582278481012667, 0.558410122871631), marker\_radius=0.035); under\_more\_45 — a Point \[gray\] drawn in under\_axes (location=(0.3417721518987342, 0.46822649278154854), marker\_radius=0.035); under\_more\_46 — a Point \[gray\] drawn in under\_axes (location=(0.4177215189873418, 0.7519166117155476), marker\_radius=0.035); under\_more\_47 — a Point \[gray\] drawn in under\_axes (location=(0.49367088607594933, 0.4486821790870714), marker\_radius=0.035); under\_more\_48 — a Point \[gray\] drawn in under\_axes (location=(0.5696202531645569, 0.8249103150582828), marker\_radius=0.035); under\_more\_49 — a Point \[gray\] drawn in under\_axes (location=(0.6455696202531644, 0.5515886875200892), marker\_radius=0.035); under\_more\_50 — a Point \[gray\] drawn in under\_axes (location=(0.721518987341772, 0.7653762280242439), marker\_radius=0.035); under\_more\_51 — a Point \[gray\] drawn in under\_axes (location=(0.7974683544303796, 0.7400419186982076), marker\_radius=0.035); under\_more\_52 — a Point \[gray\] drawn in under\_axes (location=(0.8734177215189876, 0.6520572918447377), marker\_radius=0.035); under\_more\_53 — a Point \[gray\] drawn in under\_axes (location=(0.9493670886075951, 0.908567372908395), marker\_radius=0.035); under\_more\_54 — a Point \[gray\] drawn in under\_axes (location=(1.0253164556962027, 0.5969236790559147), marker\_radius=0.035); under\_more\_55 — a Point \[gray\] drawn in under\_axes (location=(1.1012658227848098, 0.9601813224101202), marker\_radius=0.035); under\_more\_56 — a Point \[gray\] drawn in under\_axes (location=(1.1772151898734178, 0.6633657900799991), marker\_radius=0.035); under\_more\_57 — a Point \[gray\] drawn in under\_axes (location=(1.253164556962025, 0.877248504757848), marker\_radius=0.035); under\_more\_58 — a Point \[gray\] drawn in under\_axes (location=(1.3291139240506329, 0.8198803234831035), marker\_radius=0.035); under\_more\_59 — a Point \[gray\] drawn in under\_axes (location=(1.4050632911392409, 0.7343666968129939), marker\_radius=0.035); under\_more\_60 — a Point \[gray\] drawn in under\_axes (location=(1.481012658227848, 0.9630870368654897), marker\_radius=0.035); under\_more\_61 — a Point \[gray\] drawn in under\_axes (location=(1.556962025316456, 0.6438645160912888), marker\_radius=0.035); under\_more\_62 — a Point \[gray\] drawn in under\_axes (location=(1.6329113924050631, 0.9932598249323683), marker\_radius=0.035); under\_more\_63 — a Point \[gray\] drawn in under\_axes (location=(1.708860759493671, 0.6737325323347665), marker\_radius=0.035); under\_more\_64 — a Point \[gray\] drawn in under\_axes (location=(1.7848101265822782, 0.887188032931261), marker\_radius=0.035); under\_more\_65 — a Point \[gray\] drawn in under\_axes (location=(1.8607594936708862, 0.7979483885207774), marker\_radius=0.035); under\_more\_66 — a Point \[gray\] drawn in under\_axes (location=(1.9367088607594933, 0.7151344917222289), marker\_radius=0.035); under\_more\_67 — a Point \[gray\] drawn in under\_axes (location=(2.0126582278481013, 0.9154891056722809), marker\_radius=0.035); under\_more\_68 — a Point \[gray\] drawn in under\_axes (location=(2.0886075949367084, 0.5895004924386543), marker\_radius=0.035); under\_more\_69 — a Point \[gray\] drawn in under\_axes (location=(2.1645569620253164, 0.9241399106854595), marker\_radius=0.035); under\_more\_70 — a Point \[gray\] drawn in under\_axes (location=(2.2405063291139244, 0.5827038016786734), marker\_radius=0.035); under\_more\_71 — a Point \[gray\] drawn in under\_axes (location=(2.3164556962025316, 0.7951738059271779), marker\_radius=0.035); under\_more\_72 — a Point \[gray\] drawn in under\_axes (location=(2.3924050632911396, 0.6742692094119506), marker\_radius=0.035); under\_more\_73 — a Point \[gray\] drawn in under\_axes (location=(2.4683544303797467, 0.5943399230113228), marker\_radius=0.035); under\_more\_74 — a Point \[gray\] drawn in under\_axes (location=(2.5443037974683547, 0.7657880091609773), marker\_radius=0.035); under\_more\_75 — a Point \[gray\] drawn in under\_axes (location=(2.620253164556962, 0.43382627044051236), marker\_radius=0.035); under\_more\_76 — a Point \[gray\] drawn in under\_axes (location=(2.6962025316455698, 0.75281680109208), marker\_radius=0.035); under\_more\_77 — a Point \[gray\] drawn in under\_axes (location=(2.772151898734177, 0.3902935761264736), marker\_radius=0.035); under\_more\_78 — a Point \[gray\] drawn in under\_axes (location=(2.848101265822785, 0.6011853280554427), marker\_radius=0.035); under\_more\_79 — a Point \[gray\] drawn in under\_axes (location=(2.924050632911392, 0.44886586730688843), marker\_radius=0.035); under\_more\_80 — a Point \[gray\] drawn in under\_axes (location=(3.0, 0.37196175232051404), marker\_radius=0.035); under\_residuals — a Line \[red\] drawn in under\_axes (start=(-2.4, -1.9567999999999999), end=(-2.4, -1.46), dashed=True); under\_residuals\_2 — a Line \[red\] drawn in under\_axes (start=(-1.2, -0.5192), end=(-1.2, -0.8), dashed=True); under\_residuals\_3 — a Line \[red\] drawn in under\_axes (start=(0.0, 0.4), end=(0.0, -0.14), dashed=True); under\_residuals\_4 — a Line \[red\] drawn in under\_axes (start=(1.2, 0.8008000000000001), end=(1.2, 0.52), dashed=True); under\_residuals\_5 — a Line \[red\] drawn in under\_axes (start=(2.4, 0.6832000000000003), end=(2.4, 1.1800000000000002), dashed=True) Actions: - [03:29.329](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=209.32866666666663): right\_label is shown on the screen, written out. - [03:31.315](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=211.31466666666665): over\_axes is shown on the screen, written out. - [03:32.568](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=212.56766666666664): over\_truth is shown on the screen, written out. - [03:33.01](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=213.00966666666665): over\_twenty is shown on the screen, written out. - [03:33.03](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=213.02966666666663): over\_twenty\_2 is shown on the screen, written out. - [03:33.05](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=213.04966666666664): over\_twenty\_3 is shown on the screen, written out. - [03:33.07](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=213.06966666666665): over\_twenty\_4 is shown on the screen, written out. - [03:33.09](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=213.08966666666663): over\_twenty\_5 is shown on the screen, written out. - [03:33.11](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=213.10966666666664): over\_twenty\_6 is shown on the screen, written out. - [03:33.13](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=213.12966666666662): over\_twenty\_7 is shown on the screen, written out. - [03:33.15](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=213.14966666666663): over\_twenty\_8 is shown on the screen, written out. - [03:33.17](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=213.16966666666661): over\_twenty\_9 is shown on the screen, written out. - [03:33.19](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=213.18966666666662): over\_twenty\_10 is shown on the screen, written out. - [03:33.21](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=213.20966666666664): over\_twenty\_11 is shown on the screen, written out. - [03:33.23](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=213.22966666666662): over\_twenty\_12 is shown on the screen, written out. - [03:33.25](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=213.24966666666663): over\_twenty\_13 is shown on the screen, written out. - [03:33.27](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=213.26966666666664): over\_twenty\_14 is shown on the screen, written out. - [03:33.29](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=213.28966666666662): over\_twenty\_15 is shown on the screen, written out. - [03:33.31](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=213.30966666666663): over\_twenty\_16 is shown on the screen, written out. - [03:33.33](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=213.32966666666664): over\_twenty\_17 is shown on the screen, written out. - [03:33.35](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=213.34966666666662): over\_twenty\_18 is shown on the screen, written out. - [03:33.37](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=213.36966666666663): over\_twenty\_19 is shown on the screen, written out. - [03:33.39](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=213.38966666666664): over\_twenty\_20 is shown on the screen, written out. - [03:35.599](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=215.59866666666665): over\_fit is shown on the screen, written out. ##### [03:40.122](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=220.12216666666663) Narration: If we measure only at those yellow locations, every vertical residual is zero. Its mean squared training error is therefore zero. Board: left\_label — a Tex \[text\] that says "Too rigid"; under\_axes — an Axes (x\_range=(-3.0, 3.0), y\_range=(-3.6, 1.8), x\_ticks\_every=1.0); right\_label — a Tex \[text\] that says "Too responsive"; over\_axes — an Axes (x\_range=(-3.0, 3.0), y\_range=(-3.6, 1.8), x\_ticks\_every=1.0); heading — a Heading that says "Failure in Both Directions"; under\_truth — a FunctionPlot \[blue\] drawn in under\_axes (function=\, x\_range=(-3.0, 3.0)); under\_twenty — a Point \[yellow\] drawn in under\_axes (location=(3.0, 0.9300000000000002)); under\_twenty\_2 — a Point \[yellow\] drawn in under\_axes (location=(2.959083910208167, -0.048615815163021825)); under\_twenty\_3 — a Point \[yellow\] drawn in under\_axes (location=(2.837451725101904, 1.0113946361949688)); under\_twenty\_4 — a Point \[yellow\] drawn in under\_axes (location=(2.6384212536194673, 0.09810368141154147)); under\_twenty\_5 — a Point \[yellow\] drawn in under\_axes (location=(2.3674215281891806, 1.1932385959200025)); under\_twenty\_6 — a Point \[yellow\] drawn in under\_axes (location=(2.031844714877223, 0.2744038630150619)); under\_twenty\_7 — a Point \[yellow\] drawn in under\_axes (location=(1.6408444743672808, 1.31783775487091)); under\_twenty\_8 — a Point \[yellow\] drawn in under\_axes (location=(1.2050862739589085, 0.3013955236942498)); under\_twenty\_9 — a Point \[yellow\] drawn in under\_axes (location=(0.7364564614223977, 1.2074247922595749)); under\_twenty\_10 — a Point \[yellow\] drawn in under\_axes (location=(0.24773803641699718, 0.025208575785553577)); under\_twenty\_11 — a Point \[yellow\] drawn in under\_axes (location=(-0.24773803641699682, 0.7526967357268568)); under\_twenty\_12 — a Point \[yellow\] drawn in under\_axes (location=(-0.7364564614223967, -0.6026773153050619)); under\_twenty\_13 — a Point \[yellow\] drawn in under\_axes (location=(-1.2050862739589083, -0.02419937766054925)); under\_twenty\_14 — a Point \[yellow\] drawn in under\_axes (location=(-1.640844474367281, -1.4870911669330995)); under\_twenty\_15 — a Point \[yellow\] drawn in under\_axes (location=(-2.0318447148772227, -0.9606253233498832)); under\_twenty\_16 — a Point \[yellow\] drawn in under\_axes (location=(-2.3674215281891806, -2.4109250850880968)); under\_twenty\_17 — a Point \[yellow\] drawn in under\_axes (location=(-2.638421253619467, -1.8041596975698724)); under\_twenty\_18 — a Point \[yellow\] drawn in under\_axes (location=(-2.8374517251019045, -3.1098022614171272)); under\_twenty\_19 — a Point \[yellow\] drawn in under\_axes (location=(-2.959083910208167, -2.303608116392006)); under\_twenty\_20 — a Point \[yellow\] drawn in under\_axes (location=(-3.0, -3.37)); under\_line — a FunctionPlot \[green\] drawn in under\_axes (function=\, x\_range=(-3.0, 3.0)); under\_more — a Point \[gray\] drawn in under\_axes (location=(-3.0, -2.87), marker\_radius=0.035); under\_more\_2 — a Point \[gray\] drawn in under\_axes (location=(-2.9240506329113924, -2.6703124483484895), marker\_radius=0.035); under\_more\_3 — a Point \[gray\] drawn in under\_axes (location=(-2.848101265822785, -2.765655851631252), marker\_radius=0.035); under\_more\_4 — a Point \[gray\] drawn in under\_axes (location=(-2.7721518987341773, -2.3333720852896733), marker\_radius=0.035); under\_more\_5 — a Point \[gray\] drawn in under\_axes (location=(-2.6962025316455698, -2.567991370314388), marker\_radius=0.035); under\_more\_6 — a Point \[gray\] drawn in under\_axes (location=(-2.620253164556962, -2.132288840073776), marker\_radius=0.035); under\_more\_7 — a Point \[gray\] drawn in under\_axes (location=(-2.5443037974683547, -2.24962977264546), marker\_radius=0.035); under\_more\_8 — a Point \[gray\] drawn in under\_axes (location=(-2.4683544303797467, -2.0452180998191922), marker\_radius=0.035); under\_more\_9 — a Point \[gray\] drawn in under\_axes (location=(-2.392405063291139, -1.8774460539472317), marker\_radius=0.035); under\_more\_10 — a Point \[gray\] drawn in under\_axes (location=(-2.3164556962025316, -1.9730848368698612), marker\_radius=0.035); under\_more\_11 — a Point \[gray\] drawn in under\_axes (location=(-2.240506329113924, -1.563707163843767), marker\_radius=0.035); under\_more\_12 — a Point \[gray\] drawn in under\_axes (location=(-2.1645569620253164, -1.8119692356331485), marker\_radius=0.035); under\_more\_13 — a Point \[gray\] drawn in under\_axes (location=(-2.088607594936709, -1.3840481732727037), marker\_radius=0.035); under\_more\_14 — a Point \[gray\] drawn in under\_axes (location=(-2.0126582278481013, -1.5290319905132814), marker\_radius=0.035); under\_more\_15 — a Point \[gray\] drawn in under\_axes (location=(-1.9367088607594938, -1.3222122140010508), marker\_radius=0.035); under\_more\_16 — a Point \[gray\] drawn in under\_axes (location=(-1.860759493670886, -1.1865071704778114), marker\_radius=0.035); under\_more\_17 — a Point \[gray\] drawn in under\_axes (location=(-1.7848101265822784, -1.2819279781859345), marker\_radius=0.035); under\_more\_18 — a Point \[gray\] drawn in under\_axes (location=(-1.7088607594936709, -0.8962331215517699), marker\_radius=0.035); under\_more\_19 — a Point \[gray\] drawn in under\_axes (location=(-1.6329113924050633, -1.1572469075692413), marker\_radius=0.035); under\_more\_20 — a Point \[gray\] drawn in under\_axes (location=(-1.5569620253164558, -0.7379417777261894), marker\_radius=0.035); under\_more\_21 — a Point \[gray\] drawn in under\_axes (location=(-1.481012658227848, -0.9099507197082083), marker\_radius=0.035); under\_more\_22 — a Point \[gray\] drawn in under\_axes (location=(-1.4050632911392404, -0.7010053739545179), marker\_radius=0.035); under\_more\_23 — a Point \[gray\] drawn in under\_axes (location=(-1.3291139240506329, -0.5974742296665826), marker\_radius=0.035); under\_more\_24 — a Point \[gray\] drawn in under\_axes (location=(-1.2531645569620253, -0.692201242721242), marker\_radius=0.035); under\_more\_25 — a Point \[gray\] drawn in under\_axes (location=(-1.1772151898734178, -0.33094265579120674), marker\_radius=0.035); under\_more\_26 — a Point \[gray\] drawn in under\_axes (location=(-1.1012658227848102, -0.6038216231759829), marker\_radius=0.035); under\_more\_27 — a Point \[gray\] drawn in under\_axes (location=(-1.0253164556962024, -0.19398195186296024), marker\_radius=0.035); under\_more\_28 — a Point \[gray\] drawn in under\_axes (location=(-0.9493670886075951, -0.39236648584332007), marker\_radius=0.035); under\_more\_29 — a Point \[gray\] drawn in under\_axes (location=(-0.8734177215189876, -0.18162049375226394), marker\_radius=0.035); under\_more\_30 — a Point \[gray\] drawn in under\_axes (location=(-0.7974683544303796, -0.11032527395094173), marker\_radius=0.035); under\_more\_31 — a Point \[gray\] drawn in under\_axes (location=(-0.721518987341772, -0.2039214188444216), marker\_radius=0.035); under\_more\_32 — a Point \[gray\] drawn in under\_axes (location=(-0.6455696202531644, 0.13217263166811496), marker\_radius=0.035); under\_more\_33 — a Point \[gray\] drawn in under\_axes (location=(-0.5696202531645569, -0.15169177929664995), marker\_radius=0.035); under\_more\_34 — a Point \[gray\] drawn in under\_axes (location=(-0.49367088607594933, 0.24782000734812762), marker\_radius=0.035); under\_more\_35 — a Point \[gray\] drawn in under\_axes (location=(-0.4177215189873418, 0.023739534474326557), marker\_radius=0.035); under\_more\_36 — a Point \[gray\] drawn in under\_axes (location=(-0.3417721518987342, 0.2359196881642116), marker\_radius=0.035); under\_more\_37 — a Point \[gray\] drawn in under\_axes (location=(-0.26582278481012667, 0.27496198765919205), marker\_radius=0.035); under\_more\_38 — a Point \[gray\] drawn in under\_axes (location=(-0.1898734177215191, 0.18289392655972697), marker\_radius=0.035); under\_more\_39 — a Point \[gray\] drawn in under\_axes (location=(-0.11392405063291156, 0.49312221329845274), marker\_radius=0.035); under\_more\_40 — a Point \[gray\] drawn in under\_axes (location=(-0.037974683544304, 0.19914306335698384), marker\_radius=0.035); under\_more\_41 — a Point \[gray\] drawn in under\_axes (location=(0.037974683544304, 0.5874538331383163), marker\_radius=0.035); under\_more\_42 — a Point \[gray\] drawn in under\_axes (location=(0.11392405063291156, 0.3383854657557492), marker\_radius=0.035); under\_more\_43 — a Point \[gray\] drawn in under\_axes (location=(0.1898734177215191, 0.5515926668326753), marker\_radius=0.035); under\_more\_44 — a Point \[gray\] drawn in under\_axes (location=(0.26582278481012667, 0.558410122871631), marker\_radius=0.035); under\_more\_45 — a Point \[gray\] drawn in under\_axes (location=(0.3417721518987342, 0.46822649278154854), marker\_radius=0.035); under\_more\_46 — a Point \[gray\] drawn in under\_axes (location=(0.4177215189873418, 0.7519166117155476), marker\_radius=0.035); under\_more\_47 — a Point \[gray\] drawn in under\_axes (location=(0.49367088607594933, 0.4486821790870714), marker\_radius=0.035); under\_more\_48 — a Point \[gray\] drawn in under\_axes (location=(0.5696202531645569, 0.8249103150582828), marker\_radius=0.035); under\_more\_49 — a Point \[gray\] drawn in under\_axes (location=(0.6455696202531644, 0.5515886875200892), marker\_radius=0.035); under\_more\_50 — a Point \[gray\] drawn in under\_axes (location=(0.721518987341772, 0.7653762280242439), marker\_radius=0.035); under\_more\_51 — a Point \[gray\] drawn in under\_axes (location=(0.7974683544303796, 0.7400419186982076), marker\_radius=0.035); under\_more\_52 — a Point \[gray\] drawn in under\_axes (location=(0.8734177215189876, 0.6520572918447377), marker\_radius=0.035); under\_more\_53 — a Point \[gray\] drawn in under\_axes (location=(0.9493670886075951, 0.908567372908395), marker\_radius=0.035); under\_more\_54 — a Point \[gray\] drawn in under\_axes (location=(1.0253164556962027, 0.5969236790559147), marker\_radius=0.035); under\_more\_55 — a Point \[gray\] drawn in under\_axes (location=(1.1012658227848098, 0.9601813224101202), marker\_radius=0.035); under\_more\_56 — a Point \[gray\] drawn in under\_axes (location=(1.1772151898734178, 0.6633657900799991), marker\_radius=0.035); under\_more\_57 — a Point \[gray\] drawn in under\_axes (location=(1.253164556962025, 0.877248504757848), marker\_radius=0.035); under\_more\_58 — a Point \[gray\] drawn in under\_axes (location=(1.3291139240506329, 0.8198803234831035), marker\_radius=0.035); under\_more\_59 — a Point \[gray\] drawn in under\_axes (location=(1.4050632911392409, 0.7343666968129939), marker\_radius=0.035); under\_more\_60 — a Point \[gray\] drawn in under\_axes (location=(1.481012658227848, 0.9630870368654897), marker\_radius=0.035); under\_more\_61 — a Point \[gray\] drawn in under\_axes (location=(1.556962025316456, 0.6438645160912888), marker\_radius=0.035); under\_more\_62 — a Point \[gray\] drawn in under\_axes (location=(1.6329113924050631, 0.9932598249323683), marker\_radius=0.035); under\_more\_63 — a Point \[gray\] drawn in under\_axes (location=(1.708860759493671, 0.6737325323347665), marker\_radius=0.035); under\_more\_64 — a Point \[gray\] drawn in under\_axes (location=(1.7848101265822782, 0.887188032931261), marker\_radius=0.035); under\_more\_65 — a Point \[gray\] drawn in under\_axes (location=(1.8607594936708862, 0.7979483885207774), marker\_radius=0.035); under\_more\_66 — a Point \[gray\] drawn in under\_axes (location=(1.9367088607594933, 0.7151344917222289), marker\_radius=0.035); under\_more\_67 — a Point \[gray\] drawn in under\_axes (location=(2.0126582278481013, 0.9154891056722809), marker\_radius=0.035); under\_more\_68 — a Point \[gray\] drawn in under\_axes (location=(2.0886075949367084, 0.5895004924386543), marker\_radius=0.035); under\_more\_69 — a Point \[gray\] drawn in under\_axes (location=(2.1645569620253164, 0.9241399106854595), marker\_radius=0.035); under\_more\_70 — a Point \[gray\] drawn in under\_axes (location=(2.2405063291139244, 0.5827038016786734), marker\_radius=0.035); under\_more\_71 — a Point \[gray\] drawn in under\_axes (location=(2.3164556962025316, 0.7951738059271779), marker\_radius=0.035); under\_more\_72 — a Point \[gray\] drawn in under\_axes (location=(2.3924050632911396, 0.6742692094119506), marker\_radius=0.035); under\_more\_73 — a Point \[gray\] drawn in under\_axes (location=(2.4683544303797467, 0.5943399230113228), marker\_radius=0.035); under\_more\_74 — a Point \[gray\] drawn in under\_axes (location=(2.5443037974683547, 0.7657880091609773), marker\_radius=0.035); under\_more\_75 — a Point \[gray\] drawn in under\_axes (location=(2.620253164556962, 0.43382627044051236), marker\_radius=0.035); under\_more\_76 — a Point \[gray\] drawn in under\_axes (location=(2.6962025316455698, 0.75281680109208), marker\_radius=0.035); under\_more\_77 — a Point \[gray\] drawn in under\_axes (location=(2.772151898734177, 0.3902935761264736), marker\_radius=0.035); under\_more\_78 — a Point \[gray\] drawn in under\_axes (location=(2.848101265822785, 0.6011853280554427), marker\_radius=0.035); under\_more\_79 — a Point \[gray\] drawn in under\_axes (location=(2.924050632911392, 0.44886586730688843), marker\_radius=0.035); under\_more\_80 — a Point \[gray\] drawn in under\_axes (location=(3.0, 0.37196175232051404), marker\_radius=0.035); under\_residuals — a Line \[red\] drawn in under\_axes (start=(-2.4, -1.9567999999999999), end=(-2.4, -1.46), dashed=True); under\_residuals\_2 — a Line \[red\] drawn in under\_axes (start=(-1.2, -0.5192), end=(-1.2, -0.8), dashed=True); under\_residuals\_3 — a Line \[red\] drawn in under\_axes (start=(0.0, 0.4), end=(0.0, -0.14), dashed=True); under\_residuals\_4 — a Line \[red\] drawn in under\_axes (start=(1.2, 0.8008000000000001), end=(1.2, 0.52), dashed=True); under\_residuals\_5 — a Line \[red\] drawn in under\_axes (start=(2.4, 0.6832000000000003), end=(2.4, 1.1800000000000002), dashed=True); over\_truth — a FunctionPlot \[blue\] drawn in over\_axes (function=\, x\_range=(-3.0, 3.0)); over\_twenty — a Point \[yellow\] drawn in over\_axes (location=(3.0, 0.9300000000000002)); over\_twenty\_2 — a Point \[yellow\] drawn in over\_axes (location=(2.959083910208167, -0.048615815163021825)); over\_twenty\_3 — a Point \[yellow\] drawn in over\_axes (location=(2.837451725101904, 1.0113946361949688)); over\_twenty\_4 — a Point \[yellow\] drawn in over\_axes (location=(2.6384212536194673, 0.09810368141154147)); over\_twenty\_5 — a Point \[yellow\] drawn in over\_axes (location=(2.3674215281891806, 1.1932385959200025)); over\_twenty\_6 — a Point \[yellow\] drawn in over\_axes (location=(2.031844714877223, 0.2744038630150619)); over\_twenty\_7 — a Point \[yellow\] drawn in over\_axes (location=(1.6408444743672808, 1.31783775487091)); over\_twenty\_8 — a Point \[yellow\] drawn in over\_axes (location=(1.2050862739589085, 0.3013955236942498)); over\_twenty\_9 — a Point \[yellow\] drawn in over\_axes (location=(0.7364564614223977, 1.2074247922595749)); over\_twenty\_10 — a Point \[yellow\] drawn in over\_axes (location=(0.24773803641699718, 0.025208575785553577)); over\_twenty\_11 — a Point \[yellow\] drawn in over\_axes (location=(-0.24773803641699682, 0.7526967357268568)); over\_twenty\_12 — a Point \[yellow\] drawn in over\_axes (location=(-0.7364564614223967, -0.6026773153050619)); over\_twenty\_13 — a Point \[yellow\] drawn in over\_axes (location=(-1.2050862739589083, -0.02419937766054925)); over\_twenty\_14 — a Point \[yellow\] drawn in over\_axes (location=(-1.640844474367281, -1.4870911669330995)); over\_twenty\_15 — a Point \[yellow\] drawn in over\_axes (location=(-2.0318447148772227, -0.9606253233498832)); over\_twenty\_16 — a Point \[yellow\] drawn in over\_axes (location=(-2.3674215281891806, -2.4109250850880968)); over\_twenty\_17 — a Point \[yellow\] drawn in over\_axes (location=(-2.638421253619467, -1.8041596975698724)); over\_twenty\_18 — a Point \[yellow\] drawn in over\_axes (location=(-2.8374517251019045, -3.1098022614171272)); over\_twenty\_19 — a Point \[yellow\] drawn in over\_axes (location=(-2.959083910208167, -2.303608116392006)); over\_twenty\_20 — a Point \[yellow\] drawn in over\_axes (location=(-3.0, -3.37)); over\_fit — a FunctionPlot \[red\] drawn in over\_axes (function=\, x\_range=(-3.0, 3.0)) Actions: - [03:41.957](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=221.95666666666665): over\_twenty is indicated — a transient flash. - [03:41.957](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=221.95666666666665): over\_twenty\_5 is indicated — a transient flash. - [03:41.957](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=221.95666666666665): over\_twenty\_9 is indicated — a transient flash. - [03:41.957](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=221.95666666666665): over\_twenty\_13 is indicated — a transient flash. - [03:41.957](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=221.95666666666665): over\_twenty\_17 is indicated — a transient flash. - [03:44.895](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=224.89466666666664): train\_readout is shown on the screen, written out. ##### [03:50.045](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=230.04516666666663) Narration: But now place green observations at new locations between the training points. They follow the same blue relationship and were not available while the polynomial was fitted. Board: left\_label — a Tex \[text\] that says "Too rigid"; under\_axes — an Axes (x\_range=(-3.0, 3.0), y\_range=(-3.6, 1.8), x\_ticks\_every=1.0); right\_label — a Tex \[text\] that says "Too responsive"; over\_axes — an Axes (x\_range=(-3.0, 3.0), y\_range=(-3.6, 1.8), x\_ticks\_every=1.0); heading — a Heading that says "Failure in Both Directions"; under\_truth — a FunctionPlot \[blue\] drawn in under\_axes (function=\, x\_range=(-3.0, 3.0)); under\_twenty — a Point \[yellow\] drawn in under\_axes (location=(3.0, 0.9300000000000002)); under\_twenty\_2 — a Point \[yellow\] drawn in under\_axes (location=(2.959083910208167, -0.048615815163021825)); under\_twenty\_3 — a Point \[yellow\] drawn in under\_axes (location=(2.837451725101904, 1.0113946361949688)); under\_twenty\_4 — a Point \[yellow\] drawn in under\_axes (location=(2.6384212536194673, 0.09810368141154147)); under\_twenty\_5 — a Point \[yellow\] drawn in under\_axes (location=(2.3674215281891806, 1.1932385959200025)); under\_twenty\_6 — a Point \[yellow\] drawn in under\_axes (location=(2.031844714877223, 0.2744038630150619)); under\_twenty\_7 — a Point \[yellow\] drawn in under\_axes (location=(1.6408444743672808, 1.31783775487091)); under\_twenty\_8 — a Point \[yellow\] drawn in under\_axes (location=(1.2050862739589085, 0.3013955236942498)); under\_twenty\_9 — a Point \[yellow\] drawn in under\_axes (location=(0.7364564614223977, 1.2074247922595749)); under\_twenty\_10 — a Point \[yellow\] drawn in under\_axes (location=(0.24773803641699718, 0.025208575785553577)); under\_twenty\_11 — a Point \[yellow\] drawn in under\_axes (location=(-0.24773803641699682, 0.7526967357268568)); under\_twenty\_12 — a Point \[yellow\] drawn in under\_axes (location=(-0.7364564614223967, -0.6026773153050619)); under\_twenty\_13 — a Point \[yellow\] drawn in under\_axes (location=(-1.2050862739589083, -0.02419937766054925)); under\_twenty\_14 — a Point \[yellow\] drawn in under\_axes (location=(-1.640844474367281, -1.4870911669330995)); under\_twenty\_15 — a Point \[yellow\] drawn in under\_axes (location=(-2.0318447148772227, -0.9606253233498832)); under\_twenty\_16 — a Point \[yellow\] drawn in under\_axes (location=(-2.3674215281891806, -2.4109250850880968)); under\_twenty\_17 — a Point \[yellow\] drawn in under\_axes (location=(-2.638421253619467, -1.8041596975698724)); under\_twenty\_18 — a Point \[yellow\] drawn in under\_axes (location=(-2.8374517251019045, -3.1098022614171272)); under\_twenty\_19 — a Point \[yellow\] drawn in under\_axes (location=(-2.959083910208167, -2.303608116392006)); under\_twenty\_20 — a Point \[yellow\] drawn in under\_axes (location=(-3.0, -3.37)); under\_line — a FunctionPlot \[green\] drawn in under\_axes (function=\, x\_range=(-3.0, 3.0)); under\_more — a Point \[gray\] drawn in under\_axes (location=(-3.0, -2.87), marker\_radius=0.035); under\_more\_2 — a Point \[gray\] drawn in under\_axes (location=(-2.9240506329113924, -2.6703124483484895), marker\_radius=0.035); under\_more\_3 — a Point \[gray\] drawn in under\_axes (location=(-2.848101265822785, -2.765655851631252), marker\_radius=0.035); under\_more\_4 — a Point \[gray\] drawn in under\_axes (location=(-2.7721518987341773, -2.3333720852896733), marker\_radius=0.035); under\_more\_5 — a Point \[gray\] drawn in under\_axes (location=(-2.6962025316455698, -2.567991370314388), marker\_radius=0.035); under\_more\_6 — a Point \[gray\] drawn in under\_axes (location=(-2.620253164556962, -2.132288840073776), marker\_radius=0.035); under\_more\_7 — a Point \[gray\] drawn in under\_axes (location=(-2.5443037974683547, -2.24962977264546), marker\_radius=0.035); under\_more\_8 — a Point \[gray\] drawn in under\_axes (location=(-2.4683544303797467, -2.0452180998191922), marker\_radius=0.035); under\_more\_9 — a Point \[gray\] drawn in under\_axes (location=(-2.392405063291139, -1.8774460539472317), marker\_radius=0.035); under\_more\_10 — a Point \[gray\] drawn in under\_axes (location=(-2.3164556962025316, -1.9730848368698612), marker\_radius=0.035); under\_more\_11 — a Point \[gray\] drawn in under\_axes (location=(-2.240506329113924, -1.563707163843767), marker\_radius=0.035); under\_more\_12 — a Point \[gray\] drawn in under\_axes (location=(-2.1645569620253164, -1.8119692356331485), marker\_radius=0.035); under\_more\_13 — a Point \[gray\] drawn in under\_axes (location=(-2.088607594936709, -1.3840481732727037), marker\_radius=0.035); under\_more\_14 — a Point \[gray\] drawn in under\_axes (location=(-2.0126582278481013, -1.5290319905132814), marker\_radius=0.035); under\_more\_15 — a Point \[gray\] drawn in under\_axes (location=(-1.9367088607594938, -1.3222122140010508), marker\_radius=0.035); under\_more\_16 — a Point \[gray\] drawn in under\_axes (location=(-1.860759493670886, -1.1865071704778114), marker\_radius=0.035); under\_more\_17 — a Point \[gray\] drawn in under\_axes (location=(-1.7848101265822784, -1.2819279781859345), marker\_radius=0.035); under\_more\_18 — a Point \[gray\] drawn in under\_axes (location=(-1.7088607594936709, -0.8962331215517699), marker\_radius=0.035); under\_more\_19 — a Point \[gray\] drawn in under\_axes (location=(-1.6329113924050633, -1.1572469075692413), marker\_radius=0.035); under\_more\_20 — a Point \[gray\] drawn in under\_axes (location=(-1.5569620253164558, -0.7379417777261894), marker\_radius=0.035); under\_more\_21 — a Point \[gray\] drawn in under\_axes (location=(-1.481012658227848, -0.9099507197082083), marker\_radius=0.035); under\_more\_22 — a Point \[gray\] drawn in under\_axes (location=(-1.4050632911392404, -0.7010053739545179), marker\_radius=0.035); under\_more\_23 — a Point \[gray\] drawn in under\_axes (location=(-1.3291139240506329, -0.5974742296665826), marker\_radius=0.035); under\_more\_24 — a Point \[gray\] drawn in under\_axes (location=(-1.2531645569620253, -0.692201242721242), marker\_radius=0.035); under\_more\_25 — a Point \[gray\] drawn in under\_axes (location=(-1.1772151898734178, -0.33094265579120674), marker\_radius=0.035); under\_more\_26 — a Point \[gray\] drawn in under\_axes (location=(-1.1012658227848102, -0.6038216231759829), marker\_radius=0.035); under\_more\_27 — a Point \[gray\] drawn in under\_axes (location=(-1.0253164556962024, -0.19398195186296024), marker\_radius=0.035); under\_more\_28 — a Point \[gray\] drawn in under\_axes (location=(-0.9493670886075951, -0.39236648584332007), marker\_radius=0.035); under\_more\_29 — a Point \[gray\] drawn in under\_axes (location=(-0.8734177215189876, -0.18162049375226394), marker\_radius=0.035); under\_more\_30 — a Point \[gray\] drawn in under\_axes (location=(-0.7974683544303796, -0.11032527395094173), marker\_radius=0.035); under\_more\_31 — a Point \[gray\] drawn in under\_axes (location=(-0.721518987341772, -0.2039214188444216), marker\_radius=0.035); under\_more\_32 — a Point \[gray\] drawn in under\_axes (location=(-0.6455696202531644, 0.13217263166811496), marker\_radius=0.035); under\_more\_33 — a Point \[gray\] drawn in under\_axes (location=(-0.5696202531645569, -0.15169177929664995), marker\_radius=0.035); under\_more\_34 — a Point \[gray\] drawn in under\_axes (location=(-0.49367088607594933, 0.24782000734812762), marker\_radius=0.035); under\_more\_35 — a Point \[gray\] drawn in under\_axes (location=(-0.4177215189873418, 0.023739534474326557), marker\_radius=0.035); under\_more\_36 — a Point \[gray\] drawn in under\_axes (location=(-0.3417721518987342, 0.2359196881642116), marker\_radius=0.035); under\_more\_37 — a Point \[gray\] drawn in under\_axes (location=(-0.26582278481012667, 0.27496198765919205), marker\_radius=0.035); under\_more\_38 — a Point \[gray\] drawn in under\_axes (location=(-0.1898734177215191, 0.18289392655972697), marker\_radius=0.035); under\_more\_39 — a Point \[gray\] drawn in under\_axes (location=(-0.11392405063291156, 0.49312221329845274), marker\_radius=0.035); under\_more\_40 — a Point \[gray\] drawn in under\_axes (location=(-0.037974683544304, 0.19914306335698384), marker\_radius=0.035); under\_more\_41 — a Point \[gray\] drawn in under\_axes (location=(0.037974683544304, 0.5874538331383163), marker\_radius=0.035); under\_more\_42 — a Point \[gray\] drawn in under\_axes (location=(0.11392405063291156, 0.3383854657557492), marker\_radius=0.035); under\_more\_43 — a Point \[gray\] drawn in under\_axes (location=(0.1898734177215191, 0.5515926668326753), marker\_radius=0.035); under\_more\_44 — a Point \[gray\] drawn in under\_axes (location=(0.26582278481012667, 0.558410122871631), marker\_radius=0.035); under\_more\_45 — a Point \[gray\] drawn in under\_axes (location=(0.3417721518987342, 0.46822649278154854), marker\_radius=0.035); under\_more\_46 — a Point \[gray\] drawn in under\_axes (location=(0.4177215189873418, 0.7519166117155476), marker\_radius=0.035); under\_more\_47 — a Point \[gray\] drawn in under\_axes (location=(0.49367088607594933, 0.4486821790870714), marker\_radius=0.035); under\_more\_48 — a Point \[gray\] drawn in under\_axes (location=(0.5696202531645569, 0.8249103150582828), marker\_radius=0.035); under\_more\_49 — a Point \[gray\] drawn in under\_axes (location=(0.6455696202531644, 0.5515886875200892), marker\_radius=0.035); under\_more\_50 — a Point \[gray\] drawn in under\_axes (location=(0.721518987341772, 0.7653762280242439), marker\_radius=0.035); under\_more\_51 — a Point \[gray\] drawn in under\_axes (location=(0.7974683544303796, 0.7400419186982076), marker\_radius=0.035); under\_more\_52 — a Point \[gray\] drawn in under\_axes (location=(0.8734177215189876, 0.6520572918447377), marker\_radius=0.035); under\_more\_53 — a Point \[gray\] drawn in under\_axes (location=(0.9493670886075951, 0.908567372908395), marker\_radius=0.035); under\_more\_54 — a Point \[gray\] drawn in under\_axes (location=(1.0253164556962027, 0.5969236790559147), marker\_radius=0.035); under\_more\_55 — a Point \[gray\] drawn in under\_axes (location=(1.1012658227848098, 0.9601813224101202), marker\_radius=0.035); under\_more\_56 — a Point \[gray\] drawn in under\_axes (location=(1.1772151898734178, 0.6633657900799991), marker\_radius=0.035); under\_more\_57 — a Point \[gray\] drawn in under\_axes (location=(1.253164556962025, 0.877248504757848), marker\_radius=0.035); under\_more\_58 — a Point \[gray\] drawn in under\_axes (location=(1.3291139240506329, 0.8198803234831035), marker\_radius=0.035); under\_more\_59 — a Point \[gray\] drawn in under\_axes (location=(1.4050632911392409, 0.7343666968129939), marker\_radius=0.035); under\_more\_60 — a Point \[gray\] drawn in under\_axes (location=(1.481012658227848, 0.9630870368654897), marker\_radius=0.035); under\_more\_61 — a Point \[gray\] drawn in under\_axes (location=(1.556962025316456, 0.6438645160912888), marker\_radius=0.035); under\_more\_62 — a Point \[gray\] drawn in under\_axes (location=(1.6329113924050631, 0.9932598249323683), marker\_radius=0.035); under\_more\_63 — a Point \[gray\] drawn in under\_axes (location=(1.708860759493671, 0.6737325323347665), marker\_radius=0.035); under\_more\_64 — a Point \[gray\] drawn in under\_axes (location=(1.7848101265822782, 0.887188032931261), marker\_radius=0.035); under\_more\_65 — a Point \[gray\] drawn in under\_axes (location=(1.8607594936708862, 0.7979483885207774), marker\_radius=0.035); under\_more\_66 — a Point \[gray\] drawn in under\_axes (location=(1.9367088607594933, 0.7151344917222289), marker\_radius=0.035); under\_more\_67 — a Point \[gray\] drawn in under\_axes (location=(2.0126582278481013, 0.9154891056722809), marker\_radius=0.035); under\_more\_68 — a Point \[gray\] drawn in under\_axes (location=(2.0886075949367084, 0.5895004924386543), marker\_radius=0.035); under\_more\_69 — a Point \[gray\] drawn in under\_axes (location=(2.1645569620253164, 0.9241399106854595), marker\_radius=0.035); under\_more\_70 — a Point \[gray\] drawn in under\_axes (location=(2.2405063291139244, 0.5827038016786734), marker\_radius=0.035); under\_more\_71 — a Point \[gray\] drawn in under\_axes (location=(2.3164556962025316, 0.7951738059271779), marker\_radius=0.035); under\_more\_72 — a Point \[gray\] drawn in under\_axes (location=(2.3924050632911396, 0.6742692094119506), marker\_radius=0.035); under\_more\_73 — a Point \[gray\] drawn in under\_axes (location=(2.4683544303797467, 0.5943399230113228), marker\_radius=0.035); under\_more\_74 — a Point \[gray\] drawn in under\_axes (location=(2.5443037974683547, 0.7657880091609773), marker\_radius=0.035); under\_more\_75 — a Point \[gray\] drawn in under\_axes (location=(2.620253164556962, 0.43382627044051236), marker\_radius=0.035); under\_more\_76 — a Point \[gray\] drawn in under\_axes (location=(2.6962025316455698, 0.75281680109208), marker\_radius=0.035); under\_more\_77 — a Point \[gray\] drawn in under\_axes (location=(2.772151898734177, 0.3902935761264736), marker\_radius=0.035); under\_more\_78 — a Point \[gray\] drawn in under\_axes (location=(2.848101265822785, 0.6011853280554427), marker\_radius=0.035); under\_more\_79 — a Point \[gray\] drawn in under\_axes (location=(2.924050632911392, 0.44886586730688843), marker\_radius=0.035); under\_more\_80 — a Point \[gray\] drawn in under\_axes (location=(3.0, 0.37196175232051404), marker\_radius=0.035); under\_residuals — a Line \[red\] drawn in under\_axes (start=(-2.4, -1.9567999999999999), end=(-2.4, -1.46), dashed=True); under\_residuals\_2 — a Line \[red\] drawn in under\_axes (start=(-1.2, -0.5192), end=(-1.2, -0.8), dashed=True); under\_residuals\_3 — a Line \[red\] drawn in under\_axes (start=(0.0, 0.4), end=(0.0, -0.14), dashed=True); under\_residuals\_4 — a Line \[red\] drawn in under\_axes (start=(1.2, 0.8008000000000001), end=(1.2, 0.52), dashed=True); under\_residuals\_5 — a Line \[red\] drawn in under\_axes (start=(2.4, 0.6832000000000003), end=(2.4, 1.1800000000000002), dashed=True); over\_truth — a FunctionPlot \[blue\] drawn in over\_axes (function=\, x\_range=(-3.0, 3.0)); over\_twenty — a Point \[yellow\] drawn in over\_axes (location=(3.0, 0.9300000000000002)); over\_twenty\_2 — a Point \[yellow\] drawn in over\_axes (location=(2.959083910208167, -0.048615815163021825)); over\_twenty\_3 — a Point \[yellow\] drawn in over\_axes (location=(2.837451725101904, 1.0113946361949688)); over\_twenty\_4 — a Point \[yellow\] drawn in over\_axes (location=(2.6384212536194673, 0.09810368141154147)); over\_twenty\_5 — a Point \[yellow\] drawn in over\_axes (location=(2.3674215281891806, 1.1932385959200025)); over\_twenty\_6 — a Point \[yellow\] drawn in over\_axes (location=(2.031844714877223, 0.2744038630150619)); over\_twenty\_7 — a Point \[yellow\] drawn in over\_axes (location=(1.6408444743672808, 1.31783775487091)); over\_twenty\_8 — a Point \[yellow\] drawn in over\_axes (location=(1.2050862739589085, 0.3013955236942498)); over\_twenty\_9 — a Point \[yellow\] drawn in over\_axes (location=(0.7364564614223977, 1.2074247922595749)); over\_twenty\_10 — a Point \[yellow\] drawn in over\_axes (location=(0.24773803641699718, 0.025208575785553577)); over\_twenty\_11 — a Point \[yellow\] drawn in over\_axes (location=(-0.24773803641699682, 0.7526967357268568)); over\_twenty\_12 — a Point \[yellow\] drawn in over\_axes (location=(-0.7364564614223967, -0.6026773153050619)); over\_twenty\_13 — a Point \[yellow\] drawn in over\_axes (location=(-1.2050862739589083, -0.02419937766054925)); over\_twenty\_14 — a Point \[yellow\] drawn in over\_axes (location=(-1.640844474367281, -1.4870911669330995)); over\_twenty\_15 — a Point \[yellow\] drawn in over\_axes (location=(-2.0318447148772227, -0.9606253233498832)); over\_twenty\_16 — a Point \[yellow\] drawn in over\_axes (location=(-2.3674215281891806, -2.4109250850880968)); over\_twenty\_17 — a Point \[yellow\] drawn in over\_axes (location=(-2.638421253619467, -1.8041596975698724)); over\_twenty\_18 — a Point \[yellow\] drawn in over\_axes (location=(-2.8374517251019045, -3.1098022614171272)); over\_twenty\_19 — a Point \[yellow\] drawn in over\_axes (location=(-2.959083910208167, -2.303608116392006)); over\_twenty\_20 — a Point \[yellow\] drawn in over\_axes (location=(-3.0, -3.37)); over\_fit — a FunctionPlot \[red\] drawn in over\_axes (function=\, x\_range=(-3.0, 3.0)); train\_readout — a Math \[yellow\] that says "$upright("training MSE") = 0.000$" drawn in over\_axes Actions: - [03:51.369](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=231.36866666666663): test\_marks is shown on the screen, written out. - [03:51.419](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=231.41866666666664): test\_marks\_2 is shown on the screen, written out. - [03:51.469](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=231.46866666666665): test\_marks\_3 is shown on the screen, written out. - [03:51.519](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=231.51866666666663): test\_marks\_4 is shown on the screen, written out. - [03:51.569](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=231.56866666666664): test\_marks\_5 is shown on the screen, written out. - [03:51.619](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=231.61866666666663): test\_marks\_6 is shown on the screen, written out. - [03:51.669](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=231.66866666666664): test\_marks\_7 is shown on the screen, written out. - [03:51.719](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=231.71866666666665): test\_marks\_8 is shown on the screen, written out. - [03:51.769](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=231.76866666666663): test\_marks\_9 is shown on the screen, written out. - [03:51.819](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=231.81866666666664): test\_marks\_10 is shown on the screen, written out. ##### [04:1.291](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=241.29116666666664) Narration: The magenta gaps measure predictions at those unseen locations. Several are large because the red curve swings away between neighboring training points. Perfect recall of the training set did not produce reliable prediction. Board: left\_label — a Tex \[text\] that says "Too rigid"; under\_axes — an Axes (x\_range=(-3.0, 3.0), y\_range=(-3.6, 1.8), x\_ticks\_every=1.0); right\_label — a Tex \[text\] that says "Too responsive"; over\_axes — an Axes (x\_range=(-3.0, 3.0), y\_range=(-3.6, 1.8), x\_ticks\_every=1.0); heading — a Heading that says "Failure in Both Directions"; under\_truth — a FunctionPlot \[blue\] drawn in under\_axes (function=\, x\_range=(-3.0, 3.0)); under\_twenty — a Point \[yellow\] drawn in under\_axes (location=(3.0, 0.9300000000000002)); under\_twenty\_2 — a Point \[yellow\] drawn in under\_axes (location=(2.959083910208167, -0.048615815163021825)); under\_twenty\_3 — a Point \[yellow\] drawn in under\_axes (location=(2.837451725101904, 1.0113946361949688)); under\_twenty\_4 — a Point \[yellow\] drawn in under\_axes (location=(2.6384212536194673, 0.09810368141154147)); under\_twenty\_5 — a Point \[yellow\] drawn in under\_axes (location=(2.3674215281891806, 1.1932385959200025)); under\_twenty\_6 — a Point \[yellow\] drawn in under\_axes (location=(2.031844714877223, 0.2744038630150619)); under\_twenty\_7 — a Point \[yellow\] drawn in under\_axes (location=(1.6408444743672808, 1.31783775487091)); under\_twenty\_8 — a Point \[yellow\] drawn in under\_axes (location=(1.2050862739589085, 0.3013955236942498)); under\_twenty\_9 — a Point \[yellow\] drawn in under\_axes (location=(0.7364564614223977, 1.2074247922595749)); under\_twenty\_10 — a Point \[yellow\] drawn in under\_axes (location=(0.24773803641699718, 0.025208575785553577)); under\_twenty\_11 — a Point \[yellow\] drawn in under\_axes (location=(-0.24773803641699682, 0.7526967357268568)); under\_twenty\_12 — a Point \[yellow\] drawn in under\_axes (location=(-0.7364564614223967, -0.6026773153050619)); under\_twenty\_13 — a Point \[yellow\] drawn in under\_axes (location=(-1.2050862739589083, -0.02419937766054925)); under\_twenty\_14 — a Point \[yellow\] drawn in under\_axes (location=(-1.640844474367281, -1.4870911669330995)); under\_twenty\_15 — a Point \[yellow\] drawn in under\_axes (location=(-2.0318447148772227, -0.9606253233498832)); under\_twenty\_16 — a Point \[yellow\] drawn in under\_axes (location=(-2.3674215281891806, -2.4109250850880968)); under\_twenty\_17 — a Point \[yellow\] drawn in under\_axes (location=(-2.638421253619467, -1.8041596975698724)); under\_twenty\_18 — a Point \[yellow\] drawn in under\_axes (location=(-2.8374517251019045, -3.1098022614171272)); under\_twenty\_19 — a Point \[yellow\] drawn in under\_axes (location=(-2.959083910208167, -2.303608116392006)); under\_twenty\_20 — a Point \[yellow\] drawn in under\_axes (location=(-3.0, -3.37)); under\_line — a FunctionPlot \[green\] drawn in under\_axes (function=\, x\_range=(-3.0, 3.0)); under\_more — a Point \[gray\] drawn in under\_axes (location=(-3.0, -2.87), marker\_radius=0.035); under\_more\_2 — a Point \[gray\] drawn in under\_axes (location=(-2.9240506329113924, -2.6703124483484895), marker\_radius=0.035); under\_more\_3 — a Point \[gray\] drawn in under\_axes (location=(-2.848101265822785, -2.765655851631252), marker\_radius=0.035); under\_more\_4 — a Point \[gray\] drawn in under\_axes (location=(-2.7721518987341773, -2.3333720852896733), marker\_radius=0.035); under\_more\_5 — a Point \[gray\] drawn in under\_axes (location=(-2.6962025316455698, -2.567991370314388), marker\_radius=0.035); under\_more\_6 — a Point \[gray\] drawn in under\_axes (location=(-2.620253164556962, -2.132288840073776), marker\_radius=0.035); under\_more\_7 — a Point \[gray\] drawn in under\_axes (location=(-2.5443037974683547, -2.24962977264546), marker\_radius=0.035); under\_more\_8 — a Point \[gray\] drawn in under\_axes (location=(-2.4683544303797467, -2.0452180998191922), marker\_radius=0.035); under\_more\_9 — a Point \[gray\] drawn in under\_axes (location=(-2.392405063291139, -1.8774460539472317), marker\_radius=0.035); under\_more\_10 — a Point \[gray\] drawn in under\_axes (location=(-2.3164556962025316, -1.9730848368698612), marker\_radius=0.035); under\_more\_11 — a Point \[gray\] drawn in under\_axes (location=(-2.240506329113924, -1.563707163843767), marker\_radius=0.035); under\_more\_12 — a Point \[gray\] drawn in under\_axes (location=(-2.1645569620253164, -1.8119692356331485), marker\_radius=0.035); under\_more\_13 — a Point \[gray\] drawn in under\_axes (location=(-2.088607594936709, -1.3840481732727037), marker\_radius=0.035); under\_more\_14 — a Point \[gray\] drawn in under\_axes (location=(-2.0126582278481013, -1.5290319905132814), marker\_radius=0.035); under\_more\_15 — a Point \[gray\] drawn in under\_axes (location=(-1.9367088607594938, -1.3222122140010508), marker\_radius=0.035); under\_more\_16 — a Point \[gray\] drawn in under\_axes (location=(-1.860759493670886, -1.1865071704778114), marker\_radius=0.035); under\_more\_17 — a Point \[gray\] drawn in under\_axes (location=(-1.7848101265822784, -1.2819279781859345), marker\_radius=0.035); under\_more\_18 — a Point \[gray\] drawn in under\_axes (location=(-1.7088607594936709, -0.8962331215517699), marker\_radius=0.035); under\_more\_19 — a Point \[gray\] drawn in under\_axes (location=(-1.6329113924050633, -1.1572469075692413), marker\_radius=0.035); under\_more\_20 — a Point \[gray\] drawn in under\_axes (location=(-1.5569620253164558, -0.7379417777261894), marker\_radius=0.035); under\_more\_21 — a Point \[gray\] drawn in under\_axes (location=(-1.481012658227848, -0.9099507197082083), marker\_radius=0.035); under\_more\_22 — a Point \[gray\] drawn in under\_axes (location=(-1.4050632911392404, -0.7010053739545179), marker\_radius=0.035); under\_more\_23 — a Point \[gray\] drawn in under\_axes (location=(-1.3291139240506329, -0.5974742296665826), marker\_radius=0.035); under\_more\_24 — a Point \[gray\] drawn in under\_axes (location=(-1.2531645569620253, -0.692201242721242), marker\_radius=0.035); under\_more\_25 — a Point \[gray\] drawn in under\_axes (location=(-1.1772151898734178, -0.33094265579120674), marker\_radius=0.035); under\_more\_26 — a Point \[gray\] drawn in under\_axes (location=(-1.1012658227848102, -0.6038216231759829), marker\_radius=0.035); under\_more\_27 — a Point \[gray\] drawn in under\_axes (location=(-1.0253164556962024, -0.19398195186296024), marker\_radius=0.035); under\_more\_28 — a Point \[gray\] drawn in under\_axes (location=(-0.9493670886075951, -0.39236648584332007), marker\_radius=0.035); under\_more\_29 — a Point \[gray\] drawn in under\_axes (location=(-0.8734177215189876, -0.18162049375226394), marker\_radius=0.035); under\_more\_30 — a Point \[gray\] drawn in under\_axes (location=(-0.7974683544303796, -0.11032527395094173), marker\_radius=0.035); under\_more\_31 — a Point \[gray\] drawn in under\_axes (location=(-0.721518987341772, -0.2039214188444216), marker\_radius=0.035); under\_more\_32 — a Point \[gray\] drawn in under\_axes (location=(-0.6455696202531644, 0.13217263166811496), marker\_radius=0.035); under\_more\_33 — a Point \[gray\] drawn in under\_axes (location=(-0.5696202531645569, -0.15169177929664995), marker\_radius=0.035); under\_more\_34 — a Point \[gray\] drawn in under\_axes (location=(-0.49367088607594933, 0.24782000734812762), marker\_radius=0.035); under\_more\_35 — a Point \[gray\] drawn in under\_axes (location=(-0.4177215189873418, 0.023739534474326557), marker\_radius=0.035); under\_more\_36 — a Point \[gray\] drawn in under\_axes (location=(-0.3417721518987342, 0.2359196881642116), marker\_radius=0.035); under\_more\_37 — a Point \[gray\] drawn in under\_axes (location=(-0.26582278481012667, 0.27496198765919205), marker\_radius=0.035); under\_more\_38 — a Point \[gray\] drawn in under\_axes (location=(-0.1898734177215191, 0.18289392655972697), marker\_radius=0.035); under\_more\_39 — a Point \[gray\] drawn in under\_axes (location=(-0.11392405063291156, 0.49312221329845274), marker\_radius=0.035); under\_more\_40 — a Point \[gray\] drawn in under\_axes (location=(-0.037974683544304, 0.19914306335698384), marker\_radius=0.035); under\_more\_41 — a Point \[gray\] drawn in under\_axes (location=(0.037974683544304, 0.5874538331383163), marker\_radius=0.035); under\_more\_42 — a Point \[gray\] drawn in under\_axes (location=(0.11392405063291156, 0.3383854657557492), marker\_radius=0.035); under\_more\_43 — a Point \[gray\] drawn in under\_axes (location=(0.1898734177215191, 0.5515926668326753), marker\_radius=0.035); under\_more\_44 — a Point \[gray\] drawn in under\_axes (location=(0.26582278481012667, 0.558410122871631), marker\_radius=0.035); under\_more\_45 — a Point \[gray\] drawn in under\_axes (location=(0.3417721518987342, 0.46822649278154854), marker\_radius=0.035); under\_more\_46 — a Point \[gray\] drawn in under\_axes (location=(0.4177215189873418, 0.7519166117155476), marker\_radius=0.035); under\_more\_47 — a Point \[gray\] drawn in under\_axes (location=(0.49367088607594933, 0.4486821790870714), marker\_radius=0.035); under\_more\_48 — a Point \[gray\] drawn in under\_axes (location=(0.5696202531645569, 0.8249103150582828), marker\_radius=0.035); under\_more\_49 — a Point \[gray\] drawn in under\_axes (location=(0.6455696202531644, 0.5515886875200892), marker\_radius=0.035); under\_more\_50 — a Point \[gray\] drawn in under\_axes (location=(0.721518987341772, 0.7653762280242439), marker\_radius=0.035); under\_more\_51 — a Point \[gray\] drawn in under\_axes (location=(0.7974683544303796, 0.7400419186982076), marker\_radius=0.035); under\_more\_52 — a Point \[gray\] drawn in under\_axes (location=(0.8734177215189876, 0.6520572918447377), marker\_radius=0.035); under\_more\_53 — a Point \[gray\] drawn in under\_axes (location=(0.9493670886075951, 0.908567372908395), marker\_radius=0.035); under\_more\_54 — a Point \[gray\] drawn in under\_axes (location=(1.0253164556962027, 0.5969236790559147), marker\_radius=0.035); under\_more\_55 — a Point \[gray\] drawn in under\_axes (location=(1.1012658227848098, 0.9601813224101202), marker\_radius=0.035); under\_more\_56 — a Point \[gray\] drawn in under\_axes (location=(1.1772151898734178, 0.6633657900799991), marker\_radius=0.035); under\_more\_57 — a Point \[gray\] drawn in under\_axes (location=(1.253164556962025, 0.877248504757848), marker\_radius=0.035); under\_more\_58 — a Point \[gray\] drawn in under\_axes (location=(1.3291139240506329, 0.8198803234831035), marker\_radius=0.035); under\_more\_59 — a Point \[gray\] drawn in under\_axes (location=(1.4050632911392409, 0.7343666968129939), marker\_radius=0.035); under\_more\_60 — a Point \[gray\] drawn in under\_axes (location=(1.481012658227848, 0.9630870368654897), marker\_radius=0.035); under\_more\_61 — a Point \[gray\] drawn in under\_axes (location=(1.556962025316456, 0.6438645160912888), marker\_radius=0.035); under\_more\_62 — a Point \[gray\] drawn in under\_axes (location=(1.6329113924050631, 0.9932598249323683), marker\_radius=0.035); under\_more\_63 — a Point \[gray\] drawn in under\_axes (location=(1.708860759493671, 0.6737325323347665), marker\_radius=0.035); under\_more\_64 — a Point \[gray\] drawn in under\_axes (location=(1.7848101265822782, 0.887188032931261), marker\_radius=0.035); under\_more\_65 — a Point \[gray\] drawn in under\_axes (location=(1.8607594936708862, 0.7979483885207774), marker\_radius=0.035); under\_more\_66 — a Point \[gray\] drawn in under\_axes (location=(1.9367088607594933, 0.7151344917222289), marker\_radius=0.035); under\_more\_67 — a Point \[gray\] drawn in under\_axes (location=(2.0126582278481013, 0.9154891056722809), marker\_radius=0.035); under\_more\_68 — a Point \[gray\] drawn in under\_axes (location=(2.0886075949367084, 0.5895004924386543), marker\_radius=0.035); under\_more\_69 — a Point \[gray\] drawn in under\_axes (location=(2.1645569620253164, 0.9241399106854595), marker\_radius=0.035); under\_more\_70 — a Point \[gray\] drawn in under\_axes (location=(2.2405063291139244, 0.5827038016786734), marker\_radius=0.035); under\_more\_71 — a Point \[gray\] drawn in under\_axes (location=(2.3164556962025316, 0.7951738059271779), marker\_radius=0.035); under\_more\_72 — a Point \[gray\] drawn in under\_axes (location=(2.3924050632911396, 0.6742692094119506), marker\_radius=0.035); under\_more\_73 — a Point \[gray\] drawn in under\_axes (location=(2.4683544303797467, 0.5943399230113228), marker\_radius=0.035); under\_more\_74 — a Point \[gray\] drawn in under\_axes (location=(2.5443037974683547, 0.7657880091609773), marker\_radius=0.035); under\_more\_75 — a Point \[gray\] drawn in under\_axes (location=(2.620253164556962, 0.43382627044051236), marker\_radius=0.035); under\_more\_76 — a Point \[gray\] drawn in under\_axes (location=(2.6962025316455698, 0.75281680109208), marker\_radius=0.035); under\_more\_77 — a Point \[gray\] drawn in under\_axes (location=(2.772151898734177, 0.3902935761264736), marker\_radius=0.035); under\_more\_78 — a Point \[gray\] drawn in under\_axes (location=(2.848101265822785, 0.6011853280554427), marker\_radius=0.035); under\_more\_79 — a Point \[gray\] drawn in under\_axes (location=(2.924050632911392, 0.44886586730688843), marker\_radius=0.035); under\_more\_80 — a Point \[gray\] drawn in under\_axes (location=(3.0, 0.37196175232051404), marker\_radius=0.035); under\_residuals — a Line \[red\] drawn in under\_axes (start=(-2.4, -1.9567999999999999), end=(-2.4, -1.46), dashed=True); under\_residuals\_2 — a Line \[red\] drawn in under\_axes (start=(-1.2, -0.5192), end=(-1.2, -0.8), dashed=True); under\_residuals\_3 — a Line \[red\] drawn in under\_axes (start=(0.0, 0.4), end=(0.0, -0.14), dashed=True); under\_residuals\_4 — a Line \[red\] drawn in under\_axes (start=(1.2, 0.8008000000000001), end=(1.2, 0.52), dashed=True); under\_residuals\_5 — a Line \[red\] drawn in under\_axes (start=(2.4, 0.6832000000000003), end=(2.4, 1.1800000000000002), dashed=True); over\_truth — a FunctionPlot \[blue\] drawn in over\_axes (function=\, x\_range=(-3.0, 3.0)); over\_twenty — a Point \[yellow\] drawn in over\_axes (location=(3.0, 0.9300000000000002)); over\_twenty\_2 — a Point \[yellow\] drawn in over\_axes (location=(2.959083910208167, -0.048615815163021825)); over\_twenty\_3 — a Point \[yellow\] drawn in over\_axes (location=(2.837451725101904, 1.0113946361949688)); over\_twenty\_4 — a Point \[yellow\] drawn in over\_axes (location=(2.6384212536194673, 0.09810368141154147)); over\_twenty\_5 — a Point \[yellow\] drawn in over\_axes (location=(2.3674215281891806, 1.1932385959200025)); over\_twenty\_6 — a Point \[yellow\] drawn in over\_axes (location=(2.031844714877223, 0.2744038630150619)); over\_twenty\_7 — a Point \[yellow\] drawn in over\_axes (location=(1.6408444743672808, 1.31783775487091)); over\_twenty\_8 — a Point \[yellow\] drawn in over\_axes (location=(1.2050862739589085, 0.3013955236942498)); over\_twenty\_9 — a Point \[yellow\] drawn in over\_axes (location=(0.7364564614223977, 1.2074247922595749)); over\_twenty\_10 — a Point \[yellow\] drawn in over\_axes (location=(0.24773803641699718, 0.025208575785553577)); over\_twenty\_11 — a Point \[yellow\] drawn in over\_axes (location=(-0.24773803641699682, 0.7526967357268568)); over\_twenty\_12 — a Point \[yellow\] drawn in over\_axes (location=(-0.7364564614223967, -0.6026773153050619)); over\_twenty\_13 — a Point \[yellow\] drawn in over\_axes (location=(-1.2050862739589083, -0.02419937766054925)); over\_twenty\_14 — a Point \[yellow\] drawn in over\_axes (location=(-1.640844474367281, -1.4870911669330995)); over\_twenty\_15 — a Point \[yellow\] drawn in over\_axes (location=(-2.0318447148772227, -0.9606253233498832)); over\_twenty\_16 — a Point \[yellow\] drawn in over\_axes (location=(-2.3674215281891806, -2.4109250850880968)); over\_twenty\_17 — a Point \[yellow\] drawn in over\_axes (location=(-2.638421253619467, -1.8041596975698724)); over\_twenty\_18 — a Point \[yellow\] drawn in over\_axes (location=(-2.8374517251019045, -3.1098022614171272)); over\_twenty\_19 — a Point \[yellow\] drawn in over\_axes (location=(-2.959083910208167, -2.303608116392006)); over\_twenty\_20 — a Point \[yellow\] drawn in over\_axes (location=(-3.0, -3.37)); over\_fit — a FunctionPlot \[red\] drawn in over\_axes (function=\, x\_range=(-3.0, 3.0)); train\_readout — a Math \[yellow\] that says "$upright("training MSE") = 0.000$" drawn in over\_axes; test\_marks — a Point \[green\] drawn in over\_axes (location=(-2.65, -2.3215500000000002)); test\_marks\_2 — a Point \[green\] drawn in over\_axes (location=(-2.05, -1.4839499999999997)); test\_marks\_3 — a Point \[green\] drawn in over\_axes (location=(-1.45, -0.7759499999999999)); test\_marks\_4 — a Point \[green\] drawn in over\_axes (location=(-0.85, -0.19755)); test\_marks\_5 — a Point \[green\] drawn in over\_axes (location=(-0.25, 0.25125000000000003)); test\_marks\_6 — a Point \[green\] drawn in over\_axes (location=(0.35, 0.57045)); test\_marks\_7 — a Point \[green\] drawn in over\_axes (location=(0.95, 0.76005)); test\_marks\_8 — a Point \[green\] drawn in over\_axes (location=(1.55, 0.8200500000000002)); test\_marks\_9 — a Point \[green\] drawn in over\_axes (location=(2.15, 0.7504500000000002)); test\_marks\_10 — a Point \[green\] drawn in over\_axes (location=(2.65, 0.59345)) Actions: - [04:1.768](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=241.76766666666663): test\_residuals is shown on the screen, written out. - [04:1.828](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=241.82766666666663): test\_residuals\_2 is shown on the screen, written out. - [04:1.888](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=241.88766666666663): test\_residuals\_3 is shown on the screen, written out. - [04:1.948](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=241.94766666666663): test\_residuals\_4 is shown on the screen, written out. - [04:2.008](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=242.00766666666664): test\_residuals\_5 is shown on the screen, written out. - [04:2.068](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=242.06766666666664): test\_residuals\_6 is shown on the screen, written out. - [04:2.128](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=242.12766666666664): test\_residuals\_7 is shown on the screen, written out. - [04:2.188](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=242.18766666666664): test\_residuals\_8 is shown on the screen, written out. - [04:2.248](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=242.24766666666665): test\_residuals\_9 is shown on the screen, written out. - [04:2.308](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=242.30766666666665): test\_residuals\_10 is shown on the screen, written out. - [04:6.597](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=246.59666666666664): test\_readout is shown on the screen, written out. ##### [04:16.357](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=256.35716666666667) Narration: That gives us an operational definition. Overfitting is not simply a complicated-looking curve. It is a measurable gap: very small error on the data used for fitting, paired with substantially larger error on new data. Board: left\_label — a Tex \[text\] that says "Too rigid"; under\_axes — an Axes (x\_range=(-3.0, 3.0), y\_range=(-3.6, 1.8), x\_ticks\_every=1.0); right\_label — a Tex \[text\] that says "Too responsive"; over\_axes — an Axes (x\_range=(-3.0, 3.0), y\_range=(-3.6, 1.8), x\_ticks\_every=1.0); heading — a Heading that says "Failure in Both Directions"; under\_truth — a FunctionPlot \[blue\] drawn in under\_axes (function=\, x\_range=(-3.0, 3.0)); under\_twenty — a Point \[yellow\] drawn in under\_axes (location=(3.0, 0.9300000000000002)); under\_twenty\_2 — a Point \[yellow\] drawn in under\_axes (location=(2.959083910208167, -0.048615815163021825)); under\_twenty\_3 — a Point \[yellow\] drawn in under\_axes (location=(2.837451725101904, 1.0113946361949688)); under\_twenty\_4 — a Point \[yellow\] drawn in under\_axes (location=(2.6384212536194673, 0.09810368141154147)); under\_twenty\_5 — a Point \[yellow\] drawn in under\_axes (location=(2.3674215281891806, 1.1932385959200025)); under\_twenty\_6 — a Point \[yellow\] drawn in under\_axes (location=(2.031844714877223, 0.2744038630150619)); under\_twenty\_7 — a Point \[yellow\] drawn in under\_axes (location=(1.6408444743672808, 1.31783775487091)); under\_twenty\_8 — a Point \[yellow\] drawn in under\_axes (location=(1.2050862739589085, 0.3013955236942498)); under\_twenty\_9 — a Point \[yellow\] drawn in under\_axes (location=(0.7364564614223977, 1.2074247922595749)); under\_twenty\_10 — a Point \[yellow\] drawn in under\_axes (location=(0.24773803641699718, 0.025208575785553577)); under\_twenty\_11 — a Point \[yellow\] drawn in under\_axes (location=(-0.24773803641699682, 0.7526967357268568)); under\_twenty\_12 — a Point \[yellow\] drawn in under\_axes (location=(-0.7364564614223967, -0.6026773153050619)); under\_twenty\_13 — a Point \[yellow\] drawn in under\_axes (location=(-1.2050862739589083, -0.02419937766054925)); under\_twenty\_14 — a Point \[yellow\] drawn in under\_axes (location=(-1.640844474367281, -1.4870911669330995)); under\_twenty\_15 — a Point \[yellow\] drawn in under\_axes (location=(-2.0318447148772227, -0.9606253233498832)); under\_twenty\_16 — a Point \[yellow\] drawn in under\_axes (location=(-2.3674215281891806, -2.4109250850880968)); under\_twenty\_17 — a Point \[yellow\] drawn in under\_axes (location=(-2.638421253619467, -1.8041596975698724)); under\_twenty\_18 — a Point \[yellow\] drawn in under\_axes (location=(-2.8374517251019045, -3.1098022614171272)); under\_twenty\_19 — a Point \[yellow\] drawn in under\_axes (location=(-2.959083910208167, -2.303608116392006)); under\_twenty\_20 — a Point \[yellow\] drawn in under\_axes (location=(-3.0, -3.37)); under\_line — a FunctionPlot \[green\] drawn in under\_axes (function=\, x\_range=(-3.0, 3.0)); under\_more — a Point \[gray\] drawn in under\_axes (location=(-3.0, -2.87), marker\_radius=0.035); under\_more\_2 — a Point \[gray\] drawn in under\_axes (location=(-2.9240506329113924, -2.6703124483484895), marker\_radius=0.035); under\_more\_3 — a Point \[gray\] drawn in under\_axes (location=(-2.848101265822785, -2.765655851631252), marker\_radius=0.035); under\_more\_4 — a Point \[gray\] drawn in under\_axes (location=(-2.7721518987341773, -2.3333720852896733), marker\_radius=0.035); under\_more\_5 — a Point \[gray\] drawn in under\_axes (location=(-2.6962025316455698, -2.567991370314388), marker\_radius=0.035); under\_more\_6 — a Point \[gray\] drawn in under\_axes (location=(-2.620253164556962, -2.132288840073776), marker\_radius=0.035); under\_more\_7 — a Point \[gray\] drawn in under\_axes (location=(-2.5443037974683547, -2.24962977264546), marker\_radius=0.035); under\_more\_8 — a Point \[gray\] drawn in under\_axes (location=(-2.4683544303797467, -2.0452180998191922), marker\_radius=0.035); under\_more\_9 — a Point \[gray\] drawn in under\_axes (location=(-2.392405063291139, -1.8774460539472317), marker\_radius=0.035); under\_more\_10 — a Point \[gray\] drawn in under\_axes (location=(-2.3164556962025316, -1.9730848368698612), marker\_radius=0.035); under\_more\_11 — a Point \[gray\] drawn in under\_axes (location=(-2.240506329113924, -1.563707163843767), marker\_radius=0.035); under\_more\_12 — a Point \[gray\] drawn in under\_axes (location=(-2.1645569620253164, -1.8119692356331485), marker\_radius=0.035); under\_more\_13 — a Point \[gray\] drawn in under\_axes (location=(-2.088607594936709, -1.3840481732727037), marker\_radius=0.035); under\_more\_14 — a Point \[gray\] drawn in under\_axes (location=(-2.0126582278481013, -1.5290319905132814), marker\_radius=0.035); under\_more\_15 — a Point \[gray\] drawn in under\_axes (location=(-1.9367088607594938, -1.3222122140010508), marker\_radius=0.035); under\_more\_16 — a Point \[gray\] drawn in under\_axes (location=(-1.860759493670886, -1.1865071704778114), marker\_radius=0.035); under\_more\_17 — a Point \[gray\] drawn in under\_axes (location=(-1.7848101265822784, -1.2819279781859345), marker\_radius=0.035); under\_more\_18 — a Point \[gray\] drawn in under\_axes (location=(-1.7088607594936709, -0.8962331215517699), marker\_radius=0.035); under\_more\_19 — a Point \[gray\] drawn in under\_axes (location=(-1.6329113924050633, -1.1572469075692413), marker\_radius=0.035); under\_more\_20 — a Point \[gray\] drawn in under\_axes (location=(-1.5569620253164558, -0.7379417777261894), marker\_radius=0.035); under\_more\_21 — a Point \[gray\] drawn in under\_axes (location=(-1.481012658227848, -0.9099507197082083), marker\_radius=0.035); under\_more\_22 — a Point \[gray\] drawn in under\_axes (location=(-1.4050632911392404, -0.7010053739545179), marker\_radius=0.035); under\_more\_23 — a Point \[gray\] drawn in under\_axes (location=(-1.3291139240506329, -0.5974742296665826), marker\_radius=0.035); under\_more\_24 — a Point \[gray\] drawn in under\_axes (location=(-1.2531645569620253, -0.692201242721242), marker\_radius=0.035); under\_more\_25 — a Point \[gray\] drawn in under\_axes (location=(-1.1772151898734178, -0.33094265579120674), marker\_radius=0.035); under\_more\_26 — a Point \[gray\] drawn in under\_axes (location=(-1.1012658227848102, -0.6038216231759829), marker\_radius=0.035); under\_more\_27 — a Point \[gray\] drawn in under\_axes (location=(-1.0253164556962024, -0.19398195186296024), marker\_radius=0.035); under\_more\_28 — a Point \[gray\] drawn in under\_axes (location=(-0.9493670886075951, -0.39236648584332007), marker\_radius=0.035); under\_more\_29 — a Point \[gray\] drawn in under\_axes (location=(-0.8734177215189876, -0.18162049375226394), marker\_radius=0.035); under\_more\_30 — a Point \[gray\] drawn in under\_axes (location=(-0.7974683544303796, -0.11032527395094173), marker\_radius=0.035); under\_more\_31 — a Point \[gray\] drawn in under\_axes (location=(-0.721518987341772, -0.2039214188444216), marker\_radius=0.035); under\_more\_32 — a Point \[gray\] drawn in under\_axes (location=(-0.6455696202531644, 0.13217263166811496), marker\_radius=0.035); under\_more\_33 — a Point \[gray\] drawn in under\_axes (location=(-0.5696202531645569, -0.15169177929664995), marker\_radius=0.035); under\_more\_34 — a Point \[gray\] drawn in under\_axes (location=(-0.49367088607594933, 0.24782000734812762), marker\_radius=0.035); under\_more\_35 — a Point \[gray\] drawn in under\_axes (location=(-0.4177215189873418, 0.023739534474326557), marker\_radius=0.035); under\_more\_36 — a Point \[gray\] drawn in under\_axes (location=(-0.3417721518987342, 0.2359196881642116), marker\_radius=0.035); under\_more\_37 — a Point \[gray\] drawn in under\_axes (location=(-0.26582278481012667, 0.27496198765919205), marker\_radius=0.035); under\_more\_38 — a Point \[gray\] drawn in under\_axes (location=(-0.1898734177215191, 0.18289392655972697), marker\_radius=0.035); under\_more\_39 — a Point \[gray\] drawn in under\_axes (location=(-0.11392405063291156, 0.49312221329845274), marker\_radius=0.035); under\_more\_40 — a Point \[gray\] drawn in under\_axes (location=(-0.037974683544304, 0.19914306335698384), marker\_radius=0.035); under\_more\_41 — a Point \[gray\] drawn in under\_axes (location=(0.037974683544304, 0.5874538331383163), marker\_radius=0.035); under\_more\_42 — a Point \[gray\] drawn in under\_axes (location=(0.11392405063291156, 0.3383854657557492), marker\_radius=0.035); under\_more\_43 — a Point \[gray\] drawn in under\_axes (location=(0.1898734177215191, 0.5515926668326753), marker\_radius=0.035); under\_more\_44 — a Point \[gray\] drawn in under\_axes (location=(0.26582278481012667, 0.558410122871631), marker\_radius=0.035); under\_more\_45 — a Point \[gray\] drawn in under\_axes (location=(0.3417721518987342, 0.46822649278154854), marker\_radius=0.035); under\_more\_46 — a Point \[gray\] drawn in under\_axes (location=(0.4177215189873418, 0.7519166117155476), marker\_radius=0.035); under\_more\_47 — a Point \[gray\] drawn in under\_axes (location=(0.49367088607594933, 0.4486821790870714), marker\_radius=0.035); under\_more\_48 — a Point \[gray\] drawn in under\_axes (location=(0.5696202531645569, 0.8249103150582828), marker\_radius=0.035); under\_more\_49 — a Point \[gray\] drawn in under\_axes (location=(0.6455696202531644, 0.5515886875200892), marker\_radius=0.035); under\_more\_50 — a Point \[gray\] drawn in under\_axes (location=(0.721518987341772, 0.7653762280242439), marker\_radius=0.035); under\_more\_51 — a Point \[gray\] drawn in under\_axes (location=(0.7974683544303796, 0.7400419186982076), marker\_radius=0.035); under\_more\_52 — a Point \[gray\] drawn in under\_axes (location=(0.8734177215189876, 0.6520572918447377), marker\_radius=0.035); under\_more\_53 — a Point \[gray\] drawn in under\_axes (location=(0.9493670886075951, 0.908567372908395), marker\_radius=0.035); under\_more\_54 — a Point \[gray\] drawn in under\_axes (location=(1.0253164556962027, 0.5969236790559147), marker\_radius=0.035); under\_more\_55 — a Point \[gray\] drawn in under\_axes (location=(1.1012658227848098, 0.9601813224101202), marker\_radius=0.035); under\_more\_56 — a Point \[gray\] drawn in under\_axes (location=(1.1772151898734178, 0.6633657900799991), marker\_radius=0.035); under\_more\_57 — a Point \[gray\] drawn in under\_axes (location=(1.253164556962025, 0.877248504757848), marker\_radius=0.035); under\_more\_58 — a Point \[gray\] drawn in under\_axes (location=(1.3291139240506329, 0.8198803234831035), marker\_radius=0.035); under\_more\_59 — a Point \[gray\] drawn in under\_axes (location=(1.4050632911392409, 0.7343666968129939), marker\_radius=0.035); under\_more\_60 — a Point \[gray\] drawn in under\_axes (location=(1.481012658227848, 0.9630870368654897), marker\_radius=0.035); under\_more\_61 — a Point \[gray\] drawn in under\_axes (location=(1.556962025316456, 0.6438645160912888), marker\_radius=0.035); under\_more\_62 — a Point \[gray\] drawn in under\_axes (location=(1.6329113924050631, 0.9932598249323683), marker\_radius=0.035); under\_more\_63 — a Point \[gray\] drawn in under\_axes (location=(1.708860759493671, 0.6737325323347665), marker\_radius=0.035); under\_more\_64 — a Point \[gray\] drawn in under\_axes (location=(1.7848101265822782, 0.887188032931261), marker\_radius=0.035); under\_more\_65 — a Point \[gray\] drawn in under\_axes (location=(1.8607594936708862, 0.7979483885207774), marker\_radius=0.035); under\_more\_66 — a Point \[gray\] drawn in under\_axes (location=(1.9367088607594933, 0.7151344917222289), marker\_radius=0.035); under\_more\_67 — a Point \[gray\] drawn in under\_axes (location=(2.0126582278481013, 0.9154891056722809), marker\_radius=0.035); under\_more\_68 — a Point \[gray\] drawn in under\_axes (location=(2.0886075949367084, 0.5895004924386543), marker\_radius=0.035); under\_more\_69 — a Point \[gray\] drawn in under\_axes (location=(2.1645569620253164, 0.9241399106854595), marker\_radius=0.035); under\_more\_70 — a Point \[gray\] drawn in under\_axes (location=(2.2405063291139244, 0.5827038016786734), marker\_radius=0.035); under\_more\_71 — a Point \[gray\] drawn in under\_axes (location=(2.3164556962025316, 0.7951738059271779), marker\_radius=0.035); under\_more\_72 — a Point \[gray\] drawn in under\_axes (location=(2.3924050632911396, 0.6742692094119506), marker\_radius=0.035); under\_more\_73 — a Point \[gray\] drawn in under\_axes (location=(2.4683544303797467, 0.5943399230113228), marker\_radius=0.035); under\_more\_74 — a Point \[gray\] drawn in under\_axes (location=(2.5443037974683547, 0.7657880091609773), marker\_radius=0.035); under\_more\_75 — a Point \[gray\] drawn in under\_axes (location=(2.620253164556962, 0.43382627044051236), marker\_radius=0.035); under\_more\_76 — a Point \[gray\] drawn in under\_axes (location=(2.6962025316455698, 0.75281680109208), marker\_radius=0.035); under\_more\_77 — a Point \[gray\] drawn in under\_axes (location=(2.772151898734177, 0.3902935761264736), marker\_radius=0.035); under\_more\_78 — a Point \[gray\] drawn in under\_axes (location=(2.848101265822785, 0.6011853280554427), marker\_radius=0.035); under\_more\_79 — a Point \[gray\] drawn in under\_axes (location=(2.924050632911392, 0.44886586730688843), marker\_radius=0.035); under\_more\_80 — a Point \[gray\] drawn in under\_axes (location=(3.0, 0.37196175232051404), marker\_radius=0.035); under\_residuals — a Line \[red\] drawn in under\_axes (start=(-2.4, -1.9567999999999999), end=(-2.4, -1.46), dashed=True); under\_residuals\_2 — a Line \[red\] drawn in under\_axes (start=(-1.2, -0.5192), end=(-1.2, -0.8), dashed=True); under\_residuals\_3 — a Line \[red\] drawn in under\_axes (start=(0.0, 0.4), end=(0.0, -0.14), dashed=True); under\_residuals\_4 — a Line \[red\] drawn in under\_axes (start=(1.2, 0.8008000000000001), end=(1.2, 0.52), dashed=True); under\_residuals\_5 — a Line \[red\] drawn in under\_axes (start=(2.4, 0.6832000000000003), end=(2.4, 1.1800000000000002), dashed=True); over\_truth — a FunctionPlot \[blue\] drawn in over\_axes (function=\, x\_range=(-3.0, 3.0)); over\_twenty — a Point \[yellow\] drawn in over\_axes (location=(3.0, 0.9300000000000002)); over\_twenty\_2 — a Point \[yellow\] drawn in over\_axes (location=(2.959083910208167, -0.048615815163021825)); over\_twenty\_3 — a Point \[yellow\] drawn in over\_axes (location=(2.837451725101904, 1.0113946361949688)); over\_twenty\_4 — a Point \[yellow\] drawn in over\_axes (location=(2.6384212536194673, 0.09810368141154147)); over\_twenty\_5 — a Point \[yellow\] drawn in over\_axes (location=(2.3674215281891806, 1.1932385959200025)); over\_twenty\_6 — a Point \[yellow\] drawn in over\_axes (location=(2.031844714877223, 0.2744038630150619)); over\_twenty\_7 — a Point \[yellow\] drawn in over\_axes (location=(1.6408444743672808, 1.31783775487091)); over\_twenty\_8 — a Point \[yellow\] drawn in over\_axes (location=(1.2050862739589085, 0.3013955236942498)); over\_twenty\_9 — a Point \[yellow\] drawn in over\_axes (location=(0.7364564614223977, 1.2074247922595749)); over\_twenty\_10 — a Point \[yellow\] drawn in over\_axes (location=(0.24773803641699718, 0.025208575785553577)); over\_twenty\_11 — a Point \[yellow\] drawn in over\_axes (location=(-0.24773803641699682, 0.7526967357268568)); over\_twenty\_12 — a Point \[yellow\] drawn in over\_axes (location=(-0.7364564614223967, -0.6026773153050619)); over\_twenty\_13 — a Point \[yellow\] drawn in over\_axes (location=(-1.2050862739589083, -0.02419937766054925)); over\_twenty\_14 — a Point \[yellow\] drawn in over\_axes (location=(-1.640844474367281, -1.4870911669330995)); over\_twenty\_15 — a Point \[yellow\] drawn in over\_axes (location=(-2.0318447148772227, -0.9606253233498832)); over\_twenty\_16 — a Point \[yellow\] drawn in over\_axes (location=(-2.3674215281891806, -2.4109250850880968)); over\_twenty\_17 — a Point \[yellow\] drawn in over\_axes (location=(-2.638421253619467, -1.8041596975698724)); over\_twenty\_18 — a Point \[yellow\] drawn in over\_axes (location=(-2.8374517251019045, -3.1098022614171272)); over\_twenty\_19 — a Point \[yellow\] drawn in over\_axes (location=(-2.959083910208167, -2.303608116392006)); over\_twenty\_20 — a Point \[yellow\] drawn in over\_axes (location=(-3.0, -3.37)); over\_fit — a FunctionPlot \[red\] drawn in over\_axes (function=\, x\_range=(-3.0, 3.0)); train\_readout — a Math \[yellow\] that says "$upright("training MSE") = 0.000$" drawn in over\_axes; test\_marks — a Point \[green\] drawn in over\_axes (location=(-2.65, -2.3215500000000002)); test\_marks\_2 — a Point \[green\] drawn in over\_axes (location=(-2.05, -1.4839499999999997)); test\_marks\_3 — a Point \[green\] drawn in over\_axes (location=(-1.45, -0.7759499999999999)); test\_marks\_4 — a Point \[green\] drawn in over\_axes (location=(-0.85, -0.19755)); test\_marks\_5 — a Point \[green\] drawn in over\_axes (location=(-0.25, 0.25125000000000003)); test\_marks\_6 — a Point \[green\] drawn in over\_axes (location=(0.35, 0.57045)); test\_marks\_7 — a Point \[green\] drawn in over\_axes (location=(0.95, 0.76005)); test\_marks\_8 — a Point \[green\] drawn in over\_axes (location=(1.55, 0.8200500000000002)); test\_marks\_9 — a Point \[green\] drawn in over\_axes (location=(2.15, 0.7504500000000002)); test\_marks\_10 — a Point \[green\] drawn in over\_axes (location=(2.65, 0.59345)); test\_residuals — a Line \[magenta\] drawn in over\_axes (start=(-2.65, -2.3215500000000002), end=(-2.65, -1.82756352737211), dashed=True); test\_residuals\_2 — a Line \[magenta\] drawn in over\_axes (start=(-2.05, -1.4839499999999997), end=(-2.05, -0.9900905529924808), dashed=True); test\_residuals\_3 — a Line \[magenta\] drawn in over\_axes (start=(-1.45, -0.7759499999999999), end=(-1.45, -0.8555749140628064), dashed=True); test\_residuals\_4 — a Line \[magenta\] drawn in over\_axes (start=(-0.85, -0.19755), end=(-0.85, -0.5648549122571341), dashed=True); test\_residuals\_5 — a Line \[magenta\] drawn in over\_axes (start=(-0.25, 0.25125000000000003), end=(-0.25, 0.7511983384419643), dashed=True); test\_residuals\_6 — a Line \[magenta\] drawn in over\_axes (start=(0.35, 0.57045), end=(0.35, 0.1726896666190087), dashed=True); test\_residuals\_7 — a Line \[magenta\] drawn in over\_axes (start=(0.95, 0.76005), end=(0.95, 0.8402687600971048), dashed=True); test\_residuals\_8 — a Line \[magenta\] drawn in over\_axes (start=(1.55, 0.8200500000000002), end=(1.55, 1.2090045306849817), dashed=True); test\_residuals\_9 — a Line \[magenta\] drawn in over\_axes (start=(2.15, 0.7504500000000002), end=(2.15, 0.4990845185639889), dashed=True); test\_residuals\_10 — a Line \[magenta\] drawn in over\_axes (start=(2.65, 0.59345), end=(2.65, 0.09946352737211012), dashed=True); test\_readout — a Math \[green\] that says "$upright("test MSE") \> 0$" drawn in over\_axes Actions: - [04:25.89](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=265.88966666666664): train\_readout is indicated — a transient flash. - [04:29.93](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=269.92966666666666): test\_readout is indicated — a transient flash. ##### [04:32.631](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=272.6306666666666) Narration: The straight line and the degree-nineteen curve fail for opposite reasons. The line cannot express enough structure. The interpolating curve reacts to structure that was never really present. Board: Unchanged from the preceding beat in this scene. Actions: - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): heading is hidden from the screen — left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): left\_label is hidden from the screen — left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): over\_axes is hidden from the screen — left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): over\_truth is hidden from the screen — over\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): over\_twenty is hidden from the screen — over\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): over\_twenty\_2 is hidden from the screen — over\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): over\_twenty\_3 is hidden from the screen — over\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): over\_twenty\_4 is hidden from the screen — over\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): over\_twenty\_5 is hidden from the screen — over\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): over\_twenty\_6 is hidden from the screen — over\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): over\_twenty\_7 is hidden from the screen — over\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): over\_twenty\_8 is hidden from the screen — over\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): over\_twenty\_9 is hidden from the screen — over\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): over\_twenty\_10 is hidden from the screen — over\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): over\_twenty\_11 is hidden from the screen — over\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): over\_twenty\_12 is hidden from the screen — over\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): over\_twenty\_13 is hidden from the screen — over\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): over\_twenty\_14 is hidden from the screen — over\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): over\_twenty\_15 is hidden from the screen — over\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): over\_twenty\_16 is hidden from the screen — over\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): over\_twenty\_17 is hidden from the screen — over\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): over\_twenty\_18 is hidden from the screen — over\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): over\_twenty\_19 is hidden from the screen — over\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): over\_twenty\_20 is hidden from the screen — over\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): over\_fit is hidden from the screen — over\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): train\_readout is hidden from the screen — over\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): test\_marks is hidden from the screen — over\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): test\_marks\_2 is hidden from the screen — over\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): test\_marks\_3 is hidden from the screen — over\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): test\_marks\_4 is hidden from the screen — over\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): test\_marks\_5 is hidden from the screen — over\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): test\_marks\_6 is hidden from the screen — over\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): test\_marks\_7 is hidden from the screen — over\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): test\_marks\_8 is hidden from the screen — over\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): test\_marks\_9 is hidden from the screen — over\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): test\_marks\_10 is hidden from the screen — over\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): test\_residuals is hidden from the screen — over\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): test\_residuals\_2 is hidden from the screen — over\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): test\_residuals\_3 is hidden from the screen — over\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): test\_residuals\_4 is hidden from the screen — over\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): test\_residuals\_5 is hidden from the screen — over\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): test\_residuals\_6 is hidden from the screen — over\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): test\_residuals\_7 is hidden from the screen — over\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): test\_residuals\_8 is hidden from the screen — over\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): test\_residuals\_9 is hidden from the screen — over\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): test\_residuals\_10 is hidden from the screen — over\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): test\_readout is hidden from the screen — over\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): right\_label is hidden from the screen — left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_axes is hidden from the screen — left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_truth is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_twenty is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_twenty\_2 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_twenty\_3 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_twenty\_4 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_twenty\_5 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_twenty\_6 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_twenty\_7 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_twenty\_8 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_twenty\_9 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_twenty\_10 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_twenty\_11 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_twenty\_12 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_twenty\_13 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_twenty\_14 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_twenty\_15 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_twenty\_16 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_twenty\_17 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_twenty\_18 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_twenty\_19 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_twenty\_20 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_line is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_more is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_more\_2 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_more\_3 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_more\_4 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_more\_5 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_more\_6 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_more\_7 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_more\_8 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_more\_9 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_more\_10 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_more\_11 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_more\_12 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_more\_13 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_more\_14 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_more\_15 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_more\_16 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_more\_17 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_more\_18 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_more\_19 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_more\_20 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_more\_21 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_more\_22 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_more\_23 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_more\_24 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_more\_25 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_more\_26 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_more\_27 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_more\_28 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_more\_29 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_more\_30 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_more\_31 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_more\_32 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_more\_33 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_more\_34 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_more\_35 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_more\_36 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_more\_37 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_more\_38 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_more\_39 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_more\_40 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_more\_41 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_more\_42 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_more\_43 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_more\_44 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_more\_45 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_more\_46 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_more\_47 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_more\_48 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_more\_49 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_more\_50 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_more\_51 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_more\_52 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_more\_53 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_more\_54 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_more\_55 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_more\_56 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_more\_57 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_more\_58 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_more\_59 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_more\_60 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_more\_61 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_more\_62 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_more\_63 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_more\_64 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_more\_65 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_more\_66 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_more\_67 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_more\_68 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_more\_69 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_more\_70 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_more\_71 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_more\_72 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_more\_73 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_more\_74 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_more\_75 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_more\_76 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_more\_77 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_more\_78 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_more\_79 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_more\_80 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_residuals is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_residuals\_2 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_residuals\_3 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_residuals\_4 is hidden from the screen — under\_axes left the board. - [04:45.379](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=285.3786666666666): under\_residuals\_5 is hidden from the screen — under\_axes left the board. ##### [04:46.579](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=286.57866666666666) Narration: To compare models with one number, use mean squared error. For each observation, subtract prediction from observation, square that residual, add the squares, and divide by the number of observations. Board: Empty. Actions: - [04:49.586](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=289.5856666666666): mse\_train is shown on the screen, written out. - [04:55.193](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=295.19266666666664): mse\_train (the "(y\_i - hat(y)\_i)^2" part) is emphasized. - [04:58.212](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=298.2116666666666): mse\_train (the "(y\_i - hat(y)\_i)^2" part) is no longer emphasized. ##### [05:1.552](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=301.5516666666666) Narration: Training mean squared error uses the observations that selected the fitted curve. Test mean squared error repeats the same calculation on observations held aside from that selection. Board: mse\_train — a Math \[text\] that says "$upright("MSE")\_(upright("train")) = frac(1, n) sum\_i (y\_i - hat(y)\_i)^2$" Actions: - [05:7.392](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=307.39166666666665): mse\_test is shown on the screen, written out. - [05:11.919](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=311.9186666666666): definitions is shown on the screen, written out. ##### [05:14.992](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=314.99216666666666) Narration: The formulas look nearly identical. Their difference is experimental: one asks how well the model remembers its lesson, and the other asks how well it handles a new question. Board: mse\_train — a Math \[text\] that says "$upright("MSE")\_(upright("train")) = frac(1, n) sum\_i (y\_i - hat(y)\_i)^2$"; mse\_test — a Math \[text\] that says "$upright("MSE")\_(upright("test")) = frac(1, m) sum\_j (y\_j - hat(y)\_j)^2$"; definitions — a Panel that says "Training error measures predictions on observations used to fit the model. Test error measures predictions on separate observations that played no part in fitting it." Actions: - [05:20.066](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=320.06566666666663): mse\_train (the "upright("train")" part) is emphasized. - [05:23.375](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=323.3746666666666): mse\_test (the "upright("test")" part) is emphasized. - [05:23.375](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=323.3746666666666): mse\_train (the "upright("train")" part) is no longer emphasized. - [05:26.078](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=326.0777916666666): mse\_test (the "upright("test")" part) is no longer emphasized. - [05:26.328](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=326.3277916666666): definitions is hidden from the screen — left the board. - [05:26.328](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=326.3277916666666): mse\_test is hidden from the screen — left the board. - [05:26.328](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=326.3277916666666): mse\_train is hidden from the screen — left the board. ### Scene 3: [Error Against Complexity](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=327.3694583333333) Span: 05:27.369–07:41.303 (327.3694583333333s–461.30274999999995s). #### Objects - axes: an Axes (x\_range=(1.0, 19.0), y\_range=(0.0, 4.0), x\_ticks\_every=2.0) - choice: a Math \[text\] that says "$d^star = arg min\_d E\_(upright("test"))(d)$" - degree: a VariableNumber (initial\_value=1.0, format\_spec='.0f') - heading: a Heading that says "Two Errors, Two Different Curves" - minimum: a PlotPoint \[green\] labelled "upright("lowest test error")" drawn in axes (target='test\_curve', x=5.0) - point: a Point \[yellow\] drawn in axes (location=(1.0, 1.17)) - point\_2: a Point \[yellow\] drawn in axes (location=(19.0, 3.5320000000000005)) - test\_curve: a FunctionPlot \[red\] labelled "E\_(upright("test"))" drawn in axes (function=\, x\_range=(1.0, 19.0)) - test\_tracker: a PlotPoint \[red\] drawn in axes (target='test\_curve', x=\) - train\_curve: a FunctionPlot \[blue\] labelled "E\_(upright("train"))" drawn in axes (function=\, x\_range=(1.0, 19.0)) - train\_tracker: a PlotPoint \[blue\] drawn in axes (target='train\_curve', x=\) #### Beats ##### [05:27.369](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=327.3694583333333) Narration: Now repeat the fitting experiment for degree one, degree two, degree three, and so on up to degree nineteen. For every fitted model, record both errors. Board: Empty. Actions: - [05:27.369](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=327.3694583333333): heading is shown on the screen, written out. - [05:27.369](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=327.3694583333333): axes is shown on the screen, written out. ##### [05:38.662](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=338.6619583333333) Narration: The blue curve is training error. It falls as degree increases because each larger polynomial family contains more ways to accommodate the observations. Board: axes — an Axes (x\_range=(1.0, 19.0), y\_range=(0.0, 4.0), x\_ticks\_every=2.0); heading — a Heading that says "Two Errors, Two Different Curves" Actions: - [05:39.173](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=339.17345833333326): train\_curve is shown on the screen, drawn. - [05:41.135](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=341.1354583333333): train\_curve is indicated — a transient flash. ##### [05:48.55](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=348.55045833333327) Narration: That decrease is built into the fitting process. A more flexible model can imitate a simpler one if that is best, or use its extra freedom to reduce the residuals further. Board: axes — an Axes (x\_range=(1.0, 19.0), y\_range=(0.0, 4.0), x\_ticks\_every=2.0); heading — a Heading that says "Two Errors, Two Different Curves"; train\_curve — a FunctionPlot \[blue\] labelled "E\_(upright("train"))" drawn in axes (function=\, x\_range=(1.0, 19.0)) Actions: - [05:50.315](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=350.3154583333333): train\_curve is indicated — a transient flash. ##### [05:59.843](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=359.8434583333333) Narration: The red curve is test error. At first it also falls. Moving beyond a straight line lets the model capture the real curvature, so predictions improve. Board: Unchanged from the preceding beat in this scene. Actions: - [06:0.528](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=360.5284583333333): test\_curve is shown on the screen, drawn. - [06:2.919](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=362.9194583333333): train\_tracker is shown on the screen, written out. - [06:2.919](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=362.9194583333333): test\_tracker is shown on the screen, written out. - [06:9.595](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=369.5954583333333): train\_tracker is redrawn as the numbers it depends on change. - [06:9.595](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=369.5954583333333): test\_tracker is redrawn as the numbers it depends on change. - [06:9.595](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=369.5954583333333): degree ticks to 5.0. ##### [06:11.228](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=371.22845833333326) Narration: Near degree five, test error reaches its minimum. In this experiment that is the useful region: enough flexibility to represent the signal, without enough freedom to chase every disturbance. Board: axes — an Axes (x\_range=(1.0, 19.0), y\_range=(0.0, 4.0), x\_ticks\_every=2.0); heading — a Heading that says "Two Errors, Two Different Curves"; train\_curve — a FunctionPlot \[blue\] labelled "E\_(upright("train"))" drawn in axes (function=\, x\_range=(1.0, 19.0)); test\_curve — a FunctionPlot \[red\] labelled "E\_(upright("test"))" drawn in axes (function=\, x\_range=(1.0, 19.0)); train\_tracker — a PlotPoint \[blue\] drawn in axes (target='train\_curve', x=\); test\_tracker — a PlotPoint \[red\] drawn in axes (target='test\_curve', x=\) Actions: - [06:14.177](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=374.1774583333333): minimum is shown on the screen, written out. - [06:16.964](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=376.96445833333325): choice is shown on the screen, written out. ##### [06:23.88](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=383.87995833333326) Narration: Past that point, training error keeps falling but test error turns upward. The two measurements separate because extra complexity is helping on observations already seen while hurting predictions elsewhere. Board: axes — an Axes (x\_range=(1.0, 19.0), y\_range=(0.0, 4.0), x\_ticks\_every=2.0); choice — a Math \[text\] that says "$d^star = arg min\_d E\_(upright("test"))(d)$"; heading — a Heading that says "Two Errors, Two Different Curves"; train\_curve — a FunctionPlot \[blue\] labelled "E\_(upright("train"))" drawn in axes (function=\, x\_range=(1.0, 19.0)); test\_curve — a FunctionPlot \[red\] labelled "E\_(upright("test"))" drawn in axes (function=\, x\_range=(1.0, 19.0)); train\_tracker — a PlotPoint \[blue\] drawn in axes (target='train\_curve', x=\); test\_tracker — a PlotPoint \[red\] drawn in axes (target='test\_curve', x=\); minimum — a PlotPoint \[green\] labelled "upright("lowest test error")" drawn in axes (target='test\_curve', x=5.0) Actions: - [06:24.228](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=384.22845833333326): train\_tracker is redrawn as the numbers it depends on change. - [06:24.228](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=384.22845833333326): test\_tracker is redrawn as the numbers it depends on change. - [06:24.228](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=384.22845833333326): degree ticks to 12.0. - [06:25.307](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=385.3074583333333): train\_tracker is indicated — a transient flash. - [06:26.886](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=386.8864583333333): test\_tracker is indicated — a transient flash. ##### [06:37.181](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=397.18095833333325) Narration: At degree nineteen the training point is almost on the horizontal axis. The test point is high. Their vertical separation is the observable signature of the interpolation failure we just watched. Board: Unchanged from the preceding beat in this scene. Actions: - [06:38.121](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=398.1214583333333): train\_tracker is redrawn as the numbers it depends on change. - [06:38.121](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=398.1214583333333): test\_tracker is redrawn as the numbers it depends on change. - [06:38.121](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=398.1214583333333): degree ticks to 19.0. - [06:38.666](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=398.66645833333325): train\_tracker is indicated — a transient flash. - [06:42.022](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=402.02245833333325): test\_tracker is indicated — a transient flash. ##### [06:49.437](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=409.4369583333333) Narration: Low complexity and high complexity therefore have different diagnoses. On the left, both errors are large because the model misses the signal. On the right, only training error is tiny, because the model has learned details that do not transfer. Board: Unchanged from the preceding beat in this scene. Actions: - [06:54.429](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=414.4294583333333): point is shown on the screen, grown. - [06:56.429](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=416.4294583333333): point is hidden from the screen. - [06:59.003](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=419.00345833333324): point\_2 is shown on the screen, grown. - [07:1.003](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=421.00345833333324): point\_2 is hidden from the screen. ##### [07:5.722](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=425.7224583333333) Narration: Selecting the degree with the smallest training error would always reward the most flexible candidate. That is exactly the decision rule that invites overfitting. Board: Unchanged from the preceding beat in this scene. Actions: - [07:7.846](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=427.8464583333333): train\_curve is indicated — a transient flash. ##### [07:16.086](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=436.08645833333327) Narration: Instead, reserve data for validation or testing and look for low error there. The marked point is not a law that degree five is always right. It is the answer for this relationship, this noise level, and this amount of data. Board: Unchanged from the preceding beat in this scene. Actions: - [07:20.346](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=440.3464583333333): A box is drawn around choice. - [07:22.192](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=442.1924583333333): minimum is indicated — a transient flash. ##### [07:30.815](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=450.81495833333327) Narration: Change the noise, the sampling locations, or the amount of data, and the red curve can change. That dependence is the doorway to bias and variance. Board: Unchanged from the preceding beat in this scene. Actions: - [07:40.261](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=460.2610833333333): axes is hidden from the screen — left the board. - [07:40.261](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=460.2610833333333): train\_curve is hidden from the screen — axes left the board. - [07:40.261](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=460.2610833333333): test\_curve is hidden from the screen — axes left the board. - [07:40.261](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=460.2610833333333): train\_tracker is hidden from the screen — axes left the board. - [07:40.261](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=460.2610833333333): test\_tracker is hidden from the screen — axes left the board. - [07:40.261](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=460.2610833333333): minimum is hidden from the screen — axes left the board. - [07:40.261](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=460.2610833333333): choice is hidden from the screen — left the board. - [07:40.261](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=460.2610833333333): heading is hidden from the screen — left the board. ### Scene 4: [Bias and Variance](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=461.30274999999995) Span: 07:41.303–10:22.61 (461.30274999999995s–622.6099166666665s). #### Objects - bias\_equation: a Math \[text\] that says "$upright("bias")(x) = upright("average fitted prediction")(x) - f(x)$" - definition: a Panel that says "Bias is systematic error that remains after averaging many refits. Variance is sensitivity to which particular sample happened to be observed." - heading: a Heading that says "Refit on Fresh Samples" - heading\_2: a Heading that says "Two Questions About Repetition" - high\_average: a FunctionPlot \[cyan\] labelled "upright("average fit")" drawn in high\_axes (function=\, x\_range=(-3.0, 3.0)) - high\_axes: an Axes (x\_range=(-3.0, 3.0), y\_range=(-3.6, 1.8), x\_ticks\_every=1.0) - high\_fits: a FunctionPlot \[red\] drawn in high\_axes (function=\, x\_range=(-3.0, 3.0)) - high\_fits\_2: a FunctionPlot \[green\] drawn in high\_axes (function=\, x\_range=(-3.0, 3.0)) - high\_fits\_3: a FunctionPlot \[yellow\] drawn in high\_axes (function=\, x\_range=(-3.0, 3.0)) - high\_fits\_4: a FunctionPlot \[magenta\] drawn in high\_axes (function=\, x\_range=(-3.0, 3.0)) - high\_fits\_5: a FunctionPlot \[gray\] drawn in high\_axes (function=\, x\_range=(-3.0, 3.0)) - high\_truth: a FunctionPlot \[blue\] drawn in high\_axes (function=\, x\_range=(-3.0, 3.0)) - left\_label: a Tex \[text\] that says "Low flexibility" - low\_average: a FunctionPlot \[red\] labelled "upright("average fit")" drawn in low\_axes (function=\, x\_range=(-3.0, 3.0)) - low\_axes: an Axes (x\_range=(-3.0, 3.0), y\_range=(-3.6, 1.8), x\_ticks\_every=1.0) - low\_fits: a FunctionPlot \[gray\] drawn in low\_axes (function=\, x\_range=(-3.0, 3.0)) - low\_fits\_2: a FunctionPlot \[gray\] drawn in low\_axes (function=\, x\_range=(-3.0, 3.0)) - low\_fits\_3: a FunctionPlot \[gray\] drawn in low\_axes (function=\, x\_range=(-3.0, 3.0)) - low\_fits\_4: a FunctionPlot \[gray\] drawn in low\_axes (function=\, x\_range=(-3.0, 3.0)) - low\_fits\_5: a FunctionPlot \[gray\] drawn in low\_axes (function=\, x\_range=(-3.0, 3.0)) - low\_truth: a FunctionPlot \[blue\] drawn in low\_axes (function=\, x\_range=(-3.0, 3.0)) - point: a Point \[yellow\] drawn in high\_axes (location=(1.6, 1.1765380974851398)) - point\_2: a Point \[yellow\] drawn in high\_axes (location=(1.6, 0.4618619025148607)) - right\_label: a Tex \[text\] that says "High flexibility" - variance\_equation: a Math \[text\] that says "$upright("variance")(x) = upright("spread of fitted predictions at") thin x$" #### Beats ##### [07:41.303](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=461.30274999999995) Narration: Imagine repeating the entire study. Draw twenty fresh noisy observations from the same relationship, fit the model, discard that sample, and do it again. Board: Empty. Actions: - [07:41.303](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=461.30274999999995): heading is shown on the screen, written out. - [07:41.303](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=461.30274999999995): left\_label is shown on the screen, written out. - [07:41.303](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=461.30274999999995): low\_axes is shown on the screen, written out. - [07:46.667](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=466.6667499999999): low\_truth is shown on the screen, written out. ##### [07:51.62](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=471.62024999999994) Narration: On the left, each gray line is a straight model fitted to a different fresh sample. Here is the first fit, then the second, the third, the fourth, and the fifth. Board: left\_label — a Tex \[text\] that says "Low flexibility"; low\_axes — an Axes (x\_range=(-3.0, 3.0), y\_range=(-3.6, 1.8), x\_ticks\_every=1.0); heading — a Heading that says "Refit on Fresh Samples"; low\_truth — a FunctionPlot \[blue\] drawn in low\_axes (function=\, x\_range=(-3.0, 3.0)) Actions: - [07:57.867](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=477.86674999999997): low\_fits is shown on the screen, written out. - [07:59.167](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=479.1667499999999): low\_fits\_2 is shown on the screen, written out. - [07:59.864](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=479.8637499999999): low\_fits\_3 is shown on the screen, written out. - [08:0.677](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=480.67674999999997): low\_fits\_4 is shown on the screen, written out. - [08:1.548](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=481.54774999999995): low\_fits\_5 is shown on the screen, written out. ##### [08:3.018](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=483.01824999999997) Narration: The fitted lines move a little because the observations change, but they remain a tight family. Their small spread means low variance. Board: left\_label — a Tex \[text\] that says "Low flexibility"; low\_axes — an Axes (x\_range=(-3.0, 3.0), y\_range=(-3.6, 1.8), x\_ticks\_every=1.0); heading — a Heading that says "Refit on Fresh Samples"; low\_truth — a FunctionPlot \[blue\] drawn in low\_axes (function=\, x\_range=(-3.0, 3.0)); low\_fits — a FunctionPlot \[gray\] drawn in low\_axes (function=\, x\_range=(-3.0, 3.0)); low\_fits\_2 — a FunctionPlot \[gray\] drawn in low\_axes (function=\, x\_range=(-3.0, 3.0)); low\_fits\_3 — a FunctionPlot \[gray\] drawn in low\_axes (function=\, x\_range=(-3.0, 3.0)); low\_fits\_4 — a FunctionPlot \[gray\] drawn in low\_axes (function=\, x\_range=(-3.0, 3.0)); low\_fits\_5 — a FunctionPlot \[gray\] drawn in low\_axes (function=\, x\_range=(-3.0, 3.0)) Actions: - [08:7.5](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=487.49974999999995): low\_fits is indicated — a transient flash. - [08:7.5](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=487.49974999999995): low\_fits\_2 is indicated — a transient flash. - [08:7.5](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=487.49974999999995): low\_fits\_3 is indicated — a transient flash. - [08:7.5](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=487.49974999999995): low\_fits\_4 is indicated — a transient flash. - [08:7.5](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=487.49974999999995): low\_fits\_5 is indicated — a transient flash. ##### [08:12.453](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=492.45324999999997) Narration: Now average the five fitted predictions at every x. The red line is that average. It still misses the blue curve in the same organized way, especially near the ends. Board: Unchanged from the preceding beat in this scene. Actions: - [08:16.506](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=496.5057499999999): low\_average is shown on the screen, drawn. - [08:17.376](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=497.3757499999999): low\_average is indicated — a transient flash. - [08:19.048](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=499.04774999999995): low\_truth is indicated — a transient flash. ##### [08:23.572](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=503.57224999999994) Narration: That persistent difference is bias. Even if we repeated the study many more times, averaging straight lines would still produce a straight line. Repetition cannot create curvature that the model family forbids. Board: left\_label — a Tex \[text\] that says "Low flexibility"; low\_axes — an Axes (x\_range=(-3.0, 3.0), y\_range=(-3.6, 1.8), x\_ticks\_every=1.0); heading — a Heading that says "Refit on Fresh Samples"; low\_truth — a FunctionPlot \[blue\] drawn in low\_axes (function=\, x\_range=(-3.0, 3.0)); low\_fits — a FunctionPlot \[gray\] drawn in low\_axes (function=\, x\_range=(-3.0, 3.0)); low\_fits\_2 — a FunctionPlot \[gray\] drawn in low\_axes (function=\, x\_range=(-3.0, 3.0)); low\_fits\_3 — a FunctionPlot \[gray\] drawn in low\_axes (function=\, x\_range=(-3.0, 3.0)); low\_fits\_4 — a FunctionPlot \[gray\] drawn in low\_axes (function=\, x\_range=(-3.0, 3.0)); low\_fits\_5 — a FunctionPlot \[gray\] drawn in low\_axes (function=\, x\_range=(-3.0, 3.0)); low\_average — a FunctionPlot \[red\] labelled "upright("average fit")" drawn in low\_axes (function=\, x\_range=(-3.0, 3.0)) Actions: - [08:24.826](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=504.82574999999997): The segment (-2.4, -1.46) to (-2.4, -1.9567999999999999) in low\_axes is lit up. - [08:37.493](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=517.4927499999999): low\_axes: retire a lit segment (unemphasize\_line). ##### [08:38.093](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=518.0927499999999) Narration: Now repeat the same experiment with a highly flexible polynomial. The blue curve on the right is still the same relationship. Board: Unchanged from the preceding beat in this scene. Actions: - [08:38.093](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=518.0927499999999): high\_axes is shown on the screen, written out. - [08:40.833](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=520.8327499999999): right\_label is shown on the screen, written out. - [08:42.934](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=522.9337499999999): high\_truth is shown on the screen, written out. ##### [08:46.657](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=526.65725) Narration: Each colored curve is a refit on a new sample of twenty observations. One sample pulls the curve upward here, another pulls it downward, and another creates a different sequence of bends. Board: left\_label — a Tex \[text\] that says "Low flexibility"; low\_axes — an Axes (x\_range=(-3.0, 3.0), y\_range=(-3.6, 1.8), x\_ticks\_every=1.0); right\_label — a Tex \[text\] that says "High flexibility"; high\_axes — an Axes (x\_range=(-3.0, 3.0), y\_range=(-3.6, 1.8), x\_ticks\_every=1.0); heading — a Heading that says "Refit on Fresh Samples"; low\_truth — a FunctionPlot \[blue\] drawn in low\_axes (function=\, x\_range=(-3.0, 3.0)); low\_fits — a FunctionPlot \[gray\] drawn in low\_axes (function=\, x\_range=(-3.0, 3.0)); low\_fits\_2 — a FunctionPlot \[gray\] drawn in low\_axes (function=\, x\_range=(-3.0, 3.0)); low\_fits\_3 — a FunctionPlot \[gray\] drawn in low\_axes (function=\, x\_range=(-3.0, 3.0)); low\_fits\_4 — a FunctionPlot \[gray\] drawn in low\_axes (function=\, x\_range=(-3.0, 3.0)); low\_fits\_5 — a FunctionPlot \[gray\] drawn in low\_axes (function=\, x\_range=(-3.0, 3.0)); low\_average — a FunctionPlot \[red\] labelled "upright("average fit")" drawn in low\_axes (function=\, x\_range=(-3.0, 3.0)); high\_truth — a FunctionPlot \[blue\] drawn in high\_axes (function=\, x\_range=(-3.0, 3.0)) Actions: - [08:47.273](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=527.27275): high\_fits is shown on the screen, written out. - [08:54.041](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=534.0407499999999): high\_fits\_2 is shown on the screen, written out. - [08:55.818](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=535.8177499999999): high\_fits\_3 is shown on the screen, written out. - [08:56.735](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=536.73475): high\_fits\_4 is shown on the screen, written out. - [08:57.675](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=537.6747499999999): high\_fits\_5 is shown on the screen, written out. ##### [08:59.309](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=539.3087499999999) Narration: The model family is capable of following the underlying curve, so its average can be close to the truth. But the individual fits spread widely. Which data set happened to arrive has a large effect on the prediction. Board: left\_label — a Tex \[text\] that says "Low flexibility"; low\_axes — an Axes (x\_range=(-3.0, 3.0), y\_range=(-3.6, 1.8), x\_ticks\_every=1.0); right\_label — a Tex \[text\] that says "High flexibility"; high\_axes — an Axes (x\_range=(-3.0, 3.0), y\_range=(-3.6, 1.8), x\_ticks\_every=1.0); heading — a Heading that says "Refit on Fresh Samples"; low\_truth — a FunctionPlot \[blue\] drawn in low\_axes (function=\, x\_range=(-3.0, 3.0)); low\_fits — a FunctionPlot \[gray\] drawn in low\_axes (function=\, x\_range=(-3.0, 3.0)); low\_fits\_2 — a FunctionPlot \[gray\] drawn in low\_axes (function=\, x\_range=(-3.0, 3.0)); low\_fits\_3 — a FunctionPlot \[gray\] drawn in low\_axes (function=\, x\_range=(-3.0, 3.0)); low\_fits\_4 — a FunctionPlot \[gray\] drawn in low\_axes (function=\, x\_range=(-3.0, 3.0)); low\_fits\_5 — a FunctionPlot \[gray\] drawn in low\_axes (function=\, x\_range=(-3.0, 3.0)); low\_average — a FunctionPlot \[red\] labelled "upright("average fit")" drawn in low\_axes (function=\, x\_range=(-3.0, 3.0)); high\_truth — a FunctionPlot \[blue\] drawn in high\_axes (function=\, x\_range=(-3.0, 3.0)); high\_fits — a FunctionPlot \[red\] drawn in high\_axes (function=\, x\_range=(-3.0, 3.0)); high\_fits\_2 — a FunctionPlot \[green\] drawn in high\_axes (function=\, x\_range=(-3.0, 3.0)); high\_fits\_3 — a FunctionPlot \[yellow\] drawn in high\_axes (function=\, x\_range=(-3.0, 3.0)); high\_fits\_4 — a FunctionPlot \[magenta\] drawn in high\_axes (function=\, x\_range=(-3.0, 3.0)); high\_fits\_5 — a FunctionPlot \[gray\] drawn in high\_axes (function=\, x\_range=(-3.0, 3.0)) Actions: - [09:3.662](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=543.66175): high\_average is shown on the screen, drawn. - [09:4.765](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=544.7647499999999): high\_truth is indicated — a transient flash. - [09:7.203](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=547.2027499999999): high\_fits is indicated — a transient flash. - [09:7.203](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=547.2027499999999): high\_fits\_2 is indicated — a transient flash. - [09:7.203](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=547.2027499999999): high\_fits\_3 is indicated — a transient flash. - [09:7.203](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=547.2027499999999): high\_fits\_4 is indicated — a transient flash. ##### [09:13.875](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=553.87475) Narration: That sensitivity is variance. High variance means that if we repeated the data collection, our fitted answer could change substantially even though the data-generating process did not. Board: left\_label — a Tex \[text\] that says "Low flexibility"; low\_axes — an Axes (x\_range=(-3.0, 3.0), y\_range=(-3.6, 1.8), x\_ticks\_every=1.0); right\_label — a Tex \[text\] that says "High flexibility"; high\_axes — an Axes (x\_range=(-3.0, 3.0), y\_range=(-3.6, 1.8), x\_ticks\_every=1.0); heading — a Heading that says "Refit on Fresh Samples"; low\_truth — a FunctionPlot \[blue\] drawn in low\_axes (function=\, x\_range=(-3.0, 3.0)); low\_fits — a FunctionPlot \[gray\] drawn in low\_axes (function=\, x\_range=(-3.0, 3.0)); low\_fits\_2 — a FunctionPlot \[gray\] drawn in low\_axes (function=\, x\_range=(-3.0, 3.0)); low\_fits\_3 — a FunctionPlot \[gray\] drawn in low\_axes (function=\, x\_range=(-3.0, 3.0)); low\_fits\_4 — a FunctionPlot \[gray\] drawn in low\_axes (function=\, x\_range=(-3.0, 3.0)); low\_fits\_5 — a FunctionPlot \[gray\] drawn in low\_axes (function=\, x\_range=(-3.0, 3.0)); low\_average — a FunctionPlot \[red\] labelled "upright("average fit")" drawn in low\_axes (function=\, x\_range=(-3.0, 3.0)); high\_truth — a FunctionPlot \[blue\] drawn in high\_axes (function=\, x\_range=(-3.0, 3.0)); high\_fits — a FunctionPlot \[red\] drawn in high\_axes (function=\, x\_range=(-3.0, 3.0)); high\_fits\_2 — a FunctionPlot \[green\] drawn in high\_axes (function=\, x\_range=(-3.0, 3.0)); high\_fits\_3 — a FunctionPlot \[yellow\] drawn in high\_axes (function=\, x\_range=(-3.0, 3.0)); high\_fits\_4 — a FunctionPlot \[magenta\] drawn in high\_axes (function=\, x\_range=(-3.0, 3.0)); high\_fits\_5 — a FunctionPlot \[gray\] drawn in high\_axes (function=\, x\_range=(-3.0, 3.0)); high\_average — a FunctionPlot \[cyan\] labelled "upright("average fit")" drawn in high\_axes (function=\, x\_range=(-3.0, 3.0)) Actions: - [09:21.108](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=561.1077499999999): point is shown on the screen, grown. - [09:21.422](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=561.42175): point\_2 is shown on the screen, grown. - [09:23.108](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=563.1077499999999): point is hidden from the screen. - [09:23.422](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=563.42175): point\_2 is hidden from the screen. ##### [09:25.783](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=565.78275) Narration: Bias and variance are therefore not labels attached permanently to one curve. They describe behavior across repeated samples. Bias asks where the average fit lands. Variance asks how widely the individual fits scatter around that average. Board: Unchanged from the preceding beat in this scene. Actions: - [09:42.919](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=582.9192499999999): heading is hidden from the screen — left the board. - [09:42.919](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=582.9192499999999): high\_axes is hidden from the screen — left the board. - [09:42.919](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=582.9192499999999): high\_truth is hidden from the screen — high\_axes left the board. - [09:42.919](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=582.9192499999999): high\_fits is hidden from the screen — high\_axes left the board. - [09:42.919](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=582.9192499999999): high\_fits\_2 is hidden from the screen — high\_axes left the board. - [09:42.919](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=582.9192499999999): high\_fits\_3 is hidden from the screen — high\_axes left the board. - [09:42.919](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=582.9192499999999): high\_fits\_4 is hidden from the screen — high\_axes left the board. - [09:42.919](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=582.9192499999999): high\_fits\_5 is hidden from the screen — high\_axes left the board. - [09:42.919](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=582.9192499999999): high\_average is hidden from the screen — high\_axes left the board. - [09:42.919](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=582.9192499999999): left\_label is hidden from the screen — left the board. - [09:42.919](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=582.9192499999999): low\_axes is hidden from the screen — left the board. - [09:42.919](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=582.9192499999999): low\_truth is hidden from the screen — low\_axes left the board. - [09:42.919](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=582.9192499999999): low\_fits is hidden from the screen — low\_axes left the board. - [09:42.919](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=582.9192499999999): low\_fits\_2 is hidden from the screen — low\_axes left the board. - [09:42.919](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=582.9192499999999): low\_fits\_3 is hidden from the screen — low\_axes left the board. - [09:42.919](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=582.9192499999999): low\_fits\_4 is hidden from the screen — low\_axes left the board. - [09:42.919](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=582.9192499999999): low\_fits\_5 is hidden from the screen — low\_axes left the board. - [09:42.919](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=582.9192499999999): low\_average is hidden from the screen — low\_axes left the board. - [09:42.919](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=582.9192499999999): right\_label is hidden from the screen — left the board. ##### [09:44.119](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=584.11925) Narration: Bias is the difference between the average fitted prediction and the true relationship. A rigid straight model has substantial bias here because its average prediction cannot reproduce the bend. Board: Empty. Actions: - [09:44.677](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=584.67675): bias\_equation is shown on the screen, written out. - [09:46.511](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=586.5107499999999): bias\_equation (the "upright("average fitted prediction")(x)" part) is emphasized. - [09:48.264](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=588.26375): bias\_equation (the "f(x)" part) is emphasized. - [09:48.264](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=588.26375): bias\_equation (the "upright("average fitted prediction")(x)" part) is no longer emphasized. - [09:56.774](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=596.7742499999999): bias\_equation (the "f(x)" part) is no longer emphasized. ##### [09:57.374](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=597.37425) Narration: Variance is the spread of fitted predictions at the same input across fresh samples. A flexible model can have low bias and still have high variance if each sample leads it toward a different detailed curve. Board: bias\_equation — a Math \[text\] that says "$upright("bias")(x) = upright("average fitted prediction")(x) - f(x)$" Actions: - [09:57.723](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=597.7227499999999): variance\_equation is shown on the screen, written out. - [09:58.547](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=598.54675): variance\_equation (the "upright("spread of fitted predictions at") thin x" part) is emphasized. - [10:1.461](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=601.46075): definition is shown on the screen, written out. - [10:10.099](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=610.09875): variance\_equation (the "upright("spread of fitted predictions at") thin x" part) is no longer emphasized. ##### [10:10.699](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=610.69875) Narration: Underfitting is the visible face of excessive bias. Overfitting is the visible face of excessive sensitivity to the sample. The useful model must control both. Board: bias\_equation — a Math \[text\] that says "$upright("bias")(x) = upright("average fitted prediction")(x) - f(x)$"; variance\_equation — a Math \[text\] that says "$upright("variance")(x) = upright("spread of fitted predictions at") thin x$"; definition — a Panel that says "Bias is systematic error that remains after averaging many refits. Variance is sensitivity to which particular sample happened to be observed." Actions: - [10:13.358](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=613.3577499999999): definition (the "Bias" part) is indicated — a transient flash. - [10:17.073](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=617.0727499999999): definition (the "Variance" part) is indicated — a transient flash. - [10:21.568](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=621.5682499999999): bias\_equation is hidden from the screen — left the board. - [10:21.568](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=621.5682499999999): definition is hidden from the screen — left the board. - [10:21.568](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=621.5682499999999): variance\_equation is hidden from the screen — left the board. ### Scene 5: [More Data Moves the Balance](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=622.6099166666665) Span: 10:22.61–13:33.041 (622.6099166666665s–813.0411666666665s). #### Objects - balance\_heading: a Heading that says "The Balance Point Moves" - closing: a Math \[text\] that says "$upright("useful complexity") = upright("enough shape, controlled sensitivity")$" - fits\_heading: a Heading that says "Refits Become More Stable" - heading: a Heading that says "What to Carry Away" - hundred\_label: a Tex \[text\] that says "100 observations" - large\_axes: an Axes (x\_range=(1.0, 19.0), y\_range=(0.0, 1.3), x\_ticks\_every=2.0) - large\_bias: a FunctionPlot \[blue\] labelled "upright("bias error")" drawn in large\_axes (function=\, x\_range=(1.0, 19.0)) - large\_cross: a Point \[yellow\] labelled "upright("new balance")" drawn in large\_axes (location=(16.0, 0.09425980088148803)) - large\_label: a Tex \[text\] that says "100 observations" - large\_variance: a FunctionPlot \[red\] labelled "upright("variance error")" drawn in large\_axes (function=\, x\_range=(1.0, 19.0)) - narrow\_axes: an Axes (x\_range=(-3.0, 3.0), y\_range=(-3.6, 1.8), x\_ticks\_every=1.0) - narrow\_fits: a FunctionPlot \[red\] drawn in narrow\_axes (function=\, x\_range=(-3.0, 3.0)) - narrow\_fits\_2: a FunctionPlot \[green\] drawn in narrow\_axes (function=\, x\_range=(-3.0, 3.0)) - narrow\_fits\_3: a FunctionPlot \[yellow\] drawn in narrow\_axes (function=\, x\_range=(-3.0, 3.0)) - narrow\_fits\_4: a FunctionPlot \[magenta\] drawn in narrow\_axes (function=\, x\_range=(-3.0, 3.0)) - narrow\_truth: a FunctionPlot \[blue\] drawn in narrow\_axes (function=\, x\_range=(-3.0, 3.0)) - small\_axes: an Axes (x\_range=(1.0, 19.0), y\_range=(0.0, 1.3), x\_ticks\_every=2.0) - small\_bias: a FunctionPlot \[blue\] labelled "upright("bias error")" drawn in small\_axes (function=\, x\_range=(1.0, 19.0)) - small\_cross: a Point \[yellow\] labelled "upright("balance")" drawn in small\_axes (location=(11.5, 0.16911350187157478)) - small\_label: a Tex \[text\] that says "20 observations" - small\_variance: a FunctionPlot \[red\] labelled "upright("variance error")" drawn in small\_axes (function=\, x\_range=(1.0, 19.0)) - summary: a Block \[text\] that says "Training error measures fit to observations the model already used. Test error measures prediction on observations held outside fitting. Bias is the systematic miss that remains across repeated samples. Variance is the spread of refitted p…" - twenty\_label: a Tex \[text\] that says "20 observations" - wide\_axes: an Axes (x\_range=(-3.0, 3.0), y\_range=(-3.6, 1.8), x\_ticks\_every=1.0) - wide\_fits: a FunctionPlot \[red\] drawn in wide\_axes (function=\, x\_range=(-3.0, 3.0)) - wide\_fits\_2: a FunctionPlot \[green\] drawn in wide\_axes (function=\, x\_range=(-3.0, 3.0)) - wide\_fits\_3: a FunctionPlot \[yellow\] drawn in wide\_axes (function=\, x\_range=(-3.0, 3.0)) - wide\_fits\_4: a FunctionPlot \[magenta\] drawn in wide\_axes (function=\, x\_range=(-3.0, 3.0)) - wide\_truth: a FunctionPlot \[blue\] drawn in wide\_axes (function=\, x\_range=(-3.0, 3.0)) #### Beats ##### [10:22.61](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=622.6099166666665) Narration: Return to repeated fitting, but change the amount of evidence. On the left, each colored flexible curve was fitted from only twenty observations. Board: Empty. Actions: - [10:22.61](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=622.6099166666665): fits\_heading is shown on the screen, written out. - [10:22.61](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=622.6099166666665): twenty\_label is shown on the screen, written out. - [10:22.61](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=622.6099166666665): wide\_axes is shown on the screen, written out. - [10:27.138](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=627.1379166666666): wide\_truth is shown on the screen, written out. - [10:28.148](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=628.1479166666666): wide\_fits is shown on the screen, written out. - [10:28.228](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=628.2279166666666): wide\_fits\_2 is shown on the screen, written out. - [10:28.308](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=628.3079166666665): wide\_fits\_3 is shown on the screen, written out. - [10:28.388](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=628.3879166666666): wide\_fits\_4 is shown on the screen, written out. ##### [10:32.684](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=632.6839166666665) Narration: The fits disagree because a handful of noisy points can pull a flexible polynomial in noticeably different directions. The blue relationship is fixed; the fitted curves are not. Board: twenty\_label — a Tex \[text\] that says "20 observations"; wide\_axes — an Axes (x\_range=(-3.0, 3.0), y\_range=(-3.6, 1.8), x\_ticks\_every=1.0); fits\_heading — a Heading that says "Refits Become More Stable"; wide\_truth — a FunctionPlot \[blue\] drawn in wide\_axes (function=\, x\_range=(-3.0, 3.0)); wide\_fits — a FunctionPlot \[red\] drawn in wide\_axes (function=\, x\_range=(-3.0, 3.0)); wide\_fits\_2 — a FunctionPlot \[green\] drawn in wide\_axes (function=\, x\_range=(-3.0, 3.0)); wide\_fits\_3 — a FunctionPlot \[yellow\] drawn in wide\_axes (function=\, x\_range=(-3.0, 3.0)); wide\_fits\_4 — a FunctionPlot \[magenta\] drawn in wide\_axes (function=\, x\_range=(-3.0, 3.0)) Actions: - [10:33.438](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=633.4379166666665): wide\_fits is indicated — a transient flash. - [10:33.438](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=633.4379166666665): wide\_fits\_2 is indicated — a transient flash. - [10:33.438](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=633.4379166666665): wide\_fits\_3 is indicated — a transient flash. - [10:33.438](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=633.4379166666665): wide\_fits\_4 is indicated — a transient flash. - [10:40.73](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=640.7299166666666): wide\_truth is indicated — a transient flash. ##### [10:44.894](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=644.8939166666665) Narration: On the right, fit the same degree of polynomial after collecting one hundred observations. Each individual observation now has less power to redirect the entire curve. Board: Unchanged from the preceding beat in this scene. Actions: - [10:45.568](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=645.5679166666665): narrow\_axes is shown on the screen, written out. - [10:46.589](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=646.5889166666666): narrow\_truth is shown on the screen, written out. - [10:47.332](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=647.3319166666665): narrow\_fits is shown on the screen, written out. - [10:47.412](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=647.4119166666666): narrow\_fits\_2 is shown on the screen, written out. - [10:47.492](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=647.4919166666665): narrow\_fits\_3 is shown on the screen, written out. - [10:47.572](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=647.5719166666665): narrow\_fits\_4 is shown on the screen, written out. - [10:49.027](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=649.0269166666666): hundred\_label is shown on the screen, written out. ##### [10:56.489](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=656.4889166666666) Narration: The colored curves still differ, because noise has not vanished. But their spread is much narrower. More data has reduced variance without changing the basic model family. Board: twenty\_label — a Tex \[text\] that says "20 observations"; wide\_axes — an Axes (x\_range=(-3.0, 3.0), y\_range=(-3.6, 1.8), x\_ticks\_every=1.0); hundred\_label — a Tex \[text\] that says "100 observations"; narrow\_axes — an Axes (x\_range=(-3.0, 3.0), y\_range=(-3.6, 1.8), x\_ticks\_every=1.0); fits\_heading — a Heading that says "Refits Become More Stable"; wide\_truth — a FunctionPlot \[blue\] drawn in wide\_axes (function=\, x\_range=(-3.0, 3.0)); wide\_fits — a FunctionPlot \[red\] drawn in wide\_axes (function=\, x\_range=(-3.0, 3.0)); wide\_fits\_2 — a FunctionPlot \[green\] drawn in wide\_axes (function=\, x\_range=(-3.0, 3.0)); wide\_fits\_3 — a FunctionPlot \[yellow\] drawn in wide\_axes (function=\, x\_range=(-3.0, 3.0)); wide\_fits\_4 — a FunctionPlot \[magenta\] drawn in wide\_axes (function=\, x\_range=(-3.0, 3.0)); narrow\_truth — a FunctionPlot \[blue\] drawn in narrow\_axes (function=\, x\_range=(-3.0, 3.0)); narrow\_fits — a FunctionPlot \[red\] drawn in narrow\_axes (function=\, x\_range=(-3.0, 3.0)); narrow\_fits\_2 — a FunctionPlot \[green\] drawn in narrow\_axes (function=\, x\_range=(-3.0, 3.0)); narrow\_fits\_3 — a FunctionPlot \[yellow\] drawn in narrow\_axes (function=\, x\_range=(-3.0, 3.0)); narrow\_fits\_4 — a FunctionPlot \[magenta\] drawn in narrow\_axes (function=\, x\_range=(-3.0, 3.0)) Actions: - [11:2.584](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=662.5839166666666): narrow\_fits is indicated — a transient flash. - [11:2.584](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=662.5839166666666): narrow\_fits\_2 is indicated — a transient flash. - [11:2.584](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=662.5839166666666): narrow\_fits\_3 is indicated — a transient flash. - [11:2.584](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=662.5839166666666): narrow\_fits\_4 is indicated — a transient flash. ##### [11:8.617](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=668.6174166666665) Narration: Notice what more data did not do. It did not make every complex model safe, and it would not make a straight line bend. It made the fitted details less dependent on a few particular observations. Board: Unchanged from the preceding beat in this scene. Actions: - [11:21.342](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=681.3419166666665): fits\_heading is hidden from the screen — left the board. - [11:21.342](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=681.3419166666665): hundred\_label is hidden from the screen — left the board. - [11:21.342](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=681.3419166666665): narrow\_axes is hidden from the screen — left the board. - [11:21.342](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=681.3419166666665): narrow\_truth is hidden from the screen — narrow\_axes left the board. - [11:21.342](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=681.3419166666665): narrow\_fits is hidden from the screen — narrow\_axes left the board. - [11:21.342](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=681.3419166666665): narrow\_fits\_2 is hidden from the screen — narrow\_axes left the board. - [11:21.342](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=681.3419166666665): narrow\_fits\_3 is hidden from the screen — narrow\_axes left the board. - [11:21.342](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=681.3419166666665): narrow\_fits\_4 is hidden from the screen — narrow\_axes left the board. - [11:21.342](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=681.3419166666665): twenty\_label is hidden from the screen — left the board. - [11:21.342](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=681.3419166666665): wide\_axes is hidden from the screen — left the board. - [11:21.342](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=681.3419166666665): wide\_truth is hidden from the screen — wide\_axes left the board. - [11:21.342](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=681.3419166666665): wide\_fits is hidden from the screen — wide\_axes left the board. - [11:21.342](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=681.3419166666665): wide\_fits\_2 is hidden from the screen — wide\_axes left the board. - [11:21.342](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=681.3419166666665): wide\_fits\_3 is hidden from the screen — wide\_axes left the board. - [11:21.342](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=681.3419166666665): wide\_fits\_4 is hidden from the screen — wide\_axes left the board. ##### [11:22.542](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=682.5419166666666) Narration: We can summarize that change with two error contributions. Blue is error from bias. It falls as complexity gives the model more shapes to choose from. Board: Empty. Actions: - [11:22.542](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=682.5419166666666): balance\_heading is shown on the screen, written out. - [11:22.542](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=682.5419166666666): small\_label is shown on the screen, written out. - [11:22.542](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=682.5419166666666): small\_axes is shown on the screen, written out. - [11:26.721](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=686.7209166666665): small\_bias is shown on the screen, drawn. ##### [11:33.718](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=693.7179166666665) Narration: Red is error from variance. With only twenty observations, it rises as flexible models become increasingly sensitive to the sample. Board: small\_label — a Tex \[text\] that says "20 observations"; small\_axes — an Axes (x\_range=(1.0, 19.0), y\_range=(0.0, 1.3), x\_ticks\_every=2.0); balance\_heading — a Heading that says "The Balance Point Moves"; small\_bias — a FunctionPlot \[blue\] labelled "upright("bias error")" drawn in small\_axes (function=\, x\_range=(1.0, 19.0)) Actions: - [11:34.194](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=694.1939166666665): small\_variance is shown on the screen, drawn. ##### [11:43.896](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=703.8964166666665) Narration: The two contributions cross near this yellow point. To the left, missing structure dominates. To the right, sample sensitivity dominates. The crossing is a useful picture of balance, not a universal decision rule. Board: small\_label — a Tex \[text\] that says "20 observations"; small\_axes — an Axes (x\_range=(1.0, 19.0), y\_range=(0.0, 1.3), x\_ticks\_every=2.0); balance\_heading — a Heading that says "The Balance Point Moves"; small\_bias — a FunctionPlot \[blue\] labelled "upright("bias error")" drawn in small\_axes (function=\, x\_range=(1.0, 19.0)); small\_variance — a FunctionPlot \[red\] labelled "upright("variance error")" drawn in small\_axes (function=\, x\_range=(1.0, 19.0)) Actions: - [11:46.358](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=706.3579166666666): small\_cross is shown on the screen, written out. - [11:48.355](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=708.3549166666666): small\_bias is indicated — a transient flash. - [11:51.466](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=711.4659166666665): small\_variance is indicated — a transient flash. ##### [12:0.135](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=720.1349166666665) Narration: Now keep the blue bias curve and increase the data to one hundred observations. The right-hand graph shows the same declining bias contribution. Board: small\_label — a Tex \[text\] that says "20 observations"; small\_axes — an Axes (x\_range=(1.0, 19.0), y\_range=(0.0, 1.3), x\_ticks\_every=2.0); balance\_heading — a Heading that says "The Balance Point Moves"; small\_bias — a FunctionPlot \[blue\] labelled "upright("bias error")" drawn in small\_axes (function=\, x\_range=(1.0, 19.0)); small\_variance — a FunctionPlot \[red\] labelled "upright("variance error")" drawn in small\_axes (function=\, x\_range=(1.0, 19.0)); small\_cross — a Point \[yellow\] labelled "upright("balance")" drawn in small\_axes (location=(11.5, 0.16911350187157478)) Actions: - [12:0.135](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=720.1349166666665): large\_axes is shown on the screen, written out. - [12:1.272](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=721.2719166666666): large\_bias is shown on the screen, drawn. - [12:3.769](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=723.7689166666665): large\_label is shown on the screen, written out. ##### [12:10.022](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=730.0224166666666) Narration: The red variance contribution is lower because repeated fits are more stable. Its crossing with bias moves to the right, toward a more flexible polynomial. Board: small\_label — a Tex \[text\] that says "20 observations"; small\_axes — an Axes (x\_range=(1.0, 19.0), y\_range=(0.0, 1.3), x\_ticks\_every=2.0); large\_label — a Tex \[text\] that says "100 observations"; large\_axes — an Axes (x\_range=(1.0, 19.0), y\_range=(0.0, 1.3), x\_ticks\_every=2.0); balance\_heading — a Heading that says "The Balance Point Moves"; small\_bias — a FunctionPlot \[blue\] labelled "upright("bias error")" drawn in small\_axes (function=\, x\_range=(1.0, 19.0)); small\_variance — a FunctionPlot \[red\] labelled "upright("variance error")" drawn in small\_axes (function=\, x\_range=(1.0, 19.0)); small\_cross — a Point \[yellow\] labelled "upright("balance")" drawn in small\_axes (location=(11.5, 0.16911350187157478)); large\_bias — a FunctionPlot \[blue\] labelled "upright("bias error")" drawn in large\_axes (function=\, x\_range=(1.0, 19.0)) Actions: - [12:10.557](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=730.5569166666666): large\_variance is shown on the screen, drawn. - [12:15.503](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=735.5029166666666): large\_cross is shown on the screen, written out. - [12:17.035](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=737.0349166666665): large\_cross is indicated — a transient flash. ##### [12:20.503](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=740.5029166666666) Narration: This does not say that complexity is good by itself. It says complexity becomes easier to support when the data constrain it from many directions. The final choice still belongs to validation or test performance. Board: small\_label — a Tex \[text\] that says "20 observations"; small\_axes — an Axes (x\_range=(1.0, 19.0), y\_range=(0.0, 1.3), x\_ticks\_every=2.0); large\_label — a Tex \[text\] that says "100 observations"; large\_axes — an Axes (x\_range=(1.0, 19.0), y\_range=(0.0, 1.3), x\_ticks\_every=2.0); balance\_heading — a Heading that says "The Balance Point Moves"; small\_bias — a FunctionPlot \[blue\] labelled "upright("bias error")" drawn in small\_axes (function=\, x\_range=(1.0, 19.0)); small\_variance — a FunctionPlot \[red\] labelled "upright("variance error")" drawn in small\_axes (function=\, x\_range=(1.0, 19.0)); small\_cross — a Point \[yellow\] labelled "upright("balance")" drawn in small\_axes (location=(11.5, 0.16911350187157478)); large\_bias — a FunctionPlot \[blue\] labelled "upright("bias error")" drawn in large\_axes (function=\, x\_range=(1.0, 19.0)); large\_variance — a FunctionPlot \[red\] labelled "upright("variance error")" drawn in large\_axes (function=\, x\_range=(1.0, 19.0)); large\_cross — a Point \[yellow\] labelled "upright("new balance")" drawn in large\_axes (location=(16.0, 0.09425980088148803)) Actions: - [12:27.841](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=747.8409166666665): large\_variance is indicated — a transient flash. - [12:30.906](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=750.9059166666666): large\_cross is indicated — a transient flash. ##### [12:34.559](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=754.5589166666666) Narration: The point of the whole experiment is now visible. Underfitting is a systematic miss that survives more observations. Overfitting is a gap between fitting performance and new-data performance, driven by excessive sensitivity to the sample. Board: Unchanged from the preceding beat in this scene. Actions: - [12:49.489](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=769.4894166666666): balance\_heading is hidden from the screen — left the board. - [12:49.489](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=769.4894166666666): large\_axes is hidden from the screen — left the board. - [12:49.489](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=769.4894166666666): large\_bias is hidden from the screen — large\_axes left the board. - [12:49.489](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=769.4894166666666): large\_variance is hidden from the screen — large\_axes left the board. - [12:49.489](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=769.4894166666666): large\_cross is hidden from the screen — large\_axes left the board. - [12:49.489](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=769.4894166666666): large\_label is hidden from the screen — left the board. - [12:49.489](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=769.4894166666666): small\_axes is hidden from the screen — left the board. - [12:49.489](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=769.4894166666666): small\_bias is hidden from the screen — small\_axes left the board. - [12:49.489](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=769.4894166666666): small\_variance is hidden from the screen — small\_axes left the board. - [12:49.489](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=769.4894166666666): small\_cross is hidden from the screen — small\_axes left the board. - [12:49.489](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=769.4894166666666): small\_label is hidden from the screen — left the board. ##### [12:50.689](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=770.6894166666666) Narration: First, training error asks about observations the model already used. Second, test error asks about genuinely separate observations. The difference between those questions is what makes overfitting measurable. Board: Empty. Actions: - [12:51.038](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=771.0379166666665): summary is shown on the screen, written out. - [12:51.839](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=771.8389166666666): summary (the "Training error" part) is emphasized. - [12:56.959](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=776.9589166666666): summary (the "Test error" part) is emphasized. - [12:56.959](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=776.9589166666666): summary (the "Training error" part) is no longer emphasized. ##### [13:5.279](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=785.2789166666665) Narration: Third, bias is the persistent miss seen across fresh samples. Fourth, variance is the spread of the refitted predictions, and adding data can narrow that spread. Board: summary — a Block \[text\] that says "Training error measures fit to observations the model already used. Test error measures prediction on observations held outside fitting. Bias is the systematic miss that remains across repeated samples. Variance is the spread of refitted p…" Actions: - [13:6.696](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=786.6959166666666): summary (the "Bias" part) is emphasized. - [13:6.696](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=786.6959166666666): summary (the "Test error" part) is no longer emphasized. - [13:11.665](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=791.6649166666666): summary (the "Bias" part) is no longer emphasized. - [13:11.665](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=791.6649166666666): summary (the "Variance" part) is emphasized. - [13:17.296](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=797.2959166666666): summary (the "Variance" part) is no longer emphasized. ##### [13:17.896](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=797.8959166666666) Narration: A useful model has enough freedom to represent the relationship and enough evidence to keep that freedom under control. The goal is not a curve that remembers every point. It is a curve whose accuracy survives the arrival of the next point. Board: Unchanged from the preceding beat in this scene. Actions: - [13:19.51](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=799.5099166666665): closing is shown on the screen, written out. - [13:29.529](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=809.5289166666665): A box is drawn around closing. - [13:31.999](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=811.9994999999999): closing is hidden from the screen — left the board. - [13:31.999](https://academa.ai/lectures/bias-variance-and-the-shape-of-overfitting?t=811.9994999999999): summary is hidden from the screen — left the board.