# Support Vector Machines: Maximum-Margin Separation and the Kernel Trick

> A geometric introduction to support vector machines, beginning with the widest separating corridor and deriving the hard-margin optimization problem from point-to-hyperplane distance. The lecture identifies support vectors as active constraints, extends the model to overlapping classes with slack variables and soft margins, then develops a concrete nonlinear example through an explicit quadratic feature lift. It concludes by deriving the kernel trick and showing how training and prediction depend only on kernel evaluations involving the support vectors.

- Canonical watch page: [Support Vector Machines: Maximum-Margin Separation and the Kernel Trick](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor)
- Publisher: [Academa, Inc.](https://academa.ai)
- Subject: Machine Learning
- Published: 2026-08-29T00:46:39.000Z
- Updated: 2026-08-29T00:46:39.000Z
- Duration: PT965S (16 minutes 5 seconds)
- Chapters: 6
- Views: 0
- Language: en-US
- Access: Free
- Video stream: [HLS content](https://academa.ai/media/l/01M14TYW7K99A7C00QDVC7B6Y4/2/dark/master.m3u8)
- Audiovisual record: [Semantic JSON](https://academa.ai/media/l/01M14TYW7K99A7C00QDVC7B6Y4/2/semantic.json)
- Thumbnail: [Image](https://academa.ai/media/l/01M14TYW7K99A7C00QDVC7B6Y4/2/dark/poster.jpg)

## Description

A visual introduction to maximum-margin classifiers, support vectors, soft margins, nonlinear feature maps, and the kernel trick.

## Chapters

- [00:00–01:53.691 · The Widest Empty Corridor](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=0)
- [01:53.691–04:31.343 · From Geometry to Optimisation](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=113.69085416666667)
- [04:31.343–07:19.784 · When the Classes Overlap](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=271.34333333333336)
- [07:19.784–09:6.501 · No Straight Line Will Do](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=439.78439583333335)
- [09:6.501–11:19.413 · Lifting into Feature Space](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=546.5005625)
- [11:19.413–16:05 · The Kernel Trick](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=679.4129583333333)

## Transcript

### [00:00 · The Widest Empty Corridor](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=0)

Suppose red and blue are two classes in a training set. We want a line that puts every red point on one side and every blue point on the other. This data is linearly separable, so at least one such line exists. In fact, one separating line is easy to draw. Every red point lies above it, and every blue point lies below it. But this line runs rather close to the blue class. Another line also separates the data, but now it passes close to the red class. A tilted line works too. Correct classification alone does not tell us which of these choices is preferable. Imagine placing an empty corridor around a separator. Its walls remain parallel to the line, and we widen them until one wall would hit red or the other would hit blue. The corridor's width measures how much room the classifier has before a training point changes sides. Now slide the centre line inside that gap. If it moves down, the lower clearance shrinks. If it moves up, the upper clearance shrinks. The best position balances the nearest points on the two sides and maximizes the smaller clearance. The points that stop the two walls are special. Three red points touch the upper wall, and two blue points touch the lower wall. These are the support vectors, because they physically support the widest corridor. Points farther away do not determine the final position. We could move one of those distant points slightly without changing either wall. Move a support vector, however, and the maximum-margin separator generally has to move with it. So an SVM does not merely search for zero training errors. Among all separating hyperplanes, it chooses the one with the largest geometric margin. Next we will turn this corridor into an optimization problem.

### [01:53.691 · From Geometry to Optimisation](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=113.69085416666667)

A line in two dimensions, or a hyperplane in any dimension, can be written as w dot x plus b equals zero. The vector w is perpendicular to the separator, while b shifts it without changing its orientation. The expression w dot x plus b is a signed score. Positive scores lie on one side, negative scores lie on the other, and its sign becomes the predicted class. Let each training label y sub i be plus one for red and minus one for blue. Correct classification then has one compact condition: y sub i times the score must be positive. There is a scaling ambiguity. Multiplying both w and b by any positive number leaves the zero set unchanged, so it leaves the separating hyperplane unchanged. We use that freedom to make the closest signed scores equal to one. After this normalization, every red point must have score at least plus one, and every blue point must have score at most minus one. The support vectors are precisely the points where the combined inequality becomes an equality. The perpendicular distance from a point x to the central hyperplane is the absolute score divided by the length of w. Each wall has score one in magnitude, so each wall is one over the length of w from the centre. The full corridor therefore has width two over the length of w. To make that width large, we must make the length of w small. Squaring it changes neither the minimizing choice nor the geometry. We have reached the hard-margin optimization problem. Minimize one half w dot w, which is one half the squared length of the normal vector. The minimization is subject to one constraint per training example: y sub i times w dot x sub i plus b must be at least one. These inequalities say that every point lies outside the empty corridor on its proper side. Only a few of those constraints are tight at the solution. Those active constraints belong to the support vectors. The remaining points satisfy their inequalities with room to spare and do not pin the optimum. This completes the geometric derivation. A wide corridor became a small normal vector, correct classification became a family of inequalities, and the points touching equality became the support vectors.

### [04:31.343 · When the Classes Overlap](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=271.34333333333336)

Real classes often overlap. A red example may sit deep among blue examples, while a blue example may sit among red ones. No line can put every point on its requested side in this training set. If we insist on the hard-margin constraints, the optimization problem has no feasible answer. The red point below and the blue point above cannot both satisfy the same perfect separation. A soft-margin SVM relaxes each constraint with a nonnegative slack variable xi sub i. Instead of demanding a signed score of at least one, it permits the score to fall short by xi sub i. Draw the central separator and its two margin walls. A point outside its correct wall needs zero slack. A point inside the corridor needs some slack, because it has not reached the wall required by its label. This red point is not merely inside the margin. It lies on the blue side of the separator, so its slack exceeds one. The same is true in reverse for this blue point. Slack records the degree of each violation rather than pretending the data is perfectly separable. The new objective contains two costs. One half w dot w still rewards a wide margin. The sum of the slacks penalizes points that enter the margin or cross the separator, and the constant C controls how heavily that penalty is counted. This is not permission to ignore errors for free. Every violation raises the objective. The optimizer accepts a violation only when avoiding it would require a sufficiently narrower or more contorted decision boundary. With a smaller C, slack is relatively inexpensive. The optimizer is more willing to tolerate unusual training points in exchange for a broad, stable corridor. With a larger C, violations are expensive. The separator bends its linear choice as far as it can toward fitting difficult examples, often leaving a narrower margin. The drawing is schematic, but the trade is exact: margin width against total slack. Small C can underreact to genuine structure. Very large C can overreact to noise or mislabeled examples. In practice C is a regularization hyperparameter, chosen by validation rather than by demanding zero training error. Support vectors still matter in the soft-margin model. They now include points touching a margin wall and points lying inside or beyond it. These are the examples that exert force on the final decision boundary.

### [07:19.784 · No Straight Line Will Do](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=439.78439583333335)

Now consider a different training set. The red examples form a compact inner cluster, while the blue examples make a ring around them. This is a simple nonlinear pattern with no noise and no overlap. Try a slanted line. It can place some blue points opposite the red cluster, but blue points remain on both sides. Rotating the line merely changes which part of the ring is misclassified. Move the line above the centre instead. The red points may all fall below it, but so do the blue points around the lower half of the ring. No translation fixes the problem either. There is a geometric reason, not just a failure of imagination. Join the outer blue points to form their convex hull. The entire red cluster lies inside that hull. A straight line divides the plane into two half-planes. If every blue vertex lies in one half-plane, then every convex combination of those vertices lies there too. The red points inside their hull therefore cannot all occupy the opposite half-plane. The useful feature here is not left versus right or up versus down. It is distance from the centre. Square that distance and call it q of x: x one squared plus x two squared. Every red point has a small value of q. Every blue point has a much larger value. A threshold at two separates them perfectly: red below two, blue above two. Back in the original plane, the threshold q equals two is a circle, not a line. So the data is nonlinear in its original coordinates. The next move is to change coordinates rather than abandon maximum-margin separation.

### [09:6.501 · Lifting into Feature Space](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=546.5005625)

A feature map replaces each original input by a new vector. For this problem, map x one and x two to three quadratic features: x one squared, square root two times x one x two, and x two squared. At first I have placed the points in a flat copy of their familiar arrangement. Now apply the feature map. Each point moves to the three coordinates computed from its original two. The inner red points remain near the origin of feature space. The outer blue points rise onto a distant sheet because x one squared plus x two squared is large for every point on the ring. In this three-dimensional space, one flat plane separates the classes. Its equation is feature one plus feature three equals two. Red points lie below it, and blue points lie above it. This is ordinary linear separation again. We can place parallel margin planes around this separator, maximize their distance, and obtain support vectors exactly as before. The only change is that the vectors now live in feature space. Now compare the two spaces. On the left is the original input plane. On the right is the lifted feature space with its flat separating plane. The plane above is feature one plus feature three equals two. Substitute the definitions of those features and the same condition becomes x one squared plus x two squared equals two below. That equation draws this circle in the input plane. A linear decision surface in feature space has become a nonlinear decision boundary in the original coordinates. The construction works, but it seems expensive. A richer feature map may contain hundreds, millions, or even infinitely many coordinates. Writing and storing every lifted vector would defeat the purpose. The escape comes from noticing what the SVM calculation actually asks of those lifted vectors. It does not need to inspect their coordinates one at a time. It needs inner products between them.

### [11:19.413 · The Kernel Trick](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=679.4129583333333)

Return to the quadratic feature map. To compare two lifted examples phi of x and phi of z, the SVM needs their inner product. Let us calculate that inner product once and see what remains. Multiply matching lifted coordinates. The first contribution is x one squared z one squared. The middle contribution contains the factor two, and the last is x two squared z two squared. Those three terms form a familiar square. They equal x one z one plus x two z two, all squared. But x one z one plus x two z two is simply the ordinary dot product x dot z. Therefore the lifted inner product is the square of one dot product in the original two-dimensional space. Define K of x z to be that squared dot product. Evaluating K gives exactly the number we would have obtained by constructing both three-dimensional feature vectors and taking their inner product. That replacement is the kernel trick. We behave as if the data had been lifted, while every computation stays in the original input space. The feature coordinates disappear, but their geometry remains in the kernel values. Why are inner products enough? At the optimum, the normal vector in feature space can be written as a weighted sum of lifted training examples. Each weight is alpha sub i times the label y sub i. To classify a new input, begin with w dot phi of x plus b. Substitute the weighted expansion of w, and the calculation becomes a sum of inner products between training features and the new feature vector. Now replace every lifted inner product by K of x sub i and x. Prediction can be performed without forming w and without explicitly forming phi of any example. Most coefficients alpha sub i are zero. Only support vectors have nonzero coefficients and contribute to the final sum. The same small set of geometrically decisive points now makes prediction sparse. Training can also be written entirely in terms of pairwise kernels. The dual objective contains K of x sub i and x sub j for pairs of training examples, together with nonnegative coefficients whose label-weighted sum is zero. The optimization therefore needs a matrix of pairwise kernel values, often called the Gram matrix. Once those values are available, the algorithm can operate as though it knew every lifted coordinate. A kernel supplies exactly the required feature-space inner product. Different kernels encode different notions of similarity. The linear kernel is the ordinary dot product, so it gives the original linear SVM. A polynomial kernel raises a shifted dot product to a degree d. Our quadratic example used degree two with no shift. Higher degrees represent richer interactions among the original coordinates. The radial basis kernel decreases exponentially with squared distance between two inputs. It corresponds to a very rich feature space and produces flexible local boundaries, with gamma controlling how quickly similarity falls away. A kernel is not magic permission to use any similarity formula. It must behave like a genuine inner product in some feature space. When it does, the maximum-margin optimization remains mathematically well formed. Let us connect the whole argument. First, maximum margin chooses the separator with the widest empty corridor. The nearest points become support vectors because their constraints pin that corridor in place. When classes overlap, slack variables measure margin violations, and C trades a wider corridor against the cost of those violations. When no straight boundary works, a feature map can lift the examples into a space where a hyperplane does work. The decision surface is linear there even though it is curved in the original coordinates. Finally, the kernel trick computes only the inner products required by training and prediction. It preserves the lifted geometry without paying the cost of explicitly writing the lifted vectors. That is the support vector machine in one connected picture: maximize geometric separation, let a small set of points determine the answer, tolerate overlap when necessary, and use kernels when the useful separation lives in a nonlinear feature space.

## About Academa, Inc.

Academa makes technical knowledge easier to understand through visual lectures and lets learners request new lecture videos on the topics they need.

## Complete audiovisual record

Immutable source: [semantic.json](https://academa.ai/media/l/01M14TYW7K99A7C00QDVC7B6Y4/2/semantic.json)

Record version: 1. Render attempt: 2.

### How to read this timeline

Each scene owns its object identifiers. A beat's board is the complete board when listed, empty when marked empty, and unchanged from the nearest earlier listed board in the same scene when marked unchanged. Action times are absolute positions in the published video.

### Scene 1: [The Widest Empty Corridor](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=0)

Span: 00:00–01:53.691 (0s–113.69085416666667s).

#### Objects

- axes: an Axes (x\_range=(0.0, 6.0), y\_range=(0.0, 4.8), aspect=(6.0, 4.8))
- best\_separator: a Line \[green\] labelled "w dot x + b = 0" drawn in axes (start=(0.0, 2.0), end=(6.0, 2.0))
- candidate\_high: a Line \[yellow\] labelled "upright("separator")" drawn in axes (start=(0.0, 2.55), end=(6.0, 2.35))
- candidate\_low: a Line \[yellow\] labelled "upright("separator")" drawn in axes (start=(0.0, 1.55), end=(6.0, 1.75))
- candidate\_tilted: a Line \[yellow\] labelled "upright("separator")" drawn in axes (start=(0.0, 1.65), end=(6.0, 2.25))
- corridor: a Region \[yellow\] drawn in axes (predicates=(\<function \<lambda\> at 0x2ab79a3ae020\>,), x\_range=(0.0, 6.0), y\_range=(1.0, 3.0))
- lower\_wall: a Line \[gray\] labelled "w dot x + b = -1" drawn in axes (start=(0.0, 1.0), end=(6.0, 1.0), dashed=True)
- margin\_width: a Math \[text\] that says "$upright("corridor width") = frac(2, sqrt(w dot w))$"
- negative\_points: a Point \[blue\] drawn in axes (location=(0.8, 0.5))
- negative\_points\_2: a Point \[blue\] drawn in axes (location=(2.0, 1.0))
- negative\_points\_3: a Point \[blue\] drawn in axes (location=(3.0, 0.4))
- negative\_points\_4: a Point \[blue\] drawn in axes (location=(4.0, 1.0))
- negative\_points\_5: a Point \[blue\] drawn in axes (location=(5.2, 0.5))
- positive\_points: a Point \[red\] drawn in axes (location=(1.0, 3.0))
- positive\_points\_2: a Point \[red\] drawn in axes (location=(2.0, 4.0))
- positive\_points\_3: a Point \[red\] drawn in axes (location=(3.0, 3.0))
- positive\_points\_4: a Point \[red\] drawn in axes (location=(4.0, 4.2))
- positive\_points\_5: a Point \[red\] drawn in axes (location=(5.0, 3.0))
- question: a Panel that says "Many lines separate these two classes. Which separator should a classifier trust?"
- support\_definition: a Panel that says "The training points that touch the two walls of the maximum-margin corridor are the support vectors."
- upper\_wall: a Line \[gray\] labelled "w dot x + b = 1" drawn in axes (start=(0.0, 3.0), end=(6.0, 3.0), dashed=True)

#### Beats

##### [00:00](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=0)

Narration: Suppose red and blue are two classes in a training set. We want a line that puts every red point on one side and every blue point on the other. This data is linearly separable, so at least one such line exists.

Board: Empty.

Actions:
- [00:00](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=0): question is shown on the screen, written out.
- [00:00](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=0): axes is shown on the screen, written out.
- [00:0.522](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=0.522): positive\_points is shown on the screen, written out.
- [00:0.522](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=0.522): positive\_points\_2 is shown on the screen, written out.
- [00:0.522](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=0.522): positive\_points\_3 is shown on the screen, written out.
- [00:0.522](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=0.522): positive\_points\_4 is shown on the screen, written out.
- [00:0.522](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=0.522): positive\_points\_5 is shown on the screen, written out.
- [00:0.824](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=0.824): negative\_points is shown on the screen, written out.
- [00:0.824](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=0.824): negative\_points\_2 is shown on the screen, written out.
- [00:0.824](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=0.824): negative\_points\_3 is shown on the screen, written out.
- [00:0.824](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=0.824): negative\_points\_4 is shown on the screen, written out.
- [00:0.824](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=0.824): negative\_points\_5 is shown on the screen, written out.

##### [00:14.346](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=14.346)

Narration: In fact, one separating line is easy to draw. Every red point lies above it, and every blue point lies below it. But this line runs rather close to the blue class.

Board: axes — an Axes (x\_range=(0.0, 6.0), y\_range=(0.0, 4.8), aspect=(6.0, 4.8)); question — a Panel that says "Many lines separate these two classes. Which separator should a classifier trust?"; positive\_points — a Point \[red\] drawn in axes (location=(1.0, 3.0)); positive\_points\_2 — a Point \[red\] drawn in axes (location=(2.0, 4.0)); positive\_points\_3 — a Point \[red\] drawn in axes (location=(3.0, 3.0)); positive\_points\_4 — a Point \[red\] drawn in axes (location=(4.0, 4.2)); positive\_points\_5 — a Point \[red\] drawn in axes (location=(5.0, 3.0)); negative\_points — a Point \[blue\] drawn in axes (location=(0.8, 0.5)); negative\_points\_2 — a Point \[blue\] drawn in axes (location=(2.0, 1.0)); negative\_points\_3 — a Point \[blue\] drawn in axes (location=(3.0, 0.4)); negative\_points\_4 — a Point \[blue\] drawn in axes (location=(4.0, 1.0)); negative\_points\_5 — a Point \[blue\] drawn in axes (location=(5.2, 0.5))

Actions:
- [00:15.53](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=15.53): candidate\_low is shown on the screen, written out.
- [00:24.006](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=24.006): candidate\_low is indicated — a transient flash.

##### [00:26.277](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=26.2775)

Narration: Another line also separates the data, but now it passes close to the red class. A tilted line works too. Correct classification alone does not tell us which of these choices is preferable.

Board: axes — an Axes (x\_range=(0.0, 6.0), y\_range=(0.0, 4.8), aspect=(6.0, 4.8)); question — a Panel that says "Many lines separate these two classes. Which separator should a classifier trust?"; positive\_points — a Point \[red\] drawn in axes (location=(1.0, 3.0)); positive\_points\_2 — a Point \[red\] drawn in axes (location=(2.0, 4.0)); positive\_points\_3 — a Point \[red\] drawn in axes (location=(3.0, 3.0)); positive\_points\_4 — a Point \[red\] drawn in axes (location=(4.0, 4.2)); positive\_points\_5 — a Point \[red\] drawn in axes (location=(5.0, 3.0)); negative\_points — a Point \[blue\] drawn in axes (location=(0.8, 0.5)); negative\_points\_2 — a Point \[blue\] drawn in axes (location=(2.0, 1.0)); negative\_points\_3 — a Point \[blue\] drawn in axes (location=(3.0, 0.4)); negative\_points\_4 — a Point \[blue\] drawn in axes (location=(4.0, 1.0)); negative\_points\_5 — a Point \[blue\] drawn in axes (location=(5.2, 0.5)); candidate\_low — a Line \[yellow\] labelled "upright("separator")" drawn in axes (start=(0.0, 1.55), end=(6.0, 1.75))

Actions:
- [00:26.277](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=26.2775): candidate\_low is hidden from the screen.
- [00:26.626](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=26.626): candidate\_high is shown on the screen, written out.
- [00:32.373](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=32.373000000000005): candidate\_high is hidden from the screen.
- [00:32.373](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=32.373000000000005): candidate\_tilted is shown on the screen, written out.
- [00:37.945](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=37.94500000000001): candidate\_tilted is hidden from the screen.

##### [00:39.486](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=39.486)

Narration: Imagine placing an empty corridor around a separator. Its walls remain parallel to the line, and we widen them until one wall would hit red or the other would hit blue. The corridor's width measures how much room the classifier has before a training point changes sides.

Board: axes — an Axes (x\_range=(0.0, 6.0), y\_range=(0.0, 4.8), aspect=(6.0, 4.8)); question — a Panel that says "Many lines separate these two classes. Which separator should a classifier trust?"; positive\_points — a Point \[red\] drawn in axes (location=(1.0, 3.0)); positive\_points\_2 — a Point \[red\] drawn in axes (location=(2.0, 4.0)); positive\_points\_3 — a Point \[red\] drawn in axes (location=(3.0, 3.0)); positive\_points\_4 — a Point \[red\] drawn in axes (location=(4.0, 4.2)); positive\_points\_5 — a Point \[red\] drawn in axes (location=(5.0, 3.0)); negative\_points — a Point \[blue\] drawn in axes (location=(0.8, 0.5)); negative\_points\_2 — a Point \[blue\] drawn in axes (location=(2.0, 1.0)); negative\_points\_3 — a Point \[blue\] drawn in axes (location=(3.0, 0.4)); negative\_points\_4 — a Point \[blue\] drawn in axes (location=(4.0, 1.0)); negative\_points\_5 — a Point \[blue\] drawn in axes (location=(5.2, 0.5))

Actions:
- [00:41.1](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=41.099999999999994): corridor is shown on the screen, written out.
- [00:41.901](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=41.900999999999996): best\_separator is shown on the screen, written out.
- [00:43.433](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=43.433): lower\_wall is shown on the screen, written out.
- [00:43.433](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=43.433): upper\_wall is shown on the screen, written out.

##### [00:56.688](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=56.6885)

Narration: Now slide the centre line inside that gap. If it moves down, the lower clearance shrinks. If it moves up, the upper clearance shrinks. The best position balances the nearest points on the two sides and maximizes the smaller clearance.

Board: axes — an Axes (x\_range=(0.0, 6.0), y\_range=(0.0, 4.8), aspect=(6.0, 4.8)); question — a Panel that says "Many lines separate these two classes. Which separator should a classifier trust?"; positive\_points — a Point \[red\] drawn in axes (location=(1.0, 3.0)); positive\_points\_2 — a Point \[red\] drawn in axes (location=(2.0, 4.0)); positive\_points\_3 — a Point \[red\] drawn in axes (location=(3.0, 3.0)); positive\_points\_4 — a Point \[red\] drawn in axes (location=(4.0, 4.2)); positive\_points\_5 — a Point \[red\] drawn in axes (location=(5.0, 3.0)); negative\_points — a Point \[blue\] drawn in axes (location=(0.8, 0.5)); negative\_points\_2 — a Point \[blue\] drawn in axes (location=(2.0, 1.0)); negative\_points\_3 — a Point \[blue\] drawn in axes (location=(3.0, 0.4)); negative\_points\_4 — a Point \[blue\] drawn in axes (location=(4.0, 1.0)); negative\_points\_5 — a Point \[blue\] drawn in axes (location=(5.2, 0.5)); best\_separator — a Line \[green\] labelled "w dot x + b = 0" drawn in axes (start=(0.0, 2.0), end=(6.0, 2.0)); corridor — a Region \[yellow\] drawn in axes (predicates=(\<function \<lambda\> at 0x2ab79a3ae020\>,), x\_range=(0.0, 6.0), y\_range=(1.0, 3.0)); lower\_wall — a Line \[gray\] labelled "w dot x + b = -1" drawn in axes (start=(0.0, 1.0), end=(6.0, 1.0), dashed=True); upper\_wall — a Line \[gray\] labelled "w dot x + b = 1" drawn in axes (start=(0.0, 3.0), end=(6.0, 3.0), dashed=True)

Actions:
- [01:0.427](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=60.427): candidate\_low is shown on the screen, written out.
- [01:3.272](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=63.272): candidate\_low is hidden from the screen.
- [01:3.272](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=63.272): candidate\_high is shown on the screen, written out.
- [01:6.012](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=66.012): candidate\_high is hidden from the screen.
- [01:6.662](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=66.662): best\_separator is indicated — a transient flash.

##### [01:11.546](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=71.5455)

Narration: The points that stop the two walls are special. Three red points touch the upper wall, and two blue points touch the lower wall. These are the support vectors, because they physically support the widest corridor.

Board: Unchanged from the preceding beat in this scene.

Actions:
- [01:13.763](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=73.763): axes moves to a new place on the board.
- [01:13.763](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=73.763): support\_definition is shown on the screen, written out.
- [01:15.331](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=75.331): positive\_points is indicated — a transient flash.
- [01:15.331](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=75.331): positive\_points\_3 is indicated — a transient flash.
- [01:15.331](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=75.331): positive\_points\_5 is indicated — a transient flash.
- [01:17.502](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=77.50200000000001): negative\_points\_2 is indicated — a transient flash.
- [01:17.502](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=77.50200000000001): negative\_points\_4 is indicated — a transient flash.

##### [01:24.58](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=84.58)

Narration: Points farther away do not determine the final position. We could move one of those distant points slightly without changing either wall. Move a support vector, however, and the maximum-margin separator generally has to move with it.

Board: support\_definition — a Panel that says "The training points that touch the two walls of the maximum-margin corridor are the support vectors."; axes — an Axes (x\_range=(0.0, 6.0), y\_range=(0.0, 4.8), aspect=(6.0, 4.8)); question — a Panel that says "Many lines separate these two classes. Which separator should a classifier trust?"; positive\_points — a Point \[red\] drawn in axes (location=(1.0, 3.0)); positive\_points\_2 — a Point \[red\] drawn in axes (location=(2.0, 4.0)); positive\_points\_3 — a Point \[red\] drawn in axes (location=(3.0, 3.0)); positive\_points\_4 — a Point \[red\] drawn in axes (location=(4.0, 4.2)); positive\_points\_5 — a Point \[red\] drawn in axes (location=(5.0, 3.0)); negative\_points — a Point \[blue\] drawn in axes (location=(0.8, 0.5)); negative\_points\_2 — a Point \[blue\] drawn in axes (location=(2.0, 1.0)); negative\_points\_3 — a Point \[blue\] drawn in axes (location=(3.0, 0.4)); negative\_points\_4 — a Point \[blue\] drawn in axes (location=(4.0, 1.0)); negative\_points\_5 — a Point \[blue\] drawn in axes (location=(5.2, 0.5)); best\_separator — a Line \[green\] labelled "w dot x + b = 0" drawn in axes (start=(0.0, 2.0), end=(6.0, 2.0)); corridor — a Region \[yellow\] drawn in axes (predicates=(\<function \<lambda\> at 0x2ab79a3ae020\>,), x\_range=(0.0, 6.0), y\_range=(1.0, 3.0)); lower\_wall — a Line \[gray\] labelled "w dot x + b = -1" drawn in axes (start=(0.0, 1.0), end=(6.0, 1.0), dashed=True); upper\_wall — a Line \[gray\] labelled "w dot x + b = 1" drawn in axes (start=(0.0, 3.0), end=(6.0, 3.0), dashed=True)

Actions:
- [01:25.276](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=85.27600000000001): positive\_points\_2 is indicated — a transient flash.
- [01:29.41](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=89.41): negative\_points is indicated — a transient flash.
- [01:33.044](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=93.044): positive\_points is indicated — a transient flash.

##### [01:38.787](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=98.78699999999999)

Narration: So an SVM does not merely search for zero training errors. Among all separating hyperplanes, it chooses the one with the largest geometric margin. Next we will turn this corridor into an optimization problem.

Board: Unchanged from the preceding beat in this scene.

Actions:
- [01:46.415](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=106.41499999999999): best\_separator is indicated — a transient flash.
- [01:46.937](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=106.937): margin\_width is shown on the screen, written out.
- [01:52.649](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=112.6491875): axes is hidden from the screen — left the board.
- [01:52.649](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=112.6491875): positive\_points is hidden from the screen — axes left the board.
- [01:52.649](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=112.6491875): positive\_points\_2 is hidden from the screen — axes left the board.
- [01:52.649](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=112.6491875): positive\_points\_3 is hidden from the screen — axes left the board.
- [01:52.649](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=112.6491875): positive\_points\_4 is hidden from the screen — axes left the board.
- [01:52.649](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=112.6491875): positive\_points\_5 is hidden from the screen — axes left the board.
- [01:52.649](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=112.6491875): negative\_points is hidden from the screen — axes left the board.
- [01:52.649](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=112.6491875): negative\_points\_2 is hidden from the screen — axes left the board.
- [01:52.649](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=112.6491875): negative\_points\_3 is hidden from the screen — axes left the board.
- [01:52.649](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=112.6491875): negative\_points\_4 is hidden from the screen — axes left the board.
- [01:52.649](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=112.6491875): negative\_points\_5 is hidden from the screen — axes left the board.
- [01:52.649](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=112.6491875): best\_separator is hidden from the screen — axes left the board.
- [01:52.649](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=112.6491875): corridor is hidden from the screen — axes left the board.
- [01:52.649](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=112.6491875): lower\_wall is hidden from the screen — axes left the board.
- [01:52.649](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=112.6491875): upper\_wall is hidden from the screen — axes left the board.
- [01:52.649](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=112.6491875): margin\_width is hidden from the screen — left the board.
- [01:52.649](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=112.6491875): question is hidden from the screen — left the board.
- [01:52.649](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=112.6491875): support\_definition is hidden from the screen — left the board.

### Scene 2: [From Geometry to Optimisation](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=113.69085416666667)

Span: 01:53.691–04:31.343 (113.69085416666667s–271.34333333333336s).

#### Objects

- axes: an Axes (x\_range=(0.0, 6.0), y\_range=(0.0, 4.8), aspect=(6.0, 4.8))
- constraint: a Math \[text\] that says "$y\_i (w dot x\_i + b) \>= 1, quad i = 1, ..., n$"
- decision: a Math \[text\] that says "$hat(y) = op("sign")(w dot x + b)$"
- distance: a Math \[text\] that says "$upright("distance") = frac(abs(w dot x + b), sqrt(w dot w))$"
- full\_margin: a Line \[magenta\] labelled "frac(2, sqrt(w dot w))" drawn in axes (start=(5.65, 1.0), end=(5.65, 3.0))
- hard\_margin: a Panel that says "Choose the smallest squared normal vector subject to every labelled point lying on or beyond its correct margin wall."
- heading\_problem: a Heading that says "The Hard-Margin Problem"
- heading\_score: a Heading that says "Score, Distance, and Normalization"
- lower\_wall: a Line \[gray\] labelled "-1" drawn in axes (start=(0.0, 1.0), end=(6.0, 1.0), dashed=True)
- margin: a Math \[text\] that says "$M = frac(2, sqrt(w dot w))$"
- negative\_points: a Point \[blue\] drawn in axes (location=(0.8, 0.5))
- negative\_points\_2: a Point \[blue\] drawn in axes (location=(2.0, 1.0))
- negative\_points\_3: a Point \[blue\] drawn in axes (location=(3.0, 0.4))
- negative\_points\_4: a Point \[blue\] drawn in axes (location=(4.0, 1.0))
- negative\_points\_5: a Point \[blue\] drawn in axes (location=(5.2, 0.5))
- normal: a Vector \[yellow\] labelled "w" drawn in axes (start=(3.0, 2.0), end=(3.0, 3.0))
- normalized\_constraint: a Math \[text\] that says "$y\_i (w dot x\_i + b) \>= 1$"
- objective: a Math \[text\] that says "$min\_(w,b) frac(1, 2) w dot w$"
- positive\_points: a Point \[red\] drawn in axes (location=(1.0, 3.0))
- positive\_points\_2: a Point \[red\] drawn in axes (location=(2.0, 4.0))
- positive\_points\_3: a Point \[red\] drawn in axes (location=(3.0, 3.0))
- positive\_points\_4: a Point \[red\] drawn in axes (location=(4.0, 4.2))
- positive\_points\_5: a Point \[red\] drawn in axes (location=(5.0, 3.0))
- score: a Math \[text\] that says "$s(x) = w dot x + b$"
- separator: a Line \[green\] labelled "w dot x + b = 0" drawn in axes (start=(0.0, 2.0), end=(6.0, 2.0))
- signed\_constraint: a Math \[text\] that says "$y\_i (w dot x\_i + b) \> 0$"
- upper\_wall: a Line \[gray\] labelled "1" drawn in axes (start=(0.0, 3.0), end=(6.0, 3.0), dashed=True)

#### Beats

##### [01:53.691](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=113.69085416666667)

Narration: A line in two dimensions, or a hyperplane in any dimension, can be written as w dot x plus b equals zero. The vector w is perpendicular to the separator, while b shifts it without changing its orientation.

Board: Empty.

Actions:
- [01:53.691](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=113.69085416666667): heading\_score is shown on the screen, written out.
- [01:53.888](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=113.88785416666667): axes is shown on the screen, written out.
- [01:54.504](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=114.50385416666667): positive\_points is shown on the screen, written out.
- [01:54.504](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=114.50385416666667): positive\_points\_2 is shown on the screen, written out.
- [01:54.504](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=114.50385416666667): positive\_points\_3 is shown on the screen, written out.
- [01:54.504](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=114.50385416666667): positive\_points\_4 is shown on the screen, written out.
- [01:54.504](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=114.50385416666667): positive\_points\_5 is shown on the screen, written out.
- [01:54.504](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=114.50385416666667): negative\_points is shown on the screen, written out.
- [01:54.504](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=114.50385416666667): negative\_points\_2 is shown on the screen, written out.
- [01:54.504](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=114.50385416666667): negative\_points\_3 is shown on the screen, written out.
- [01:54.504](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=114.50385416666667): negative\_points\_4 is shown on the screen, written out.
- [01:54.504](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=114.50385416666667): negative\_points\_5 is shown on the screen, written out.
- [02:0.158](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=120.15785416666667): separator is shown on the screen, written out.
- [02:2.851](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=122.85085416666666): normal is shown on the screen, written out.

##### [02:8.536](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=128.53635416666665)

Narration: The expression w dot x plus b is a signed score. Positive scores lie on one side, negative scores lie on the other, and its sign becomes the predicted class.

Board: axes — an Axes (x\_range=(0.0, 6.0), y\_range=(0.0, 4.8), aspect=(6.0, 4.8)); heading\_score — a Heading that says "Score, Distance, and Normalization"; positive\_points — a Point \[red\] drawn in axes (location=(1.0, 3.0)); positive\_points\_2 — a Point \[red\] drawn in axes (location=(2.0, 4.0)); positive\_points\_3 — a Point \[red\] drawn in axes (location=(3.0, 3.0)); positive\_points\_4 — a Point \[red\] drawn in axes (location=(4.0, 4.2)); positive\_points\_5 — a Point \[red\] drawn in axes (location=(5.0, 3.0)); negative\_points — a Point \[blue\] drawn in axes (location=(0.8, 0.5)); negative\_points\_2 — a Point \[blue\] drawn in axes (location=(2.0, 1.0)); negative\_points\_3 — a Point \[blue\] drawn in axes (location=(3.0, 0.4)); negative\_points\_4 — a Point \[blue\] drawn in axes (location=(4.0, 1.0)); negative\_points\_5 — a Point \[blue\] drawn in axes (location=(5.2, 0.5)); separator — a Line \[green\] labelled "w dot x + b = 0" drawn in axes (start=(0.0, 2.0), end=(6.0, 2.0)); normal — a Vector \[yellow\] labelled "w" drawn in axes (start=(3.0, 2.0), end=(3.0, 3.0))

Actions:
- [02:11.753](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=131.75285416666668): axes moves to a new place on the board.
- [02:11.753](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=131.75285416666668): score is shown on the screen, written out.
- [02:17.418](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=137.41785416666667): The segment (0.0, 2.0) to (6.0, 2.0) in axes is lit up.
- [02:18.591](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=138.59085416666667): decision is shown on the screen, written out.
- [02:20.831](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=140.83135416666667): axes: retire a lit segment (unemphasize\_line).

##### [02:21.431](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=141.43135416666667)

Narration: Let each training label y sub i be plus one for red and minus one for blue. Correct classification then has one compact condition: y sub i times the score must be positive.

Board: score — a Math \[text\] that says "$s(x) = w dot x + b$"; decision — a Math \[text\] that says "$hat(y) = op("sign")(w dot x + b)$"; axes — an Axes (x\_range=(0.0, 6.0), y\_range=(0.0, 4.8), aspect=(6.0, 4.8)); heading\_score — a Heading that says "Score, Distance, and Normalization"; positive\_points — a Point \[red\] drawn in axes (location=(1.0, 3.0)); positive\_points\_2 — a Point \[red\] drawn in axes (location=(2.0, 4.0)); positive\_points\_3 — a Point \[red\] drawn in axes (location=(3.0, 3.0)); positive\_points\_4 — a Point \[red\] drawn in axes (location=(4.0, 4.2)); positive\_points\_5 — a Point \[red\] drawn in axes (location=(5.0, 3.0)); negative\_points — a Point \[blue\] drawn in axes (location=(0.8, 0.5)); negative\_points\_2 — a Point \[blue\] drawn in axes (location=(2.0, 1.0)); negative\_points\_3 — a Point \[blue\] drawn in axes (location=(3.0, 0.4)); negative\_points\_4 — a Point \[blue\] drawn in axes (location=(4.0, 1.0)); negative\_points\_5 — a Point \[blue\] drawn in axes (location=(5.2, 0.5)); separator — a Line \[green\] labelled "w dot x + b = 0" drawn in axes (start=(0.0, 2.0), end=(6.0, 2.0)); normal — a Vector \[yellow\] labelled "w" drawn in axes (start=(3.0, 2.0), end=(3.0, 3.0))

Actions:
- [02:21.431](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=141.43135416666667): signed\_constraint is shown on the screen, written out.
- [02:31.637](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=151.63685416666667): signed\_constraint (the "y\_i" part) is emphasized.
- [02:34.435](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=154.43485416666667): signed\_constraint (the "w dot x\_i + b" part) is emphasized.
- [02:34.435](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=154.43485416666667): signed\_constraint (the "y\_i" part) is no longer emphasized.
- [02:36.188](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=156.18835416666667): signed\_constraint (the "w dot x\_i + b" part) is no longer emphasized.

##### [02:36.788](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=156.78835416666666)

Narration: There is a scaling ambiguity. Multiplying both w and b by any positive number leaves the zero set unchanged, so it leaves the separating hyperplane unchanged. We use that freedom to make the closest signed scores equal to one.

Board: score — a Math \[text\] that says "$s(x) = w dot x + b$"; decision — a Math \[text\] that says "$hat(y) = op("sign")(w dot x + b)$"; signed\_constraint — a Math \[text\] that says "$y\_i (w dot x\_i + b) \> 0$"; axes — an Axes (x\_range=(0.0, 6.0), y\_range=(0.0, 4.8), aspect=(6.0, 4.8)); heading\_score — a Heading that says "Score, Distance, and Normalization"; positive\_points — a Point \[red\] drawn in axes (location=(1.0, 3.0)); positive\_points\_2 — a Point \[red\] drawn in axes (location=(2.0, 4.0)); positive\_points\_3 — a Point \[red\] drawn in axes (location=(3.0, 3.0)); positive\_points\_4 — a Point \[red\] drawn in axes (location=(4.0, 4.2)); positive\_points\_5 — a Point \[red\] drawn in axes (location=(5.0, 3.0)); negative\_points — a Point \[blue\] drawn in axes (location=(0.8, 0.5)); negative\_points\_2 — a Point \[blue\] drawn in axes (location=(2.0, 1.0)); negative\_points\_3 — a Point \[blue\] drawn in axes (location=(3.0, 0.4)); negative\_points\_4 — a Point \[blue\] drawn in axes (location=(4.0, 1.0)); negative\_points\_5 — a Point \[blue\] drawn in axes (location=(5.2, 0.5)); separator — a Line \[green\] labelled "w dot x + b = 0" drawn in axes (start=(0.0, 2.0), end=(6.0, 2.0)); normal — a Vector \[yellow\] labelled "w" drawn in axes (start=(3.0, 2.0), end=(3.0, 3.0))

Actions:
- [02:51.452](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=171.45185416666666): normalized\_constraint is shown on the screen, written out.
- [02:51.452](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=171.45185416666666): upper\_wall is shown on the screen, written out.
- [02:51.452](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=171.45185416666666): lower\_wall is shown on the screen, written out.

##### [02:52.818](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=172.81835416666667)

Narration: After this normalization, every red point must have score at least plus one, and every blue point must have score at most minus one. The support vectors are precisely the points where the combined inequality becomes an equality.

Board: score — a Math \[text\] that says "$s(x) = w dot x + b$"; decision — a Math \[text\] that says "$hat(y) = op("sign")(w dot x + b)$"; signed\_constraint — a Math \[text\] that says "$y\_i (w dot x\_i + b) \> 0$"; normalized\_constraint — a Math \[text\] that says "$y\_i (w dot x\_i + b) \>= 1$"; axes — an Axes (x\_range=(0.0, 6.0), y\_range=(0.0, 4.8), aspect=(6.0, 4.8)); heading\_score — a Heading that says "Score, Distance, and Normalization"; positive\_points — a Point \[red\] drawn in axes (location=(1.0, 3.0)); positive\_points\_2 — a Point \[red\] drawn in axes (location=(2.0, 4.0)); positive\_points\_3 — a Point \[red\] drawn in axes (location=(3.0, 3.0)); positive\_points\_4 — a Point \[red\] drawn in axes (location=(4.0, 4.2)); positive\_points\_5 — a Point \[red\] drawn in axes (location=(5.0, 3.0)); negative\_points — a Point \[blue\] drawn in axes (location=(0.8, 0.5)); negative\_points\_2 — a Point \[blue\] drawn in axes (location=(2.0, 1.0)); negative\_points\_3 — a Point \[blue\] drawn in axes (location=(3.0, 0.4)); negative\_points\_4 — a Point \[blue\] drawn in axes (location=(4.0, 1.0)); negative\_points\_5 — a Point \[blue\] drawn in axes (location=(5.2, 0.5)); separator — a Line \[green\] labelled "w dot x + b = 0" drawn in axes (start=(0.0, 2.0), end=(6.0, 2.0)); normal — a Vector \[yellow\] labelled "w" drawn in axes (start=(3.0, 2.0), end=(3.0, 3.0)); upper\_wall — a Line \[gray\] labelled "1" drawn in axes (start=(0.0, 3.0), end=(6.0, 3.0), dashed=True); lower\_wall — a Line \[gray\] labelled "-1" drawn in axes (start=(0.0, 1.0), end=(6.0, 1.0), dashed=True)

Actions:
- [02:55.082](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=175.08185416666666): positive\_points is indicated — a transient flash.
- [02:55.082](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=175.08185416666666): positive\_points\_3 is indicated — a transient flash.
- [02:55.082](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=175.08185416666666): positive\_points\_5 is indicated — a transient flash.
- [02:56.301](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=176.30085416666668): normalized\_constraint (the "\>= 1" part) is emphasized.
- [02:58.124](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=178.12385416666666): negative\_points\_2 is indicated — a transient flash.
- [02:58.124](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=178.12385416666666): negative\_points\_4 is indicated — a transient flash.
- [03:6.785](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=186.78535416666665): normalized\_constraint (the "\>= 1" part) is no longer emphasized.

##### [03:7.385](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=187.38535416666667)

Narration: The perpendicular distance from a point x to the central hyperplane is the absolute score divided by the length of w. Each wall has score one in magnitude, so each wall is one over the length of w from the centre.

Board: Unchanged from the preceding beat in this scene.

Actions:
- [03:8.628](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=188.62785416666668): distance is shown on the screen, written out.
- [03:12.412](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=192.41185416666667): distance (the "abs(w dot x + b)" part) is emphasized.
- [03:13.306](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=193.30585416666668): distance (the "abs(w dot x + b)" part) is no longer emphasized.
- [03:13.306](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=193.30585416666668): distance (the "sqrt(w dot w)" part) is emphasized.
- [03:21.34](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=201.34035416666666): distance (the "sqrt(w dot w)" part) is no longer emphasized.

##### [03:21.94](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=201.94035416666668)

Narration: The full corridor therefore has width two over the length of w. To make that width large, we must make the length of w small. Squaring it changes neither the minimizing choice nor the geometry.

Board: score — a Math \[text\] that says "$s(x) = w dot x + b$"; decision — a Math \[text\] that says "$hat(y) = op("sign")(w dot x + b)$"; signed\_constraint — a Math \[text\] that says "$y\_i (w dot x\_i + b) \> 0$"; normalized\_constraint — a Math \[text\] that says "$y\_i (w dot x\_i + b) \>= 1$"; distance — a Math \[text\] that says "$upright("distance") = frac(abs(w dot x + b), sqrt(w dot w))$"; axes — an Axes (x\_range=(0.0, 6.0), y\_range=(0.0, 4.8), aspect=(6.0, 4.8)); heading\_score — a Heading that says "Score, Distance, and Normalization"; positive\_points — a Point \[red\] drawn in axes (location=(1.0, 3.0)); positive\_points\_2 — a Point \[red\] drawn in axes (location=(2.0, 4.0)); positive\_points\_3 — a Point \[red\] drawn in axes (location=(3.0, 3.0)); positive\_points\_4 — a Point \[red\] drawn in axes (location=(4.0, 4.2)); positive\_points\_5 — a Point \[red\] drawn in axes (location=(5.0, 3.0)); negative\_points — a Point \[blue\] drawn in axes (location=(0.8, 0.5)); negative\_points\_2 — a Point \[blue\] drawn in axes (location=(2.0, 1.0)); negative\_points\_3 — a Point \[blue\] drawn in axes (location=(3.0, 0.4)); negative\_points\_4 — a Point \[blue\] drawn in axes (location=(4.0, 1.0)); negative\_points\_5 — a Point \[blue\] drawn in axes (location=(5.2, 0.5)); separator — a Line \[green\] labelled "w dot x + b = 0" drawn in axes (start=(0.0, 2.0), end=(6.0, 2.0)); normal — a Vector \[yellow\] labelled "w" drawn in axes (start=(3.0, 2.0), end=(3.0, 3.0)); upper\_wall — a Line \[gray\] labelled "1" drawn in axes (start=(0.0, 3.0), end=(6.0, 3.0), dashed=True); lower\_wall — a Line \[gray\] labelled "-1" drawn in axes (start=(0.0, 1.0), end=(6.0, 1.0), dashed=True)

Actions:
- [03:23.914](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=203.91385416666668): margin is shown on the screen, written out.
- [03:24.17](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=204.16985416666668): full\_margin is shown on the screen, written out.
- [03:24.878](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=204.87785416666668): margin (the "sqrt(w dot w)" part) is emphasized.
- [03:35.28](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=215.27985416666667): margin moves to a new place on the board.
- [03:35.28](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=215.27985416666667): decision is hidden from the screen — left the board.
- [03:35.28](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=215.27985416666667): distance is hidden from the screen — left the board.
- [03:35.28](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=215.27985416666667): heading\_score is hidden from the screen — left the board.
- [03:35.28](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=215.27985416666667): normalized\_constraint is hidden from the screen — left the board.
- [03:35.28](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=215.27985416666667): score is hidden from the screen — left the board.
- [03:35.28](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=215.27985416666667): signed\_constraint is hidden from the screen — left the board.
- [03:35.28](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=215.27985416666667): margin (the "sqrt(w dot w)" part) is no longer emphasized.

##### [03:35.88](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=215.8798541666667)

Narration: We have reached the hard-margin optimization problem. Minimize one half w dot w, which is one half the squared length of the normal vector.

Board: margin — a Math \[text\] that says "$M = frac(2, sqrt(w dot w))$"; axes — an Axes (x\_range=(0.0, 6.0), y\_range=(0.0, 4.8), aspect=(6.0, 4.8)); positive\_points — a Point \[red\] drawn in axes (location=(1.0, 3.0)); positive\_points\_2 — a Point \[red\] drawn in axes (location=(2.0, 4.0)); positive\_points\_3 — a Point \[red\] drawn in axes (location=(3.0, 3.0)); positive\_points\_4 — a Point \[red\] drawn in axes (location=(4.0, 4.2)); positive\_points\_5 — a Point \[red\] drawn in axes (location=(5.0, 3.0)); negative\_points — a Point \[blue\] drawn in axes (location=(0.8, 0.5)); negative\_points\_2 — a Point \[blue\] drawn in axes (location=(2.0, 1.0)); negative\_points\_3 — a Point \[blue\] drawn in axes (location=(3.0, 0.4)); negative\_points\_4 — a Point \[blue\] drawn in axes (location=(4.0, 1.0)); negative\_points\_5 — a Point \[blue\] drawn in axes (location=(5.2, 0.5)); separator — a Line \[green\] labelled "w dot x + b = 0" drawn in axes (start=(0.0, 2.0), end=(6.0, 2.0)); normal — a Vector \[yellow\] labelled "w" drawn in axes (start=(3.0, 2.0), end=(3.0, 3.0)); upper\_wall — a Line \[gray\] labelled "1" drawn in axes (start=(0.0, 3.0), end=(6.0, 3.0), dashed=True); lower\_wall — a Line \[gray\] labelled "-1" drawn in axes (start=(0.0, 1.0), end=(6.0, 1.0), dashed=True); full\_margin — a Line \[magenta\] labelled "frac(2, sqrt(w dot w))" drawn in axes (start=(5.65, 1.0), end=(5.65, 3.0))

Actions:
- [03:35.88](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=215.8798541666667): heading\_problem is shown on the screen, written out.
- [03:39.978](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=219.9778541666667): objective is shown on the screen, written out.
- [03:43.566](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=223.56585416666667): objective (the "w dot w" part) is emphasized.
- [03:45.737](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=225.73685416666666): objective (the "w dot w" part) is no longer emphasized.

##### [03:46.337](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=226.33685416666668)

Narration: The minimization is subject to one constraint per training example: y sub i times w dot x sub i plus b must be at least one. These inequalities say that every point lies outside the empty corridor on its proper side.

Board: margin — a Math \[text\] that says "$M = frac(2, sqrt(w dot w))$"; axes — an Axes (x\_range=(0.0, 6.0), y\_range=(0.0, 4.8), aspect=(6.0, 4.8)); positive\_points — a Point \[red\] drawn in axes (location=(1.0, 3.0)); positive\_points\_2 — a Point \[red\] drawn in axes (location=(2.0, 4.0)); positive\_points\_3 — a Point \[red\] drawn in axes (location=(3.0, 3.0)); positive\_points\_4 — a Point \[red\] drawn in axes (location=(4.0, 4.2)); positive\_points\_5 — a Point \[red\] drawn in axes (location=(5.0, 3.0)); negative\_points — a Point \[blue\] drawn in axes (location=(0.8, 0.5)); negative\_points\_2 — a Point \[blue\] drawn in axes (location=(2.0, 1.0)); negative\_points\_3 — a Point \[blue\] drawn in axes (location=(3.0, 0.4)); negative\_points\_4 — a Point \[blue\] drawn in axes (location=(4.0, 1.0)); negative\_points\_5 — a Point \[blue\] drawn in axes (location=(5.2, 0.5)); separator — a Line \[green\] labelled "w dot x + b = 0" drawn in axes (start=(0.0, 2.0), end=(6.0, 2.0)); normal — a Vector \[yellow\] labelled "w" drawn in axes (start=(3.0, 2.0), end=(3.0, 3.0)); upper\_wall — a Line \[gray\] labelled "1" drawn in axes (start=(0.0, 3.0), end=(6.0, 3.0), dashed=True); lower\_wall — a Line \[gray\] labelled "-1" drawn in axes (start=(0.0, 1.0), end=(6.0, 1.0), dashed=True); full\_margin — a Line \[magenta\] labelled "frac(2, sqrt(w dot w))" drawn in axes (start=(5.65, 1.0), end=(5.65, 3.0)); objective — a Math \[text\] that says "$min\_(w,b) frac(1, 2) w dot w$"; heading\_problem — a Heading that says "The Hard-Margin Problem"

Actions:
- [03:48.647](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=228.64685416666669): constraint is shown on the screen, written out.
- [03:49.17](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=229.16985416666668): constraint (the "i = 1, ..., n" part) is emphasized.
- [03:57.192](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=237.19185416666667): constraint (the "i = 1, ..., n" part) is no longer emphasized.

##### [04:2.889](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=242.88885416666668)

Narration: Only a few of those constraints are tight at the solution. Those active constraints belong to the support vectors. The remaining points satisfy their inequalities with room to spare and do not pin the optimum.

Board: margin — a Math \[text\] that says "$M = frac(2, sqrt(w dot w))$"; axes — an Axes (x\_range=(0.0, 6.0), y\_range=(0.0, 4.8), aspect=(6.0, 4.8)); positive\_points — a Point \[red\] drawn in axes (location=(1.0, 3.0)); positive\_points\_2 — a Point \[red\] drawn in axes (location=(2.0, 4.0)); positive\_points\_3 — a Point \[red\] drawn in axes (location=(3.0, 3.0)); positive\_points\_4 — a Point \[red\] drawn in axes (location=(4.0, 4.2)); positive\_points\_5 — a Point \[red\] drawn in axes (location=(5.0, 3.0)); negative\_points — a Point \[blue\] drawn in axes (location=(0.8, 0.5)); negative\_points\_2 — a Point \[blue\] drawn in axes (location=(2.0, 1.0)); negative\_points\_3 — a Point \[blue\] drawn in axes (location=(3.0, 0.4)); negative\_points\_4 — a Point \[blue\] drawn in axes (location=(4.0, 1.0)); negative\_points\_5 — a Point \[blue\] drawn in axes (location=(5.2, 0.5)); separator — a Line \[green\] labelled "w dot x + b = 0" drawn in axes (start=(0.0, 2.0), end=(6.0, 2.0)); normal — a Vector \[yellow\] labelled "w" drawn in axes (start=(3.0, 2.0), end=(3.0, 3.0)); upper\_wall — a Line \[gray\] labelled "1" drawn in axes (start=(0.0, 3.0), end=(6.0, 3.0), dashed=True); lower\_wall — a Line \[gray\] labelled "-1" drawn in axes (start=(0.0, 1.0), end=(6.0, 1.0), dashed=True); full\_margin — a Line \[magenta\] labelled "frac(2, sqrt(w dot w))" drawn in axes (start=(5.65, 1.0), end=(5.65, 3.0)); objective — a Math \[text\] that says "$min\_(w,b) frac(1, 2) w dot w$"; constraint — a Math \[text\] that says "$y\_i (w dot x\_i + b) \>= 1, quad i = 1, ..., n$"; heading\_problem — a Heading that says "The Hard-Margin Problem"

Actions:
- [04:7.092](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=247.09185416666668): hard\_margin is shown on the screen, written out.
- [04:8.555](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=248.55485416666667): positive\_points is indicated — a transient flash.
- [04:8.555](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=248.55485416666667): positive\_points\_3 is indicated — a transient flash.
- [04:8.555](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=248.55485416666667): positive\_points\_5 is indicated — a transient flash.
- [04:8.555](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=248.55485416666667): negative\_points\_2 is indicated — a transient flash.
- [04:8.555](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=248.55485416666667): negative\_points\_4 is indicated — a transient flash.

##### [04:16.631](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=256.6313541666667)

Narration: This completes the geometric derivation. A wide corridor became a small normal vector, correct classification became a family of inequalities, and the points touching equality became the support vectors.

Board: margin — a Math \[text\] that says "$M = frac(2, sqrt(w dot w))$"; axes — an Axes (x\_range=(0.0, 6.0), y\_range=(0.0, 4.8), aspect=(6.0, 4.8)); positive\_points — a Point \[red\] drawn in axes (location=(1.0, 3.0)); positive\_points\_2 — a Point \[red\] drawn in axes (location=(2.0, 4.0)); positive\_points\_3 — a Point \[red\] drawn in axes (location=(3.0, 3.0)); positive\_points\_4 — a Point \[red\] drawn in axes (location=(4.0, 4.2)); positive\_points\_5 — a Point \[red\] drawn in axes (location=(5.0, 3.0)); negative\_points — a Point \[blue\] drawn in axes (location=(0.8, 0.5)); negative\_points\_2 — a Point \[blue\] drawn in axes (location=(2.0, 1.0)); negative\_points\_3 — a Point \[blue\] drawn in axes (location=(3.0, 0.4)); negative\_points\_4 — a Point \[blue\] drawn in axes (location=(4.0, 1.0)); negative\_points\_5 — a Point \[blue\] drawn in axes (location=(5.2, 0.5)); separator — a Line \[green\] labelled "w dot x + b = 0" drawn in axes (start=(0.0, 2.0), end=(6.0, 2.0)); normal — a Vector \[yellow\] labelled "w" drawn in axes (start=(3.0, 2.0), end=(3.0, 3.0)); upper\_wall — a Line \[gray\] labelled "1" drawn in axes (start=(0.0, 3.0), end=(6.0, 3.0), dashed=True); lower\_wall — a Line \[gray\] labelled "-1" drawn in axes (start=(0.0, 1.0), end=(6.0, 1.0), dashed=True); full\_margin — a Line \[magenta\] labelled "frac(2, sqrt(w dot w))" drawn in axes (start=(5.65, 1.0), end=(5.65, 3.0)); objective — a Math \[text\] that says "$min\_(w,b) frac(1, 2) w dot w$"; constraint — a Math \[text\] that says "$y\_i (w dot x\_i + b) \>= 1, quad i = 1, ..., n$"; hard\_margin — a Panel that says "Choose the smallest squared normal vector subject to every labelled point lying on or beyond its correct margin wall."; heading\_problem — a Heading that says "The Hard-Margin Problem"

Actions:
- [04:20.126](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=260.1258541666667): A box is drawn around margin.
- [04:21.519](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=261.51885416666676): objective is indicated — a transient flash.
- [04:25.722](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=265.72185416666673): constraint is indicated — a transient flash.
- [04:30.302](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=270.3016666666667): axes is hidden from the screen — left the board.
- [04:30.302](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=270.3016666666667): positive\_points is hidden from the screen — axes left the board.
- [04:30.302](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=270.3016666666667): positive\_points\_2 is hidden from the screen — axes left the board.
- [04:30.302](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=270.3016666666667): positive\_points\_3 is hidden from the screen — axes left the board.
- [04:30.302](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=270.3016666666667): positive\_points\_4 is hidden from the screen — axes left the board.
- [04:30.302](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=270.3016666666667): positive\_points\_5 is hidden from the screen — axes left the board.
- [04:30.302](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=270.3016666666667): negative\_points is hidden from the screen — axes left the board.
- [04:30.302](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=270.3016666666667): negative\_points\_2 is hidden from the screen — axes left the board.
- [04:30.302](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=270.3016666666667): negative\_points\_3 is hidden from the screen — axes left the board.
- [04:30.302](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=270.3016666666667): negative\_points\_4 is hidden from the screen — axes left the board.
- [04:30.302](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=270.3016666666667): negative\_points\_5 is hidden from the screen — axes left the board.
- [04:30.302](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=270.3016666666667): separator is hidden from the screen — axes left the board.
- [04:30.302](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=270.3016666666667): normal is hidden from the screen — axes left the board.
- [04:30.302](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=270.3016666666667): upper\_wall is hidden from the screen — axes left the board.
- [04:30.302](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=270.3016666666667): lower\_wall is hidden from the screen — axes left the board.
- [04:30.302](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=270.3016666666667): full\_margin is hidden from the screen — axes left the board.
- [04:30.302](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=270.3016666666667): constraint is hidden from the screen — left the board.
- [04:30.302](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=270.3016666666667): hard\_margin is hidden from the screen — left the board.
- [04:30.302](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=270.3016666666667): heading\_problem is hidden from the screen — left the board.
- [04:30.302](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=270.3016666666667): margin is hidden from the screen — left the board.
- [04:30.302](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=270.3016666666667): objective is hidden from the screen — left the board.

### Scene 3: [When the Classes Overlap](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=271.34333333333336)

Span: 04:31.343–07:19.784 (271.34333333333336s–439.78439583333335s).

#### Objects

- axes: an Axes (x\_range=(0.0, 6.0), y\_range=(0.0, 4.8), aspect=(6.0, 4.8))
- blue\_slack: a Line \[magenta\] labelled "xi\_j" drawn in axes (start=(1.5, 3.3), end=(1.5, 1.0), dashed=True)
- hard\_constraint: a Math \[text\] that says "$y\_i (w dot x\_i + b) \>= 1$"
- heading\_tradeoff: a Heading that says "What the Parameter $C$ Trades"
- high\_blue: a Point \[blue\] drawn in high\_c (location=(0.8, 0.5))
- high\_blue\_2: a Point \[blue\] drawn in high\_c (location=(1.5, 3.3))
- high\_blue\_3: a Point \[blue\] drawn in high\_c (location=(2.5, 1.0))
- high\_blue\_4: a Point \[blue\] drawn in high\_c (location=(4.0, 0.8))
- high\_blue\_5: a Point \[blue\] drawn in high\_c (location=(5.2, 1.2))
- high\_c: a Figure (x\_range=(0.0, 6.0), y\_range=(0.0, 4.8), aspect=(6.0, 4.8))
- high\_label: a Tex \[text\] that says "Larger $C$: violations cost more"
- high\_line: a Line \[green\] drawn in high\_c (start=(0.0, 1.45), end=(6.0, 2.55))
- high\_lower: a Line \[gray\] drawn in high\_c (start=(0.0, 0.45), end=(6.0, 1.55), dashed=True)
- high\_red: a Point \[red\] drawn in high\_c (location=(0.8, 3.4))
- high\_red\_2: a Point \[red\] drawn in high\_c (location=(2.0, 4.0))
- high\_red\_3: a Point \[red\] drawn in high\_c (location=(3.0, 3.0))
- high\_red\_4: a Point \[red\] drawn in high\_c (location=(4.2, 3.6))
- high\_red\_5: a Point \[red\] drawn in high\_c (location=(5.0, 0.7))
- high\_upper: a Line \[gray\] drawn in high\_c (start=(0.0, 2.45), end=(6.0, 3.55), dashed=True)
- interpretation: a Panel that says "Slack measures how far a training example falls inside its required margin wall. Values greater than one correspond to misclassification."
- low\_blue: a Point \[blue\] drawn in low\_c (location=(0.8, 0.5))
- low\_blue\_2: a Point \[blue\] drawn in low\_c (location=(1.5, 3.3))
- low\_blue\_3: a Point \[blue\] drawn in low\_c (location=(2.5, 1.0))
- low\_blue\_4: a Point \[blue\] drawn in low\_c (location=(4.0, 0.8))
- low\_blue\_5: a Point \[blue\] drawn in low\_c (location=(5.2, 1.2))
- low\_c: a Figure (x\_range=(0.0, 6.0), y\_range=(0.0, 4.8), aspect=(6.0, 4.8))
- low\_label: a Tex \[text\] that says "Smaller $C$: wider margin, more tolerance"
- low\_line: a Line \[green\] drawn in low\_c (start=(0.0, 2.0), end=(6.0, 2.0))
- low\_lower: a Line \[gray\] drawn in low\_c (start=(0.0, 1.0), end=(6.0, 1.0), dashed=True)
- low\_red: a Point \[red\] drawn in low\_c (location=(0.8, 3.4))
- low\_red\_2: a Point \[red\] drawn in low\_c (location=(2.0, 4.0))
- low\_red\_3: a Point \[red\] drawn in low\_c (location=(3.0, 3.0))
- low\_red\_4: a Point \[red\] drawn in low\_c (location=(4.2, 3.6))
- low\_red\_5: a Point \[red\] drawn in low\_c (location=(5.0, 0.7))
- low\_upper: a Line \[gray\] drawn in low\_c (start=(0.0, 3.0), end=(6.0, 3.0), dashed=True)
- lower\_wall: a Line \[gray\] labelled "-1" drawn in axes (start=(0.0, 1.0), end=(6.0, 1.0), dashed=True)
- negative\_points: a Point \[blue\] drawn in axes (location=(0.8, 0.5))
- negative\_points\_2: a Point \[blue\] drawn in axes (location=(1.5, 3.3))
- negative\_points\_3: a Point \[blue\] drawn in axes (location=(2.5, 1.0))
- negative\_points\_4: a Point \[blue\] drawn in axes (location=(4.0, 0.8))
- negative\_points\_5: a Point \[blue\] drawn in axes (location=(5.2, 1.2))
- positive\_points: a Point \[red\] drawn in axes (location=(0.8, 3.4))
- positive\_points\_2: a Point \[red\] drawn in axes (location=(2.0, 4.0))
- positive\_points\_3: a Point \[red\] drawn in axes (location=(3.0, 3.0))
- positive\_points\_4: a Point \[red\] drawn in axes (location=(4.2, 3.6))
- positive\_points\_5: a Point \[red\] drawn in axes (location=(5.0, 0.7))
- question: a Panel that says "What should the classifier do when no hyperplane separates every training point?"
- red\_slack: a Line \[magenta\] labelled "xi\_i" drawn in axes (start=(5.0, 0.7), end=(5.0, 3.0), dashed=True)
- separator: a Line \[green\] labelled "w dot x + b = 0" drawn in axes (start=(0.0, 2.0), end=(6.0, 2.0))
- soft\_constraint: a Math \[text\] that says "$y\_i (w dot x\_i + b) \>= 1 - xi\_i, quad xi\_i \>= 0$"
- soft\_objective: a Math \[text\] that says "$min\_(w,b,xi) frac(1, 2) w dot w + C sum\_i xi\_i$"
- upper\_wall: a Line \[gray\] labelled "1" drawn in axes (start=(0.0, 3.0), end=(6.0, 3.0), dashed=True)

#### Beats

##### [04:31.343](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=271.34333333333336)

Narration: Real classes often overlap. A red example may sit deep among blue examples, while a blue example may sit among red ones. No line can put every point on its requested side in this training set.

Board: Empty.

Actions:
- [04:31.343](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=271.34333333333336): question is shown on the screen, written out.
- [04:31.343](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=271.34333333333336): axes is shown on the screen, written out.
- [04:34.083](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=274.08333333333337): positive\_points is shown on the screen, written out.
- [04:34.083](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=274.08333333333337): positive\_points\_2 is shown on the screen, written out.
- [04:34.083](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=274.08333333333337): positive\_points\_3 is shown on the screen, written out.
- [04:34.083](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=274.08333333333337): positive\_points\_4 is shown on the screen, written out.
- [04:34.083](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=274.08333333333337): positive\_points\_5 is shown on the screen, written out.
- [04:35.917](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=275.9173333333334): negative\_points is shown on the screen, written out.
- [04:35.917](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=275.9173333333334): negative\_points\_2 is shown on the screen, written out.
- [04:35.917](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=275.9173333333334): negative\_points\_3 is shown on the screen, written out.
- [04:35.917](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=275.9173333333334): negative\_points\_4 is shown on the screen, written out.
- [04:35.917](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=275.9173333333334): negative\_points\_5 is shown on the screen, written out.

##### [04:45.411](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=285.41083333333336)

Narration: If we insist on the hard-margin constraints, the optimization problem has no feasible answer. The red point below and the blue point above cannot both satisfy the same perfect separation.

Board: axes — an Axes (x\_range=(0.0, 6.0), y\_range=(0.0, 4.8), aspect=(6.0, 4.8)); question — a Panel that says "What should the classifier do when no hyperplane separates every training point?"; positive\_points — a Point \[red\] drawn in axes (location=(0.8, 3.4)); positive\_points\_2 — a Point \[red\] drawn in axes (location=(2.0, 4.0)); positive\_points\_3 — a Point \[red\] drawn in axes (location=(3.0, 3.0)); positive\_points\_4 — a Point \[red\] drawn in axes (location=(4.2, 3.6)); positive\_points\_5 — a Point \[red\] drawn in axes (location=(5.0, 0.7)); negative\_points — a Point \[blue\] drawn in axes (location=(0.8, 0.5)); negative\_points\_2 — a Point \[blue\] drawn in axes (location=(1.5, 3.3)); negative\_points\_3 — a Point \[blue\] drawn in axes (location=(2.5, 1.0)); negative\_points\_4 — a Point \[blue\] drawn in axes (location=(4.0, 0.8)); negative\_points\_5 — a Point \[blue\] drawn in axes (location=(5.2, 1.2))

Actions:
- [04:47.791](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=287.79133333333334): axes moves to a new place on the board.
- [04:47.791](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=287.79133333333334): hard\_constraint is shown on the screen, written out.
- [04:52.377](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=292.37733333333335): positive\_points\_5 is indicated — a transient flash.
- [04:53.573](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=293.5733333333334): negative\_points\_2 is indicated — a transient flash.

##### [04:58.492](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=298.4918333333334)

Narration: A soft-margin SVM relaxes each constraint with a nonnegative slack variable xi sub i. Instead of demanding a signed score of at least one, it permits the score to fall short by xi sub i.

Board: hard\_constraint — a Math \[text\] that says "$y\_i (w dot x\_i + b) \>= 1$"; axes — an Axes (x\_range=(0.0, 6.0), y\_range=(0.0, 4.8), aspect=(6.0, 4.8)); question — a Panel that says "What should the classifier do when no hyperplane separates every training point?"; positive\_points — a Point \[red\] drawn in axes (location=(0.8, 3.4)); positive\_points\_2 — a Point \[red\] drawn in axes (location=(2.0, 4.0)); positive\_points\_3 — a Point \[red\] drawn in axes (location=(3.0, 3.0)); positive\_points\_4 — a Point \[red\] drawn in axes (location=(4.2, 3.6)); positive\_points\_5 — a Point \[red\] drawn in axes (location=(5.0, 0.7)); negative\_points — a Point \[blue\] drawn in axes (location=(0.8, 0.5)); negative\_points\_2 — a Point \[blue\] drawn in axes (location=(1.5, 3.3)); negative\_points\_3 — a Point \[blue\] drawn in axes (location=(2.5, 1.0)); negative\_points\_4 — a Point \[blue\] drawn in axes (location=(4.0, 0.8)); negative\_points\_5 — a Point \[blue\] drawn in axes (location=(5.2, 1.2))

Actions:
- [05:0.5](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=300.50033333333334): soft\_constraint is shown on the screen, written out.
- [05:3.112](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=303.11233333333337): soft\_constraint (the "xi\_i" part) is emphasized.
- [05:12.738](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=312.73783333333336): soft\_constraint (the "xi\_i" part) is no longer emphasized.

##### [05:13.338](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=313.3378333333334)

Narration: Draw the central separator and its two margin walls. A point outside its correct wall needs zero slack. A point inside the corridor needs some slack, because it has not reached the wall required by its label.

Board: hard\_constraint — a Math \[text\] that says "$y\_i (w dot x\_i + b) \>= 1$"; soft\_constraint — a Math \[text\] that says "$y\_i (w dot x\_i + b) \>= 1 - xi\_i, quad xi\_i \>= 0$"; axes — an Axes (x\_range=(0.0, 6.0), y\_range=(0.0, 4.8), aspect=(6.0, 4.8)); question — a Panel that says "What should the classifier do when no hyperplane separates every training point?"; positive\_points — a Point \[red\] drawn in axes (location=(0.8, 3.4)); positive\_points\_2 — a Point \[red\] drawn in axes (location=(2.0, 4.0)); positive\_points\_3 — a Point \[red\] drawn in axes (location=(3.0, 3.0)); positive\_points\_4 — a Point \[red\] drawn in axes (location=(4.2, 3.6)); positive\_points\_5 — a Point \[red\] drawn in axes (location=(5.0, 0.7)); negative\_points — a Point \[blue\] drawn in axes (location=(0.8, 0.5)); negative\_points\_2 — a Point \[blue\] drawn in axes (location=(1.5, 3.3)); negative\_points\_3 — a Point \[blue\] drawn in axes (location=(2.5, 1.0)); negative\_points\_4 — a Point \[blue\] drawn in axes (location=(4.0, 0.8)); negative\_points\_5 — a Point \[blue\] drawn in axes (location=(5.2, 1.2))

Actions:
- [05:14.464](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=314.46433333333334): separator is shown on the screen, written out.
- [05:16.066](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=316.0663333333334): lower\_wall is shown on the screen, written out.
- [05:16.066](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=316.0663333333334): upper\_wall is shown on the screen, written out.
- [05:21.894](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=321.89433333333335): interpretation is shown on the screen, written out.

##### [05:28.311](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=328.3113333333334)

Narration: This red point is not merely inside the margin. It lies on the blue side of the separator, so its slack exceeds one. The same is true in reverse for this blue point. Slack records the degree of each violation rather than pretending the data is perfectly separable.

Board: hard\_constraint — a Math \[text\] that says "$y\_i (w dot x\_i + b) \>= 1$"; soft\_constraint — a Math \[text\] that says "$y\_i (w dot x\_i + b) \>= 1 - xi\_i, quad xi\_i \>= 0$"; interpretation — a Panel that says "Slack measures how far a training example falls inside its required margin wall. Values greater than one correspond to misclassification."; axes — an Axes (x\_range=(0.0, 6.0), y\_range=(0.0, 4.8), aspect=(6.0, 4.8)); question — a Panel that says "What should the classifier do when no hyperplane separates every training point?"; positive\_points — a Point \[red\] drawn in axes (location=(0.8, 3.4)); positive\_points\_2 — a Point \[red\] drawn in axes (location=(2.0, 4.0)); positive\_points\_3 — a Point \[red\] drawn in axes (location=(3.0, 3.0)); positive\_points\_4 — a Point \[red\] drawn in axes (location=(4.2, 3.6)); positive\_points\_5 — a Point \[red\] drawn in axes (location=(5.0, 0.7)); negative\_points — a Point \[blue\] drawn in axes (location=(0.8, 0.5)); negative\_points\_2 — a Point \[blue\] drawn in axes (location=(1.5, 3.3)); negative\_points\_3 — a Point \[blue\] drawn in axes (location=(2.5, 1.0)); negative\_points\_4 — a Point \[blue\] drawn in axes (location=(4.0, 0.8)); negative\_points\_5 — a Point \[blue\] drawn in axes (location=(5.2, 1.2)); separator — a Line \[green\] labelled "w dot x + b = 0" drawn in axes (start=(0.0, 2.0), end=(6.0, 2.0)); lower\_wall — a Line \[gray\] labelled "-1" drawn in axes (start=(0.0, 1.0), end=(6.0, 1.0), dashed=True); upper\_wall — a Line \[gray\] labelled "1" drawn in axes (start=(0.0, 3.0), end=(6.0, 3.0), dashed=True)

Actions:
- [05:28.961](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=328.96133333333336): red\_slack is shown on the screen, written out.
- [05:34.859](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=334.8593333333334): interpretation (the "greater than one" part) is emphasized.
- [05:38.086](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=338.08633333333336): blue\_slack is shown on the screen, written out.
- [05:45.157](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=345.15683333333334): interpretation (the "greater than one" part) is no longer emphasized.

##### [05:45.757](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=345.75683333333336)

Narration: The new objective contains two costs. One half w dot w still rewards a wide margin. The sum of the slacks penalizes points that enter the margin or cross the separator, and the constant C controls how heavily that penalty is counted.

Board: hard\_constraint — a Math \[text\] that says "$y\_i (w dot x\_i + b) \>= 1$"; soft\_constraint — a Math \[text\] that says "$y\_i (w dot x\_i + b) \>= 1 - xi\_i, quad xi\_i \>= 0$"; interpretation — a Panel that says "Slack measures how far a training example falls inside its required margin wall. Values greater than one correspond to misclassification."; axes — an Axes (x\_range=(0.0, 6.0), y\_range=(0.0, 4.8), aspect=(6.0, 4.8)); question — a Panel that says "What should the classifier do when no hyperplane separates every training point?"; positive\_points — a Point \[red\] drawn in axes (location=(0.8, 3.4)); positive\_points\_2 — a Point \[red\] drawn in axes (location=(2.0, 4.0)); positive\_points\_3 — a Point \[red\] drawn in axes (location=(3.0, 3.0)); positive\_points\_4 — a Point \[red\] drawn in axes (location=(4.2, 3.6)); positive\_points\_5 — a Point \[red\] drawn in axes (location=(5.0, 0.7)); negative\_points — a Point \[blue\] drawn in axes (location=(0.8, 0.5)); negative\_points\_2 — a Point \[blue\] drawn in axes (location=(1.5, 3.3)); negative\_points\_3 — a Point \[blue\] drawn in axes (location=(2.5, 1.0)); negative\_points\_4 — a Point \[blue\] drawn in axes (location=(4.0, 0.8)); negative\_points\_5 — a Point \[blue\] drawn in axes (location=(5.2, 1.2)); separator — a Line \[green\] labelled "w dot x + b = 0" drawn in axes (start=(0.0, 2.0), end=(6.0, 2.0)); lower\_wall — a Line \[gray\] labelled "-1" drawn in axes (start=(0.0, 1.0), end=(6.0, 1.0), dashed=True); upper\_wall — a Line \[gray\] labelled "1" drawn in axes (start=(0.0, 3.0), end=(6.0, 3.0), dashed=True); red\_slack — a Line \[magenta\] labelled "xi\_i" drawn in axes (start=(5.0, 0.7), end=(5.0, 3.0), dashed=True); blue\_slack — a Line \[magenta\] labelled "xi\_j" drawn in axes (start=(1.5, 3.3), end=(1.5, 1.0), dashed=True)

Actions:
- [05:46.349](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=346.34933333333333): soft\_objective is shown on the screen, written out.
- [05:51.48](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=351.48033333333336): soft\_objective (the "frac(1, 2) w dot w" part) is emphasized.
- [05:54.441](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=354.4413333333334): soft\_objective (the "C sum\_i xi\_i" part) is emphasized.
- [05:54.441](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=354.4413333333334): soft\_objective (the "frac(1, 2) w dot w" part) is no longer emphasized.
- [06:2.219](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=362.21933333333334): soft\_objective (the "C sum\_i xi\_i" part) is no longer emphasized.

##### [06:2.819](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=362.81933333333336)

Narration: This is not permission to ignore errors for free. Every violation raises the objective. The optimizer accepts a violation only when avoiding it would require a sufficiently narrower or more contorted decision boundary.

Board: hard\_constraint — a Math \[text\] that says "$y\_i (w dot x\_i + b) \>= 1$"; soft\_constraint — a Math \[text\] that says "$y\_i (w dot x\_i + b) \>= 1 - xi\_i, quad xi\_i \>= 0$"; soft\_objective — a Math \[text\] that says "$min\_(w,b,xi) frac(1, 2) w dot w + C sum\_i xi\_i$"; interpretation — a Panel that says "Slack measures how far a training example falls inside its required margin wall. Values greater than one correspond to misclassification."; axes — an Axes (x\_range=(0.0, 6.0), y\_range=(0.0, 4.8), aspect=(6.0, 4.8)); question — a Panel that says "What should the classifier do when no hyperplane separates every training point?"; positive\_points — a Point \[red\] drawn in axes (location=(0.8, 3.4)); positive\_points\_2 — a Point \[red\] drawn in axes (location=(2.0, 4.0)); positive\_points\_3 — a Point \[red\] drawn in axes (location=(3.0, 3.0)); positive\_points\_4 — a Point \[red\] drawn in axes (location=(4.2, 3.6)); positive\_points\_5 — a Point \[red\] drawn in axes (location=(5.0, 0.7)); negative\_points — a Point \[blue\] drawn in axes (location=(0.8, 0.5)); negative\_points\_2 — a Point \[blue\] drawn in axes (location=(1.5, 3.3)); negative\_points\_3 — a Point \[blue\] drawn in axes (location=(2.5, 1.0)); negative\_points\_4 — a Point \[blue\] drawn in axes (location=(4.0, 0.8)); negative\_points\_5 — a Point \[blue\] drawn in axes (location=(5.2, 1.2)); separator — a Line \[green\] labelled "w dot x + b = 0" drawn in axes (start=(0.0, 2.0), end=(6.0, 2.0)); lower\_wall — a Line \[gray\] labelled "-1" drawn in axes (start=(0.0, 1.0), end=(6.0, 1.0), dashed=True); upper\_wall — a Line \[gray\] labelled "1" drawn in axes (start=(0.0, 3.0), end=(6.0, 3.0), dashed=True); red\_slack — a Line \[magenta\] labelled "xi\_i" drawn in axes (start=(5.0, 0.7), end=(5.0, 3.0), dashed=True); blue\_slack — a Line \[magenta\] labelled "xi\_j" drawn in axes (start=(1.5, 3.3), end=(1.5, 1.0), dashed=True)

Actions:
- [06:17.553](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=377.55283333333335): axes is hidden from the screen — left the board.
- [06:17.553](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=377.55283333333335): positive\_points is hidden from the screen — axes left the board.
- [06:17.553](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=377.55283333333335): positive\_points\_2 is hidden from the screen — axes left the board.
- [06:17.553](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=377.55283333333335): positive\_points\_3 is hidden from the screen — axes left the board.
- [06:17.553](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=377.55283333333335): positive\_points\_4 is hidden from the screen — axes left the board.
- [06:17.553](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=377.55283333333335): positive\_points\_5 is hidden from the screen — axes left the board.
- [06:17.553](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=377.55283333333335): negative\_points is hidden from the screen — axes left the board.
- [06:17.553](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=377.55283333333335): negative\_points\_2 is hidden from the screen — axes left the board.
- [06:17.553](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=377.55283333333335): negative\_points\_3 is hidden from the screen — axes left the board.
- [06:17.553](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=377.55283333333335): negative\_points\_4 is hidden from the screen — axes left the board.
- [06:17.553](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=377.55283333333335): negative\_points\_5 is hidden from the screen — axes left the board.
- [06:17.553](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=377.55283333333335): separator is hidden from the screen — axes left the board.
- [06:17.553](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=377.55283333333335): lower\_wall is hidden from the screen — axes left the board.
- [06:17.553](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=377.55283333333335): upper\_wall is hidden from the screen — axes left the board.
- [06:17.553](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=377.55283333333335): red\_slack is hidden from the screen — axes left the board.
- [06:17.553](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=377.55283333333335): blue\_slack is hidden from the screen — axes left the board.
- [06:17.553](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=377.55283333333335): hard\_constraint is hidden from the screen — left the board.
- [06:17.553](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=377.55283333333335): interpretation is hidden from the screen — left the board.
- [06:17.553](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=377.55283333333335): question is hidden from the screen — left the board.
- [06:17.553](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=377.55283333333335): soft\_constraint is hidden from the screen — left the board.
- [06:17.553](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=377.55283333333335): soft\_objective is hidden from the screen — left the board.

##### [06:18.153](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=378.1528333333334)

Narration: With a smaller C, slack is relatively inexpensive. The optimizer is more willing to tolerate unusual training points in exchange for a broad, stable corridor.

Board: Empty.

Actions:
- [06:18.153](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=378.1528333333334): heading\_tradeoff is shown on the screen, written out.
- [06:18.791](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=378.79133333333334): low\_c is shown on the screen, written out.
- [06:18.791](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=378.79133333333334): low\_label is shown on the screen, written out.
- [06:25.536](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=385.53633333333335): low\_red is shown on the screen, written out.
- [06:25.536](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=385.53633333333335): low\_red\_2 is shown on the screen, written out.
- [06:25.536](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=385.53633333333335): low\_red\_3 is shown on the screen, written out.
- [06:25.536](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=385.53633333333335): low\_red\_4 is shown on the screen, written out.
- [06:25.536](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=385.53633333333335): low\_red\_5 is shown on the screen, written out.
- [06:25.536](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=385.53633333333335): low\_blue is shown on the screen, written out.
- [06:25.536](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=385.53633333333335): low\_blue\_2 is shown on the screen, written out.
- [06:25.536](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=385.53633333333335): low\_blue\_3 is shown on the screen, written out.
- [06:25.536](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=385.53633333333335): low\_blue\_4 is shown on the screen, written out.
- [06:25.536](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=385.53633333333335): low\_blue\_5 is shown on the screen, written out.
- [06:28.613](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=388.61333333333334): low\_line is shown on the screen, written out.
- [06:28.613](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=388.61333333333334): low\_upper is shown on the screen, written out.
- [06:28.613](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=388.61333333333334): low\_lower is shown on the screen, written out.

##### [06:30.281](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=390.28133333333335)

Narration: With a larger C, violations are expensive. The separator bends its linear choice as far as it can toward fitting difficult examples, often leaving a narrower margin. The drawing is schematic, but the trade is exact: margin width against total slack.

Board: low\_label — a Tex \[text\] that says "Smaller $C$: wider margin, more tolerance"; low\_c — a Figure (x\_range=(0.0, 6.0), y\_range=(0.0, 4.8), aspect=(6.0, 4.8)); heading\_tradeoff — a Heading that says "What the Parameter $C$ Trades"; low\_red — a Point \[red\] drawn in low\_c (location=(0.8, 3.4)); low\_red\_2 — a Point \[red\] drawn in low\_c (location=(2.0, 4.0)); low\_red\_3 — a Point \[red\] drawn in low\_c (location=(3.0, 3.0)); low\_red\_4 — a Point \[red\] drawn in low\_c (location=(4.2, 3.6)); low\_red\_5 — a Point \[red\] drawn in low\_c (location=(5.0, 0.7)); low\_blue — a Point \[blue\] drawn in low\_c (location=(0.8, 0.5)); low\_blue\_2 — a Point \[blue\] drawn in low\_c (location=(1.5, 3.3)); low\_blue\_3 — a Point \[blue\] drawn in low\_c (location=(2.5, 1.0)); low\_blue\_4 — a Point \[blue\] drawn in low\_c (location=(4.0, 0.8)); low\_blue\_5 — a Point \[blue\] drawn in low\_c (location=(5.2, 1.2)); low\_line — a Line \[green\] drawn in low\_c (start=(0.0, 2.0), end=(6.0, 2.0)); low\_upper — a Line \[gray\] drawn in low\_c (start=(0.0, 3.0), end=(6.0, 3.0), dashed=True); low\_lower — a Line \[gray\] drawn in low\_c (start=(0.0, 1.0), end=(6.0, 1.0), dashed=True)

Actions:
- [06:31.047](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=391.04733333333337): high\_c is shown on the screen, written out.
- [06:31.047](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=391.04733333333337): high\_label is shown on the screen, written out.
- [06:34.229](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=394.22933333333333): high\_line is shown on the screen, written out.
- [06:34.229](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=394.22933333333333): high\_upper is shown on the screen, written out.
- [06:34.229](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=394.22933333333333): high\_lower is shown on the screen, written out.
- [06:38.141](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=398.14133333333336): high\_red is shown on the screen, written out.
- [06:38.141](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=398.14133333333336): high\_red\_2 is shown on the screen, written out.
- [06:38.141](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=398.14133333333336): high\_red\_3 is shown on the screen, written out.
- [06:38.141](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=398.14133333333336): high\_red\_4 is shown on the screen, written out.
- [06:38.141](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=398.14133333333336): high\_red\_5 is shown on the screen, written out.
- [06:38.141](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=398.14133333333336): high\_blue is shown on the screen, written out.
- [06:38.141](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=398.14133333333336): high\_blue\_2 is shown on the screen, written out.
- [06:38.141](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=398.14133333333336): high\_blue\_3 is shown on the screen, written out.
- [06:38.141](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=398.14133333333336): high\_blue\_4 is shown on the screen, written out.
- [06:38.141](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=398.14133333333336): high\_blue\_5 is shown on the screen, written out.

##### [06:48.157](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=408.15683333333334)

Narration: Small C can underreact to genuine structure. Very large C can overreact to noise or mislabeled examples. In practice C is a regularization hyperparameter, chosen by validation rather than by demanding zero training error.

Board: low\_label — a Tex \[text\] that says "Smaller $C$: wider margin, more tolerance"; low\_c — a Figure (x\_range=(0.0, 6.0), y\_range=(0.0, 4.8), aspect=(6.0, 4.8)); high\_label — a Tex \[text\] that says "Larger $C$: violations cost more"; high\_c — a Figure (x\_range=(0.0, 6.0), y\_range=(0.0, 4.8), aspect=(6.0, 4.8)); heading\_tradeoff — a Heading that says "What the Parameter $C$ Trades"; low\_red — a Point \[red\] drawn in low\_c (location=(0.8, 3.4)); low\_red\_2 — a Point \[red\] drawn in low\_c (location=(2.0, 4.0)); low\_red\_3 — a Point \[red\] drawn in low\_c (location=(3.0, 3.0)); low\_red\_4 — a Point \[red\] drawn in low\_c (location=(4.2, 3.6)); low\_red\_5 — a Point \[red\] drawn in low\_c (location=(5.0, 0.7)); low\_blue — a Point \[blue\] drawn in low\_c (location=(0.8, 0.5)); low\_blue\_2 — a Point \[blue\] drawn in low\_c (location=(1.5, 3.3)); low\_blue\_3 — a Point \[blue\] drawn in low\_c (location=(2.5, 1.0)); low\_blue\_4 — a Point \[blue\] drawn in low\_c (location=(4.0, 0.8)); low\_blue\_5 — a Point \[blue\] drawn in low\_c (location=(5.2, 1.2)); low\_line — a Line \[green\] drawn in low\_c (start=(0.0, 2.0), end=(6.0, 2.0)); low\_upper — a Line \[gray\] drawn in low\_c (start=(0.0, 3.0), end=(6.0, 3.0), dashed=True); low\_lower — a Line \[gray\] drawn in low\_c (start=(0.0, 1.0), end=(6.0, 1.0), dashed=True); high\_red — a Point \[red\] drawn in high\_c (location=(0.8, 3.4)); high\_red\_2 — a Point \[red\] drawn in high\_c (location=(2.0, 4.0)); high\_red\_3 — a Point \[red\] drawn in high\_c (location=(3.0, 3.0)); high\_red\_4 — a Point \[red\] drawn in high\_c (location=(4.2, 3.6)); high\_red\_5 — a Point \[red\] drawn in high\_c (location=(5.0, 0.7)); high\_blue — a Point \[blue\] drawn in high\_c (location=(0.8, 0.5)); high\_blue\_2 — a Point \[blue\] drawn in high\_c (location=(1.5, 3.3)); high\_blue\_3 — a Point \[blue\] drawn in high\_c (location=(2.5, 1.0)); high\_blue\_4 — a Point \[blue\] drawn in high\_c (location=(4.0, 0.8)); high\_blue\_5 — a Point \[blue\] drawn in high\_c (location=(5.2, 1.2)); high\_line — a Line \[green\] drawn in high\_c (start=(0.0, 1.45), end=(6.0, 2.55)); high\_upper — a Line \[gray\] drawn in high\_c (start=(0.0, 2.45), end=(6.0, 3.55), dashed=True); high\_lower — a Line \[gray\] drawn in high\_c (start=(0.0, 0.45), end=(6.0, 1.55), dashed=True)

Actions:
- [06:48.458](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=408.45833333333337): low\_line is indicated — a transient flash.
- [06:52.348](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=412.34833333333336): high\_line is indicated — a transient flash.

##### [07:4.918](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=424.91783333333336)

Narration: Support vectors still matter in the soft-margin model. They now include points touching a margin wall and points lying inside or beyond it. These are the examples that exert force on the final decision boundary.

Board: Unchanged from the preceding beat in this scene.

Actions:
- [07:9.864](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=429.8643333333334): low\_red\_3 is indicated — a transient flash.
- [07:9.864](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=429.8643333333334): low\_red\_5 is indicated — a transient flash.
- [07:9.864](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=429.8643333333334): low\_blue\_2 is indicated — a transient flash.
- [07:9.864](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=429.8643333333334): low\_blue\_3 is indicated — a transient flash.
- [07:18.743](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=438.74272916666666): heading\_tradeoff is hidden from the screen — left the board.
- [07:18.743](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=438.74272916666666): high\_c is hidden from the screen — left the board.
- [07:18.743](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=438.74272916666666): high\_red is hidden from the screen — high\_c left the board.
- [07:18.743](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=438.74272916666666): high\_red\_2 is hidden from the screen — high\_c left the board.
- [07:18.743](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=438.74272916666666): high\_red\_3 is hidden from the screen — high\_c left the board.
- [07:18.743](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=438.74272916666666): high\_red\_4 is hidden from the screen — high\_c left the board.
- [07:18.743](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=438.74272916666666): high\_red\_5 is hidden from the screen — high\_c left the board.
- [07:18.743](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=438.74272916666666): high\_blue is hidden from the screen — high\_c left the board.
- [07:18.743](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=438.74272916666666): high\_blue\_2 is hidden from the screen — high\_c left the board.
- [07:18.743](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=438.74272916666666): high\_blue\_3 is hidden from the screen — high\_c left the board.
- [07:18.743](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=438.74272916666666): high\_blue\_4 is hidden from the screen — high\_c left the board.
- [07:18.743](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=438.74272916666666): high\_blue\_5 is hidden from the screen — high\_c left the board.
- [07:18.743](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=438.74272916666666): high\_line is hidden from the screen — high\_c left the board.
- [07:18.743](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=438.74272916666666): high\_upper is hidden from the screen — high\_c left the board.
- [07:18.743](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=438.74272916666666): high\_lower is hidden from the screen — high\_c left the board.
- [07:18.743](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=438.74272916666666): high\_label is hidden from the screen — left the board.
- [07:18.743](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=438.74272916666666): low\_c is hidden from the screen — left the board.
- [07:18.743](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=438.74272916666666): low\_red is hidden from the screen — low\_c left the board.
- [07:18.743](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=438.74272916666666): low\_red\_2 is hidden from the screen — low\_c left the board.
- [07:18.743](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=438.74272916666666): low\_red\_3 is hidden from the screen — low\_c left the board.
- [07:18.743](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=438.74272916666666): low\_red\_4 is hidden from the screen — low\_c left the board.
- [07:18.743](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=438.74272916666666): low\_red\_5 is hidden from the screen — low\_c left the board.
- [07:18.743](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=438.74272916666666): low\_blue is hidden from the screen — low\_c left the board.
- [07:18.743](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=438.74272916666666): low\_blue\_2 is hidden from the screen — low\_c left the board.
- [07:18.743](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=438.74272916666666): low\_blue\_3 is hidden from the screen — low\_c left the board.
- [07:18.743](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=438.74272916666666): low\_blue\_4 is hidden from the screen — low\_c left the board.
- [07:18.743](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=438.74272916666666): low\_blue\_5 is hidden from the screen — low\_c left the board.
- [07:18.743](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=438.74272916666666): low\_line is hidden from the screen — low\_c left the board.
- [07:18.743](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=438.74272916666666): low\_upper is hidden from the screen — low\_c left the board.
- [07:18.743](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=438.74272916666666): low\_lower is hidden from the screen — low\_c left the board.
- [07:18.743](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=438.74272916666666): low\_label is hidden from the screen — left the board.

### Scene 4: [No Straight Line Will Do](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=439.78439583333335)

Span: 07:19.784–09:6.501 (439.78439583333335s–546.5005625s).

#### Objects

- axes: an Axes (x\_range=(-3.0, 3.0), y\_range=(-3.0, 3.0), aspect=(1.0, 1.0))
- boundary\_rule: a Math \[text\] that says "$q(x) = 2$"
- first\_try: a Line \[yellow\] drawn in axes (start=(-3.0, -0.8), end=(3.0, 0.8))
- inner\_points: a Point \[red\] drawn in axes (location=(0.6, 0.0))
- inner\_points\_2: a Point \[red\] drawn in axes (location=(-0.6, 0.0))
- inner\_points\_3: a Point \[red\] drawn in axes (location=(0.0, 0.6))
- inner\_points\_4: a Point \[red\] drawn in axes (location=(0.0, -0.6))
- inner\_rule: a Math \[text\] that says "$q(x) \< 2$"
- nonlinear\_boundary: a Circle \[green\] labelled "x\_1^2 + x\_2^2 = 2" drawn in axes (radius=1.4142135623730951)
- outer\_hull: a Polygon \[gray\] drawn in axes (vertices=((2.0, 0.0), (1.4, 1.4), (0.0, 2.0), (-1.4, 1.4), (-2.0, 0.0), …, fill\_opacity=0.12)
- outer\_points: a Point \[blue\] drawn in axes (location=(2.0, 0.0))
- outer\_points\_2: a Point \[blue\] drawn in axes (location=(1.4, 1.4))
- outer\_points\_3: a Point \[blue\] drawn in axes (location=(0.0, 2.0))
- outer\_points\_4: a Point \[blue\] drawn in axes (location=(-1.4, 1.4))
- outer\_points\_5: a Point \[blue\] drawn in axes (location=(-2.0, 0.0))
- outer\_points\_6: a Point \[blue\] drawn in axes (location=(-1.4, -1.4))
- outer\_points\_7: a Point \[blue\] drawn in axes (location=(0.0, -2.0))
- outer\_points\_8: a Point \[blue\] drawn in axes (location=(1.4, -1.4))
- outer\_rule: a Math \[text\] that says "$q(x) \> 2$"
- question: a Panel that says "Can one straight line separate the inner red cluster from the outer blue ring?"
- radius\_score: a Math \[text\] that says "$q(x) = x\_1^2 + x\_2^2$"
- second\_try: a Line \[yellow\] drawn in axes (start=(-1.1, -3.0), end=(1.1, 3.0))
- third\_try: a Line \[yellow\] drawn in axes (start=(-3.0, 1.25), end=(3.0, 1.25))

#### Beats

##### [07:19.784](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=439.78439583333335)

Narration: Now consider a different training set. The red examples form a compact inner cluster, while the blue examples make a ring around them. This is a simple nonlinear pattern with no noise and no overlap.

Board: Empty.

Actions:
- [07:19.784](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=439.78439583333335): question is shown on the screen, written out.
- [07:19.784](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=439.78439583333335): axes is shown on the screen, written out.
- [07:22.594](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=442.59439583333335): inner\_points is shown on the screen, written out.
- [07:22.594](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=442.59439583333335): inner\_points\_2 is shown on the screen, written out.
- [07:22.594](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=442.59439583333335): inner\_points\_3 is shown on the screen, written out.
- [07:22.594](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=442.59439583333335): inner\_points\_4 is shown on the screen, written out.
- [07:25.508](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=445.50839583333334): outer\_points is shown on the screen, written out.
- [07:25.508](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=445.50839583333334): outer\_points\_2 is shown on the screen, written out.
- [07:25.508](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=445.50839583333334): outer\_points\_3 is shown on the screen, written out.
- [07:25.508](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=445.50839583333334): outer\_points\_4 is shown on the screen, written out.
- [07:25.508](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=445.50839583333334): outer\_points\_5 is shown on the screen, written out.
- [07:25.508](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=445.50839583333334): outer\_points\_6 is shown on the screen, written out.
- [07:25.508](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=445.50839583333334): outer\_points\_7 is shown on the screen, written out.
- [07:25.508](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=445.50839583333334): outer\_points\_8 is shown on the screen, written out.

##### [07:32.865](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=452.86539583333337)

Narration: Try a slanted line. It can place some blue points opposite the red cluster, but blue points remain on both sides. Rotating the line merely changes which part of the ring is misclassified.

Board: axes — an Axes (x\_range=(-3.0, 3.0), y\_range=(-3.0, 3.0), aspect=(1.0, 1.0)); question — a Panel that says "Can one straight line separate the inner red cluster from the outer blue ring?"; inner\_points — a Point \[red\] drawn in axes (location=(0.6, 0.0)); inner\_points\_2 — a Point \[red\] drawn in axes (location=(-0.6, 0.0)); inner\_points\_3 — a Point \[red\] drawn in axes (location=(0.0, 0.6)); inner\_points\_4 — a Point \[red\] drawn in axes (location=(0.0, -0.6)); outer\_points — a Point \[blue\] drawn in axes (location=(2.0, 0.0)); outer\_points\_2 — a Point \[blue\] drawn in axes (location=(1.4, 1.4)); outer\_points\_3 — a Point \[blue\] drawn in axes (location=(0.0, 2.0)); outer\_points\_4 — a Point \[blue\] drawn in axes (location=(-1.4, 1.4)); outer\_points\_5 — a Point \[blue\] drawn in axes (location=(-2.0, 0.0)); outer\_points\_6 — a Point \[blue\] drawn in axes (location=(-1.4, -1.4)); outer\_points\_7 — a Point \[blue\] drawn in axes (location=(0.0, -2.0)); outer\_points\_8 — a Point \[blue\] drawn in axes (location=(1.4, -1.4))

Actions:
- [07:33.643](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=453.64339583333333): first\_try is shown on the screen, written out.
- [07:41.027](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=461.02739583333334): first\_try is hidden from the screen.
- [07:41.027](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=461.02739583333334): second\_try is shown on the screen, written out.

##### [07:45.748](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=465.74839583333335)

Narration: Move the line above the centre instead. The red points may all fall below it, but so do the blue points around the lower half of the ring. No translation fixes the problem either.

Board: axes — an Axes (x\_range=(-3.0, 3.0), y\_range=(-3.0, 3.0), aspect=(1.0, 1.0)); question — a Panel that says "Can one straight line separate the inner red cluster from the outer blue ring?"; inner\_points — a Point \[red\] drawn in axes (location=(0.6, 0.0)); inner\_points\_2 — a Point \[red\] drawn in axes (location=(-0.6, 0.0)); inner\_points\_3 — a Point \[red\] drawn in axes (location=(0.0, 0.6)); inner\_points\_4 — a Point \[red\] drawn in axes (location=(0.0, -0.6)); outer\_points — a Point \[blue\] drawn in axes (location=(2.0, 0.0)); outer\_points\_2 — a Point \[blue\] drawn in axes (location=(1.4, 1.4)); outer\_points\_3 — a Point \[blue\] drawn in axes (location=(0.0, 2.0)); outer\_points\_4 — a Point \[blue\] drawn in axes (location=(-1.4, 1.4)); outer\_points\_5 — a Point \[blue\] drawn in axes (location=(-2.0, 0.0)); outer\_points\_6 — a Point \[blue\] drawn in axes (location=(-1.4, -1.4)); outer\_points\_7 — a Point \[blue\] drawn in axes (location=(0.0, -2.0)); outer\_points\_8 — a Point \[blue\] drawn in axes (location=(1.4, -1.4)); second\_try — a Line \[yellow\] drawn in axes (start=(-1.1, -3.0), end=(1.1, 3.0))

Actions:
- [07:45.748](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=465.74839583333335): second\_try is hidden from the screen.
- [07:46.677](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=466.6773958333333): third\_try is shown on the screen, written out.
- [07:55.907](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=475.90739583333334): third\_try is hidden from the screen.

##### [07:57.308](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=477.30839583333335)

Narration: There is a geometric reason, not just a failure of imagination. Join the outer blue points to form their convex hull. The entire red cluster lies inside that hull.

Board: axes — an Axes (x\_range=(-3.0, 3.0), y\_range=(-3.0, 3.0), aspect=(1.0, 1.0)); question — a Panel that says "Can one straight line separate the inner red cluster from the outer blue ring?"; inner\_points — a Point \[red\] drawn in axes (location=(0.6, 0.0)); inner\_points\_2 — a Point \[red\] drawn in axes (location=(-0.6, 0.0)); inner\_points\_3 — a Point \[red\] drawn in axes (location=(0.0, 0.6)); inner\_points\_4 — a Point \[red\] drawn in axes (location=(0.0, -0.6)); outer\_points — a Point \[blue\] drawn in axes (location=(2.0, 0.0)); outer\_points\_2 — a Point \[blue\] drawn in axes (location=(1.4, 1.4)); outer\_points\_3 — a Point \[blue\] drawn in axes (location=(0.0, 2.0)); outer\_points\_4 — a Point \[blue\] drawn in axes (location=(-1.4, 1.4)); outer\_points\_5 — a Point \[blue\] drawn in axes (location=(-2.0, 0.0)); outer\_points\_6 — a Point \[blue\] drawn in axes (location=(-1.4, -1.4)); outer\_points\_7 — a Point \[blue\] drawn in axes (location=(0.0, -2.0)); outer\_points\_8 — a Point \[blue\] drawn in axes (location=(1.4, -1.4))

Actions:
- [08:3.647](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=483.64739583333335): outer\_hull is shown on the screen, drawn.
- [08:5.76](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=485.76039583333335): inner\_points is indicated — a transient flash.
- [08:5.76](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=485.76039583333335): inner\_points\_2 is indicated — a transient flash.
- [08:5.76](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=485.76039583333335): inner\_points\_3 is indicated — a transient flash.
- [08:5.76](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=485.76039583333335): inner\_points\_4 is indicated — a transient flash.

##### [08:8.647](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=488.64689583333336)

Narration: A straight line divides the plane into two half-planes. If every blue vertex lies in one half-plane, then every convex combination of those vertices lies there too. The red points inside their hull therefore cannot all occupy the opposite half-plane.

Board: axes — an Axes (x\_range=(-3.0, 3.0), y\_range=(-3.0, 3.0), aspect=(1.0, 1.0)); question — a Panel that says "Can one straight line separate the inner red cluster from the outer blue ring?"; inner\_points — a Point \[red\] drawn in axes (location=(0.6, 0.0)); inner\_points\_2 — a Point \[red\] drawn in axes (location=(-0.6, 0.0)); inner\_points\_3 — a Point \[red\] drawn in axes (location=(0.0, 0.6)); inner\_points\_4 — a Point \[red\] drawn in axes (location=(0.0, -0.6)); outer\_points — a Point \[blue\] drawn in axes (location=(2.0, 0.0)); outer\_points\_2 — a Point \[blue\] drawn in axes (location=(1.4, 1.4)); outer\_points\_3 — a Point \[blue\] drawn in axes (location=(0.0, 2.0)); outer\_points\_4 — a Point \[blue\] drawn in axes (location=(-1.4, 1.4)); outer\_points\_5 — a Point \[blue\] drawn in axes (location=(-2.0, 0.0)); outer\_points\_6 — a Point \[blue\] drawn in axes (location=(-1.4, -1.4)); outer\_points\_7 — a Point \[blue\] drawn in axes (location=(0.0, -2.0)); outer\_points\_8 — a Point \[blue\] drawn in axes (location=(1.4, -1.4)); outer\_hull — a Polygon \[gray\] drawn in axes (vertices=((2.0, 0.0), (1.4, 1.4), (0.0, 2.0), (-1.4, 1.4), (-2.0, 0.0), …, fill\_opacity=0.12)

Actions:
- [08:15.276](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=495.27639583333337): outer\_hull is indicated — a transient flash.

##### [08:24.015](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=504.01489583333336)

Narration: The useful feature here is not left versus right or up versus down. It is distance from the centre. Square that distance and call it q of x: x one squared plus x two squared.

Board: Unchanged from the preceding beat in this scene.

Actions:
- [08:33.268](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=513.2683958333333): axes moves to a new place on the board.
- [08:33.268](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=513.2683958333333): radius\_score is shown on the screen, written out.
- [08:35.624](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=515.6243958333333): radius\_score (the "x\_1^2 + x\_2^2" part) is emphasized.
- [08:38.237](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=518.2368958333334): radius\_score (the "x\_1^2 + x\_2^2" part) is no longer emphasized.

##### [08:38.837](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=518.8368958333333)

Narration: Every red point has a small value of q. Every blue point has a much larger value. A threshold at two separates them perfectly: red below two, blue above two.

Board: radius\_score — a Math \[text\] that says "$q(x) = x\_1^2 + x\_2^2$"; axes — an Axes (x\_range=(-3.0, 3.0), y\_range=(-3.0, 3.0), aspect=(1.0, 1.0)); question — a Panel that says "Can one straight line separate the inner red cluster from the outer blue ring?"; inner\_points — a Point \[red\] drawn in axes (location=(0.6, 0.0)); inner\_points\_2 — a Point \[red\] drawn in axes (location=(-0.6, 0.0)); inner\_points\_3 — a Point \[red\] drawn in axes (location=(0.0, 0.6)); inner\_points\_4 — a Point \[red\] drawn in axes (location=(0.0, -0.6)); outer\_points — a Point \[blue\] drawn in axes (location=(2.0, 0.0)); outer\_points\_2 — a Point \[blue\] drawn in axes (location=(1.4, 1.4)); outer\_points\_3 — a Point \[blue\] drawn in axes (location=(0.0, 2.0)); outer\_points\_4 — a Point \[blue\] drawn in axes (location=(-1.4, 1.4)); outer\_points\_5 — a Point \[blue\] drawn in axes (location=(-2.0, 0.0)); outer\_points\_6 — a Point \[blue\] drawn in axes (location=(-1.4, -1.4)); outer\_points\_7 — a Point \[blue\] drawn in axes (location=(0.0, -2.0)); outer\_points\_8 — a Point \[blue\] drawn in axes (location=(1.4, -1.4)); outer\_hull — a Polygon \[gray\] drawn in axes (vertices=((2.0, 0.0), (1.4, 1.4), (0.0, 2.0), (-1.4, 1.4), (-2.0, 0.0), …, fill\_opacity=0.12)

Actions:
- [08:39.545](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=519.5453958333334): inner\_rule is shown on the screen, written out.
- [08:40.311](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=520.3113958333333): inner\_points is indicated — a transient flash.
- [08:40.311](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=520.3113958333333): inner\_points\_2 is indicated — a transient flash.
- [08:40.311](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=520.3113958333333): inner\_points\_3 is indicated — a transient flash.
- [08:40.311](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=520.3113958333333): inner\_points\_4 is indicated — a transient flash.
- [08:42.621](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=522.6213958333334): outer\_rule is shown on the screen, written out.
- [08:43.666](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=523.6663958333334): outer\_points is indicated — a transient flash.
- [08:43.666](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=523.6663958333334): outer\_points\_2 is indicated — a transient flash.
- [08:43.666](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=523.6663958333334): outer\_points\_3 is indicated — a transient flash.
- [08:43.666](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=523.6663958333334): outer\_points\_4 is indicated — a transient flash.
- [08:43.666](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=523.6663958333334): outer\_points\_5 is indicated — a transient flash.
- [08:43.666](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=523.6663958333334): outer\_points\_6 is indicated — a transient flash.
- [08:43.666](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=523.6663958333334): outer\_points\_7 is indicated — a transient flash.
- [08:43.666](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=523.6663958333334): outer\_points\_8 is indicated — a transient flash.

##### [08:50.849](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=530.8493958333333)

Narration: Back in the original plane, the threshold q equals two is a circle, not a line. So the data is nonlinear in its original coordinates. The next move is to change coordinates rather than abandon maximum-margin separation.

Board: radius\_score — a Math \[text\] that says "$q(x) = x\_1^2 + x\_2^2$"; inner\_rule — a Math \[text\] that says "$q(x) \< 2$"; outer\_rule — a Math \[text\] that says "$q(x) \> 2$"; axes — an Axes (x\_range=(-3.0, 3.0), y\_range=(-3.0, 3.0), aspect=(1.0, 1.0)); question — a Panel that says "Can one straight line separate the inner red cluster from the outer blue ring?"; inner\_points — a Point \[red\] drawn in axes (location=(0.6, 0.0)); inner\_points\_2 — a Point \[red\] drawn in axes (location=(-0.6, 0.0)); inner\_points\_3 — a Point \[red\] drawn in axes (location=(0.0, 0.6)); inner\_points\_4 — a Point \[red\] drawn in axes (location=(0.0, -0.6)); outer\_points — a Point \[blue\] drawn in axes (location=(2.0, 0.0)); outer\_points\_2 — a Point \[blue\] drawn in axes (location=(1.4, 1.4)); outer\_points\_3 — a Point \[blue\] drawn in axes (location=(0.0, 2.0)); outer\_points\_4 — a Point \[blue\] drawn in axes (location=(-1.4, 1.4)); outer\_points\_5 — a Point \[blue\] drawn in axes (location=(-2.0, 0.0)); outer\_points\_6 — a Point \[blue\] drawn in axes (location=(-1.4, -1.4)); outer\_points\_7 — a Point \[blue\] drawn in axes (location=(0.0, -2.0)); outer\_points\_8 — a Point \[blue\] drawn in axes (location=(1.4, -1.4)); outer\_hull — a Polygon \[gray\] drawn in axes (vertices=((2.0, 0.0), (1.4, 1.4), (0.0, 2.0), (-1.4, 1.4), (-2.0, 0.0), …, fill\_opacity=0.12)

Actions:
- [08:53.833](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=533.8333958333334): boundary\_rule is shown on the screen, written out.
- [08:54.774](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=534.7743958333333): nonlinear\_boundary is shown on the screen, drawn.
- [08:57.548](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=537.5483958333334): nonlinear\_boundary is indicated — a transient flash.
- [09:5.459](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=545.4588958333334): axes is hidden from the screen — left the board.
- [09:5.459](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=545.4588958333334): inner\_points is hidden from the screen — axes left the board.
- [09:5.459](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=545.4588958333334): inner\_points\_2 is hidden from the screen — axes left the board.
- [09:5.459](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=545.4588958333334): inner\_points\_3 is hidden from the screen — axes left the board.
- [09:5.459](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=545.4588958333334): inner\_points\_4 is hidden from the screen — axes left the board.
- [09:5.459](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=545.4588958333334): outer\_points is hidden from the screen — axes left the board.
- [09:5.459](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=545.4588958333334): outer\_points\_2 is hidden from the screen — axes left the board.
- [09:5.459](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=545.4588958333334): outer\_points\_3 is hidden from the screen — axes left the board.
- [09:5.459](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=545.4588958333334): outer\_points\_4 is hidden from the screen — axes left the board.
- [09:5.459](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=545.4588958333334): outer\_points\_5 is hidden from the screen — axes left the board.
- [09:5.459](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=545.4588958333334): outer\_points\_6 is hidden from the screen — axes left the board.
- [09:5.459](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=545.4588958333334): outer\_points\_7 is hidden from the screen — axes left the board.
- [09:5.459](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=545.4588958333334): outer\_points\_8 is hidden from the screen — axes left the board.
- [09:5.459](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=545.4588958333334): outer\_hull is hidden from the screen — axes left the board.
- [09:5.459](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=545.4588958333334): nonlinear\_boundary is hidden from the screen — axes left the board.
- [09:5.459](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=545.4588958333334): boundary\_rule is hidden from the screen — left the board.
- [09:5.459](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=545.4588958333334): inner\_rule is hidden from the screen — left the board.
- [09:5.459](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=545.4588958333334): outer\_rule is hidden from the screen — left the board.
- [09:5.459](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=545.4588958333334): question is hidden from the screen — left the board.
- [09:5.459](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=545.4588958333334): radius\_score is hidden from the screen — left the board.

### Scene 5: [Lifting into Feature Space](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=546.5005625)

Span: 09:6.501–11:19.413 (546.5005625s–679.4129583333333s).

#### Objects

- circle: a Circle \[green\] labelled "x\_1^2 + x\_2^2 = 2" drawn in flat (radius=1.4142135623730951)
- feature\_map: a Math \[text\] that says "$phi(x) = (x\_1^2, thin sqrt(2) x\_1 x\_2, thin x\_2^2)$"
- flat: an Axes (x\_range=(-3.0, 3.0), y\_range=(-3.0, 3.0), aspect=(1.0, 1.0))
- flat\_inner: a Point \[red\] drawn in flat (location=(0.6, 0.0))
- flat\_inner\_2: a Point \[red\] drawn in flat (location=(-0.6, 0.0))
- flat\_inner\_3: a Point \[red\] drawn in flat (location=(0.0, 0.6))
- flat\_inner\_4: a Point \[red\] drawn in flat (location=(0.0, -0.6))
- flat\_outer: a Point \[blue\] drawn in flat (location=(2.0, 0.0))
- flat\_outer\_2: a Point \[blue\] drawn in flat (location=(1.4, 1.4))
- flat\_outer\_3: a Point \[blue\] drawn in flat (location=(0.0, 2.0))
- flat\_outer\_4: a Point \[blue\] drawn in flat (location=(-1.4, 1.4))
- flat\_outer\_5: a Point \[blue\] drawn in flat (location=(-2.0, 0.0))
- flat\_outer\_6: a Point \[blue\] drawn in flat (location=(-1.4, -1.4))
- flat\_outer\_7: a Point \[blue\] drawn in flat (location=(0.0, -2.0))
- flat\_outer\_8: a Point \[blue\] drawn in flat (location=(1.4, -1.4))
- heading\_lift: a Heading that says "Lift the Data into Three Features"
- heading\_return: a Heading that says "A Plane Above, a Curve Below"
- lift: a VariableNumber
- lifted\_points: a Point \[red\] drawn in space (location=((((1.0 - lift) \* 0.6) + ((lift \* 0.6) \* 0.6)), (((1.0 - lift) …)
- lifted\_points\_10: a Point \[blue\] drawn in space (location=((((1.0 - lift) \* -1.4) + ((lift \* -1.4) \* -1.4)), (((1.0 - lif…)
- lifted\_points\_11: a Point \[blue\] drawn in space (location=((((1.0 - lift) \* 0.0) + ((lift \* 0.0) \* 0.0)), (((1.0 - lift) …)
- lifted\_points\_12: a Point \[blue\] drawn in space (location=((((1.0 - lift) \* 1.4) + ((lift \* 1.4) \* 1.4)), (((1.0 - lift) …)
- lifted\_points\_2: a Point \[red\] drawn in space (location=((((1.0 - lift) \* -0.6) + ((lift \* -0.6) \* -0.6)), (((1.0 - lif…)
- lifted\_points\_3: a Point \[red\] drawn in space (location=((((1.0 - lift) \* 0.0) + ((lift \* 0.0) \* 0.0)), (((1.0 - lift) …)
- lifted\_points\_4: a Point \[red\] drawn in space (location=((((1.0 - lift) \* 0.0) + ((lift \* 0.0) \* 0.0)), (((1.0 - lift) …)
- lifted\_points\_5: a Point \[blue\] drawn in space (location=((((1.0 - lift) \* 2.0) + ((lift \* 2.0) \* 2.0)), (((1.0 - lift) …)
- lifted\_points\_6: a Point \[blue\] drawn in space (location=((((1.0 - lift) \* 1.4) + ((lift \* 1.4) \* 1.4)), (((1.0 - lift) …)
- lifted\_points\_7: a Point \[blue\] drawn in space (location=((((1.0 - lift) \* 0.0) + ((lift \* 0.0) \* 0.0)), (((1.0 - lift) …)
- lifted\_points\_8: a Point \[blue\] drawn in space (location=((((1.0 - lift) \* -1.4) + ((lift \* -1.4) \* -1.4)), (((1.0 - lif…)
- lifted\_points\_9: a Point \[blue\] drawn in space (location=((((1.0 - lift) \* -2.0) + ((lift \* -2.0) \* -2.0)), (((1.0 - lif…)
- plane\_rule: a Math \[text\] that says "$phi\_1 + phi\_3 = 2$"
- same\_rule: a Math \[text\] that says "$x\_1^2 + x\_2^2 = 2$"
- separating\_plane: a Plane \[green\] labelled "phi\_1 + phi\_3 = 2" drawn in space (point=(1.0, 0.0, 1.0), normal=(1.0, 0.0, 1.0), size=5.0)
- space: an Axes3D (x\_range=(-2.6, 4.4), y\_range=(-3.2, 3.2), z\_range=(-0.6, 4.4))
- tex: a Tex \[text\] that says "Original input space"
- tex\_2: a Tex \[text\] that says "Quadratic feature space"

#### Beats

##### [09:6.501](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=546.5005625)

Narration: A feature map replaces each original input by a new vector. For this problem, map x one and x two to three quadratic features: x one squared, square root two times x one x two, and x two squared.

Board: Empty.

Actions:
- [09:6.501](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=546.5005625): heading\_lift is shown on the screen, written out.
- [09:6.501](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=546.5005625): space is shown on the screen, written out.
- [09:6.687](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=546.6865625): feature\_map is shown on the screen, written out.
- [09:8.777](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=548.7765625): lifted\_points is shown on the screen, written out.
- [09:8.777](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=548.7765625): lifted\_points\_2 is shown on the screen, written out.
- [09:8.777](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=548.7765625): lifted\_points\_3 is shown on the screen, written out.
- [09:8.777](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=548.7765625): lifted\_points\_4 is shown on the screen, written out.
- [09:8.777](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=548.7765625): lifted\_points\_5 is shown on the screen, written out.
- [09:8.777](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=548.7765625): lifted\_points\_6 is shown on the screen, written out.
- [09:8.777](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=548.7765625): lifted\_points\_7 is shown on the screen, written out.
- [09:8.777](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=548.7765625): lifted\_points\_8 is shown on the screen, written out.
- [09:8.777](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=548.7765625): lifted\_points\_9 is shown on the screen, written out.
- [09:8.777](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=548.7765625): lifted\_points\_10 is shown on the screen, written out.
- [09:8.777](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=548.7765625): lifted\_points\_11 is shown on the screen, written out.
- [09:8.777](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=548.7765625): lifted\_points\_12 is shown on the screen, written out.

##### [09:22.983](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=562.9825625)

Narration: At first I have placed the points in a flat copy of their familiar arrangement. Now apply the feature map. Each point moves to the three coordinates computed from its original two.

Board: feature\_map — a Math \[text\] that says "$phi(x) = (x\_1^2, thin sqrt(2) x\_1 x\_2, thin x\_2^2)$"; space — an Axes3D (x\_range=(-2.6, 4.4), y\_range=(-3.2, 3.2), z\_range=(-0.6, 4.4)); heading\_lift — a Heading that says "Lift the Data into Three Features"; lifted\_points — a Point \[red\] drawn in space (location=((((1.0 - lift) \* 0.6) + ((lift \* 0.6) \* 0.6)), (((1.0 - lift) …); lifted\_points\_2 — a Point \[red\] drawn in space (location=((((1.0 - lift) \* -0.6) + ((lift \* -0.6) \* -0.6)), (((1.0 - lif…); lifted\_points\_3 — a Point \[red\] drawn in space (location=((((1.0 - lift) \* 0.0) + ((lift \* 0.0) \* 0.0)), (((1.0 - lift) …); lifted\_points\_4 — a Point \[red\] drawn in space (location=((((1.0 - lift) \* 0.0) + ((lift \* 0.0) \* 0.0)), (((1.0 - lift) …); lifted\_points\_5 — a Point \[blue\] drawn in space (location=((((1.0 - lift) \* 2.0) + ((lift \* 2.0) \* 2.0)), (((1.0 - lift) …); lifted\_points\_6 — a Point \[blue\] drawn in space (location=((((1.0 - lift) \* 1.4) + ((lift \* 1.4) \* 1.4)), (((1.0 - lift) …); lifted\_points\_7 — a Point \[blue\] drawn in space (location=((((1.0 - lift) \* 0.0) + ((lift \* 0.0) \* 0.0)), (((1.0 - lift) …); lifted\_points\_8 — a Point \[blue\] drawn in space (location=((((1.0 - lift) \* -1.4) + ((lift \* -1.4) \* -1.4)), (((1.0 - lif…); lifted\_points\_9 — a Point \[blue\] drawn in space (location=((((1.0 - lift) \* -2.0) + ((lift \* -2.0) \* -2.0)), (((1.0 - lif…); lifted\_points\_10 — a Point \[blue\] drawn in space (location=((((1.0 - lift) \* -1.4) + ((lift \* -1.4) \* -1.4)), (((1.0 - lif…); lifted\_points\_11 — a Point \[blue\] drawn in space (location=((((1.0 - lift) \* 0.0) + ((lift \* 0.0) \* 0.0)), (((1.0 - lift) …); lifted\_points\_12 — a Point \[blue\] drawn in space (location=((((1.0 - lift) \* 1.4) + ((lift \* 1.4) \* 1.4)), (((1.0 - lift) …)

Actions:
- [09:28.917](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=568.9165625): lifted\_points is redrawn as the numbers it depends on change.
- [09:28.917](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=568.9165625): lifted\_points\_2 is redrawn as the numbers it depends on change.
- [09:28.917](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=568.9165625): lifted\_points\_3 is redrawn as the numbers it depends on change.
- [09:28.917](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=568.9165625): lifted\_points\_4 is redrawn as the numbers it depends on change.
- [09:28.917](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=568.9165625): lifted\_points\_5 is redrawn as the numbers it depends on change.
- [09:28.917](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=568.9165625): lifted\_points\_6 is redrawn as the numbers it depends on change.
- [09:28.917](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=568.9165625): lifted\_points\_7 is redrawn as the numbers it depends on change.
- [09:28.917](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=568.9165625): lifted\_points\_8 is redrawn as the numbers it depends on change.
- [09:28.917](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=568.9165625): lifted\_points\_9 is redrawn as the numbers it depends on change.
- [09:28.917](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=568.9165625): lifted\_points\_10 is redrawn as the numbers it depends on change.
- [09:28.917](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=568.9165625): lifted\_points\_11 is redrawn as the numbers it depends on change.
- [09:28.917](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=568.9165625): lifted\_points\_12 is redrawn as the numbers it depends on change.
- [09:28.917](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=568.9165625): lift ticks to 1.0.

##### [09:35.171](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=575.1705625)

Narration: The inner red points remain near the origin of feature space. The outer blue points rise onto a distant sheet because x one squared plus x two squared is large for every point on the ring.

Board: Unchanged from the preceding beat in this scene.

Actions:
- [09:35.972](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=575.9715625): lifted\_points is indicated — a transient flash.
- [09:35.972](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=575.9715625): lifted\_points\_2 is indicated — a transient flash.
- [09:35.972](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=575.9715625): lifted\_points\_3 is indicated — a transient flash.
- [09:35.972](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=575.9715625): lifted\_points\_4 is indicated — a transient flash.
- [09:37.992](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=577.9915625): space turns in its own slot.
- [09:40.105](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=580.1045625): lifted\_points\_5 is indicated — a transient flash.
- [09:40.105](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=580.1045625): lifted\_points\_6 is indicated — a transient flash.
- [09:40.105](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=580.1045625): lifted\_points\_7 is indicated — a transient flash.
- [09:40.105](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=580.1045625): lifted\_points\_8 is indicated — a transient flash.
- [09:40.105](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=580.1045625): lifted\_points\_9 is indicated — a transient flash.
- [09:40.105](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=580.1045625): lifted\_points\_10 is indicated — a transient flash.
- [09:40.105](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=580.1045625): lifted\_points\_11 is indicated — a transient flash.
- [09:40.105](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=580.1045625): lifted\_points\_12 is indicated — a transient flash.

##### [09:48.669](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=588.6690625)

Narration: In this three-dimensional space, one flat plane separates the classes. Its equation is feature one plus feature three equals two. Red points lie below it, and blue points lie above it.

Board: Unchanged from the preceding beat in this scene.

Actions:
- [09:51.595](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=591.5945624999999): separating\_plane is shown on the screen, written out.
- [09:54.103](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=594.1025625): plane\_rule is shown on the screen, written out.
- [09:58.526](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=598.5255625): lifted\_points is indicated — a transient flash.
- [09:58.526](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=598.5255625): lifted\_points\_2 is indicated — a transient flash.
- [09:58.526](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=598.5255625): lifted\_points\_3 is indicated — a transient flash.
- [09:58.526](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=598.5255625): lifted\_points\_4 is indicated — a transient flash.
- [10:0.442](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=600.4415625): lifted\_points\_5 is indicated — a transient flash.
- [10:0.442](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=600.4415625): lifted\_points\_6 is indicated — a transient flash.
- [10:0.442](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=600.4415625): lifted\_points\_7 is indicated — a transient flash.
- [10:0.442](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=600.4415625): lifted\_points\_8 is indicated — a transient flash.
- [10:0.442](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=600.4415625): lifted\_points\_9 is indicated — a transient flash.
- [10:0.442](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=600.4415625): lifted\_points\_10 is indicated — a transient flash.
- [10:0.442](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=600.4415625): lifted\_points\_11 is indicated — a transient flash.
- [10:0.442](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=600.4415625): lifted\_points\_12 is indicated — a transient flash.

##### [10:2.806](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=602.8060625)

Narration: This is ordinary linear separation again. We can place parallel margin planes around this separator, maximize their distance, and obtain support vectors exactly as before. The only change is that the vectors now live in feature space.

Board: feature\_map — a Math \[text\] that says "$phi(x) = (x\_1^2, thin sqrt(2) x\_1 x\_2, thin x\_2^2)$"; plane\_rule — a Math \[text\] that says "$phi\_1 + phi\_3 = 2$"; space — an Axes3D (x\_range=(-2.6, 4.4), y\_range=(-3.2, 3.2), z\_range=(-0.6, 4.4)); heading\_lift — a Heading that says "Lift the Data into Three Features"; lifted\_points — a Point \[red\] drawn in space (location=((((1.0 - lift) \* 0.6) + ((lift \* 0.6) \* 0.6)), (((1.0 - lift) …); lifted\_points\_2 — a Point \[red\] drawn in space (location=((((1.0 - lift) \* -0.6) + ((lift \* -0.6) \* -0.6)), (((1.0 - lif…); lifted\_points\_3 — a Point \[red\] drawn in space (location=((((1.0 - lift) \* 0.0) + ((lift \* 0.0) \* 0.0)), (((1.0 - lift) …); lifted\_points\_4 — a Point \[red\] drawn in space (location=((((1.0 - lift) \* 0.0) + ((lift \* 0.0) \* 0.0)), (((1.0 - lift) …); lifted\_points\_5 — a Point \[blue\] drawn in space (location=((((1.0 - lift) \* 2.0) + ((lift \* 2.0) \* 2.0)), (((1.0 - lift) …); lifted\_points\_6 — a Point \[blue\] drawn in space (location=((((1.0 - lift) \* 1.4) + ((lift \* 1.4) \* 1.4)), (((1.0 - lift) …); lifted\_points\_7 — a Point \[blue\] drawn in space (location=((((1.0 - lift) \* 0.0) + ((lift \* 0.0) \* 0.0)), (((1.0 - lift) …); lifted\_points\_8 — a Point \[blue\] drawn in space (location=((((1.0 - lift) \* -1.4) + ((lift \* -1.4) \* -1.4)), (((1.0 - lif…); lifted\_points\_9 — a Point \[blue\] drawn in space (location=((((1.0 - lift) \* -2.0) + ((lift \* -2.0) \* -2.0)), (((1.0 - lif…); lifted\_points\_10 — a Point \[blue\] drawn in space (location=((((1.0 - lift) \* -1.4) + ((lift \* -1.4) \* -1.4)), (((1.0 - lif…); lifted\_points\_11 — a Point \[blue\] drawn in space (location=((((1.0 - lift) \* 0.0) + ((lift \* 0.0) \* 0.0)), (((1.0 - lift) …); lifted\_points\_12 — a Point \[blue\] drawn in space (location=((((1.0 - lift) \* 1.4) + ((lift \* 1.4) \* 1.4)), (((1.0 - lift) …); separating\_plane — a Plane \[green\] labelled "phi\_1 + phi\_3 = 2" drawn in space (point=(1.0, 0.0, 1.0), normal=(1.0, 0.0, 1.0), size=5.0)

Actions:
- [10:4.235](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=604.2345625): separating\_plane is indicated — a transient flash.
- [10:18.237](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=618.2365625): space moves to a new place on the board.
- [10:18.237](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=618.2365625): feature\_map is hidden from the screen — left the board.
- [10:18.237](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=618.2365625): heading\_lift is hidden from the screen — left the board.
- [10:18.237](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=618.2365625): plane\_rule is hidden from the screen — left the board.

##### [10:18.837](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=618.8365625)

Narration: Now compare the two spaces. On the left is the original input plane. On the right is the lifted feature space with its flat separating plane.

Board: space — an Axes3D (x\_range=(-2.6, 4.4), y\_range=(-3.2, 3.2), z\_range=(-0.6, 4.4)); lifted\_points — a Point \[red\] drawn in space (location=((((1.0 - lift) \* 0.6) + ((lift \* 0.6) \* 0.6)), (((1.0 - lift) …); lifted\_points\_2 — a Point \[red\] drawn in space (location=((((1.0 - lift) \* -0.6) + ((lift \* -0.6) \* -0.6)), (((1.0 - lif…); lifted\_points\_3 — a Point \[red\] drawn in space (location=((((1.0 - lift) \* 0.0) + ((lift \* 0.0) \* 0.0)), (((1.0 - lift) …); lifted\_points\_4 — a Point \[red\] drawn in space (location=((((1.0 - lift) \* 0.0) + ((lift \* 0.0) \* 0.0)), (((1.0 - lift) …); lifted\_points\_5 — a Point \[blue\] drawn in space (location=((((1.0 - lift) \* 2.0) + ((lift \* 2.0) \* 2.0)), (((1.0 - lift) …); lifted\_points\_6 — a Point \[blue\] drawn in space (location=((((1.0 - lift) \* 1.4) + ((lift \* 1.4) \* 1.4)), (((1.0 - lift) …); lifted\_points\_7 — a Point \[blue\] drawn in space (location=((((1.0 - lift) \* 0.0) + ((lift \* 0.0) \* 0.0)), (((1.0 - lift) …); lifted\_points\_8 — a Point \[blue\] drawn in space (location=((((1.0 - lift) \* -1.4) + ((lift \* -1.4) \* -1.4)), (((1.0 - lif…); lifted\_points\_9 — a Point \[blue\] drawn in space (location=((((1.0 - lift) \* -2.0) + ((lift \* -2.0) \* -2.0)), (((1.0 - lif…); lifted\_points\_10 — a Point \[blue\] drawn in space (location=((((1.0 - lift) \* -1.4) + ((lift \* -1.4) \* -1.4)), (((1.0 - lif…); lifted\_points\_11 — a Point \[blue\] drawn in space (location=((((1.0 - lift) \* 0.0) + ((lift \* 0.0) \* 0.0)), (((1.0 - lift) …); lifted\_points\_12 — a Point \[blue\] drawn in space (location=((((1.0 - lift) \* 1.4) + ((lift \* 1.4) \* 1.4)), (((1.0 - lift) …); separating\_plane — a Plane \[green\] labelled "phi\_1 + phi\_3 = 2" drawn in space (point=(1.0, 0.0, 1.0), normal=(1.0, 0.0, 1.0), size=5.0)

Actions:
- [10:18.837](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=618.8365625): heading\_return is shown on the screen, written out.
- [10:21.879](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=621.8785625): flat is shown on the screen, written out.
- [10:22.401](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=622.4005625): flat\_inner is shown on the screen, written out.
- [10:22.401](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=622.4005625): flat\_inner\_2 is shown on the screen, written out.
- [10:22.401](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=622.4005625): flat\_inner\_3 is shown on the screen, written out.
- [10:22.401](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=622.4005625): flat\_inner\_4 is shown on the screen, written out.
- [10:22.401](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=622.4005625): flat\_outer is shown on the screen, written out.
- [10:22.401](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=622.4005625): flat\_outer\_2 is shown on the screen, written out.
- [10:22.401](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=622.4005625): flat\_outer\_3 is shown on the screen, written out.
- [10:22.401](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=622.4005625): flat\_outer\_4 is shown on the screen, written out.
- [10:22.401](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=622.4005625): flat\_outer\_5 is shown on the screen, written out.
- [10:22.401](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=622.4005625): flat\_outer\_6 is shown on the screen, written out.
- [10:22.401](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=622.4005625): flat\_outer\_7 is shown on the screen, written out.
- [10:22.401](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=622.4005625): flat\_outer\_8 is shown on the screen, written out.

##### [10:28.736](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=628.7360625)

Narration: The plane above is feature one plus feature three equals two. Substitute the definitions of those features and the same condition becomes x one squared plus x two squared equals two below.

Board: space — an Axes3D (x\_range=(-2.6, 4.4), y\_range=(-3.2, 3.2), z\_range=(-0.6, 4.4)); lifted\_points — a Point \[red\] drawn in space (location=((((1.0 - lift) \* 0.6) + ((lift \* 0.6) \* 0.6)), (((1.0 - lift) …); lifted\_points\_2 — a Point \[red\] drawn in space (location=((((1.0 - lift) \* -0.6) + ((lift \* -0.6) \* -0.6)), (((1.0 - lif…); lifted\_points\_3 — a Point \[red\] drawn in space (location=((((1.0 - lift) \* 0.0) + ((lift \* 0.0) \* 0.0)), (((1.0 - lift) …); lifted\_points\_4 — a Point \[red\] drawn in space (location=((((1.0 - lift) \* 0.0) + ((lift \* 0.0) \* 0.0)), (((1.0 - lift) …); lifted\_points\_5 — a Point \[blue\] drawn in space (location=((((1.0 - lift) \* 2.0) + ((lift \* 2.0) \* 2.0)), (((1.0 - lift) …); lifted\_points\_6 — a Point \[blue\] drawn in space (location=((((1.0 - lift) \* 1.4) + ((lift \* 1.4) \* 1.4)), (((1.0 - lift) …); lifted\_points\_7 — a Point \[blue\] drawn in space (location=((((1.0 - lift) \* 0.0) + ((lift \* 0.0) \* 0.0)), (((1.0 - lift) …); lifted\_points\_8 — a Point \[blue\] drawn in space (location=((((1.0 - lift) \* -1.4) + ((lift \* -1.4) \* -1.4)), (((1.0 - lif…); lifted\_points\_9 — a Point \[blue\] drawn in space (location=((((1.0 - lift) \* -2.0) + ((lift \* -2.0) \* -2.0)), (((1.0 - lif…); lifted\_points\_10 — a Point \[blue\] drawn in space (location=((((1.0 - lift) \* -1.4) + ((lift \* -1.4) \* -1.4)), (((1.0 - lif…); lifted\_points\_11 — a Point \[blue\] drawn in space (location=((((1.0 - lift) \* 0.0) + ((lift \* 0.0) \* 0.0)), (((1.0 - lift) …); lifted\_points\_12 — a Point \[blue\] drawn in space (location=((((1.0 - lift) \* 1.4) + ((lift \* 1.4) \* 1.4)), (((1.0 - lift) …); separating\_plane — a Plane \[green\] labelled "phi\_1 + phi\_3 = 2" drawn in space (point=(1.0, 0.0, 1.0), normal=(1.0, 0.0, 1.0), size=5.0); flat — an Axes (x\_range=(-3.0, 3.0), y\_range=(-3.0, 3.0), aspect=(1.0, 1.0)); heading\_return — a Heading that says "A Plane Above, a Curve Below"; flat\_inner — a Point \[red\] drawn in flat (location=(0.6, 0.0)); flat\_inner\_2 — a Point \[red\] drawn in flat (location=(-0.6, 0.0)); flat\_inner\_3 — a Point \[red\] drawn in flat (location=(0.0, 0.6)); flat\_inner\_4 — a Point \[red\] drawn in flat (location=(0.0, -0.6)); flat\_outer — a Point \[blue\] drawn in flat (location=(2.0, 0.0)); flat\_outer\_2 — a Point \[blue\] drawn in flat (location=(1.4, 1.4)); flat\_outer\_3 — a Point \[blue\] drawn in flat (location=(0.0, 2.0)); flat\_outer\_4 — a Point \[blue\] drawn in flat (location=(-1.4, 1.4)); flat\_outer\_5 — a Point \[blue\] drawn in flat (location=(-2.0, 0.0)); flat\_outer\_6 — a Point \[blue\] drawn in flat (location=(-1.4, -1.4)); flat\_outer\_7 — a Point \[blue\] drawn in flat (location=(0.0, -2.0)); flat\_outer\_8 — a Point \[blue\] drawn in flat (location=(1.4, -1.4))

Actions:
- [10:33.299](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=633.2985625): same\_rule is shown on the screen, written out.
- [10:37.362](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=637.3615625): same\_rule (the "x\_1^2 + x\_2^2" part) is emphasized.
- [10:40.903](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=640.9030625): same\_rule (the "x\_1^2 + x\_2^2" part) is no longer emphasized.

##### [10:41.503](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=641.5030624999999)

Narration: That equation draws this circle in the input plane. A linear decision surface in feature space has become a nonlinear decision boundary in the original coordinates.

Board: space — an Axes3D (x\_range=(-2.6, 4.4), y\_range=(-3.2, 3.2), z\_range=(-0.6, 4.4)); lifted\_points — a Point \[red\] drawn in space (location=((((1.0 - lift) \* 0.6) + ((lift \* 0.6) \* 0.6)), (((1.0 - lift) …); lifted\_points\_2 — a Point \[red\] drawn in space (location=((((1.0 - lift) \* -0.6) + ((lift \* -0.6) \* -0.6)), (((1.0 - lif…); lifted\_points\_3 — a Point \[red\] drawn in space (location=((((1.0 - lift) \* 0.0) + ((lift \* 0.0) \* 0.0)), (((1.0 - lift) …); lifted\_points\_4 — a Point \[red\] drawn in space (location=((((1.0 - lift) \* 0.0) + ((lift \* 0.0) \* 0.0)), (((1.0 - lift) …); lifted\_points\_5 — a Point \[blue\] drawn in space (location=((((1.0 - lift) \* 2.0) + ((lift \* 2.0) \* 2.0)), (((1.0 - lift) …); lifted\_points\_6 — a Point \[blue\] drawn in space (location=((((1.0 - lift) \* 1.4) + ((lift \* 1.4) \* 1.4)), (((1.0 - lift) …); lifted\_points\_7 — a Point \[blue\] drawn in space (location=((((1.0 - lift) \* 0.0) + ((lift \* 0.0) \* 0.0)), (((1.0 - lift) …); lifted\_points\_8 — a Point \[blue\] drawn in space (location=((((1.0 - lift) \* -1.4) + ((lift \* -1.4) \* -1.4)), (((1.0 - lif…); lifted\_points\_9 — a Point \[blue\] drawn in space (location=((((1.0 - lift) \* -2.0) + ((lift \* -2.0) \* -2.0)), (((1.0 - lif…); lifted\_points\_10 — a Point \[blue\] drawn in space (location=((((1.0 - lift) \* -1.4) + ((lift \* -1.4) \* -1.4)), (((1.0 - lif…); lifted\_points\_11 — a Point \[blue\] drawn in space (location=((((1.0 - lift) \* 0.0) + ((lift \* 0.0) \* 0.0)), (((1.0 - lift) …); lifted\_points\_12 — a Point \[blue\] drawn in space (location=((((1.0 - lift) \* 1.4) + ((lift \* 1.4) \* 1.4)), (((1.0 - lift) …); separating\_plane — a Plane \[green\] labelled "phi\_1 + phi\_3 = 2" drawn in space (point=(1.0, 0.0, 1.0), normal=(1.0, 0.0, 1.0), size=5.0); flat — an Axes (x\_range=(-3.0, 3.0), y\_range=(-3.0, 3.0), aspect=(1.0, 1.0)); same\_rule — a Math \[text\] that says "$x\_1^2 + x\_2^2 = 2$"; heading\_return — a Heading that says "A Plane Above, a Curve Below"; flat\_inner — a Point \[red\] drawn in flat (location=(0.6, 0.0)); flat\_inner\_2 — a Point \[red\] drawn in flat (location=(-0.6, 0.0)); flat\_inner\_3 — a Point \[red\] drawn in flat (location=(0.0, 0.6)); flat\_inner\_4 — a Point \[red\] drawn in flat (location=(0.0, -0.6)); flat\_outer — a Point \[blue\] drawn in flat (location=(2.0, 0.0)); flat\_outer\_2 — a Point \[blue\] drawn in flat (location=(1.4, 1.4)); flat\_outer\_3 — a Point \[blue\] drawn in flat (location=(0.0, 2.0)); flat\_outer\_4 — a Point \[blue\] drawn in flat (location=(-1.4, 1.4)); flat\_outer\_5 — a Point \[blue\] drawn in flat (location=(-2.0, 0.0)); flat\_outer\_6 — a Point \[blue\] drawn in flat (location=(-1.4, -1.4)); flat\_outer\_7 — a Point \[blue\] drawn in flat (location=(0.0, -2.0)); flat\_outer\_8 — a Point \[blue\] drawn in flat (location=(1.4, -1.4))

Actions:
- [10:43.129](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=643.1285625): circle is shown on the screen, drawn.
- [10:48.597](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=648.5965625): circle is indicated — a transient flash.

##### [10:52.563](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=652.5630625)

Narration: The construction works, but it seems expensive. A richer feature map may contain hundreds, millions, or even infinitely many coordinates. Writing and storing every lifted vector would defeat the purpose.

Board: space — an Axes3D (x\_range=(-2.6, 4.4), y\_range=(-3.2, 3.2), z\_range=(-0.6, 4.4)); lifted\_points — a Point \[red\] drawn in space (location=((((1.0 - lift) \* 0.6) + ((lift \* 0.6) \* 0.6)), (((1.0 - lift) …); lifted\_points\_2 — a Point \[red\] drawn in space (location=((((1.0 - lift) \* -0.6) + ((lift \* -0.6) \* -0.6)), (((1.0 - lif…); lifted\_points\_3 — a Point \[red\] drawn in space (location=((((1.0 - lift) \* 0.0) + ((lift \* 0.0) \* 0.0)), (((1.0 - lift) …); lifted\_points\_4 — a Point \[red\] drawn in space (location=((((1.0 - lift) \* 0.0) + ((lift \* 0.0) \* 0.0)), (((1.0 - lift) …); lifted\_points\_5 — a Point \[blue\] drawn in space (location=((((1.0 - lift) \* 2.0) + ((lift \* 2.0) \* 2.0)), (((1.0 - lift) …); lifted\_points\_6 — a Point \[blue\] drawn in space (location=((((1.0 - lift) \* 1.4) + ((lift \* 1.4) \* 1.4)), (((1.0 - lift) …); lifted\_points\_7 — a Point \[blue\] drawn in space (location=((((1.0 - lift) \* 0.0) + ((lift \* 0.0) \* 0.0)), (((1.0 - lift) …); lifted\_points\_8 — a Point \[blue\] drawn in space (location=((((1.0 - lift) \* -1.4) + ((lift \* -1.4) \* -1.4)), (((1.0 - lif…); lifted\_points\_9 — a Point \[blue\] drawn in space (location=((((1.0 - lift) \* -2.0) + ((lift \* -2.0) \* -2.0)), (((1.0 - lif…); lifted\_points\_10 — a Point \[blue\] drawn in space (location=((((1.0 - lift) \* -1.4) + ((lift \* -1.4) \* -1.4)), (((1.0 - lif…); lifted\_points\_11 — a Point \[blue\] drawn in space (location=((((1.0 - lift) \* 0.0) + ((lift \* 0.0) \* 0.0)), (((1.0 - lift) …); lifted\_points\_12 — a Point \[blue\] drawn in space (location=((((1.0 - lift) \* 1.4) + ((lift \* 1.4) \* 1.4)), (((1.0 - lift) …); separating\_plane — a Plane \[green\] labelled "phi\_1 + phi\_3 = 2" drawn in space (point=(1.0, 0.0, 1.0), normal=(1.0, 0.0, 1.0), size=5.0); flat — an Axes (x\_range=(-3.0, 3.0), y\_range=(-3.0, 3.0), aspect=(1.0, 1.0)); same\_rule — a Math \[text\] that says "$x\_1^2 + x\_2^2 = 2$"; heading\_return — a Heading that says "A Plane Above, a Curve Below"; flat\_inner — a Point \[red\] drawn in flat (location=(0.6, 0.0)); flat\_inner\_2 — a Point \[red\] drawn in flat (location=(-0.6, 0.0)); flat\_inner\_3 — a Point \[red\] drawn in flat (location=(0.0, 0.6)); flat\_inner\_4 — a Point \[red\] drawn in flat (location=(0.0, -0.6)); flat\_outer — a Point \[blue\] drawn in flat (location=(2.0, 0.0)); flat\_outer\_2 — a Point \[blue\] drawn in flat (location=(1.4, 1.4)); flat\_outer\_3 — a Point \[blue\] drawn in flat (location=(0.0, 2.0)); flat\_outer\_4 — a Point \[blue\] drawn in flat (location=(-1.4, 1.4)); flat\_outer\_5 — a Point \[blue\] drawn in flat (location=(-2.0, 0.0)); flat\_outer\_6 — a Point \[blue\] drawn in flat (location=(-1.4, -1.4)); flat\_outer\_7 — a Point \[blue\] drawn in flat (location=(0.0, -2.0)); flat\_outer\_8 — a Point \[blue\] drawn in flat (location=(1.4, -1.4)); circle — a Circle \[green\] labelled "x\_1^2 + x\_2^2 = 2" drawn in flat (radius=1.4142135623730951)

Actions:
- None.

##### [11:6.422](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=666.4220625)

Narration: The escape comes from noticing what the SVM calculation actually asks of those lifted vectors. It does not need to inspect their coordinates one at a time. It needs inner products between them.

Board: Unchanged from the preceding beat in this scene.

Actions:
- [11:11.089](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=671.0885625): lifted\_points is indicated — a transient flash.
- [11:11.089](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=671.0885625): lifted\_points\_2 is indicated — a transient flash.
- [11:11.089](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=671.0885625): lifted\_points\_3 is indicated — a transient flash.
- [11:11.089](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=671.0885625): lifted\_points\_4 is indicated — a transient flash.
- [11:11.089](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=671.0885625): lifted\_points\_5 is indicated — a transient flash.
- [11:11.089](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=671.0885625): lifted\_points\_6 is indicated — a transient flash.
- [11:11.089](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=671.0885625): lifted\_points\_7 is indicated — a transient flash.
- [11:11.089](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=671.0885625): lifted\_points\_8 is indicated — a transient flash.
- [11:11.089](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=671.0885625): lifted\_points\_9 is indicated — a transient flash.
- [11:11.089](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=671.0885625): lifted\_points\_10 is indicated — a transient flash.
- [11:11.089](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=671.0885625): lifted\_points\_11 is indicated — a transient flash.
- [11:11.089](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=671.0885625): lifted\_points\_12 is indicated — a transient flash.
- [11:18.371](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=678.3712916666666): flat is hidden from the screen — left the board.
- [11:18.371](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=678.3712916666666): flat\_inner is hidden from the screen — flat left the board.
- [11:18.371](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=678.3712916666666): flat\_inner\_2 is hidden from the screen — flat left the board.
- [11:18.371](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=678.3712916666666): flat\_inner\_3 is hidden from the screen — flat left the board.
- [11:18.371](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=678.3712916666666): flat\_inner\_4 is hidden from the screen — flat left the board.
- [11:18.371](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=678.3712916666666): flat\_outer is hidden from the screen — flat left the board.
- [11:18.371](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=678.3712916666666): flat\_outer\_2 is hidden from the screen — flat left the board.
- [11:18.371](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=678.3712916666666): flat\_outer\_3 is hidden from the screen — flat left the board.
- [11:18.371](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=678.3712916666666): flat\_outer\_4 is hidden from the screen — flat left the board.
- [11:18.371](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=678.3712916666666): flat\_outer\_5 is hidden from the screen — flat left the board.
- [11:18.371](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=678.3712916666666): flat\_outer\_6 is hidden from the screen — flat left the board.
- [11:18.371](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=678.3712916666666): flat\_outer\_7 is hidden from the screen — flat left the board.
- [11:18.371](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=678.3712916666666): flat\_outer\_8 is hidden from the screen — flat left the board.
- [11:18.371](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=678.3712916666666): circle is hidden from the screen — flat left the board.
- [11:18.371](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=678.3712916666666): heading\_return is hidden from the screen — left the board.
- [11:18.371](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=678.3712916666666): same\_rule is hidden from the screen — left the board.
- [11:18.371](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=678.3712916666666): space is hidden from the screen — left the board.
- [11:18.371](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=678.3712916666666): lifted\_points is hidden from the screen — space left the board.
- [11:18.371](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=678.3712916666666): lifted\_points\_2 is hidden from the screen — space left the board.
- [11:18.371](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=678.3712916666666): lifted\_points\_3 is hidden from the screen — space left the board.
- [11:18.371](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=678.3712916666666): lifted\_points\_4 is hidden from the screen — space left the board.
- [11:18.371](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=678.3712916666666): lifted\_points\_5 is hidden from the screen — space left the board.
- [11:18.371](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=678.3712916666666): lifted\_points\_6 is hidden from the screen — space left the board.
- [11:18.371](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=678.3712916666666): lifted\_points\_7 is hidden from the screen — space left the board.
- [11:18.371](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=678.3712916666666): lifted\_points\_8 is hidden from the screen — space left the board.
- [11:18.371](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=678.3712916666666): lifted\_points\_9 is hidden from the screen — space left the board.
- [11:18.371](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=678.3712916666666): lifted\_points\_10 is hidden from the screen — space left the board.
- [11:18.371](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=678.3712916666666): lifted\_points\_11 is hidden from the screen — space left the board.
- [11:18.371](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=678.3712916666666): lifted\_points\_12 is hidden from the screen — space left the board.
- [11:18.371](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=678.3712916666666): separating\_plane is hidden from the screen — space left the board.

### Scene 6: [The Kernel Trick](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=679.4129583333333)

Span: 11:19.413–16:4.874 (679.4129583333333s–964.8744166666667s).

#### Objects

- boundary: a Circle \[green\] drawn in plot (radius=1.4142135624)
- classifier: a Derivation \[text\] that says "$f(x) &= w dot phi(x) + b \\ &= sum\_i alpha\_i y\_i (phi(x\_i) dot phi(x)) + b \\ &= sum\_(i in upright("SV")) alpha\_i y\_i K(x\_i,x) + b$"
- dual\_constraints: a Math \[text\] that says "$alpha\_i \>= 0, quad sum\_i alpha\_i y\_i = 0$"
- dual\_objective: a Math \[text\] that says "$max\_alpha sum\_i alpha\_i - frac(1, 2) sum\_(i,j) alpha\_i alpha\_j y\_i y\_j K(x\_i,x\_j)$"
- heading: a Heading that says "Common Kernel Choices"
- heading\_classifier: a Heading that says "The Classifier Needs Only Kernels"
- heading\_identity: a Heading that says "Compute the Lifted Inner Product Directly"
- heading\_recap: a Heading that says "The Complete SVM Story"
- heading\_training: a Heading that says "Training Uses a Kernel Matrix"
- kernel: a Math \[text\] that says "$K(x,z) = (x dot z)^2$"
- kernel\_definition: a Panel that says "A kernel returns an inner product in some feature space: $K(x,z)=phi(x) dot phi(z)$."
- kernel\_derivation: a Derivation \[text\] that says "$phi(x) = (x\_1^2, thin sqrt(2) x\_1 x\_2, thin x\_2^2) \\ phi(x) dot phi(z) &= x\_1^2 z\_1^2 + 2 x\_1 x\_2 z\_1 z\_2 + x\_2^2 z\_2^2 \\ &= (x\_1 z\_1 + x\_2 z\_2)^2 \\ &= (x dot z)^2$"
- linear\_kernel: a Math \[text\] that says "$K(x,z) = x dot z$"
- ordinary\_inner: a Point \[gray\] drawn in plot (location=(0.45, 0.0))
- ordinary\_inner\_2: a Point \[gray\] drawn in plot (location=(-0.45, 0.0))
- ordinary\_outer: a Point \[gray\] drawn in plot (location=(2.25, 0.0))
- ordinary\_outer\_2: a Point \[gray\] drawn in plot (location=(-2.25, 0.0))
- ordinary\_outer\_3: a Point \[gray\] drawn in plot (location=(0.0, 2.25))
- ordinary\_outer\_4: a Point \[gray\] drawn in plot (location=(0.0, -2.25))
- plot: an Axes (x\_range=(-3.0, 3.0), y\_range=(-3.0, 3.0), aspect=(1.0, 1.0))
- polynomial\_kernel: a Math \[text\] that says "$K(x,z) = (x dot z + c)^d$"
- rbf\_kernel: a Math \[text\] that says "$K(x,z) = exp(-gamma (x-z) dot (x-z))$"
- recap: a Block \[text\] that says "Choose a separator with a wide margin. The closest training points become support vectors. Slack variables trade margin width against violations. A feature map can make nonlinear data linearly separable. A kernel computes feature-space inn…"
- support\_inner: a Point \[red\] drawn in plot (location=(0.75, 0.0))
- support\_inner\_2: a Point \[red\] drawn in plot (location=(-0.75, 0.0))
- support\_inner\_3: a Point \[red\] drawn in plot (location=(0.0, 0.75))
- support\_inner\_4: a Point \[red\] drawn in plot (location=(0.0, -0.75))
- support\_outer: a Point \[blue\] drawn in plot (location=(1.4, 1.4))
- support\_outer\_2: a Point \[blue\] drawn in plot (location=(-1.4, 1.4))
- support\_outer\_3: a Point \[blue\] drawn in plot (location=(-1.4, -1.4))
- support\_outer\_4: a Point \[blue\] drawn in plot (location=(1.4, -1.4))
- w\_expansion: a Math \[text\] that says "$w = sum\_i alpha\_i y\_i phi(x\_i)$"

#### Beats

##### [11:19.413](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=679.4129583333333)

Narration: Return to the quadratic feature map. To compare two lifted examples phi of x and phi of z, the SVM needs their inner product. Let us calculate that inner product once and see what remains.

Board: Empty.

Actions:
- [11:19.413](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=679.4129583333333): heading\_identity is shown on the screen, written out.
- [11:19.413](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=679.4129583333333): plot is shown on the screen, written out.
- [11:20.075](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=680.0749583333334): boundary is shown on the screen, written out.
- [11:20.667](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=680.6669583333334): kernel\_derivation is shown on the screen, written out.
- [11:23.175](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=683.1749583333333): ordinary\_inner is shown on the screen, written out.
- [11:23.175](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=683.1749583333333): ordinary\_inner\_2 is shown on the screen, written out.
- [11:23.175](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=683.1749583333333): support\_inner is shown on the screen, written out.
- [11:23.175](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=683.1749583333333): support\_inner\_2 is shown on the screen, written out.
- [11:23.175](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=683.1749583333333): support\_inner\_3 is shown on the screen, written out.
- [11:23.175](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=683.1749583333333): support\_inner\_4 is shown on the screen, written out.
- [11:23.175](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=683.1749583333333): ordinary\_outer is shown on the screen, written out.
- [11:23.175](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=683.1749583333333): ordinary\_outer\_2 is shown on the screen, written out.
- [11:23.175](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=683.1749583333333): ordinary\_outer\_3 is shown on the screen, written out.
- [11:23.175](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=683.1749583333333): ordinary\_outer\_4 is shown on the screen, written out.
- [11:23.175](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=683.1749583333333): support\_outer is shown on the screen, written out.
- [11:23.175](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=683.1749583333333): support\_outer\_2 is shown on the screen, written out.
- [11:23.175](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=683.1749583333333): support\_outer\_3 is shown on the screen, written out.
- [11:23.175](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=683.1749583333333): support\_outer\_4 is shown on the screen, written out.

##### [11:33.19](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=693.1904583333334)

Narration: Multiply matching lifted coordinates. The first contribution is x one squared z one squared. The middle contribution contains the factor two, and the last is x two squared z two squared.

Board: plot — an Axes (x\_range=(-3.0, 3.0), y\_range=(-3.0, 3.0), aspect=(1.0, 1.0)); heading\_identity — a Heading that says "Compute the Lifted Inner Product Directly"; boundary — a Circle \[green\] drawn in plot (radius=1.4142135624); ordinary\_inner — a Point \[gray\] drawn in plot (location=(0.45, 0.0)); ordinary\_inner\_2 — a Point \[gray\] drawn in plot (location=(-0.45, 0.0)); support\_inner — a Point \[red\] drawn in plot (location=(0.75, 0.0)); support\_inner\_2 — a Point \[red\] drawn in plot (location=(-0.75, 0.0)); support\_inner\_3 — a Point \[red\] drawn in plot (location=(0.0, 0.75)); support\_inner\_4 — a Point \[red\] drawn in plot (location=(0.0, -0.75)); ordinary\_outer — a Point \[gray\] drawn in plot (location=(2.25, 0.0)); ordinary\_outer\_2 — a Point \[gray\] drawn in plot (location=(-2.25, 0.0)); ordinary\_outer\_3 — a Point \[gray\] drawn in plot (location=(0.0, 2.25)); ordinary\_outer\_4 — a Point \[gray\] drawn in plot (location=(0.0, -2.25)); support\_outer — a Point \[blue\] drawn in plot (location=(1.4, 1.4)); support\_outer\_2 — a Point \[blue\] drawn in plot (location=(-1.4, 1.4)); support\_outer\_3 — a Point \[blue\] drawn in plot (location=(-1.4, -1.4)); support\_outer\_4 — a Point \[blue\] drawn in plot (location=(1.4, -1.4))

Actions:
- [11:33.539](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=693.5389583333333): kernel\_derivation is shown on the screen, written out.
- [11:36.452](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=696.4519583333333): kernel\_derivation (the "x\_1^2 z\_1^2" part) is emphasized.
- [11:40.783](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=700.7829583333333): kernel\_derivation (the "2 x\_1 x\_2 z\_1 z\_2" part) is emphasized.
- [11:40.783](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=700.7829583333333): kernel\_derivation (the "x\_1^2 z\_1^2" part) is no longer emphasized.
- [11:43.662](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=703.6619583333334): kernel\_derivation (the "2 x\_1 x\_2 z\_1 z\_2" part) is no longer emphasized.
- [11:43.662](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=703.6619583333334): kernel\_derivation (the "x\_2^2 z\_2^2" part) is emphasized.
- [11:46.634](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=706.6344583333333): kernel\_derivation (the "x\_2^2 z\_2^2" part) is no longer emphasized.

##### [11:47.234](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=707.2344583333334)

Narration: Those three terms form a familiar square. They equal x one z one plus x two z two, all squared.

Board: Unchanged from the preceding beat in this scene.

Actions:
- [11:49.324](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=709.3239583333333): kernel\_derivation is shown on the screen, written out.
- [11:52.192](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=712.1919583333333): kernel\_derivation (the "x\_1 z\_1 + x\_2 z\_2" part) is emphasized.
- [11:55.489](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=715.4889583333334): kernel\_derivation (the "x\_1 z\_1 + x\_2 z\_2" part) is no longer emphasized.

##### [11:56.089](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=716.0889583333334)

Narration: But x one z one plus x two z two is simply the ordinary dot product x dot z. Therefore the lifted inner product is the square of one dot product in the original two-dimensional space.

Board: Unchanged from the preceding beat in this scene.

Actions:
- [12:0.396](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=720.3959583333333): kernel\_derivation is shown on the screen, written out.
- [12:4.808](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=724.8079583333333): kernel\_derivation (the "(x dot z)^2" part) is emphasized.
- [12:8.372](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=728.3724583333334): kernel\_derivation (the "(x dot z)^2" part) is no longer emphasized.

##### [12:8.972](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=728.9724583333333)

Narration: Define K of x z to be that squared dot product. Evaluating K gives exactly the number we would have obtained by constructing both three-dimensional feature vectors and taking their inner product.

Board: Unchanged from the preceding beat in this scene.

Actions:
- [12:9.321](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=729.3209583333334): kernel is shown on the screen, written out.
- [12:15.416](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=735.4159583333334): A box is drawn around kernel.

##### [12:22.773](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=742.7734583333333)

Narration: That replacement is the kernel trick. We behave as if the data had been lifted, while every computation stays in the original input space. The feature coordinates disappear, but their geometry remains in the kernel values.

Board: kernel — a Math \[text\] that says "$K(x,z) = (x dot z)^2$"; plot — an Axes (x\_range=(-3.0, 3.0), y\_range=(-3.0, 3.0), aspect=(1.0, 1.0)); heading\_identity — a Heading that says "Compute the Lifted Inner Product Directly"; boundary — a Circle \[green\] drawn in plot (radius=1.4142135624); ordinary\_inner — a Point \[gray\] drawn in plot (location=(0.45, 0.0)); ordinary\_inner\_2 — a Point \[gray\] drawn in plot (location=(-0.45, 0.0)); support\_inner — a Point \[red\] drawn in plot (location=(0.75, 0.0)); support\_inner\_2 — a Point \[red\] drawn in plot (location=(-0.75, 0.0)); support\_inner\_3 — a Point \[red\] drawn in plot (location=(0.0, 0.75)); support\_inner\_4 — a Point \[red\] drawn in plot (location=(0.0, -0.75)); ordinary\_outer — a Point \[gray\] drawn in plot (location=(2.25, 0.0)); ordinary\_outer\_2 — a Point \[gray\] drawn in plot (location=(-2.25, 0.0)); ordinary\_outer\_3 — a Point \[gray\] drawn in plot (location=(0.0, 2.25)); ordinary\_outer\_4 — a Point \[gray\] drawn in plot (location=(0.0, -2.25)); support\_outer — a Point \[blue\] drawn in plot (location=(1.4, 1.4)); support\_outer\_2 — a Point \[blue\] drawn in plot (location=(-1.4, 1.4)); support\_outer\_3 — a Point \[blue\] drawn in plot (location=(-1.4, -1.4)); support\_outer\_4 — a Point \[blue\] drawn in plot (location=(1.4, -1.4))

Actions:
- [12:24.399](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=744.3989583333333): kernel is indicated — a transient flash.
- [12:37.25](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=757.2504583333333): kernel moves to a new place on the board.
- [12:37.25](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=757.2504583333333): heading\_identity is hidden from the screen — left the board.
- [12:37.25](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=757.2504583333333): kernel\_derivation is hidden from the screen — left the board.
- [12:37.25](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=757.2504583333333): The box around kernel is lifted.

##### [12:37.85](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=757.8504583333333)

Narration: Why are inner products enough? At the optimum, the normal vector in feature space can be written as a weighted sum of lifted training examples. Each weight is alpha sub i times the label y sub i.

Board: kernel — a Math \[text\] that says "$K(x,z) = (x dot z)^2$"; plot — an Axes (x\_range=(-3.0, 3.0), y\_range=(-3.0, 3.0), aspect=(1.0, 1.0)); boundary — a Circle \[green\] drawn in plot (radius=1.4142135624); ordinary\_inner — a Point \[gray\] drawn in plot (location=(0.45, 0.0)); ordinary\_inner\_2 — a Point \[gray\] drawn in plot (location=(-0.45, 0.0)); support\_inner — a Point \[red\] drawn in plot (location=(0.75, 0.0)); support\_inner\_2 — a Point \[red\] drawn in plot (location=(-0.75, 0.0)); support\_inner\_3 — a Point \[red\] drawn in plot (location=(0.0, 0.75)); support\_inner\_4 — a Point \[red\] drawn in plot (location=(0.0, -0.75)); ordinary\_outer — a Point \[gray\] drawn in plot (location=(2.25, 0.0)); ordinary\_outer\_2 — a Point \[gray\] drawn in plot (location=(-2.25, 0.0)); ordinary\_outer\_3 — a Point \[gray\] drawn in plot (location=(0.0, 2.25)); ordinary\_outer\_4 — a Point \[gray\] drawn in plot (location=(0.0, -2.25)); support\_outer — a Point \[blue\] drawn in plot (location=(1.4, 1.4)); support\_outer\_2 — a Point \[blue\] drawn in plot (location=(-1.4, 1.4)); support\_outer\_3 — a Point \[blue\] drawn in plot (location=(-1.4, -1.4)); support\_outer\_4 — a Point \[blue\] drawn in plot (location=(1.4, -1.4))

Actions:
- [12:37.85](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=757.8504583333333): heading\_classifier is shown on the screen, written out.
- [12:44.364](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=764.3639583333334): w\_expansion is shown on the screen, written out.
- [12:47.626](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=767.6259583333333): w\_expansion (the "alpha\_i" part) is indicated — a transient flash.
- [12:49.67](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=769.6699583333334): w\_expansion (the "y\_i" part) is indicated — a transient flash.

##### [12:52.173](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=772.1734583333333)

Narration: To classify a new input, begin with w dot phi of x plus b. Substitute the weighted expansion of w, and the calculation becomes a sum of inner products between training features and the new feature vector.

Board: kernel — a Math \[text\] that says "$K(x,z) = (x dot z)^2$"; plot — an Axes (x\_range=(-3.0, 3.0), y\_range=(-3.0, 3.0), aspect=(1.0, 1.0)); boundary — a Circle \[green\] drawn in plot (radius=1.4142135624); ordinary\_inner — a Point \[gray\] drawn in plot (location=(0.45, 0.0)); ordinary\_inner\_2 — a Point \[gray\] drawn in plot (location=(-0.45, 0.0)); support\_inner — a Point \[red\] drawn in plot (location=(0.75, 0.0)); support\_inner\_2 — a Point \[red\] drawn in plot (location=(-0.75, 0.0)); support\_inner\_3 — a Point \[red\] drawn in plot (location=(0.0, 0.75)); support\_inner\_4 — a Point \[red\] drawn in plot (location=(0.0, -0.75)); ordinary\_outer — a Point \[gray\] drawn in plot (location=(2.25, 0.0)); ordinary\_outer\_2 — a Point \[gray\] drawn in plot (location=(-2.25, 0.0)); ordinary\_outer\_3 — a Point \[gray\] drawn in plot (location=(0.0, 2.25)); ordinary\_outer\_4 — a Point \[gray\] drawn in plot (location=(0.0, -2.25)); support\_outer — a Point \[blue\] drawn in plot (location=(1.4, 1.4)); support\_outer\_2 — a Point \[blue\] drawn in plot (location=(-1.4, 1.4)); support\_outer\_3 — a Point \[blue\] drawn in plot (location=(-1.4, -1.4)); support\_outer\_4 — a Point \[blue\] drawn in plot (location=(1.4, -1.4)); w\_expansion — a Math \[text\] that says "$w = sum\_i alpha\_i y\_i phi(x\_i)$"; heading\_classifier — a Heading that says "The Classifier Needs Only Kernels"

Actions:
- [12:54.484](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=774.4839583333334): classifier is shown on the screen, written out.
- [12:58.153](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=778.1529583333333): classifier is shown on the screen, written out.
- [13:2.448](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=782.4479583333333): classifier (the "phi(x\_i) dot phi(x)" part) is emphasized.
- [13:6.105](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=786.1054583333333): classifier (the "phi(x\_i) dot phi(x)" part) is no longer emphasized.

##### [13:6.705](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=786.7054583333334)

Narration: Now replace every lifted inner product by K of x sub i and x. Prediction can be performed without forming w and without explicitly forming phi of any example.

Board: Unchanged from the preceding beat in this scene.

Actions:
- [13:7.286](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=787.2859583333334): classifier is shown on the screen, written out.
- [13:9.202](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=789.2019583333333): classifier (the "K(x\_i,x)" part) is emphasized.
- [13:17.642](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=797.6424583333334): classifier (the "K(x\_i,x)" part) is no longer emphasized.

##### [13:18.242](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=798.2424583333334)

Narration: Most coefficients alpha sub i are zero. Only support vectors have nonzero coefficients and contribute to the final sum. The same small set of geometrically decisive points now makes prediction sparse.

Board: Unchanged from the preceding beat in this scene.

Actions:
- [13:21.064](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=801.0639583333333): ordinary\_inner is dimmed out.
- [13:21.064](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=801.0639583333333): ordinary\_inner\_2 is dimmed out.
- [13:21.064](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=801.0639583333333): ordinary\_outer is dimmed out.
- [13:21.064](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=801.0639583333333): ordinary\_outer\_2 is dimmed out.
- [13:21.064](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=801.0639583333333): ordinary\_outer\_3 is dimmed out.
- [13:21.064](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=801.0639583333333): ordinary\_outer\_4 is dimmed out.
- [13:22.689](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=802.6889583333333): support\_inner is indicated — a transient flash.
- [13:22.689](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=802.6889583333333): support\_inner\_2 is indicated — a transient flash.
- [13:22.689](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=802.6889583333333): support\_inner\_3 is indicated — a transient flash.
- [13:22.689](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=802.6889583333333): support\_inner\_4 is indicated — a transient flash.
- [13:22.689](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=802.6889583333333): support\_outer is indicated — a transient flash.
- [13:22.689](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=802.6889583333333): support\_outer\_2 is indicated — a transient flash.
- [13:22.689](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=802.6889583333333): support\_outer\_3 is indicated — a transient flash.
- [13:22.689](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=802.6889583333333): support\_outer\_4 is indicated — a transient flash.
- [13:32.789](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=812.7894583333333): kernel moves to a new place on the board.
- [13:32.789](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=812.7894583333333): classifier is hidden from the screen — left the board.
- [13:32.789](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=812.7894583333333): heading\_classifier is hidden from the screen — left the board.
- [13:32.789](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=812.7894583333333): plot is hidden from the screen — left the board.
- [13:32.789](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=812.7894583333333): boundary is hidden from the screen — plot left the board.
- [13:32.789](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=812.7894583333333): ordinary\_inner is hidden from the screen — plot left the board.
- [13:32.789](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=812.7894583333333): ordinary\_inner\_2 is hidden from the screen — plot left the board.
- [13:32.789](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=812.7894583333333): support\_inner is hidden from the screen — plot left the board.
- [13:32.789](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=812.7894583333333): support\_inner\_2 is hidden from the screen — plot left the board.
- [13:32.789](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=812.7894583333333): support\_inner\_3 is hidden from the screen — plot left the board.
- [13:32.789](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=812.7894583333333): support\_inner\_4 is hidden from the screen — plot left the board.
- [13:32.789](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=812.7894583333333): ordinary\_outer is hidden from the screen — plot left the board.
- [13:32.789](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=812.7894583333333): ordinary\_outer\_2 is hidden from the screen — plot left the board.
- [13:32.789](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=812.7894583333333): ordinary\_outer\_3 is hidden from the screen — plot left the board.
- [13:32.789](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=812.7894583333333): ordinary\_outer\_4 is hidden from the screen — plot left the board.
- [13:32.789](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=812.7894583333333): support\_outer is hidden from the screen — plot left the board.
- [13:32.789](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=812.7894583333333): support\_outer\_2 is hidden from the screen — plot left the board.
- [13:32.789](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=812.7894583333333): support\_outer\_3 is hidden from the screen — plot left the board.
- [13:32.789](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=812.7894583333333): support\_outer\_4 is hidden from the screen — plot left the board.
- [13:32.789](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=812.7894583333333): w\_expansion is hidden from the screen — left the board.

##### [13:33.389](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=813.3894583333333)

Narration: Training can also be written entirely in terms of pairwise kernels. The dual objective contains K of x sub i and x sub j for pairs of training examples, together with nonnegative coefficients whose label-weighted sum is zero.

Board: kernel — a Math \[text\] that says "$K(x,z) = (x dot z)^2$"

Actions:
- [13:33.389](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=813.3894583333333): heading\_training is shown on the screen, written out.
- [13:38.278](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=818.2779583333333): dual\_objective is shown on the screen, written out.
- [13:42.654](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=822.6539583333333): dual\_objective (the "K(x\_i,x\_j)" part) is emphasized.
- [13:45.139](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=825.1389583333333): dual\_constraints is shown on the screen, written out.
- [13:48.924](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=828.9239583333333): dual\_objective (the "K(x\_i,x\_j)" part) is no longer emphasized.

##### [13:49.524](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=829.5239583333333)

Narration: The optimization therefore needs a matrix of pairwise kernel values, often called the Gram matrix. Once those values are available, the algorithm can operate as though it knew every lifted coordinate. A kernel supplies exactly the required feature-space inner product.

Board: kernel — a Math \[text\] that says "$K(x,z) = (x dot z)^2$"; dual\_objective — a Math \[text\] that says "$max\_alpha sum\_i alpha\_i - frac(1, 2) sum\_(i,j) alpha\_i alpha\_j y\_i y\_j K(x\_i,x\_j)$"; dual\_constraints — a Math \[text\] that says "$alpha\_i \>= 0, quad sum\_i alpha\_i y\_i = 0$"; heading\_training — a Heading that says "Training Uses a Kernel Matrix"

Actions:
- [13:49.524](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=829.5239583333333): kernel\_definition is shown on the screen, written out.
- [14:6.462](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=846.4624583333333): dual\_constraints is hidden from the screen — left the board.
- [14:6.462](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=846.4624583333333): dual\_objective is hidden from the screen — left the board.
- [14:6.462](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=846.4624583333333): heading\_training is hidden from the screen — left the board.
- [14:6.462](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=846.4624583333333): kernel is hidden from the screen — left the board.
- [14:6.462](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=846.4624583333333): kernel\_definition is hidden from the screen — left the board.

##### [14:7.062](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=847.0624583333333)

Narration: Different kernels encode different notions of similarity. The linear kernel is the ordinary dot product, so it gives the original linear SVM.

Board: Empty.

Actions:
- [14:11.138](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=851.1379583333334): linear\_kernel is shown on the screen, written out.
- [14:12.798](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=852.7979583333333): linear\_kernel (the "x dot z" part) is emphasized.
- [14:16.443](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=856.4434583333333): linear\_kernel (the "x dot z" part) is no longer emphasized.

##### [14:17.043](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=857.0434583333333)

Narration: A polynomial kernel raises a shifted dot product to a degree d. Our quadratic example used degree two with no shift. Higher degrees represent richer interactions among the original coordinates.

Board: linear\_kernel — a Math \[text\] that says "$K(x,z) = x dot z$"

Actions:
- [14:17.554](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=857.5539583333333): polynomial\_kernel is shown on the screen, written out.
- [14:20.422](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=860.4219583333334): polynomial\_kernel (the "d" part) is emphasized.
- [14:29.884](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=869.8844583333333): polynomial\_kernel (the "d" part) is no longer emphasized.

##### [14:30.484](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=870.4844583333334)

Narration: The radial basis kernel decreases exponentially with squared distance between two inputs. It corresponds to a very rich feature space and produces flexible local boundaries, with gamma controlling how quickly similarity falls away.

Board: linear\_kernel — a Math \[text\] that says "$K(x,z) = x dot z$"; polynomial\_kernel — a Math \[text\] that says "$K(x,z) = (x dot z + c)^d$"

Actions:
- [14:31.018](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=871.0179583333334): rbf\_kernel is shown on the screen, written out.
- [14:41.514](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=881.5139583333333): rbf\_kernel (the "gamma" part) is emphasized.
- [14:44.985](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=884.9854583333333): rbf\_kernel (the "gamma" part) is no longer emphasized.

##### [14:45.585](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=885.5854583333333)

Narration: A kernel is not magic permission to use any similarity formula. It must behave like a genuine inner product in some feature space. When it does, the maximum-margin optimization remains mathematically well formed.

Board: linear\_kernel — a Math \[text\] that says "$K(x,z) = x dot z$"; polynomial\_kernel — a Math \[text\] that says "$K(x,z) = (x dot z + c)^d$"; rbf\_kernel — a Math \[text\] that says "$K(x,z) = exp(-gamma (x-z) dot (x-z))$"

Actions:
- [14:51.889](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=891.8889583333333): linear\_kernel is indicated — a transient flash.
- [15:0.202](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=900.2019583333333): linear\_kernel is hidden from the screen — left the board.
- [15:0.202](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=900.2019583333333): polynomial\_kernel is hidden from the screen — left the board.
- [15:0.202](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=900.2019583333333): rbf\_kernel is hidden from the screen — left the board.

##### [15:0.802](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=900.8019583333333)

Narration: Let us connect the whole argument. First, maximum margin chooses the separator with the widest empty corridor. The nearest points become support vectors because their constraints pin that corridor in place.

Board: Empty.

Actions:
- [15:0.802](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=900.8019583333333): heading\_recap is shown on the screen, written out.
- [15:0.802](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=900.8019583333333): recap is shown on the screen, written out.
- [15:0.802](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=900.8019583333333): classifier is shown on the screen, written out.
- [15:1.002](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=901.0019583333333): classifier is shown on the screen, written out.
- [15:1.202](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=901.2019583333333): classifier is shown on the screen, written out.

##### [15:14.475](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=914.4749583333333)

Narration: When classes overlap, slack variables measure margin violations, and C trades a wider corridor against the cost of those violations.

Board: recap — a Block \[text\] that says "Choose a separator with a wide margin. The closest training points become support vectors. Slack variables trade margin width against violations. A feature map can make nonlinear data linearly separable. A kernel computes feature-space inn…"; heading\_recap — a Heading that says "The Complete SVM Story"

Actions:
- None.

##### [15:23.596](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=923.5964583333333)

Narration: When no straight boundary works, a feature map can lift the examples into a space where a hyperplane does work. The decision surface is linear there even though it is curved in the original coordinates.

Board: Unchanged from the preceding beat in this scene.

Actions:
- None.

##### [15:36.503](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=936.5029583333333)

Narration: Finally, the kernel trick computes only the inner products required by training and prediction. It preserves the lifted geometry without paying the cost of explicitly writing the lifted vectors.

Board: Unchanged from the preceding beat in this scene.

Actions:
- None.

##### [15:47.97](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=947.9699583333334)

Narration: That is the support vector machine in one connected picture: maximize geometric separation, let a small set of points determine the answer, tolerate overlap when necessary, and use kernels when the useful separation lives in a nonlinear feature space.

Board: Unchanged from the preceding beat in this scene.

Actions:
- [15:50.42](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=950.4199583333334): A box is drawn around classifier.
- [16:3.833](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=963.83275): classifier is hidden from the screen — left the board.
- [16:3.833](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=963.83275): heading\_recap is hidden from the screen — left the board.
- [16:3.833](https://academa.ai/lectures/support-vector-machines-and-the-widest-corridor?t=963.83275): recap is hidden from the screen — left the board.