# PCA from Geometric Intuition to Eigenvectors

> A geometric explanation of principal component analysis for working data scientists. Rotate a direction through an elongated point cloud, watch projected variance reach its maximum, and build the remaining orthogonal components. The lecture then derives the covariance eigenvalue problem as the algebraic form of that same search, before using eigenvalues for scree plots, component selection, and low-rank reconstruction. It closes with a direct account of why measurement units can dominate PCA and when scaling or standardization is needed.

- Canonical watch page: [PCA from Geometric Intuition to Eigenvectors](https://academa.ai/lectures/principal-component-analysis-seen-geometrically)
- Publisher: [Academa, Inc.](https://academa.ai)
- Subject: Machine Learning
- Published: 2026-08-29T00:02:47.000Z
- Updated: 2026-08-29T00:02:47.000Z
- Duration: PT1071S (17 minutes 51 seconds)
- Chapters: 5
- Views: 1
- Language: en-US
- Access: Free
- Video stream: [HLS content](https://academa.ai/media/l/01M14TYV1ZG691V3XR3GYR0P92/1/dark/master.m3u8)
- Audiovisual record: [Semantic JSON](https://academa.ai/media/l/01M14TYV1ZG691V3XR3GYR0P92/1/semantic.json)
- Thumbnail: [Image](https://academa.ai/media/l/01M14TYV1ZG691V3XR3GYR0P92/1/dark/poster.jpg)

## Description

Understand PCA geometrically, connect maximum projected variance to covariance eigenvectors, and use scree plots, reconstruction, and scaling.

## Chapters

- [00:00–04:12.261 · Variance in a Direction](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=0)
- [04:12.261–06:57.722 · More Than Two Dimensions](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=252.26064583333334)
- [06:57.722–10:57.495 · The Covariance Eigenproblem](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=417.72210416666667)
- [10:57.495–14:39.662 · Scree Plots and Reconstruction](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=657.4953958333333)
- [14:39.662–17:51 · Units Can Choose the Components](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=879.6621666666666)

## Transcript

### [00:00 · Variance in a Direction](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=0)

Suppose you use PCA every week. You call a library, ask for two components, and get a transformed table. But what did the algorithm actually choose? Here is the question that organizes the whole method: if we flatten a centered cloud onto one line, which line preserves the most variation? Begin with a two-dimensional data set. Each blue point is one observation, and the two axes are two measured variables. The cloud is not round. It has a long direction running roughly from lower left to upper right. PCA starts by centering each variable. Subtract its sample mean, so the red mean moves to the origin. Centering matters because PCA is supposed to describe variation around the typical observation, not distance from an arbitrary zero. Now choose a candidate direction u. We require unit length, not because short arrows are virtuous, but because otherwise we could manufacture unlimited variance merely by multiplying the direction by a huge number. Project every observation onto that line. Each gray segment drops one blue point to its green projected position. The signed coordinate along the line is its score, z sub i, equal to u transpose times the centered observation. Those scores form a one-dimensional data set. Measure their variance in the ordinary way: square each score, average the squares, and call the result V of u. At the moment the line is horizontal. The green points have some spread, and the marker reports variance four. That is neither especially small nor the largest value available. Turn the candidate line toward the visible long axis of the cloud. The projected points separate, so their one-dimensional variance rises. It reaches seven at this direction, where the spread is greatest. Continue turning. The projections draw closer together again. On the vertical axis the variance has fallen back to four, even though no data point has moved. Keep turning toward the short axis of the cloud. The green points bunch tightly around the origin, and the variance falls to one. This direction preserves very little of the cloud's visible spread. Now return to the maximum. The data stayed fixed throughout. Only the question we asked of them changed: how much of their variation can be seen from this direction? This is PCA's central geometric operation. It searches over all unit directions, projects the centered observations, and chooses the direction whose scores have the largest variance. That maximizing direction is the first principal component. A component is a direction in variable space. The projected coordinate is the score of an observation on that component. Those are related objects, but they are not the same thing. Write the first component as v one: the unit direction that maximizes projected variance. Its variance, seven in this example, tells us how much variation the component captures. After choosing v one, PCA looks for another direction that captures as much remaining variance as possible without repeating the first answer. In two dimensions, there is only one perpendicular direction available. That green direction is the second principal component. The cloud is narrow along it, so its variance is only one. The first component describes the long way through the cloud, and the second describes the remaining side-to-side variation. The two component directions are perpendicular and have unit length. Together they form a rotated coordinate system. PCA has not bent the data or invented a nonlinear feature. It has rotated the axes to line up with the directions of greatest and least variation. So the first answer to carry forward is geometric. PCA asks where the centered cloud is widest, calls that direction component one, and then repeats the question in directions perpendicular to what it has already chosen.

### [04:12.261 · More Than Two Dimensions](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=252.26064583333334)

A real data set may have fifty variables, five thousand, or five million. We cannot rotate a literal line through all those dimensions on a screen, but the geometric question does not change. Use three dimensions as the bridge. This cloud has a long spatial direction, plus smaller deviations in two independent sideways directions. The first search is exactly the one we just performed. Among every unit direction in the space, find the one on which the projected coordinates have maximum variance. The red arrow is the answer. Projecting onto it compresses every observation to one score. Across all observations, those scores vary more than scores along any other unit direction. Now remove the possibility of choosing that answer again. Every direction perpendicular to v one lies in this gray plane through the origin. Search inside that plane for the direction with greatest projected variance. The green arrow is component two. It is the strongest remaining direction after component one has been excluded. In three dimensions, one perpendicular direction remains after the first two have been fixed. That yellow arrow becomes component three. In p dimensions the process continues the same way. Component k maximizes projected variance subject to unit length and orthogonality to every earlier component. Collect the first k component directions as columns of V sub k. Multiplying the centered data matrix X by V sub k produces the score matrix Z sub k. Each row is now the same observation expressed in principal-component coordinates. Because the component directions are mutually perpendicular unit vectors, V transpose V is the identity. This orthonormal structure is why moving into component coordinates and reconstructing from them remains simple. There cannot be more useful components than variables. There also cannot be more than n minus one nonzero directions of variation after n observations have been centered. The crucial point is that higher-dimensional PCA is not a new algorithmic idea. It is the same sequence of questions: widest direction first, then the widest perpendicular direction left, then the next. We now have the geometry. Only at this point do we need the covariance matrix, because it gives an efficient algebraic way to answer exactly the optimization question we have already understood.

### [06:57.722 · The Covariance Eigenproblem](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=417.72210416666667)

Return to the centered two-dimensional cloud. We already know the geometric problem: choose a unit direction u that makes the projected scores vary as much as possible. For one centered observation x sub i, its coordinate on u is the dot product u transpose x sub i. Call that score z sub i. The scores are centered too, so their variance is the average squared score. Substitute the dot product into that familiar calculation. A scalar square can be written as u transpose x sub i, times x sub i transpose u. Put the direction vectors outside the observation-specific middle. The average of those middle outer products is the covariance matrix Sigma. With centered observations stored in the rows of X, Sigma is one over n times X transpose X. So the variance seen along u is exactly the quadratic form u transpose Sigma u. This line is not a different objective. It is the projected spread from the rotating picture, written compactly. We want to maximize that quadratic form while keeping u at unit length. Use a Lagrange multiplier to attach the constraint u transpose u equals one. At a maximum, the derivative with respect to u is zero. Differentiating gives two Sigma u minus two lambda u. Cancel the common factor of two and rearrange. Sigma u equals lambda u. The maximizing direction must therefore be an eigenvector of the covariance matrix. This is the promised restatement. The geometry asked for a direction that keeps its identity while covariance assigns it a variance. The algebra calls that direction an eigenvector. Multiply the eigenvalue equation on the left by u transpose. Because u has unit length, u transpose Sigma u equals lambda. The eigenvalue is the projected variance along its eigenvector. Therefore the first component is the eigenvector with the largest eigenvalue. The second is the perpendicular eigenvector with the next eigenvalue, and so on in descending order. Covariance matrices are symmetric and positive semidefinite. Their eigenvectors can be chosen orthonormal, and their eigenvalues are nonnegative. Those facts are exactly what the geometric construction required. Now calculate the example. Use the covariance matrix with fours on the diagonal and threes off the diagonal. Its positive covariance says the two variables tend to rise and fall together. Eigenvalues make Sigma minus lambda I singular. Subtract lambda from both diagonal entries and set the determinant to zero. Multiply the main diagonal and subtract the other product. We get four minus lambda squared, minus nine, equal to zero. Expand the square and collect terms. The characteristic polynomial is lambda squared minus eight lambda plus seven. Factor it. Lambda minus seven times lambda minus one equals zero. The two eigenvalues are seven and one, exactly the maximum and minimum projected variances from the rotating line. The eigenvector for seven points along one, one. The other eigenvector points along one, minus one. Normalize both by dividing by square root two. Seven of the total eight variance units lie along component one. So one component explains eighty-seven point five percent of this cloud's variance. The eigenvalue calculation has returned every fact we first discovered by rotating and projecting.

### [10:57.495 · Scree Plots and Reconstruction](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=657.4953958333333)

PCA orders components from most variance to least, but a model still needs a decision: how many components should we keep? The eigenvalues give the evidence for that decision. A scree plot puts the component number along the horizontal axis and its eigenvalue on the vertical axis. Here are five eigenvalues in descending order. The first point is high because component one captures a great deal of variation. The second captures less. After the third, the remaining eigenvalues are small. An individual explained-variance ratio divides one eigenvalue by the sum of all eigenvalues. It reports the fraction of total variance assigned to that component. Cumulative explained variance adds the first k eigenvalues before dividing by the total. It asks how much variation remains visible after reducing the data to k component coordinates. With one component, this example retains sixty-seven point six percent. With two, the total reaches eighty-eight point nine percent. With three components it reaches ninety-six point six percent. Components four and five add very little. The visible bend near component three is often called the elbow. One common heuristic keeps components before the curve settles into a shallow tail. But the elbow is not a theorem and ninety-five percent is not a universal law. Keep enough components for the downstream task. For prediction, choose k with cross-validation. For compression, choose an acceptable reconstruction error. For interpretation, check stability and domain meaning. A scree plot is diagnostic evidence, not an automatic command. A weak elbow may honestly mean there is no clean low-dimensional cutoff. Now make component retention concrete. In principal-component coordinates, a centered observation is built from one contribution along each component direction. This observation has a large score on component one, a smaller score on component two, and a still smaller score on component three. The general reconstruction formula starts at the original mean and adds back the first k score times direction contributions. Keep only component one. Starting from the mean, move along v one by the first score. The green point is the rank-one reconstruction. It preserves the strongest coordinate and discards everything in the omitted perpendicular subspace. The red dashed segment is the information lost for this observation. Now retain component two as well. Add the yellow contribution from the first reconstruction. The new point moves closer to the original. The remaining error lies entirely along component three. If we retain that final red contribution, the reconstruction reaches the original point exactly. The displayed error falls from one point three four with one component to zero point six with two. Every additional component can only reduce squared reconstruction error on the training data. Averaged over the data set, the squared error after keeping k components equals the sum of the omitted eigenvalues. The scree plot and reconstruction error are therefore two readings of the same tradeoff. Keeping a component means preserving one orthogonal pattern of variation. Dropping it means replacing every observation's score on that pattern by zero, then mapping the reduced coordinates back to the original variable space.

### [14:39.662 · Units Can Choose the Components](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=879.6621666666666)

One warning belongs in every practical explanation of PCA. PCA does not understand variables. It understands numbers, and it rewards whichever direction has the greatest numerical variance. On the left, salary deviation is measured in dollars and age deviation in years. These are centered observations, so subtracting the means has already been done. The salary coordinates contain tens of thousands while the age coordinates contain tens. The raw cloud therefore looks overwhelmingly horizontal, and raw PCA chooses an almost horizontal first component. That result does not prove salary is the most important scientific variable. It proves salary has the largest variance in the numbers handed to the algorithm. On the right, each variable has been centered and divided by its sample standard deviation. Both begin with variance one, so their relationship, rather than their original unit sizes, determines the component. The two answers differ even though they describe the same observations. Scaling has changed the geometry, and changing the geometry changes PCA. Here is the raw covariance matrix. Salary variance is four hundred million, while age variance is one hundred. The larger numerical scale dominates the optimization. Now perform a scientifically meaningless change: write the same salaries in cents instead of dollars. Every salary value is multiplied by one hundred, and its variance is multiplied by ten thousand. Raw PCA can rotate even though nothing about the people changed. That is the blunt warning. PCA maximizes numerical variance. It cannot distinguish signal from noise, importance from nuisance, or measurement scale from genuine structure. Standardization replaces each variable by its centered value divided by its sample standard deviation. The covariance matrix of standardized variables is their correlation matrix. Do not turn that into another automatic ritual. If a variable's original scale carries real meaning, raw covariance may be appropriate. If unlike units should receive equal starting weight, standardization is often necessary. Heavy tails, outliers, counts, and strongly skewed variables may call for robust scaling or a domain-specific transformation. Whatever you choose, document it and validate it as part of the model. PCA is now one connected idea. Center the observations, find the direction of maximum projected variance, continue in perpendicular directions, and compute those directions as covariance eigenvectors. Use eigenvalues to judge compression, reconstruct from retained scores, and never forget that the units help define the answer.

## About Academa, Inc.

Academa makes technical knowledge easier to understand through visual lectures and lets learners request new lecture videos on the topics they need.

## Complete audiovisual record

Immutable source: [semantic.json](https://academa.ai/media/l/01M14TYV1ZG691V3XR3GYR0P92/1/semantic.json)

Record version: 1. Render attempt: 1.

### How to read this timeline

Each scene owns its object identifiers. A beat's board is the complete board when listed, empty when marked empty, and unchanged from the nearest earlier listed board in the same scene when marked unchanged. Action times are absolute positions in the published video.

### Scene 1: [Variance in a Direction](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=0)

Span: 00:00–04:12.261 (0s–252.26064583333334s).

#### Objects

- angle: a VariableNumber
- candidate: a Line \[yellow\] labelled "u" drawn in plane (start=((-4.0 \* cos(angle)), (-4.0 \* sin(angle))), end=((4.0 \* cos(angle)), (4.0 \* sin(angle))))
- centering: a Math \[text\] that says "$x\_i arrow.r x\_i - overline(x)$"
- centre: a Point \[red\] labelled "overline(x)" drawn in plane
- cloud: a Point \[blue\] drawn in plane (location=(-3.1, -2.4), marker\_radius=0.075)
- cloud\_10: a Point \[blue\] drawn in plane (location=(2.0, 1.5), marker\_radius=0.075)
- cloud\_11: a Point \[blue\] drawn in plane (location=(2.6, 3.0), marker\_radius=0.075)
- cloud\_12: a Point \[blue\] drawn in plane (location=(3.1, 2.4), marker\_radius=0.075)
- cloud\_2: a Point \[blue\] drawn in plane (location=(-2.7, -3.0), marker\_radius=0.075)
- cloud\_3: a Point \[blue\] drawn in plane (location=(-2.2, -1.5), marker\_radius=0.075)
- cloud\_4: a Point \[blue\] drawn in plane (location=(-1.6, -2.1), marker\_radius=0.075)
- cloud\_5: a Point \[blue\] drawn in plane (location=(-1.1, -0.5), marker\_radius=0.075)
- cloud\_6: a Point \[blue\] drawn in plane (location=(-0.5, -1.0), marker\_radius=0.075)
- cloud\_7: a Point \[blue\] drawn in plane (location=(0.4, 0.9), marker\_radius=0.075)
- cloud\_8: a Point \[blue\] drawn in plane (location=(0.9, 0.4), marker\_radius=0.075)
- cloud\_9: a Point \[blue\] drawn in plane (location=(1.4, 2.0), marker\_radius=0.075)
- constraint: a Math \[text\] that says "$u^T u = 1$"
- heading: a Heading that says "Rotate a Direction and Watch the Spread"
- pc1: a Math \[text\] that says "$v\_1 = arg max\_(u^T u = 1) V(u)$"
- pc2: a Math \[text\] that says "$v\_2 bot v\_1$"
- pc2\_line: a Line \[green\] labelled "v\_2" drawn in plane (start=(2.8, -2.8), end=(-2.8, 2.8))
- plane: an Axes (x\_range=(-4.2, 4.2), y\_range=(-4.2, 4.2), aspect=(1.0, 1.0))
- projected: a Point \[green\] drawn in plane (location=((((-3.1 \* cos(angle)) + (-2.4 \* sin(angle))) \* cos(angle)), ((…, marker\_radius=0.065)
- projected\_10: a Point \[green\] drawn in plane (location=((((2.0 \* cos(angle)) + (1.5 \* sin(angle))) \* cos(angle)), (((2…, marker\_radius=0.065)
- projected\_11: a Point \[green\] drawn in plane (location=((((2.6 \* cos(angle)) + (3.0 \* sin(angle))) \* cos(angle)), (((2…, marker\_radius=0.065)
- projected\_12: a Point \[green\] drawn in plane (location=((((3.1 \* cos(angle)) + (2.4 \* sin(angle))) \* cos(angle)), (((3…, marker\_radius=0.065)
- projected\_2: a Point \[green\] drawn in plane (location=((((-2.7 \* cos(angle)) + (-3.0 \* sin(angle))) \* cos(angle)), ((…, marker\_radius=0.065)
- projected\_3: a Point \[green\] drawn in plane (location=((((-2.2 \* cos(angle)) + (-1.5 \* sin(angle))) \* cos(angle)), ((…, marker\_radius=0.065)
- projected\_4: a Point \[green\] drawn in plane (location=((((-1.6 \* cos(angle)) + (-2.1 \* sin(angle))) \* cos(angle)), ((…, marker\_radius=0.065)
- projected\_5: a Point \[green\] drawn in plane (location=((((-1.1 \* cos(angle)) + (-0.5 \* sin(angle))) \* cos(angle)), ((…, marker\_radius=0.065)
- projected\_6: a Point \[green\] drawn in plane (location=((((-0.5 \* cos(angle)) + (-1.0 \* sin(angle))) \* cos(angle)), ((…, marker\_radius=0.065)
- projected\_7: a Point \[green\] drawn in plane (location=((((0.4 \* cos(angle)) + (0.9 \* sin(angle))) \* cos(angle)), (((0…, marker\_radius=0.065)
- projected\_8: a Point \[green\] drawn in plane (location=((((0.9 \* cos(angle)) + (0.4 \* sin(angle))) \* cos(angle)), (((0…, marker\_radius=0.065)
- projected\_9: a Point \[green\] drawn in plane (location=((((1.4 \* cos(angle)) + (2.0 \* sin(angle))) \* cos(angle)), (((1…, marker\_radius=0.065)
- question: a Panel that says "For a centered cloud of points, which line preserves the greatest amount of variation when the points are projected onto it?"
- residuals: a Line \[gray\] drawn in plane (start=(-3.1, -2.4), end=((((-3.1 \* cos(angle)) + (-2.4 \* sin(angle))) \* cos(angle)), ((…, dashed=True)
- residuals\_10: a Line \[gray\] drawn in plane (start=(2.0, 1.5), end=((((2.0 \* cos(angle)) + (1.5 \* sin(angle))) \* cos(angle)), (((2…, dashed=True)
- residuals\_11: a Line \[gray\] drawn in plane (start=(2.6, 3.0), end=((((2.6 \* cos(angle)) + (3.0 \* sin(angle))) \* cos(angle)), (((2…, dashed=True)
- residuals\_12: a Line \[gray\] drawn in plane (start=(3.1, 2.4), end=((((3.1 \* cos(angle)) + (2.4 \* sin(angle))) \* cos(angle)), (((3…, dashed=True)
- residuals\_2: a Line \[gray\] drawn in plane (start=(-2.7, -3.0), end=((((-2.7 \* cos(angle)) + (-3.0 \* sin(angle))) \* cos(angle)), ((…, dashed=True)
- residuals\_3: a Line \[gray\] drawn in plane (start=(-2.2, -1.5), end=((((-2.2 \* cos(angle)) + (-1.5 \* sin(angle))) \* cos(angle)), ((…, dashed=True)
- residuals\_4: a Line \[gray\] drawn in plane (start=(-1.6, -2.1), end=((((-1.6 \* cos(angle)) + (-2.1 \* sin(angle))) \* cos(angle)), ((…, dashed=True)
- residuals\_5: a Line \[gray\] drawn in plane (start=(-1.1, -0.5), end=((((-1.1 \* cos(angle)) + (-0.5 \* sin(angle))) \* cos(angle)), ((…, dashed=True)
- residuals\_6: a Line \[gray\] drawn in plane (start=(-0.5, -1.0), end=((((-0.5 \* cos(angle)) + (-1.0 \* sin(angle))) \* cos(angle)), ((…, dashed=True)
- residuals\_7: a Line \[gray\] drawn in plane (start=(0.4, 0.9), end=((((0.4 \* cos(angle)) + (0.9 \* sin(angle))) \* cos(angle)), (((0…, dashed=True)
- residuals\_8: a Line \[gray\] drawn in plane (start=(0.9, 0.4), end=((((0.9 \* cos(angle)) + (0.4 \* sin(angle))) \* cos(angle)), (((0…, dashed=True)
- residuals\_9: a Line \[gray\] drawn in plane (start=(1.4, 2.0), end=((((1.4 \* cos(angle)) + (2.0 \* sin(angle))) \* cos(angle)), (((1…, dashed=True)
- score: a Math \[text\] that says "$z\_i = u^T (x\_i - overline(x))$"
- variance: a Math \[text\] that says "$V(u) = frac(1, n) sum\_i z\_i^2$"
- variance\_line: a NumberLine labelled "V(u)" (x\_range=(0.0, 8.0), include\_numbers=True)
- variance\_marker: a Point \[yellow\] labelled "V(u)" drawn in variance\_line (location=((4.0 + (3.0 \* sin((2.0 \* angle)))), 0.0))
- variance\_split: a Math \[text\] that says "$lambda\_1 = 7, quad lambda\_2 = 1$"

#### Beats

##### [00:00](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=0)

Narration: Suppose you use PCA every week. You call a library, ask for two components, and get a transformed table. But what did the algorithm actually choose? Here is the question that organizes the whole method: if we flatten a centered cloud onto one line, which line preserves the most variation?

Board: Empty.

Actions:
- [00:00](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=0): question is shown on the screen, written out.
- [00:18.379](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=18.379): question is hidden from the screen — left the board.

##### [00:19.579](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=19.579)

Narration: Begin with a two-dimensional data set. Each blue point is one observation, and the two axes are two measured variables. The cloud is not round. It has a long direction running roughly from lower left to upper right.

Board: Unchanged from the preceding beat in this scene.

Actions:
- [00:19.579](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=19.579): plane is shown on the screen, written out.
- [00:22.934](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=22.934): cloud is shown on the screen, written out.
- [00:22.934](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=22.934): cloud\_2 is shown on the screen, written out.
- [00:22.934](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=22.934): cloud\_3 is shown on the screen, written out.
- [00:22.934](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=22.934): cloud\_4 is shown on the screen, written out.
- [00:22.934](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=22.934): cloud\_5 is shown on the screen, written out.
- [00:22.934](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=22.934): cloud\_6 is shown on the screen, written out.
- [00:22.934](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=22.934): cloud\_7 is shown on the screen, written out.
- [00:22.934](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=22.934): cloud\_8 is shown on the screen, written out.
- [00:22.934](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=22.934): cloud\_9 is shown on the screen, written out.
- [00:22.934](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=22.934): cloud\_10 is shown on the screen, written out.
- [00:22.934](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=22.934): cloud\_11 is shown on the screen, written out.
- [00:22.934](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=22.934): cloud\_12 is shown on the screen, written out.
- [00:28.24](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=28.24): centre is shown on the screen, written out.

##### [00:34.366](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=34.366)

Narration: PCA starts by centering each variable. Subtract its sample mean, so the red mean moves to the origin. Centering matters because PCA is supposed to describe variation around the typical observation, not distance from an arbitrary zero.

Board: plane — an Axes (x\_range=(-4.2, 4.2), y\_range=(-4.2, 4.2), aspect=(1.0, 1.0)); cloud — a Point \[blue\] drawn in plane (location=(-3.1, -2.4), marker\_radius=0.075); cloud\_2 — a Point \[blue\] drawn in plane (location=(-2.7, -3.0), marker\_radius=0.075); cloud\_3 — a Point \[blue\] drawn in plane (location=(-2.2, -1.5), marker\_radius=0.075); cloud\_4 — a Point \[blue\] drawn in plane (location=(-1.6, -2.1), marker\_radius=0.075); cloud\_5 — a Point \[blue\] drawn in plane (location=(-1.1, -0.5), marker\_radius=0.075); cloud\_6 — a Point \[blue\] drawn in plane (location=(-0.5, -1.0), marker\_radius=0.075); cloud\_7 — a Point \[blue\] drawn in plane (location=(0.4, 0.9), marker\_radius=0.075); cloud\_8 — a Point \[blue\] drawn in plane (location=(0.9, 0.4), marker\_radius=0.075); cloud\_9 — a Point \[blue\] drawn in plane (location=(1.4, 2.0), marker\_radius=0.075); cloud\_10 — a Point \[blue\] drawn in plane (location=(2.0, 1.5), marker\_radius=0.075); cloud\_11 — a Point \[blue\] drawn in plane (location=(2.6, 3.0), marker\_radius=0.075); cloud\_12 — a Point \[blue\] drawn in plane (location=(3.1, 2.4), marker\_radius=0.075); centre — a Point \[red\] labelled "overline(x)" drawn in plane

Actions:
- [00:35.933](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=35.933): plane moves to a new place on the board.
- [00:35.933](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=35.933): centering is shown on the screen, written out.
- [00:41.158](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=41.158): centre is indicated — a transient flash.

##### [00:50.942](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=50.941500000000005)

Narration: Now choose a candidate direction u. We require unit length, not because short arrows are virtuous, but because otherwise we could manufacture unlimited variance merely by multiplying the direction by a huge number.

Board: centering — a Math \[text\] that says "$x\_i arrow.r x\_i - overline(x)$"; plane — an Axes (x\_range=(-4.2, 4.2), y\_range=(-4.2, 4.2), aspect=(1.0, 1.0)); cloud — a Point \[blue\] drawn in plane (location=(-3.1, -2.4), marker\_radius=0.075); cloud\_2 — a Point \[blue\] drawn in plane (location=(-2.7, -3.0), marker\_radius=0.075); cloud\_3 — a Point \[blue\] drawn in plane (location=(-2.2, -1.5), marker\_radius=0.075); cloud\_4 — a Point \[blue\] drawn in plane (location=(-1.6, -2.1), marker\_radius=0.075); cloud\_5 — a Point \[blue\] drawn in plane (location=(-1.1, -0.5), marker\_radius=0.075); cloud\_6 — a Point \[blue\] drawn in plane (location=(-0.5, -1.0), marker\_radius=0.075); cloud\_7 — a Point \[blue\] drawn in plane (location=(0.4, 0.9), marker\_radius=0.075); cloud\_8 — a Point \[blue\] drawn in plane (location=(0.9, 0.4), marker\_radius=0.075); cloud\_9 — a Point \[blue\] drawn in plane (location=(1.4, 2.0), marker\_radius=0.075); cloud\_10 — a Point \[blue\] drawn in plane (location=(2.0, 1.5), marker\_radius=0.075); cloud\_11 — a Point \[blue\] drawn in plane (location=(2.6, 3.0), marker\_radius=0.075); cloud\_12 — a Point \[blue\] drawn in plane (location=(3.1, 2.4), marker\_radius=0.075); centre — a Point \[red\] labelled "overline(x)" drawn in plane

Actions:
- [00:52.3](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=52.30000000000001): candidate is shown on the screen, written out.
- [00:54.645](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=54.64500000000001): constraint is shown on the screen, written out.

##### [01:4.232](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=64.23150000000001)

Narration: Project every observation onto that line. Each gray segment drops one blue point to its green projected position. The signed coordinate along the line is its score, z sub i, equal to u transpose times the centered observation.

Board: centering — a Math \[text\] that says "$x\_i arrow.r x\_i - overline(x)$"; constraint — a Math \[text\] that says "$u^T u = 1$"; plane — an Axes (x\_range=(-4.2, 4.2), y\_range=(-4.2, 4.2), aspect=(1.0, 1.0)); cloud — a Point \[blue\] drawn in plane (location=(-3.1, -2.4), marker\_radius=0.075); cloud\_2 — a Point \[blue\] drawn in plane (location=(-2.7, -3.0), marker\_radius=0.075); cloud\_3 — a Point \[blue\] drawn in plane (location=(-2.2, -1.5), marker\_radius=0.075); cloud\_4 — a Point \[blue\] drawn in plane (location=(-1.6, -2.1), marker\_radius=0.075); cloud\_5 — a Point \[blue\] drawn in plane (location=(-1.1, -0.5), marker\_radius=0.075); cloud\_6 — a Point \[blue\] drawn in plane (location=(-0.5, -1.0), marker\_radius=0.075); cloud\_7 — a Point \[blue\] drawn in plane (location=(0.4, 0.9), marker\_radius=0.075); cloud\_8 — a Point \[blue\] drawn in plane (location=(0.9, 0.4), marker\_radius=0.075); cloud\_9 — a Point \[blue\] drawn in plane (location=(1.4, 2.0), marker\_radius=0.075); cloud\_10 — a Point \[blue\] drawn in plane (location=(2.0, 1.5), marker\_radius=0.075); cloud\_11 — a Point \[blue\] drawn in plane (location=(2.6, 3.0), marker\_radius=0.075); cloud\_12 — a Point \[blue\] drawn in plane (location=(3.1, 2.4), marker\_radius=0.075); centre — a Point \[red\] labelled "overline(x)" drawn in plane; candidate — a Line \[yellow\] labelled "u" drawn in plane (start=((-4.0 \* cos(angle)), (-4.0 \* sin(angle))), end=((4.0 \* cos(angle)), (4.0 \* sin(angle))))

Actions:
- [01:7.726](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=67.72600000000001): residuals is shown on the screen, written out.
- [01:7.726](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=67.72600000000001): residuals\_2 is shown on the screen, written out.
- [01:7.726](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=67.72600000000001): residuals\_3 is shown on the screen, written out.
- [01:7.726](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=67.72600000000001): residuals\_4 is shown on the screen, written out.
- [01:7.726](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=67.72600000000001): residuals\_5 is shown on the screen, written out.
- [01:7.726](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=67.72600000000001): residuals\_6 is shown on the screen, written out.
- [01:7.726](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=67.72600000000001): residuals\_7 is shown on the screen, written out.
- [01:7.726](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=67.72600000000001): residuals\_8 is shown on the screen, written out.
- [01:7.726](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=67.72600000000001): residuals\_9 is shown on the screen, written out.
- [01:7.726](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=67.72600000000001): residuals\_10 is shown on the screen, written out.
- [01:7.726](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=67.72600000000001): residuals\_11 is shown on the screen, written out.
- [01:7.726](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=67.72600000000001): residuals\_12 is shown on the screen, written out.
- [01:10.025](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=70.02500000000002): projected is shown on the screen, written out.
- [01:10.025](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=70.02500000000002): projected\_2 is shown on the screen, written out.
- [01:10.025](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=70.02500000000002): projected\_3 is shown on the screen, written out.
- [01:10.025](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=70.02500000000002): projected\_4 is shown on the screen, written out.
- [01:10.025](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=70.02500000000002): projected\_5 is shown on the screen, written out.
- [01:10.025](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=70.02500000000002): projected\_6 is shown on the screen, written out.
- [01:10.025](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=70.02500000000002): projected\_7 is shown on the screen, written out.
- [01:10.025](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=70.02500000000002): projected\_8 is shown on the screen, written out.
- [01:10.025](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=70.02500000000002): projected\_9 is shown on the screen, written out.
- [01:10.025](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=70.02500000000002): projected\_10 is shown on the screen, written out.
- [01:10.025](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=70.02500000000002): projected\_11 is shown on the screen, written out.
- [01:10.025](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=70.02500000000002): projected\_12 is shown on the screen, written out.
- [01:14.193](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=74.19300000000001): score is shown on the screen, written out.

##### [01:20.087](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=80.08700000000002)

Narration: Those scores form a one-dimensional data set. Measure their variance in the ordinary way: square each score, average the squares, and call the result V of u.

Board: centering — a Math \[text\] that says "$x\_i arrow.r x\_i - overline(x)$"; score — a Math \[text\] that says "$z\_i = u^T (x\_i - overline(x))$"; constraint — a Math \[text\] that says "$u^T u = 1$"; plane — an Axes (x\_range=(-4.2, 4.2), y\_range=(-4.2, 4.2), aspect=(1.0, 1.0)); cloud — a Point \[blue\] drawn in plane (location=(-3.1, -2.4), marker\_radius=0.075); cloud\_2 — a Point \[blue\] drawn in plane (location=(-2.7, -3.0), marker\_radius=0.075); cloud\_3 — a Point \[blue\] drawn in plane (location=(-2.2, -1.5), marker\_radius=0.075); cloud\_4 — a Point \[blue\] drawn in plane (location=(-1.6, -2.1), marker\_radius=0.075); cloud\_5 — a Point \[blue\] drawn in plane (location=(-1.1, -0.5), marker\_radius=0.075); cloud\_6 — a Point \[blue\] drawn in plane (location=(-0.5, -1.0), marker\_radius=0.075); cloud\_7 — a Point \[blue\] drawn in plane (location=(0.4, 0.9), marker\_radius=0.075); cloud\_8 — a Point \[blue\] drawn in plane (location=(0.9, 0.4), marker\_radius=0.075); cloud\_9 — a Point \[blue\] drawn in plane (location=(1.4, 2.0), marker\_radius=0.075); cloud\_10 — a Point \[blue\] drawn in plane (location=(2.0, 1.5), marker\_radius=0.075); cloud\_11 — a Point \[blue\] drawn in plane (location=(2.6, 3.0), marker\_radius=0.075); cloud\_12 — a Point \[blue\] drawn in plane (location=(3.1, 2.4), marker\_radius=0.075); centre — a Point \[red\] labelled "overline(x)" drawn in plane; candidate — a Line \[yellow\] labelled "u" drawn in plane (start=((-4.0 \* cos(angle)), (-4.0 \* sin(angle))), end=((4.0 \* cos(angle)), (4.0 \* sin(angle)))); residuals — a Line \[gray\] drawn in plane (start=(-3.1, -2.4), end=((((-3.1 \* cos(angle)) + (-2.4 \* sin(angle))) \* cos(angle)), ((…, dashed=True); residuals\_2 — a Line \[gray\] drawn in plane (start=(-2.7, -3.0), end=((((-2.7 \* cos(angle)) + (-3.0 \* sin(angle))) \* cos(angle)), ((…, dashed=True); residuals\_3 — a Line \[gray\] drawn in plane (start=(-2.2, -1.5), end=((((-2.2 \* cos(angle)) + (-1.5 \* sin(angle))) \* cos(angle)), ((…, dashed=True); residuals\_4 — a Line \[gray\] drawn in plane (start=(-1.6, -2.1), end=((((-1.6 \* cos(angle)) + (-2.1 \* sin(angle))) \* cos(angle)), ((…, dashed=True); residuals\_5 — a Line \[gray\] drawn in plane (start=(-1.1, -0.5), end=((((-1.1 \* cos(angle)) + (-0.5 \* sin(angle))) \* cos(angle)), ((…, dashed=True); residuals\_6 — a Line \[gray\] drawn in plane (start=(-0.5, -1.0), end=((((-0.5 \* cos(angle)) + (-1.0 \* sin(angle))) \* cos(angle)), ((…, dashed=True); residuals\_7 — a Line \[gray\] drawn in plane (start=(0.4, 0.9), end=((((0.4 \* cos(angle)) + (0.9 \* sin(angle))) \* cos(angle)), (((0…, dashed=True); residuals\_8 — a Line \[gray\] drawn in plane (start=(0.9, 0.4), end=((((0.9 \* cos(angle)) + (0.4 \* sin(angle))) \* cos(angle)), (((0…, dashed=True); residuals\_9 — a Line \[gray\] drawn in plane (start=(1.4, 2.0), end=((((1.4 \* cos(angle)) + (2.0 \* sin(angle))) \* cos(angle)), (((1…, dashed=True); residuals\_10 — a Line \[gray\] drawn in plane (start=(2.0, 1.5), end=((((2.0 \* cos(angle)) + (1.5 \* sin(angle))) \* cos(angle)), (((2…, dashed=True); residuals\_11 — a Line \[gray\] drawn in plane (start=(2.6, 3.0), end=((((2.6 \* cos(angle)) + (3.0 \* sin(angle))) \* cos(angle)), (((2…, dashed=True); residuals\_12 — a Line \[gray\] drawn in plane (start=(3.1, 2.4), end=((((3.1 \* cos(angle)) + (2.4 \* sin(angle))) \* cos(angle)), (((3…, dashed=True); projected — a Point \[green\] drawn in plane (location=((((-3.1 \* cos(angle)) + (-2.4 \* sin(angle))) \* cos(angle)), ((…, marker\_radius=0.065); projected\_2 — a Point \[green\] drawn in plane (location=((((-2.7 \* cos(angle)) + (-3.0 \* sin(angle))) \* cos(angle)), ((…, marker\_radius=0.065); projected\_3 — a Point \[green\] drawn in plane (location=((((-2.2 \* cos(angle)) + (-1.5 \* sin(angle))) \* cos(angle)), ((…, marker\_radius=0.065); projected\_4 — a Point \[green\] drawn in plane (location=((((-1.6 \* cos(angle)) + (-2.1 \* sin(angle))) \* cos(angle)), ((…, marker\_radius=0.065); projected\_5 — a Point \[green\] drawn in plane (location=((((-1.1 \* cos(angle)) + (-0.5 \* sin(angle))) \* cos(angle)), ((…, marker\_radius=0.065); projected\_6 — a Point \[green\] drawn in plane (location=((((-0.5 \* cos(angle)) + (-1.0 \* sin(angle))) \* cos(angle)), ((…, marker\_radius=0.065); projected\_7 — a Point \[green\] drawn in plane (location=((((0.4 \* cos(angle)) + (0.9 \* sin(angle))) \* cos(angle)), (((0…, marker\_radius=0.065); projected\_8 — a Point \[green\] drawn in plane (location=((((0.9 \* cos(angle)) + (0.4 \* sin(angle))) \* cos(angle)), (((0…, marker\_radius=0.065); projected\_9 — a Point \[green\] drawn in plane (location=((((1.4 \* cos(angle)) + (2.0 \* sin(angle))) \* cos(angle)), (((1…, marker\_radius=0.065); projected\_10 — a Point \[green\] drawn in plane (location=((((2.0 \* cos(angle)) + (1.5 \* sin(angle))) \* cos(angle)), (((2…, marker\_radius=0.065); projected\_11 — a Point \[green\] drawn in plane (location=((((2.6 \* cos(angle)) + (3.0 \* sin(angle))) \* cos(angle)), (((2…, marker\_radius=0.065); projected\_12 — a Point \[green\] drawn in plane (location=((((3.1 \* cos(angle)) + (2.4 \* sin(angle))) \* cos(angle)), (((3…, marker\_radius=0.065)

Actions:
- [01:24.151](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=84.15100000000002): variance is shown on the screen, written out.
- [01:29.503](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=89.50300000000003): variance\_line is shown on the screen, written out.
- [01:29.503](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=89.50300000000003): variance\_marker is shown on the screen, written out.

##### [01:31.833](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=91.83300000000001)

Narration: At the moment the line is horizontal. The green points have some spread, and the marker reports variance four. That is neither especially small nor the largest value available.

Board: centering — a Math \[text\] that says "$x\_i arrow.r x\_i - overline(x)$"; score — a Math \[text\] that says "$z\_i = u^T (x\_i - overline(x))$"; variance — a Math \[text\] that says "$V(u) = frac(1, n) sum\_i z\_i^2$"; constraint — a Math \[text\] that says "$u^T u = 1$"; variance\_line — a NumberLine labelled "V(u)" (x\_range=(0.0, 8.0), include\_numbers=True); plane — an Axes (x\_range=(-4.2, 4.2), y\_range=(-4.2, 4.2), aspect=(1.0, 1.0)); cloud — a Point \[blue\] drawn in plane (location=(-3.1, -2.4), marker\_radius=0.075); cloud\_2 — a Point \[blue\] drawn in plane (location=(-2.7, -3.0), marker\_radius=0.075); cloud\_3 — a Point \[blue\] drawn in plane (location=(-2.2, -1.5), marker\_radius=0.075); cloud\_4 — a Point \[blue\] drawn in plane (location=(-1.6, -2.1), marker\_radius=0.075); cloud\_5 — a Point \[blue\] drawn in plane (location=(-1.1, -0.5), marker\_radius=0.075); cloud\_6 — a Point \[blue\] drawn in plane (location=(-0.5, -1.0), marker\_radius=0.075); cloud\_7 — a Point \[blue\] drawn in plane (location=(0.4, 0.9), marker\_radius=0.075); cloud\_8 — a Point \[blue\] drawn in plane (location=(0.9, 0.4), marker\_radius=0.075); cloud\_9 — a Point \[blue\] drawn in plane (location=(1.4, 2.0), marker\_radius=0.075); cloud\_10 — a Point \[blue\] drawn in plane (location=(2.0, 1.5), marker\_radius=0.075); cloud\_11 — a Point \[blue\] drawn in plane (location=(2.6, 3.0), marker\_radius=0.075); cloud\_12 — a Point \[blue\] drawn in plane (location=(3.1, 2.4), marker\_radius=0.075); centre — a Point \[red\] labelled "overline(x)" drawn in plane; candidate — a Line \[yellow\] labelled "u" drawn in plane (start=((-4.0 \* cos(angle)), (-4.0 \* sin(angle))), end=((4.0 \* cos(angle)), (4.0 \* sin(angle)))); residuals — a Line \[gray\] drawn in plane (start=(-3.1, -2.4), end=((((-3.1 \* cos(angle)) + (-2.4 \* sin(angle))) \* cos(angle)), ((…, dashed=True); residuals\_2 — a Line \[gray\] drawn in plane (start=(-2.7, -3.0), end=((((-2.7 \* cos(angle)) + (-3.0 \* sin(angle))) \* cos(angle)), ((…, dashed=True); residuals\_3 — a Line \[gray\] drawn in plane (start=(-2.2, -1.5), end=((((-2.2 \* cos(angle)) + (-1.5 \* sin(angle))) \* cos(angle)), ((…, dashed=True); residuals\_4 — a Line \[gray\] drawn in plane (start=(-1.6, -2.1), end=((((-1.6 \* cos(angle)) + (-2.1 \* sin(angle))) \* cos(angle)), ((…, dashed=True); residuals\_5 — a Line \[gray\] drawn in plane (start=(-1.1, -0.5), end=((((-1.1 \* cos(angle)) + (-0.5 \* sin(angle))) \* cos(angle)), ((…, dashed=True); residuals\_6 — a Line \[gray\] drawn in plane (start=(-0.5, -1.0), end=((((-0.5 \* cos(angle)) + (-1.0 \* sin(angle))) \* cos(angle)), ((…, dashed=True); residuals\_7 — a Line \[gray\] drawn in plane (start=(0.4, 0.9), end=((((0.4 \* cos(angle)) + (0.9 \* sin(angle))) \* cos(angle)), (((0…, dashed=True); residuals\_8 — a Line \[gray\] drawn in plane (start=(0.9, 0.4), end=((((0.9 \* cos(angle)) + (0.4 \* sin(angle))) \* cos(angle)), (((0…, dashed=True); residuals\_9 — a Line \[gray\] drawn in plane (start=(1.4, 2.0), end=((((1.4 \* cos(angle)) + (2.0 \* sin(angle))) \* cos(angle)), (((1…, dashed=True); residuals\_10 — a Line \[gray\] drawn in plane (start=(2.0, 1.5), end=((((2.0 \* cos(angle)) + (1.5 \* sin(angle))) \* cos(angle)), (((2…, dashed=True); residuals\_11 — a Line \[gray\] drawn in plane (start=(2.6, 3.0), end=((((2.6 \* cos(angle)) + (3.0 \* sin(angle))) \* cos(angle)), (((2…, dashed=True); residuals\_12 — a Line \[gray\] drawn in plane (start=(3.1, 2.4), end=((((3.1 \* cos(angle)) + (2.4 \* sin(angle))) \* cos(angle)), (((3…, dashed=True); projected — a Point \[green\] drawn in plane (location=((((-3.1 \* cos(angle)) + (-2.4 \* sin(angle))) \* cos(angle)), ((…, marker\_radius=0.065); projected\_2 — a Point \[green\] drawn in plane (location=((((-2.7 \* cos(angle)) + (-3.0 \* sin(angle))) \* cos(angle)), ((…, marker\_radius=0.065); projected\_3 — a Point \[green\] drawn in plane (location=((((-2.2 \* cos(angle)) + (-1.5 \* sin(angle))) \* cos(angle)), ((…, marker\_radius=0.065); projected\_4 — a Point \[green\] drawn in plane (location=((((-1.6 \* cos(angle)) + (-2.1 \* sin(angle))) \* cos(angle)), ((…, marker\_radius=0.065); projected\_5 — a Point \[green\] drawn in plane (location=((((-1.1 \* cos(angle)) + (-0.5 \* sin(angle))) \* cos(angle)), ((…, marker\_radius=0.065); projected\_6 — a Point \[green\] drawn in plane (location=((((-0.5 \* cos(angle)) + (-1.0 \* sin(angle))) \* cos(angle)), ((…, marker\_radius=0.065); projected\_7 — a Point \[green\] drawn in plane (location=((((0.4 \* cos(angle)) + (0.9 \* sin(angle))) \* cos(angle)), (((0…, marker\_radius=0.065); projected\_8 — a Point \[green\] drawn in plane (location=((((0.9 \* cos(angle)) + (0.4 \* sin(angle))) \* cos(angle)), (((0…, marker\_radius=0.065); projected\_9 — a Point \[green\] drawn in plane (location=((((1.4 \* cos(angle)) + (2.0 \* sin(angle))) \* cos(angle)), (((1…, marker\_radius=0.065); projected\_10 — a Point \[green\] drawn in plane (location=((((2.0 \* cos(angle)) + (1.5 \* sin(angle))) \* cos(angle)), (((2…, marker\_radius=0.065); projected\_11 — a Point \[green\] drawn in plane (location=((((2.6 \* cos(angle)) + (3.0 \* sin(angle))) \* cos(angle)), (((2…, marker\_radius=0.065); projected\_12 — a Point \[green\] drawn in plane (location=((((3.1 \* cos(angle)) + (2.4 \* sin(angle))) \* cos(angle)), (((3…, marker\_radius=0.065); variance\_marker — a Point \[yellow\] labelled "V(u)" drawn in variance\_line (location=((4.0 + (3.0 \* sin((2.0 \* angle)))), 0.0))

Actions:
- [01:38.149](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=98.14900000000002): variance\_marker is indicated — a transient flash.

##### [01:43.533](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=103.53250000000001)

Narration: Turn the candidate line toward the visible long axis of the cloud. The projected points separate, so their one-dimensional variance rises. It reaches seven at this direction, where the spread is greatest.

Board: Unchanged from the preceding beat in this scene.

Actions:
- [01:43.881](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=103.88100000000001): candidate is redrawn as the numbers it depends on change.
- [01:43.881](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=103.88100000000001): residuals is redrawn as the numbers it depends on change.
- [01:43.881](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=103.88100000000001): residuals\_2 is redrawn as the numbers it depends on change.
- [01:43.881](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=103.88100000000001): residuals\_3 is redrawn as the numbers it depends on change.
- [01:43.881](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=103.88100000000001): residuals\_4 is redrawn as the numbers it depends on change.
- [01:43.881](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=103.88100000000001): residuals\_5 is redrawn as the numbers it depends on change.
- [01:43.881](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=103.88100000000001): residuals\_6 is redrawn as the numbers it depends on change.
- [01:43.881](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=103.88100000000001): residuals\_7 is redrawn as the numbers it depends on change.
- [01:43.881](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=103.88100000000001): residuals\_8 is redrawn as the numbers it depends on change.
- [01:43.881](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=103.88100000000001): residuals\_9 is redrawn as the numbers it depends on change.
- [01:43.881](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=103.88100000000001): residuals\_10 is redrawn as the numbers it depends on change.
- [01:43.881](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=103.88100000000001): residuals\_11 is redrawn as the numbers it depends on change.
- [01:43.881](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=103.88100000000001): residuals\_12 is redrawn as the numbers it depends on change.
- [01:43.881](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=103.88100000000001): projected is redrawn as the numbers it depends on change.
- [01:43.881](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=103.88100000000001): projected\_2 is redrawn as the numbers it depends on change.
- [01:43.881](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=103.88100000000001): projected\_3 is redrawn as the numbers it depends on change.
- [01:43.881](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=103.88100000000001): projected\_4 is redrawn as the numbers it depends on change.
- [01:43.881](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=103.88100000000001): projected\_5 is redrawn as the numbers it depends on change.
- [01:43.881](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=103.88100000000001): projected\_6 is redrawn as the numbers it depends on change.
- [01:43.881](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=103.88100000000001): projected\_7 is redrawn as the numbers it depends on change.
- [01:43.881](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=103.88100000000001): projected\_8 is redrawn as the numbers it depends on change.
- [01:43.881](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=103.88100000000001): projected\_9 is redrawn as the numbers it depends on change.
- [01:43.881](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=103.88100000000001): projected\_10 is redrawn as the numbers it depends on change.
- [01:43.881](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=103.88100000000001): projected\_11 is redrawn as the numbers it depends on change.
- [01:43.881](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=103.88100000000001): projected\_12 is redrawn as the numbers it depends on change.
- [01:43.881](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=103.88100000000001): variance\_marker is redrawn as the numbers it depends on change.
- [01:43.881](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=103.88100000000001): angle ticks to 0.7853981633974483.
- [01:52.972](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=112.97200000000001): variance\_marker is indicated — a transient flash.

##### [01:56.474](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=116.47400000000002)

Narration: Continue turning. The projections draw closer together again. On the vertical axis the variance has fallen back to four, even though no data point has moved.

Board: Unchanged from the preceding beat in this scene.

Actions:
- [01:56.776](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=116.77600000000002): candidate is redrawn as the numbers it depends on change.
- [01:56.776](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=116.77600000000002): residuals is redrawn as the numbers it depends on change.
- [01:56.776](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=116.77600000000002): residuals\_2 is redrawn as the numbers it depends on change.
- [01:56.776](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=116.77600000000002): residuals\_3 is redrawn as the numbers it depends on change.
- [01:56.776](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=116.77600000000002): residuals\_4 is redrawn as the numbers it depends on change.
- [01:56.776](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=116.77600000000002): residuals\_5 is redrawn as the numbers it depends on change.
- [01:56.776](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=116.77600000000002): residuals\_6 is redrawn as the numbers it depends on change.
- [01:56.776](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=116.77600000000002): residuals\_7 is redrawn as the numbers it depends on change.
- [01:56.776](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=116.77600000000002): residuals\_8 is redrawn as the numbers it depends on change.
- [01:56.776](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=116.77600000000002): residuals\_9 is redrawn as the numbers it depends on change.
- [01:56.776](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=116.77600000000002): residuals\_10 is redrawn as the numbers it depends on change.
- [01:56.776](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=116.77600000000002): residuals\_11 is redrawn as the numbers it depends on change.
- [01:56.776](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=116.77600000000002): residuals\_12 is redrawn as the numbers it depends on change.
- [01:56.776](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=116.77600000000002): projected is redrawn as the numbers it depends on change.
- [01:56.776](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=116.77600000000002): projected\_2 is redrawn as the numbers it depends on change.
- [01:56.776](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=116.77600000000002): projected\_3 is redrawn as the numbers it depends on change.
- [01:56.776](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=116.77600000000002): projected\_4 is redrawn as the numbers it depends on change.
- [01:56.776](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=116.77600000000002): projected\_5 is redrawn as the numbers it depends on change.
- [01:56.776](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=116.77600000000002): projected\_6 is redrawn as the numbers it depends on change.
- [01:56.776](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=116.77600000000002): projected\_7 is redrawn as the numbers it depends on change.
- [01:56.776](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=116.77600000000002): projected\_8 is redrawn as the numbers it depends on change.
- [01:56.776](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=116.77600000000002): projected\_9 is redrawn as the numbers it depends on change.
- [01:56.776](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=116.77600000000002): projected\_10 is redrawn as the numbers it depends on change.
- [01:56.776](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=116.77600000000002): projected\_11 is redrawn as the numbers it depends on change.
- [01:56.776](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=116.77600000000002): projected\_12 is redrawn as the numbers it depends on change.
- [01:56.776](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=116.77600000000002): variance\_marker is redrawn as the numbers it depends on change.
- [01:56.776](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=116.77600000000002): angle ticks to 1.5707963267948966.
- [02:4.102](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=124.10200000000002): variance\_marker is indicated — a transient flash.

##### [02:7.431](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=127.43050000000002)

Narration: Keep turning toward the short axis of the cloud. The green points bunch tightly around the origin, and the variance falls to one. This direction preserves very little of the cloud's visible spread.

Board: Unchanged from the preceding beat in this scene.

Actions:
- [02:7.779](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=127.77900000000002): candidate is redrawn as the numbers it depends on change.
- [02:7.779](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=127.77900000000002): residuals is redrawn as the numbers it depends on change.
- [02:7.779](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=127.77900000000002): residuals\_2 is redrawn as the numbers it depends on change.
- [02:7.779](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=127.77900000000002): residuals\_3 is redrawn as the numbers it depends on change.
- [02:7.779](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=127.77900000000002): residuals\_4 is redrawn as the numbers it depends on change.
- [02:7.779](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=127.77900000000002): residuals\_5 is redrawn as the numbers it depends on change.
- [02:7.779](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=127.77900000000002): residuals\_6 is redrawn as the numbers it depends on change.
- [02:7.779](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=127.77900000000002): residuals\_7 is redrawn as the numbers it depends on change.
- [02:7.779](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=127.77900000000002): residuals\_8 is redrawn as the numbers it depends on change.
- [02:7.779](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=127.77900000000002): residuals\_9 is redrawn as the numbers it depends on change.
- [02:7.779](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=127.77900000000002): residuals\_10 is redrawn as the numbers it depends on change.
- [02:7.779](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=127.77900000000002): residuals\_11 is redrawn as the numbers it depends on change.
- [02:7.779](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=127.77900000000002): residuals\_12 is redrawn as the numbers it depends on change.
- [02:7.779](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=127.77900000000002): projected is redrawn as the numbers it depends on change.
- [02:7.779](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=127.77900000000002): projected\_2 is redrawn as the numbers it depends on change.
- [02:7.779](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=127.77900000000002): projected\_3 is redrawn as the numbers it depends on change.
- [02:7.779](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=127.77900000000002): projected\_4 is redrawn as the numbers it depends on change.
- [02:7.779](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=127.77900000000002): projected\_5 is redrawn as the numbers it depends on change.
- [02:7.779](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=127.77900000000002): projected\_6 is redrawn as the numbers it depends on change.
- [02:7.779](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=127.77900000000002): projected\_7 is redrawn as the numbers it depends on change.
- [02:7.779](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=127.77900000000002): projected\_8 is redrawn as the numbers it depends on change.
- [02:7.779](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=127.77900000000002): projected\_9 is redrawn as the numbers it depends on change.
- [02:7.779](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=127.77900000000002): projected\_10 is redrawn as the numbers it depends on change.
- [02:7.779](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=127.77900000000002): projected\_11 is redrawn as the numbers it depends on change.
- [02:7.779](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=127.77900000000002): projected\_12 is redrawn as the numbers it depends on change.
- [02:7.779](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=127.77900000000002): variance\_marker is redrawn as the numbers it depends on change.
- [02:7.779](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=127.77900000000002): angle ticks to 2.356194490192345.
- [02:14.559](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=134.55900000000003): variance\_marker is indicated — a transient flash.

##### [02:19.885](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=139.88500000000002)

Narration: Now return to the maximum. The data stayed fixed throughout. Only the question we asked of them changed: how much of their variation can be seen from this direction?

Board: Unchanged from the preceding beat in this scene.

Actions:
- [02:20.442](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=140.44200000000004): candidate is redrawn as the numbers it depends on change.
- [02:20.442](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=140.44200000000004): residuals is redrawn as the numbers it depends on change.
- [02:20.442](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=140.44200000000004): residuals\_2 is redrawn as the numbers it depends on change.
- [02:20.442](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=140.44200000000004): residuals\_3 is redrawn as the numbers it depends on change.
- [02:20.442](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=140.44200000000004): residuals\_4 is redrawn as the numbers it depends on change.
- [02:20.442](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=140.44200000000004): residuals\_5 is redrawn as the numbers it depends on change.
- [02:20.442](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=140.44200000000004): residuals\_6 is redrawn as the numbers it depends on change.
- [02:20.442](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=140.44200000000004): residuals\_7 is redrawn as the numbers it depends on change.
- [02:20.442](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=140.44200000000004): residuals\_8 is redrawn as the numbers it depends on change.
- [02:20.442](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=140.44200000000004): residuals\_9 is redrawn as the numbers it depends on change.
- [02:20.442](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=140.44200000000004): residuals\_10 is redrawn as the numbers it depends on change.
- [02:20.442](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=140.44200000000004): residuals\_11 is redrawn as the numbers it depends on change.
- [02:20.442](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=140.44200000000004): residuals\_12 is redrawn as the numbers it depends on change.
- [02:20.442](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=140.44200000000004): projected is redrawn as the numbers it depends on change.
- [02:20.442](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=140.44200000000004): projected\_2 is redrawn as the numbers it depends on change.
- [02:20.442](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=140.44200000000004): projected\_3 is redrawn as the numbers it depends on change.
- [02:20.442](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=140.44200000000004): projected\_4 is redrawn as the numbers it depends on change.
- [02:20.442](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=140.44200000000004): projected\_5 is redrawn as the numbers it depends on change.
- [02:20.442](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=140.44200000000004): projected\_6 is redrawn as the numbers it depends on change.
- [02:20.442](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=140.44200000000004): projected\_7 is redrawn as the numbers it depends on change.
- [02:20.442](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=140.44200000000004): projected\_8 is redrawn as the numbers it depends on change.
- [02:20.442](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=140.44200000000004): projected\_9 is redrawn as the numbers it depends on change.
- [02:20.442](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=140.44200000000004): projected\_10 is redrawn as the numbers it depends on change.
- [02:20.442](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=140.44200000000004): projected\_11 is redrawn as the numbers it depends on change.
- [02:20.442](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=140.44200000000004): projected\_12 is redrawn as the numbers it depends on change.
- [02:20.442](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=140.44200000000004): variance\_marker is redrawn as the numbers it depends on change.
- [02:20.442](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=140.44200000000004): angle ticks to 0.7853981633974483.

##### [02:30.586](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=150.58550000000002)

Narration: This is PCA's central geometric operation. It searches over all unit directions, projects the centered observations, and chooses the direction whose scores have the largest variance.

Board: Unchanged from the preceding beat in this scene.

Actions:
- [02:39.49](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=159.49): candidate is indicated — a transient flash.
- [02:40.814](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=160.81400000000002): variance\_marker is indicated — a transient flash.

##### [02:42.819](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=162.81850000000003)

Narration: That maximizing direction is the first principal component. A component is a direction in variable space. The projected coordinate is the score of an observation on that component. Those are related objects, but they are not the same thing.

Board: Unchanged from the preceding beat in this scene.

Actions:
- [02:44.966](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=164.96600000000004): residuals is hidden from the screen.
- [02:44.966](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=164.96600000000004): residuals\_2 is hidden from the screen.
- [02:44.966](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=164.96600000000004): residuals\_3 is hidden from the screen.
- [02:44.966](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=164.96600000000004): residuals\_4 is hidden from the screen.
- [02:44.966](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=164.96600000000004): residuals\_5 is hidden from the screen.
- [02:44.966](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=164.96600000000004): residuals\_6 is hidden from the screen.
- [02:44.966](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=164.96600000000004): residuals\_7 is hidden from the screen.
- [02:44.966](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=164.96600000000004): residuals\_8 is hidden from the screen.
- [02:44.966](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=164.96600000000004): residuals\_9 is hidden from the screen.
- [02:44.966](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=164.96600000000004): residuals\_10 is hidden from the screen.
- [02:44.966](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=164.96600000000004): residuals\_11 is hidden from the screen.
- [02:44.966](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=164.96600000000004): residuals\_12 is hidden from the screen.
- [02:57.934](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=177.93400000000003): centering is hidden from the screen — left the board.
- [02:57.934](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=177.93400000000003): constraint is hidden from the screen — left the board.
- [02:57.934](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=177.93400000000003): score is hidden from the screen — left the board.
- [02:57.934](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=177.93400000000003): variance is hidden from the screen — left the board.
- [02:57.934](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=177.93400000000003): variance\_line is hidden from the screen — left the board.
- [02:57.934](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=177.93400000000003): variance\_marker is hidden from the screen — variance\_line left the board.

##### [02:59.134](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=179.13400000000001)

Narration: Write the first component as v one: the unit direction that maximizes projected variance. Its variance, seven in this example, tells us how much variation the component captures.

Board: plane — an Axes (x\_range=(-4.2, 4.2), y\_range=(-4.2, 4.2), aspect=(1.0, 1.0)); cloud — a Point \[blue\] drawn in plane (location=(-3.1, -2.4), marker\_radius=0.075); cloud\_2 — a Point \[blue\] drawn in plane (location=(-2.7, -3.0), marker\_radius=0.075); cloud\_3 — a Point \[blue\] drawn in plane (location=(-2.2, -1.5), marker\_radius=0.075); cloud\_4 — a Point \[blue\] drawn in plane (location=(-1.6, -2.1), marker\_radius=0.075); cloud\_5 — a Point \[blue\] drawn in plane (location=(-1.1, -0.5), marker\_radius=0.075); cloud\_6 — a Point \[blue\] drawn in plane (location=(-0.5, -1.0), marker\_radius=0.075); cloud\_7 — a Point \[blue\] drawn in plane (location=(0.4, 0.9), marker\_radius=0.075); cloud\_8 — a Point \[blue\] drawn in plane (location=(0.9, 0.4), marker\_radius=0.075); cloud\_9 — a Point \[blue\] drawn in plane (location=(1.4, 2.0), marker\_radius=0.075); cloud\_10 — a Point \[blue\] drawn in plane (location=(2.0, 1.5), marker\_radius=0.075); cloud\_11 — a Point \[blue\] drawn in plane (location=(2.6, 3.0), marker\_radius=0.075); cloud\_12 — a Point \[blue\] drawn in plane (location=(3.1, 2.4), marker\_radius=0.075); centre — a Point \[red\] labelled "overline(x)" drawn in plane; candidate — a Line \[yellow\] labelled "u" drawn in plane (start=((-4.0 \* cos(angle)), (-4.0 \* sin(angle))), end=((4.0 \* cos(angle)), (4.0 \* sin(angle)))); projected — a Point \[green\] drawn in plane (location=((((-3.1 \* cos(angle)) + (-2.4 \* sin(angle))) \* cos(angle)), ((…, marker\_radius=0.065); projected\_2 — a Point \[green\] drawn in plane (location=((((-2.7 \* cos(angle)) + (-3.0 \* sin(angle))) \* cos(angle)), ((…, marker\_radius=0.065); projected\_3 — a Point \[green\] drawn in plane (location=((((-2.2 \* cos(angle)) + (-1.5 \* sin(angle))) \* cos(angle)), ((…, marker\_radius=0.065); projected\_4 — a Point \[green\] drawn in plane (location=((((-1.6 \* cos(angle)) + (-2.1 \* sin(angle))) \* cos(angle)), ((…, marker\_radius=0.065); projected\_5 — a Point \[green\] drawn in plane (location=((((-1.1 \* cos(angle)) + (-0.5 \* sin(angle))) \* cos(angle)), ((…, marker\_radius=0.065); projected\_6 — a Point \[green\] drawn in plane (location=((((-0.5 \* cos(angle)) + (-1.0 \* sin(angle))) \* cos(angle)), ((…, marker\_radius=0.065); projected\_7 — a Point \[green\] drawn in plane (location=((((0.4 \* cos(angle)) + (0.9 \* sin(angle))) \* cos(angle)), (((0…, marker\_radius=0.065); projected\_8 — a Point \[green\] drawn in plane (location=((((0.9 \* cos(angle)) + (0.4 \* sin(angle))) \* cos(angle)), (((0…, marker\_radius=0.065); projected\_9 — a Point \[green\] drawn in plane (location=((((1.4 \* cos(angle)) + (2.0 \* sin(angle))) \* cos(angle)), (((1…, marker\_radius=0.065); projected\_10 — a Point \[green\] drawn in plane (location=((((2.0 \* cos(angle)) + (1.5 \* sin(angle))) \* cos(angle)), (((2…, marker\_radius=0.065); projected\_11 — a Point \[green\] drawn in plane (location=((((2.6 \* cos(angle)) + (3.0 \* sin(angle))) \* cos(angle)), (((2…, marker\_radius=0.065); projected\_12 — a Point \[green\] drawn in plane (location=((((3.1 \* cos(angle)) + (2.4 \* sin(angle))) \* cos(angle)), (((3…, marker\_radius=0.065)

Actions:
- [02:59.134](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=179.13400000000001): heading is shown on the screen, written out.
- [03:0.017](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=180.01700000000005): pc1 is shown on the screen, written out.
- [03:6.774](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=186.77400000000006): variance\_split is shown on the screen, written out.

##### [03:11.786](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=191.786)

Narration: After choosing v one, PCA looks for another direction that captures as much remaining variance as possible without repeating the first answer. In two dimensions, there is only one perpendicular direction available.

Board: plane — an Axes (x\_range=(-4.2, 4.2), y\_range=(-4.2, 4.2), aspect=(1.0, 1.0)); cloud — a Point \[blue\] drawn in plane (location=(-3.1, -2.4), marker\_radius=0.075); cloud\_2 — a Point \[blue\] drawn in plane (location=(-2.7, -3.0), marker\_radius=0.075); cloud\_3 — a Point \[blue\] drawn in plane (location=(-2.2, -1.5), marker\_radius=0.075); cloud\_4 — a Point \[blue\] drawn in plane (location=(-1.6, -2.1), marker\_radius=0.075); cloud\_5 — a Point \[blue\] drawn in plane (location=(-1.1, -0.5), marker\_radius=0.075); cloud\_6 — a Point \[blue\] drawn in plane (location=(-0.5, -1.0), marker\_radius=0.075); cloud\_7 — a Point \[blue\] drawn in plane (location=(0.4, 0.9), marker\_radius=0.075); cloud\_8 — a Point \[blue\] drawn in plane (location=(0.9, 0.4), marker\_radius=0.075); cloud\_9 — a Point \[blue\] drawn in plane (location=(1.4, 2.0), marker\_radius=0.075); cloud\_10 — a Point \[blue\] drawn in plane (location=(2.0, 1.5), marker\_radius=0.075); cloud\_11 — a Point \[blue\] drawn in plane (location=(2.6, 3.0), marker\_radius=0.075); cloud\_12 — a Point \[blue\] drawn in plane (location=(3.1, 2.4), marker\_radius=0.075); centre — a Point \[red\] labelled "overline(x)" drawn in plane; candidate — a Line \[yellow\] labelled "u" drawn in plane (start=((-4.0 \* cos(angle)), (-4.0 \* sin(angle))), end=((4.0 \* cos(angle)), (4.0 \* sin(angle)))); projected — a Point \[green\] drawn in plane (location=((((-3.1 \* cos(angle)) + (-2.4 \* sin(angle))) \* cos(angle)), ((…, marker\_radius=0.065); projected\_2 — a Point \[green\] drawn in plane (location=((((-2.7 \* cos(angle)) + (-3.0 \* sin(angle))) \* cos(angle)), ((…, marker\_radius=0.065); projected\_3 — a Point \[green\] drawn in plane (location=((((-2.2 \* cos(angle)) + (-1.5 \* sin(angle))) \* cos(angle)), ((…, marker\_radius=0.065); projected\_4 — a Point \[green\] drawn in plane (location=((((-1.6 \* cos(angle)) + (-2.1 \* sin(angle))) \* cos(angle)), ((…, marker\_radius=0.065); projected\_5 — a Point \[green\] drawn in plane (location=((((-1.1 \* cos(angle)) + (-0.5 \* sin(angle))) \* cos(angle)), ((…, marker\_radius=0.065); projected\_6 — a Point \[green\] drawn in plane (location=((((-0.5 \* cos(angle)) + (-1.0 \* sin(angle))) \* cos(angle)), ((…, marker\_radius=0.065); projected\_7 — a Point \[green\] drawn in plane (location=((((0.4 \* cos(angle)) + (0.9 \* sin(angle))) \* cos(angle)), (((0…, marker\_radius=0.065); projected\_8 — a Point \[green\] drawn in plane (location=((((0.9 \* cos(angle)) + (0.4 \* sin(angle))) \* cos(angle)), (((0…, marker\_radius=0.065); projected\_9 — a Point \[green\] drawn in plane (location=((((1.4 \* cos(angle)) + (2.0 \* sin(angle))) \* cos(angle)), (((1…, marker\_radius=0.065); projected\_10 — a Point \[green\] drawn in plane (location=((((2.0 \* cos(angle)) + (1.5 \* sin(angle))) \* cos(angle)), (((2…, marker\_radius=0.065); projected\_11 — a Point \[green\] drawn in plane (location=((((2.6 \* cos(angle)) + (3.0 \* sin(angle))) \* cos(angle)), (((2…, marker\_radius=0.065); projected\_12 — a Point \[green\] drawn in plane (location=((((3.1 \* cos(angle)) + (2.4 \* sin(angle))) \* cos(angle)), (((3…, marker\_radius=0.065); heading — a Heading that says "Rotate a Direction and Watch the Spread"; pc1 — a Math \[text\] that says "$v\_1 = arg max\_(u^T u = 1) V(u)$"; variance\_split — a Math \[text\] that says "$lambda\_1 = 7, quad lambda\_2 = 1$"

Actions:
- [03:14.967](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=194.967): pc2\_line is shown on the screen, written out.
- [03:22.478](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=202.47800000000004): pc2 is shown on the screen, written out.

##### [03:25.11](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=205.11)

Narration: That green direction is the second principal component. The cloud is narrow along it, so its variance is only one. The first component describes the long way through the cloud, and the second describes the remaining side-to-side variation.

Board: plane — an Axes (x\_range=(-4.2, 4.2), y\_range=(-4.2, 4.2), aspect=(1.0, 1.0)); cloud — a Point \[blue\] drawn in plane (location=(-3.1, -2.4), marker\_radius=0.075); cloud\_2 — a Point \[blue\] drawn in plane (location=(-2.7, -3.0), marker\_radius=0.075); cloud\_3 — a Point \[blue\] drawn in plane (location=(-2.2, -1.5), marker\_radius=0.075); cloud\_4 — a Point \[blue\] drawn in plane (location=(-1.6, -2.1), marker\_radius=0.075); cloud\_5 — a Point \[blue\] drawn in plane (location=(-1.1, -0.5), marker\_radius=0.075); cloud\_6 — a Point \[blue\] drawn in plane (location=(-0.5, -1.0), marker\_radius=0.075); cloud\_7 — a Point \[blue\] drawn in plane (location=(0.4, 0.9), marker\_radius=0.075); cloud\_8 — a Point \[blue\] drawn in plane (location=(0.9, 0.4), marker\_radius=0.075); cloud\_9 — a Point \[blue\] drawn in plane (location=(1.4, 2.0), marker\_radius=0.075); cloud\_10 — a Point \[blue\] drawn in plane (location=(2.0, 1.5), marker\_radius=0.075); cloud\_11 — a Point \[blue\] drawn in plane (location=(2.6, 3.0), marker\_radius=0.075); cloud\_12 — a Point \[blue\] drawn in plane (location=(3.1, 2.4), marker\_radius=0.075); centre — a Point \[red\] labelled "overline(x)" drawn in plane; candidate — a Line \[yellow\] labelled "u" drawn in plane (start=((-4.0 \* cos(angle)), (-4.0 \* sin(angle))), end=((4.0 \* cos(angle)), (4.0 \* sin(angle)))); projected — a Point \[green\] drawn in plane (location=((((-3.1 \* cos(angle)) + (-2.4 \* sin(angle))) \* cos(angle)), ((…, marker\_radius=0.065); projected\_2 — a Point \[green\] drawn in plane (location=((((-2.7 \* cos(angle)) + (-3.0 \* sin(angle))) \* cos(angle)), ((…, marker\_radius=0.065); projected\_3 — a Point \[green\] drawn in plane (location=((((-2.2 \* cos(angle)) + (-1.5 \* sin(angle))) \* cos(angle)), ((…, marker\_radius=0.065); projected\_4 — a Point \[green\] drawn in plane (location=((((-1.6 \* cos(angle)) + (-2.1 \* sin(angle))) \* cos(angle)), ((…, marker\_radius=0.065); projected\_5 — a Point \[green\] drawn in plane (location=((((-1.1 \* cos(angle)) + (-0.5 \* sin(angle))) \* cos(angle)), ((…, marker\_radius=0.065); projected\_6 — a Point \[green\] drawn in plane (location=((((-0.5 \* cos(angle)) + (-1.0 \* sin(angle))) \* cos(angle)), ((…, marker\_radius=0.065); projected\_7 — a Point \[green\] drawn in plane (location=((((0.4 \* cos(angle)) + (0.9 \* sin(angle))) \* cos(angle)), (((0…, marker\_radius=0.065); projected\_8 — a Point \[green\] drawn in plane (location=((((0.9 \* cos(angle)) + (0.4 \* sin(angle))) \* cos(angle)), (((0…, marker\_radius=0.065); projected\_9 — a Point \[green\] drawn in plane (location=((((1.4 \* cos(angle)) + (2.0 \* sin(angle))) \* cos(angle)), (((1…, marker\_radius=0.065); projected\_10 — a Point \[green\] drawn in plane (location=((((2.0 \* cos(angle)) + (1.5 \* sin(angle))) \* cos(angle)), (((2…, marker\_radius=0.065); projected\_11 — a Point \[green\] drawn in plane (location=((((2.6 \* cos(angle)) + (3.0 \* sin(angle))) \* cos(angle)), (((2…, marker\_radius=0.065); projected\_12 — a Point \[green\] drawn in plane (location=((((3.1 \* cos(angle)) + (2.4 \* sin(angle))) \* cos(angle)), (((3…, marker\_radius=0.065); heading — a Heading that says "Rotate a Direction and Watch the Spread"; pc1 — a Math \[text\] that says "$v\_1 = arg max\_(u^T u = 1) V(u)$"; variance\_split — a Math \[text\] that says "$lambda\_1 = 7, quad lambda\_2 = 1$"; pc2 — a Math \[text\] that says "$v\_2 bot v\_1$"; pc2\_line — a Line \[green\] labelled "v\_2" drawn in plane (start=(2.8, -2.8), end=(-2.8, 2.8))

Actions:
- [03:25.714](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=205.71400000000006): pc2\_line is indicated — a transient flash.
- [03:33.005](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=213.00500000000005): variance\_split (the "lambda\_1 = 7" part) is emphasized.
- [03:36.197](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=216.19700000000006): variance\_split (the "lambda\_1 = 7" part) is no longer emphasized.
- [03:36.197](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=216.19700000000006): variance\_split (the "lambda\_2 = 1" part) is emphasized.
- [03:39.39](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=219.39000000000001): variance\_split (the "lambda\_2 = 1" part) is no longer emphasized.

##### [03:39.99](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=219.99)

Narration: The two component directions are perpendicular and have unit length. Together they form a rotated coordinate system. PCA has not bent the data or invented a nonlinear feature. It has rotated the axes to line up with the directions of greatest and least variation.

Board: Unchanged from the preceding beat in this scene.

Actions:
- [03:45.575](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=225.57500000000007): candidate is indicated — a transient flash.
- [03:53.075](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=233.07500000000007): pc2\_line is indicated — a transient flash.

##### [03:57.634](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=237.63350000000003)

Narration: So the first answer to carry forward is geometric. PCA asks where the centered cloud is widest, calls that direction component one, and then repeats the question in directions perpendicular to what it has already chosen.

Board: Unchanged from the preceding beat in this scene.

Actions:
- [04:11.219](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=251.21897916666668): heading is hidden from the screen — left the board.
- [04:11.219](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=251.21897916666668): pc1 is hidden from the screen — left the board.
- [04:11.219](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=251.21897916666668): pc2 is hidden from the screen — left the board.
- [04:11.219](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=251.21897916666668): plane is hidden from the screen — left the board.
- [04:11.219](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=251.21897916666668): cloud is hidden from the screen — plane left the board.
- [04:11.219](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=251.21897916666668): cloud\_2 is hidden from the screen — plane left the board.
- [04:11.219](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=251.21897916666668): cloud\_3 is hidden from the screen — plane left the board.
- [04:11.219](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=251.21897916666668): cloud\_4 is hidden from the screen — plane left the board.
- [04:11.219](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=251.21897916666668): cloud\_5 is hidden from the screen — plane left the board.
- [04:11.219](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=251.21897916666668): cloud\_6 is hidden from the screen — plane left the board.
- [04:11.219](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=251.21897916666668): cloud\_7 is hidden from the screen — plane left the board.
- [04:11.219](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=251.21897916666668): cloud\_8 is hidden from the screen — plane left the board.
- [04:11.219](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=251.21897916666668): cloud\_9 is hidden from the screen — plane left the board.
- [04:11.219](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=251.21897916666668): cloud\_10 is hidden from the screen — plane left the board.
- [04:11.219](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=251.21897916666668): cloud\_11 is hidden from the screen — plane left the board.
- [04:11.219](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=251.21897916666668): cloud\_12 is hidden from the screen — plane left the board.
- [04:11.219](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=251.21897916666668): centre is hidden from the screen — plane left the board.
- [04:11.219](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=251.21897916666668): candidate is hidden from the screen — plane left the board.
- [04:11.219](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=251.21897916666668): projected is hidden from the screen — plane left the board.
- [04:11.219](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=251.21897916666668): projected\_2 is hidden from the screen — plane left the board.
- [04:11.219](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=251.21897916666668): projected\_3 is hidden from the screen — plane left the board.
- [04:11.219](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=251.21897916666668): projected\_4 is hidden from the screen — plane left the board.
- [04:11.219](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=251.21897916666668): projected\_5 is hidden from the screen — plane left the board.
- [04:11.219](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=251.21897916666668): projected\_6 is hidden from the screen — plane left the board.
- [04:11.219](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=251.21897916666668): projected\_7 is hidden from the screen — plane left the board.
- [04:11.219](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=251.21897916666668): projected\_8 is hidden from the screen — plane left the board.
- [04:11.219](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=251.21897916666668): projected\_9 is hidden from the screen — plane left the board.
- [04:11.219](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=251.21897916666668): projected\_10 is hidden from the screen — plane left the board.
- [04:11.219](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=251.21897916666668): projected\_11 is hidden from the screen — plane left the board.
- [04:11.219](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=251.21897916666668): projected\_12 is hidden from the screen — plane left the board.
- [04:11.219](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=251.21897916666668): pc2\_line is hidden from the screen — plane left the board.
- [04:11.219](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=251.21897916666668): variance\_split is hidden from the screen — left the board.

### Scene 2: [More Than Two Dimensions](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=252.26064583333334)

Span: 04:12.261–06:57.722 (252.26064583333334s–417.72210416666667s).

#### Objects

- axes: an Axes3D (x\_range=(-3.2, 3.2), y\_range=(-3.2, 3.2), z\_range=(-3.2, 3.2))
- basis: a Math \[text\] that says "$V^T V = I$"
- cloud: a Point \[blue\] drawn in axes (location=(-2.6, -1.8, -1.2), marker\_radius=0.07)
- cloud\_10: a Point \[blue\] drawn in axes (location=(1.8, 1.5, 0.5), marker\_radius=0.07)
- cloud\_11: a Point \[blue\] drawn in axes (location=(2.3, 1.4, 1.4), marker\_radius=0.07)
- cloud\_12: a Point \[blue\] drawn in axes (location=(2.7, 2.1, 0.8), marker\_radius=0.07)
- cloud\_2: a Point \[blue\] drawn in axes (location=(-2.2, -1.8, -0.5), marker\_radius=0.07)
- cloud\_3: a Point \[blue\] drawn in axes (location=(-1.8, -1.0, -1.1), marker\_radius=0.07)
- cloud\_4: a Point \[blue\] drawn in axes (location=(-1.3, -1.2, 0.0), marker\_radius=0.07)
- cloud\_5: a Point \[blue\] drawn in axes (location=(-0.8, -0.2, -0.9), marker\_radius=0.07)
- cloud\_6: a Point \[blue\] drawn in axes (location=(-0.3, -0.5, 0.5), marker\_radius=0.07)
- cloud\_7: a Point \[blue\] drawn in axes (location=(0.4, 0.1, 0.8), marker\_radius=0.07)
- cloud\_8: a Point \[blue\] drawn in axes (location=(0.8, 0.9, 0.1), marker\_radius=0.07)
- cloud\_9: a Point \[blue\] drawn in axes (location=(1.3, 0.7, 1.1), marker\_radius=0.07)
- constraints: a Math \[text\] that says "$u^T u = 1, quad u^T v\_j = 0 thin upright("for") thin j \< k$"
- first\_rule: a Math \[text\] that says "$v\_1 = arg max\_(u^T u = 1) upright("Var")(X u)$"
- heading: a Heading that says "The Same Search in Higher Dimensions"
- limit: a Math \[text\] that says "$k \<= min(p, n - 1)$"
- next\_rule: a Math \[text\] that says "$v\_k = arg max upright("Var")(X u)$"
- orthogonal\_plane: a Plane \[gray\] labelled "v\_1^bot" drawn in axes (normal=(2.4, 1.8, 0.9), edge\_direction=(-0.6, 0.8, 0.0), size=4.5)
- pc1: a Vector \[red\] labelled "v\_1" drawn in axes (start=(0.0, 0.0, 0.0), end=(2.4, 1.8, 0.9))
- pc2: a Vector \[green\] labelled "v\_2" drawn in axes (start=(0.0, 0.0, 0.0), end=(-1.2, 1.6, 0.0))
- pc3: a Vector \[yellow\] labelled "v\_3" drawn in axes (start=(0.0, 0.0, 0.0), end=(-0.6, -0.45, 2.5))
- scores: a Math \[text\] that says "$Z\_k = X V\_k$"

#### Beats

##### [04:12.261](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=252.26064583333334)

Narration: A real data set may have fifty variables, five thousand, or five million. We cannot rotate a literal line through all those dimensions on a screen, but the geometric question does not change.

Board: Empty.

Actions:
- [04:12.261](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=252.26064583333334): heading is shown on the screen, written out.
- [04:20.202](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=260.2016458333333): axes is shown on the screen, written out.

##### [04:25.086](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=265.0861458333333)

Narration: Use three dimensions as the bridge. This cloud has a long spatial direction, plus smaller deviations in two independent sideways directions.

Board: axes — an Axes3D (x\_range=(-3.2, 3.2), y\_range=(-3.2, 3.2), z\_range=(-3.2, 3.2)); heading — a Heading that says "The Same Search in Higher Dimensions"

Actions:
- [04:28.267](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=268.2666458333333): cloud is shown on the screen, written out.
- [04:28.267](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=268.2666458333333): cloud\_2 is shown on the screen, written out.
- [04:28.267](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=268.2666458333333): cloud\_3 is shown on the screen, written out.
- [04:28.267](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=268.2666458333333): cloud\_4 is shown on the screen, written out.
- [04:28.267](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=268.2666458333333): cloud\_5 is shown on the screen, written out.
- [04:28.267](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=268.2666458333333): cloud\_6 is shown on the screen, written out.
- [04:28.267](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=268.2666458333333): cloud\_7 is shown on the screen, written out.
- [04:28.267](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=268.2666458333333): cloud\_8 is shown on the screen, written out.
- [04:28.267](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=268.2666458333333): cloud\_9 is shown on the screen, written out.
- [04:28.267](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=268.2666458333333): cloud\_10 is shown on the screen, written out.
- [04:28.267](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=268.2666458333333): cloud\_11 is shown on the screen, written out.
- [04:28.267](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=268.2666458333333): cloud\_12 is shown on the screen, written out.
- [04:29.231](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=269.23064583333337): axes turns in its own slot.

##### [04:35.439](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=275.43864583333334)

Narration: The first search is exactly the one we just performed. Among every unit direction in the space, find the one on which the projected coordinates have maximum variance.

Board: axes — an Axes3D (x\_range=(-3.2, 3.2), y\_range=(-3.2, 3.2), z\_range=(-3.2, 3.2)); heading — a Heading that says "The Same Search in Higher Dimensions"; cloud — a Point \[blue\] drawn in axes (location=(-2.6, -1.8, -1.2), marker\_radius=0.07); cloud\_2 — a Point \[blue\] drawn in axes (location=(-2.2, -1.8, -0.5), marker\_radius=0.07); cloud\_3 — a Point \[blue\] drawn in axes (location=(-1.8, -1.0, -1.1), marker\_radius=0.07); cloud\_4 — a Point \[blue\] drawn in axes (location=(-1.3, -1.2, 0.0), marker\_radius=0.07); cloud\_5 — a Point \[blue\] drawn in axes (location=(-0.8, -0.2, -0.9), marker\_radius=0.07); cloud\_6 — a Point \[blue\] drawn in axes (location=(-0.3, -0.5, 0.5), marker\_radius=0.07); cloud\_7 — a Point \[blue\] drawn in axes (location=(0.4, 0.1, 0.8), marker\_radius=0.07); cloud\_8 — a Point \[blue\] drawn in axes (location=(0.8, 0.9, 0.1), marker\_radius=0.07); cloud\_9 — a Point \[blue\] drawn in axes (location=(1.3, 0.7, 1.1), marker\_radius=0.07); cloud\_10 — a Point \[blue\] drawn in axes (location=(1.8, 1.5, 0.5), marker\_radius=0.07); cloud\_11 — a Point \[blue\] drawn in axes (location=(2.3, 1.4, 1.4), marker\_radius=0.07); cloud\_12 — a Point \[blue\] drawn in axes (location=(2.7, 2.1, 0.8), marker\_radius=0.07)

Actions:
- [04:36.1](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=276.09964583333334): axes moves to a new place on the board.
- [04:36.1](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=276.09964583333334): first\_rule is shown on the screen, written out.
- [04:40.605](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=280.60464583333334): pc1 is shown on the screen, written out.

##### [04:46.581](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=286.58064583333334)

Narration: The red arrow is the answer. Projecting onto it compresses every observation to one score. Across all observations, those scores vary more than scores along any other unit direction.

Board: first\_rule — a Math \[text\] that says "$v\_1 = arg max\_(u^T u = 1) upright("Var")(X u)$"; axes — an Axes3D (x\_range=(-3.2, 3.2), y\_range=(-3.2, 3.2), z\_range=(-3.2, 3.2)); heading — a Heading that says "The Same Search in Higher Dimensions"; cloud — a Point \[blue\] drawn in axes (location=(-2.6, -1.8, -1.2), marker\_radius=0.07); cloud\_2 — a Point \[blue\] drawn in axes (location=(-2.2, -1.8, -0.5), marker\_radius=0.07); cloud\_3 — a Point \[blue\] drawn in axes (location=(-1.8, -1.0, -1.1), marker\_radius=0.07); cloud\_4 — a Point \[blue\] drawn in axes (location=(-1.3, -1.2, 0.0), marker\_radius=0.07); cloud\_5 — a Point \[blue\] drawn in axes (location=(-0.8, -0.2, -0.9), marker\_radius=0.07); cloud\_6 — a Point \[blue\] drawn in axes (location=(-0.3, -0.5, 0.5), marker\_radius=0.07); cloud\_7 — a Point \[blue\] drawn in axes (location=(0.4, 0.1, 0.8), marker\_radius=0.07); cloud\_8 — a Point \[blue\] drawn in axes (location=(0.8, 0.9, 0.1), marker\_radius=0.07); cloud\_9 — a Point \[blue\] drawn in axes (location=(1.3, 0.7, 1.1), marker\_radius=0.07); cloud\_10 — a Point \[blue\] drawn in axes (location=(1.8, 1.5, 0.5), marker\_radius=0.07); cloud\_11 — a Point \[blue\] drawn in axes (location=(2.3, 1.4, 1.4), marker\_radius=0.07); cloud\_12 — a Point \[blue\] drawn in axes (location=(2.7, 2.1, 0.8), marker\_radius=0.07); pc1 — a Vector \[red\] labelled "v\_1" drawn in axes (start=(0.0, 0.0, 0.0), end=(2.4, 1.8, 0.9))

Actions:
- [04:47.185](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=287.1846458333333): pc1 is indicated — a transient flash.

##### [04:59.847](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=299.84714583333334)

Narration: Now remove the possibility of choosing that answer again. Every direction perpendicular to v one lies in this gray plane through the origin.

Board: Unchanged from the preceding beat in this scene.

Actions:
- [05:7.231](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=307.2306458333333): orthogonal\_plane is shown on the screen, written out.
- [05:7.522](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=307.5216458333333): A quad of data points in axes is lit up.
- [05:9.147](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=309.1466458333333): axes: retire a lit plane (unemphasize\_plane).

##### [05:9.747](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=309.74664583333333)

Narration: Search inside that plane for the direction with greatest projected variance. The green arrow is component two. It is the strongest remaining direction after component one has been excluded.

Board: first\_rule — a Math \[text\] that says "$v\_1 = arg max\_(u^T u = 1) upright("Var")(X u)$"; axes — an Axes3D (x\_range=(-3.2, 3.2), y\_range=(-3.2, 3.2), z\_range=(-3.2, 3.2)); heading — a Heading that says "The Same Search in Higher Dimensions"; cloud — a Point \[blue\] drawn in axes (location=(-2.6, -1.8, -1.2), marker\_radius=0.07); cloud\_2 — a Point \[blue\] drawn in axes (location=(-2.2, -1.8, -0.5), marker\_radius=0.07); cloud\_3 — a Point \[blue\] drawn in axes (location=(-1.8, -1.0, -1.1), marker\_radius=0.07); cloud\_4 — a Point \[blue\] drawn in axes (location=(-1.3, -1.2, 0.0), marker\_radius=0.07); cloud\_5 — a Point \[blue\] drawn in axes (location=(-0.8, -0.2, -0.9), marker\_radius=0.07); cloud\_6 — a Point \[blue\] drawn in axes (location=(-0.3, -0.5, 0.5), marker\_radius=0.07); cloud\_7 — a Point \[blue\] drawn in axes (location=(0.4, 0.1, 0.8), marker\_radius=0.07); cloud\_8 — a Point \[blue\] drawn in axes (location=(0.8, 0.9, 0.1), marker\_radius=0.07); cloud\_9 — a Point \[blue\] drawn in axes (location=(1.3, 0.7, 1.1), marker\_radius=0.07); cloud\_10 — a Point \[blue\] drawn in axes (location=(1.8, 1.5, 0.5), marker\_radius=0.07); cloud\_11 — a Point \[blue\] drawn in axes (location=(2.3, 1.4, 1.4), marker\_radius=0.07); cloud\_12 — a Point \[blue\] drawn in axes (location=(2.7, 2.1, 0.8), marker\_radius=0.07); pc1 — a Vector \[red\] labelled "v\_1" drawn in axes (start=(0.0, 0.0, 0.0), end=(2.4, 1.8, 0.9)); orthogonal\_plane — a Plane \[gray\] labelled "v\_1^bot" drawn in axes (normal=(2.4, 1.8, 0.9), edge\_direction=(-0.6, 0.8, 0.0), size=4.5)

Actions:
- [05:10.095](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=310.09464583333335): next\_rule is shown on the screen, written out.
- [05:14.67](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=314.66964583333333): pc2 is shown on the screen, written out.
- [05:20.429](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=320.42864583333335): constraints is shown on the screen, written out.

##### [05:22.05](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=322.0501458333333)

Narration: In three dimensions, one perpendicular direction remains after the first two have been fixed. That yellow arrow becomes component three.

Board: first\_rule — a Math \[text\] that says "$v\_1 = arg max\_(u^T u = 1) upright("Var")(X u)$"; next\_rule — a Math \[text\] that says "$v\_k = arg max upright("Var")(X u)$"; constraints — a Math \[text\] that says "$u^T u = 1, quad u^T v\_j = 0 thin upright("for") thin j \< k$"; axes — an Axes3D (x\_range=(-3.2, 3.2), y\_range=(-3.2, 3.2), z\_range=(-3.2, 3.2)); heading — a Heading that says "The Same Search in Higher Dimensions"; cloud — a Point \[blue\] drawn in axes (location=(-2.6, -1.8, -1.2), marker\_radius=0.07); cloud\_2 — a Point \[blue\] drawn in axes (location=(-2.2, -1.8, -0.5), marker\_radius=0.07); cloud\_3 — a Point \[blue\] drawn in axes (location=(-1.8, -1.0, -1.1), marker\_radius=0.07); cloud\_4 — a Point \[blue\] drawn in axes (location=(-1.3, -1.2, 0.0), marker\_radius=0.07); cloud\_5 — a Point \[blue\] drawn in axes (location=(-0.8, -0.2, -0.9), marker\_radius=0.07); cloud\_6 — a Point \[blue\] drawn in axes (location=(-0.3, -0.5, 0.5), marker\_radius=0.07); cloud\_7 — a Point \[blue\] drawn in axes (location=(0.4, 0.1, 0.8), marker\_radius=0.07); cloud\_8 — a Point \[blue\] drawn in axes (location=(0.8, 0.9, 0.1), marker\_radius=0.07); cloud\_9 — a Point \[blue\] drawn in axes (location=(1.3, 0.7, 1.1), marker\_radius=0.07); cloud\_10 — a Point \[blue\] drawn in axes (location=(1.8, 1.5, 0.5), marker\_radius=0.07); cloud\_11 — a Point \[blue\] drawn in axes (location=(2.3, 1.4, 1.4), marker\_radius=0.07); cloud\_12 — a Point \[blue\] drawn in axes (location=(2.7, 2.1, 0.8), marker\_radius=0.07); pc1 — a Vector \[red\] labelled "v\_1" drawn in axes (start=(0.0, 0.0, 0.0), end=(2.4, 1.8, 0.9)); orthogonal\_plane — a Plane \[gray\] labelled "v\_1^bot" drawn in axes (normal=(2.4, 1.8, 0.9), edge\_direction=(-0.6, 0.8, 0.0), size=4.5); pc2 — a Vector \[green\] labelled "v\_2" drawn in axes (start=(0.0, 0.0, 0.0), end=(-1.2, 1.6, 0.0))

Actions:
- [05:28.749](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=328.74864583333334): pc3 is shown on the screen, written out.

##### [05:31.613](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=331.6126458333333)

Narration: In p dimensions the process continues the same way. Component k maximizes projected variance subject to unit length and orthogonality to every earlier component.

Board: first\_rule — a Math \[text\] that says "$v\_1 = arg max\_(u^T u = 1) upright("Var")(X u)$"; next\_rule — a Math \[text\] that says "$v\_k = arg max upright("Var")(X u)$"; constraints — a Math \[text\] that says "$u^T u = 1, quad u^T v\_j = 0 thin upright("for") thin j \< k$"; axes — an Axes3D (x\_range=(-3.2, 3.2), y\_range=(-3.2, 3.2), z\_range=(-3.2, 3.2)); heading — a Heading that says "The Same Search in Higher Dimensions"; cloud — a Point \[blue\] drawn in axes (location=(-2.6, -1.8, -1.2), marker\_radius=0.07); cloud\_2 — a Point \[blue\] drawn in axes (location=(-2.2, -1.8, -0.5), marker\_radius=0.07); cloud\_3 — a Point \[blue\] drawn in axes (location=(-1.8, -1.0, -1.1), marker\_radius=0.07); cloud\_4 — a Point \[blue\] drawn in axes (location=(-1.3, -1.2, 0.0), marker\_radius=0.07); cloud\_5 — a Point \[blue\] drawn in axes (location=(-0.8, -0.2, -0.9), marker\_radius=0.07); cloud\_6 — a Point \[blue\] drawn in axes (location=(-0.3, -0.5, 0.5), marker\_radius=0.07); cloud\_7 — a Point \[blue\] drawn in axes (location=(0.4, 0.1, 0.8), marker\_radius=0.07); cloud\_8 — a Point \[blue\] drawn in axes (location=(0.8, 0.9, 0.1), marker\_radius=0.07); cloud\_9 — a Point \[blue\] drawn in axes (location=(1.3, 0.7, 1.1), marker\_radius=0.07); cloud\_10 — a Point \[blue\] drawn in axes (location=(1.8, 1.5, 0.5), marker\_radius=0.07); cloud\_11 — a Point \[blue\] drawn in axes (location=(2.3, 1.4, 1.4), marker\_radius=0.07); cloud\_12 — a Point \[blue\] drawn in axes (location=(2.7, 2.1, 0.8), marker\_radius=0.07); pc1 — a Vector \[red\] labelled "v\_1" drawn in axes (start=(0.0, 0.0, 0.0), end=(2.4, 1.8, 0.9)); orthogonal\_plane — a Plane \[gray\] labelled "v\_1^bot" drawn in axes (normal=(2.4, 1.8, 0.9), edge\_direction=(-0.6, 0.8, 0.0), size=4.5); pc2 — a Vector \[green\] labelled "v\_2" drawn in axes (start=(0.0, 0.0, 0.0), end=(-1.2, 1.6, 0.0)); pc3 — a Vector \[yellow\] labelled "v\_3" drawn in axes (start=(0.0, 0.0, 0.0), end=(-0.6, -0.45, 2.5))

Actions:
- [05:35.514](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=335.5136458333333): next\_rule is indicated — a transient flash.
- [05:39.276](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=339.27564583333333): constraints (the "u^T u = 1" part) is emphasized.
- [05:40.518](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=340.51764583333335): constraints (the "u^T u = 1" part) is no longer emphasized.
- [05:40.518](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=340.51764583333335): constraints (the "u^T v\_j = 0 thin upright("for") thin j \< k" part) is emphasized.
- [05:43.35](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=343.35014583333333): constraints (the "u^T v\_j = 0 thin upright("for") thin j \< k" part) is no longer emphasized.

##### [05:43.95](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=343.95014583333335)

Narration: Collect the first k component directions as columns of V sub k. Multiplying the centered data matrix X by V sub k produces the score matrix Z sub k. Each row is now the same observation expressed in principal-component coordinates.

Board: Unchanged from the preceding beat in this scene.

Actions:
- [05:44.299](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=344.29864583333335): scores is shown on the screen, written out.

##### [06:2.267](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=362.2671458333333)

Narration: Because the component directions are mutually perpendicular unit vectors, V transpose V is the identity. This orthonormal structure is why moving into component coordinates and reconstructing from them remains simple.

Board: first\_rule — a Math \[text\] that says "$v\_1 = arg max\_(u^T u = 1) upright("Var")(X u)$"; next\_rule — a Math \[text\] that says "$v\_k = arg max upright("Var")(X u)$"; constraints — a Math \[text\] that says "$u^T u = 1, quad u^T v\_j = 0 thin upright("for") thin j \< k$"; scores — a Math \[text\] that says "$Z\_k = X V\_k$"; axes — an Axes3D (x\_range=(-3.2, 3.2), y\_range=(-3.2, 3.2), z\_range=(-3.2, 3.2)); heading — a Heading that says "The Same Search in Higher Dimensions"; cloud — a Point \[blue\] drawn in axes (location=(-2.6, -1.8, -1.2), marker\_radius=0.07); cloud\_2 — a Point \[blue\] drawn in axes (location=(-2.2, -1.8, -0.5), marker\_radius=0.07); cloud\_3 — a Point \[blue\] drawn in axes (location=(-1.8, -1.0, -1.1), marker\_radius=0.07); cloud\_4 — a Point \[blue\] drawn in axes (location=(-1.3, -1.2, 0.0), marker\_radius=0.07); cloud\_5 — a Point \[blue\] drawn in axes (location=(-0.8, -0.2, -0.9), marker\_radius=0.07); cloud\_6 — a Point \[blue\] drawn in axes (location=(-0.3, -0.5, 0.5), marker\_radius=0.07); cloud\_7 — a Point \[blue\] drawn in axes (location=(0.4, 0.1, 0.8), marker\_radius=0.07); cloud\_8 — a Point \[blue\] drawn in axes (location=(0.8, 0.9, 0.1), marker\_radius=0.07); cloud\_9 — a Point \[blue\] drawn in axes (location=(1.3, 0.7, 1.1), marker\_radius=0.07); cloud\_10 — a Point \[blue\] drawn in axes (location=(1.8, 1.5, 0.5), marker\_radius=0.07); cloud\_11 — a Point \[blue\] drawn in axes (location=(2.3, 1.4, 1.4), marker\_radius=0.07); cloud\_12 — a Point \[blue\] drawn in axes (location=(2.7, 2.1, 0.8), marker\_radius=0.07); pc1 — a Vector \[red\] labelled "v\_1" drawn in axes (start=(0.0, 0.0, 0.0), end=(2.4, 1.8, 0.9)); orthogonal\_plane — a Plane \[gray\] labelled "v\_1^bot" drawn in axes (normal=(2.4, 1.8, 0.9), edge\_direction=(-0.6, 0.8, 0.0), size=4.5); pc2 — a Vector \[green\] labelled "v\_2" drawn in axes (start=(0.0, 0.0, 0.0), end=(-1.2, 1.6, 0.0)); pc3 — a Vector \[yellow\] labelled "v\_3" drawn in axes (start=(0.0, 0.0, 0.0), end=(-0.6, -0.45, 2.5))

Actions:
- [06:8.525](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=368.52464583333335): basis is shown on the screen, written out.
- [06:10.232](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=370.23164583333335): pc1 is indicated — a transient flash.
- [06:10.432](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=370.43164583333333): pc2 is indicated — a transient flash.
- [06:10.632](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=370.6316458333333): pc3 is indicated — a transient flash.

##### [06:16.799](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=376.79914583333334)

Narration: There cannot be more useful components than variables. There also cannot be more than n minus one nonzero directions of variation after n observations have been centered.

Board: first\_rule — a Math \[text\] that says "$v\_1 = arg max\_(u^T u = 1) upright("Var")(X u)$"; next\_rule — a Math \[text\] that says "$v\_k = arg max upright("Var")(X u)$"; constraints — a Math \[text\] that says "$u^T u = 1, quad u^T v\_j = 0 thin upright("for") thin j \< k$"; scores — a Math \[text\] that says "$Z\_k = X V\_k$"; basis — a Math \[text\] that says "$V^T V = I$"; axes — an Axes3D (x\_range=(-3.2, 3.2), y\_range=(-3.2, 3.2), z\_range=(-3.2, 3.2)); heading — a Heading that says "The Same Search in Higher Dimensions"; cloud — a Point \[blue\] drawn in axes (location=(-2.6, -1.8, -1.2), marker\_radius=0.07); cloud\_2 — a Point \[blue\] drawn in axes (location=(-2.2, -1.8, -0.5), marker\_radius=0.07); cloud\_3 — a Point \[blue\] drawn in axes (location=(-1.8, -1.0, -1.1), marker\_radius=0.07); cloud\_4 — a Point \[blue\] drawn in axes (location=(-1.3, -1.2, 0.0), marker\_radius=0.07); cloud\_5 — a Point \[blue\] drawn in axes (location=(-0.8, -0.2, -0.9), marker\_radius=0.07); cloud\_6 — a Point \[blue\] drawn in axes (location=(-0.3, -0.5, 0.5), marker\_radius=0.07); cloud\_7 — a Point \[blue\] drawn in axes (location=(0.4, 0.1, 0.8), marker\_radius=0.07); cloud\_8 — a Point \[blue\] drawn in axes (location=(0.8, 0.9, 0.1), marker\_radius=0.07); cloud\_9 — a Point \[blue\] drawn in axes (location=(1.3, 0.7, 1.1), marker\_radius=0.07); cloud\_10 — a Point \[blue\] drawn in axes (location=(1.8, 1.5, 0.5), marker\_radius=0.07); cloud\_11 — a Point \[blue\] drawn in axes (location=(2.3, 1.4, 1.4), marker\_radius=0.07); cloud\_12 — a Point \[blue\] drawn in axes (location=(2.7, 2.1, 0.8), marker\_radius=0.07); pc1 — a Vector \[red\] labelled "v\_1" drawn in axes (start=(0.0, 0.0, 0.0), end=(2.4, 1.8, 0.9)); orthogonal\_plane — a Plane \[gray\] labelled "v\_1^bot" drawn in axes (normal=(2.4, 1.8, 0.9), edge\_direction=(-0.6, 0.8, 0.0), size=4.5); pc2 — a Vector \[green\] labelled "v\_2" drawn in axes (start=(0.0, 0.0, 0.0), end=(-1.2, 1.6, 0.0)); pc3 — a Vector \[yellow\] labelled "v\_3" drawn in axes (start=(0.0, 0.0, 0.0), end=(-0.6, -0.45, 2.5))

Actions:
- [06:17.392](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=377.3916458333333): limit is shown on the screen, written out.

##### [06:29.265](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=389.26514583333335)

Narration: The crucial point is that higher-dimensional PCA is not a new algorithmic idea. It is the same sequence of questions: widest direction first, then the widest perpendicular direction left, then the next.

Board: first\_rule — a Math \[text\] that says "$v\_1 = arg max\_(u^T u = 1) upright("Var")(X u)$"; next\_rule — a Math \[text\] that says "$v\_k = arg max upright("Var")(X u)$"; constraints — a Math \[text\] that says "$u^T u = 1, quad u^T v\_j = 0 thin upright("for") thin j \< k$"; scores — a Math \[text\] that says "$Z\_k = X V\_k$"; basis — a Math \[text\] that says "$V^T V = I$"; limit — a Math \[text\] that says "$k \<= min(p, n - 1)$"; axes — an Axes3D (x\_range=(-3.2, 3.2), y\_range=(-3.2, 3.2), z\_range=(-3.2, 3.2)); heading — a Heading that says "The Same Search in Higher Dimensions"; cloud — a Point \[blue\] drawn in axes (location=(-2.6, -1.8, -1.2), marker\_radius=0.07); cloud\_2 — a Point \[blue\] drawn in axes (location=(-2.2, -1.8, -0.5), marker\_radius=0.07); cloud\_3 — a Point \[blue\] drawn in axes (location=(-1.8, -1.0, -1.1), marker\_radius=0.07); cloud\_4 — a Point \[blue\] drawn in axes (location=(-1.3, -1.2, 0.0), marker\_radius=0.07); cloud\_5 — a Point \[blue\] drawn in axes (location=(-0.8, -0.2, -0.9), marker\_radius=0.07); cloud\_6 — a Point \[blue\] drawn in axes (location=(-0.3, -0.5, 0.5), marker\_radius=0.07); cloud\_7 — a Point \[blue\] drawn in axes (location=(0.4, 0.1, 0.8), marker\_radius=0.07); cloud\_8 — a Point \[blue\] drawn in axes (location=(0.8, 0.9, 0.1), marker\_radius=0.07); cloud\_9 — a Point \[blue\] drawn in axes (location=(1.3, 0.7, 1.1), marker\_radius=0.07); cloud\_10 — a Point \[blue\] drawn in axes (location=(1.8, 1.5, 0.5), marker\_radius=0.07); cloud\_11 — a Point \[blue\] drawn in axes (location=(2.3, 1.4, 1.4), marker\_radius=0.07); cloud\_12 — a Point \[blue\] drawn in axes (location=(2.7, 2.1, 0.8), marker\_radius=0.07); pc1 — a Vector \[red\] labelled "v\_1" drawn in axes (start=(0.0, 0.0, 0.0), end=(2.4, 1.8, 0.9)); orthogonal\_plane — a Plane \[gray\] labelled "v\_1^bot" drawn in axes (normal=(2.4, 1.8, 0.9), edge\_direction=(-0.6, 0.8, 0.0), size=4.5); pc2 — a Vector \[green\] labelled "v\_2" drawn in axes (start=(0.0, 0.0, 0.0), end=(-1.2, 1.6, 0.0)); pc3 — a Vector \[yellow\] labelled "v\_3" drawn in axes (start=(0.0, 0.0, 0.0), end=(-0.6, -0.45, 2.5))

Actions:
- [06:39.076](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=399.0756458333333): pc1 is indicated — a transient flash.
- [06:39.865](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=399.86464583333327): pc2 is indicated — a transient flash.
- [06:43.07](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=403.0696458333333): pc3 is indicated — a transient flash.

##### [06:44.482](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=404.48214583333333)

Narration: We now have the geometry. Only at this point do we need the covariance matrix, because it gives an efficient algebraic way to answer exactly the optimization question we have already understood.

Board: Unchanged from the preceding beat in this scene.

Actions:
- [06:56.68](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=416.68043750000004): axes is hidden from the screen — left the board.
- [06:56.68](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=416.68043750000004): cloud is hidden from the screen — axes left the board.
- [06:56.68](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=416.68043750000004): cloud\_2 is hidden from the screen — axes left the board.
- [06:56.68](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=416.68043750000004): cloud\_3 is hidden from the screen — axes left the board.
- [06:56.68](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=416.68043750000004): cloud\_4 is hidden from the screen — axes left the board.
- [06:56.68](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=416.68043750000004): cloud\_5 is hidden from the screen — axes left the board.
- [06:56.68](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=416.68043750000004): cloud\_6 is hidden from the screen — axes left the board.
- [06:56.68](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=416.68043750000004): cloud\_7 is hidden from the screen — axes left the board.
- [06:56.68](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=416.68043750000004): cloud\_8 is hidden from the screen — axes left the board.
- [06:56.68](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=416.68043750000004): cloud\_9 is hidden from the screen — axes left the board.
- [06:56.68](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=416.68043750000004): cloud\_10 is hidden from the screen — axes left the board.
- [06:56.68](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=416.68043750000004): cloud\_11 is hidden from the screen — axes left the board.
- [06:56.68](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=416.68043750000004): cloud\_12 is hidden from the screen — axes left the board.
- [06:56.68](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=416.68043750000004): pc1 is hidden from the screen — axes left the board.
- [06:56.68](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=416.68043750000004): orthogonal\_plane is hidden from the screen — axes left the board.
- [06:56.68](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=416.68043750000004): pc2 is hidden from the screen — axes left the board.
- [06:56.68](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=416.68043750000004): pc3 is hidden from the screen — axes left the board.
- [06:56.68](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=416.68043750000004): basis is hidden from the screen — left the board.
- [06:56.68](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=416.68043750000004): constraints is hidden from the screen — left the board.
- [06:56.68](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=416.68043750000004): first\_rule is hidden from the screen — left the board.
- [06:56.68](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=416.68043750000004): heading is hidden from the screen — left the board.
- [06:56.68](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=416.68043750000004): limit is hidden from the screen — left the board.
- [06:56.68](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=416.68043750000004): next\_rule is hidden from the screen — left the board.
- [06:56.68](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=416.68043750000004): scores is hidden from the screen — left the board.

### Scene 3: [The Covariance Eigenproblem](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=417.72210416666667)

Span: 06:57.722–10:57.495 (417.72210416666667s–657.4953958333333s).

#### Objects

- axes: a Figure (x\_range=(-4.2, 4.2), y\_range=(-4.2, 4.2), aspect=(1.0, 1.0))
- cloud: a Point \[blue\] drawn in axes (location=(-3.1, -2.4), marker\_radius=0.07)
- cloud\_10: a Point \[blue\] drawn in axes (location=(2.0, 1.5), marker\_radius=0.07)
- cloud\_11: a Point \[blue\] drawn in axes (location=(2.6, 3.0), marker\_radius=0.07)
- cloud\_12: a Point \[blue\] drawn in axes (location=(3.1, 2.4), marker\_radius=0.07)
- cloud\_2: a Point \[blue\] drawn in axes (location=(-2.7, -3.0), marker\_radius=0.07)
- cloud\_3: a Point \[blue\] drawn in axes (location=(-2.2, -1.5), marker\_radius=0.07)
- cloud\_4: a Point \[blue\] drawn in axes (location=(-1.6, -2.1), marker\_radius=0.07)
- cloud\_5: a Point \[blue\] drawn in axes (location=(-1.1, -0.5), marker\_radius=0.07)
- cloud\_6: a Point \[blue\] drawn in axes (location=(-0.5, -1.0), marker\_radius=0.07)
- cloud\_7: a Point \[blue\] drawn in axes (location=(0.4, 0.9), marker\_radius=0.07)
- cloud\_8: a Point \[blue\] drawn in axes (location=(0.9, 0.4), marker\_radius=0.07)
- cloud\_9: a Point \[blue\] drawn in axes (location=(1.4, 2.0), marker\_radius=0.07)
- covariance: a Math \[text\] that says "$Sigma = frac(1, n) X^T X$"
- d3: a Math \[text\] that says "$&= u^T Sigma u$"
- determinant: a Math \[text\] that says "$op("det")(Sigma - lambda I) = op("det") mat(4-lambda, 3; 3, 4-lambda)$"
- eigenproblem: a Math \[text\] that says "$Sigma u = lambda u$"
- gradient: a Math \[text\] that says "$frac(partial L, partial u) = 2 Sigma u - 2 lambda u = 0$"
- heading\_example: a Heading that says "The Elongated Cloud, Calculated"
- heading\_lagrange: a Heading that says "The Maximizer Must Be an Eigenvector"
- heading\_share: a Heading that says "The Eigenvalues Measure Explained Variance"
- heading\_variance: a Heading that says "Write the Geometric Question Algebraically"
- lagrangian: a Math \[text\] that says "$L(u, lambda) = u^T Sigma u - lambda (u^T u - 1)$"
- meaning: a Math \[text\] that says "$u^T Sigma u = lambda$"
- polynomial: a Math \[text\] that says "$(4-lambda)^2 - 9 = 0$"
- ranking: a Math \[text\] that says "$lambda\_1 \>= lambda\_2 \>= dots \>= 0$"
- share: a Math \[text\] that says "$frac(lambda\_1, lambda\_1 + lambda\_2) = frac(7, 8) = 87.5%$"
- sigma: a Math \[text\] that says "$Sigma = mat(4, 3; 3, 4)$"
- v1\_line: a Line \[red\] labelled "v\_1" drawn in axes (start=(-3.7, -3.7), end=(3.7, 3.7))
- v2\_line: a Line \[green\] labelled "v\_2" drawn in axes (start=(3.7, -3.7), end=(-3.7, 3.7))
- values: a Math \[text\] that says "$lambda\_1 = 7, quad lambda\_2 = 1$"
- variance\_derivation: a Derivation \[text\] that says "$z\_i &= u^T x\_i \\ upright("Var")(z) &= frac(1, n) sum\_i (u^T x\_i)^2 \\ &= frac(1, n) sum\_i u^T x\_i x\_i^T u \\ &= u^T Sigma u$"
- vectors: a Math \[text\] that says "$v\_1 = frac(1, sqrt(2)) vec(1, 1), quad v\_2 = frac(1, sqrt(2)) vec(1, -1)$"

#### Beats

##### [06:57.722](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=417.72210416666667)

Narration: Return to the centered two-dimensional cloud. We already know the geometric problem: choose a unit direction u that makes the projected scores vary as much as possible.

Board: Empty.

Actions:
- [06:57.722](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=417.72210416666667): heading\_variance is shown on the screen, written out.
- [06:59.522](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=419.5221041666667): axes is shown on the screen, written out.
- [06:59.522](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=419.5221041666667): cloud is shown on the screen, written out.
- [06:59.522](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=419.5221041666667): cloud\_2 is shown on the screen, written out.
- [06:59.522](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=419.5221041666667): cloud\_3 is shown on the screen, written out.
- [06:59.522](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=419.5221041666667): cloud\_4 is shown on the screen, written out.
- [06:59.522](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=419.5221041666667): cloud\_5 is shown on the screen, written out.
- [06:59.522](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=419.5221041666667): cloud\_6 is shown on the screen, written out.
- [06:59.522](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=419.5221041666667): cloud\_7 is shown on the screen, written out.
- [06:59.522](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=419.5221041666667): cloud\_8 is shown on the screen, written out.
- [06:59.522](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=419.5221041666667): cloud\_9 is shown on the screen, written out.
- [06:59.522](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=419.5221041666667): cloud\_10 is shown on the screen, written out.
- [06:59.522](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=419.5221041666667): cloud\_11 is shown on the screen, written out.
- [06:59.522](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=419.5221041666667): cloud\_12 is shown on the screen, written out.

##### [07:8.62](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=428.6201041666667)

Narration: For one centered observation x sub i, its coordinate on u is the dot product u transpose x sub i. Call that score z sub i.

Board: axes — a Figure (x\_range=(-4.2, 4.2), y\_range=(-4.2, 4.2), aspect=(1.0, 1.0)); heading\_variance — a Heading that says "Write the Geometric Question Algebraically"; cloud — a Point \[blue\] drawn in axes (location=(-3.1, -2.4), marker\_radius=0.07); cloud\_2 — a Point \[blue\] drawn in axes (location=(-2.7, -3.0), marker\_radius=0.07); cloud\_3 — a Point \[blue\] drawn in axes (location=(-2.2, -1.5), marker\_radius=0.07); cloud\_4 — a Point \[blue\] drawn in axes (location=(-1.6, -2.1), marker\_radius=0.07); cloud\_5 — a Point \[blue\] drawn in axes (location=(-1.1, -0.5), marker\_radius=0.07); cloud\_6 — a Point \[blue\] drawn in axes (location=(-0.5, -1.0), marker\_radius=0.07); cloud\_7 — a Point \[blue\] drawn in axes (location=(0.4, 0.9), marker\_radius=0.07); cloud\_8 — a Point \[blue\] drawn in axes (location=(0.9, 0.4), marker\_radius=0.07); cloud\_9 — a Point \[blue\] drawn in axes (location=(1.4, 2.0), marker\_radius=0.07); cloud\_10 — a Point \[blue\] drawn in axes (location=(2.0, 1.5), marker\_radius=0.07); cloud\_11 — a Point \[blue\] drawn in axes (location=(2.6, 3.0), marker\_radius=0.07); cloud\_12 — a Point \[blue\] drawn in axes (location=(3.1, 2.4), marker\_radius=0.07)

Actions:
- [07:17.293](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=437.2931041666667): axes moves to a new place on the board.
- [07:17.293](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=437.2931041666667): variance\_derivation is shown on the screen, written out.

##### [07:19.554](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=439.5536041666667)

Narration: The scores are centered too, so their variance is the average squared score. Substitute the dot product into that familiar calculation.

Board: Unchanged from the preceding beat in this scene.

Actions:
- [07:22.166](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=442.1661041666667): variance\_derivation is shown on the screen, written out.

##### [07:29.175](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=449.17460416666665)

Narration: A scalar square can be written as u transpose x sub i, times x sub i transpose u. Put the direction vectors outside the observation-specific middle.

Board: Unchanged from the preceding beat in this scene.

Actions:
- [07:30.811](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=450.81110416666667): variance\_derivation is shown on the screen, written out.
- [07:36.906](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=456.90610416666664): variance\_derivation (the "u^T" part) is emphasized.
- [07:39.878](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=459.8781041666667): variance\_derivation (the "u#2" part) is emphasized.
- [07:39.878](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=459.8781041666667): variance\_derivation (the "u^T" part) is no longer emphasized.
- [07:40.349](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=460.3486041666667): variance\_derivation (the "u#2" part) is no longer emphasized.

##### [07:40.949](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=460.94860416666666)

Narration: The average of those middle outer products is the covariance matrix Sigma. With centered observations stored in the rows of X, Sigma is one over n times X transpose X.

Board: Unchanged from the preceding beat in this scene.

Actions:
- [07:43.265](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=463.2651041666667): covariance is shown on the screen, written out.
- [07:44.553](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=464.5531041666667): variance\_derivation is shown on the screen, written out.

##### [07:52.509](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=472.50860416666666)

Narration: So the variance seen along u is exactly the quadratic form u transpose Sigma u. This line is not a different objective. It is the projected spread from the rotating picture, written compactly.

Board: covariance — a Math \[text\] that says "$Sigma = frac(1, n) X^T X$"; axes — a Figure (x\_range=(-4.2, 4.2), y\_range=(-4.2, 4.2), aspect=(1.0, 1.0)); heading\_variance — a Heading that says "Write the Geometric Question Algebraically"; cloud — a Point \[blue\] drawn in axes (location=(-3.1, -2.4), marker\_radius=0.07); cloud\_2 — a Point \[blue\] drawn in axes (location=(-2.7, -3.0), marker\_radius=0.07); cloud\_3 — a Point \[blue\] drawn in axes (location=(-2.2, -1.5), marker\_radius=0.07); cloud\_4 — a Point \[blue\] drawn in axes (location=(-1.6, -2.1), marker\_radius=0.07); cloud\_5 — a Point \[blue\] drawn in axes (location=(-1.1, -0.5), marker\_radius=0.07); cloud\_6 — a Point \[blue\] drawn in axes (location=(-0.5, -1.0), marker\_radius=0.07); cloud\_7 — a Point \[blue\] drawn in axes (location=(0.4, 0.9), marker\_radius=0.07); cloud\_8 — a Point \[blue\] drawn in axes (location=(0.9, 0.4), marker\_radius=0.07); cloud\_9 — a Point \[blue\] drawn in axes (location=(1.4, 2.0), marker\_radius=0.07); cloud\_10 — a Point \[blue\] drawn in axes (location=(2.0, 1.5), marker\_radius=0.07); cloud\_11 — a Point \[blue\] drawn in axes (location=(2.6, 3.0), marker\_radius=0.07); cloud\_12 — a Point \[blue\] drawn in axes (location=(3.1, 2.4), marker\_radius=0.07)

Actions:
- [07:54.813](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=474.8131041666667): variance\_derivation (the "u^T Sigma u" part) is emphasized.
- [08:5.552](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=485.55210416666665): covariance is hidden from the screen — left the board.
- [08:5.552](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=485.55210416666665): heading\_variance is hidden from the screen — left the board.
- [08:5.552](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=485.55210416666665): variance\_derivation is hidden from the screen — left the board.
- [08:5.552](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=485.55210416666665): variance\_derivation (the "u^T Sigma u" part) is no longer emphasized.

##### [08:6.152](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=486.1521041666667)

Narration: We want to maximize that quadratic form while keeping u at unit length. Use a Lagrange multiplier to attach the constraint u transpose u equals one.

Board: axes — a Figure (x\_range=(-4.2, 4.2), y\_range=(-4.2, 4.2), aspect=(1.0, 1.0)); cloud — a Point \[blue\] drawn in axes (location=(-3.1, -2.4), marker\_radius=0.07); cloud\_2 — a Point \[blue\] drawn in axes (location=(-2.7, -3.0), marker\_radius=0.07); cloud\_3 — a Point \[blue\] drawn in axes (location=(-2.2, -1.5), marker\_radius=0.07); cloud\_4 — a Point \[blue\] drawn in axes (location=(-1.6, -2.1), marker\_radius=0.07); cloud\_5 — a Point \[blue\] drawn in axes (location=(-1.1, -0.5), marker\_radius=0.07); cloud\_6 — a Point \[blue\] drawn in axes (location=(-0.5, -1.0), marker\_radius=0.07); cloud\_7 — a Point \[blue\] drawn in axes (location=(0.4, 0.9), marker\_radius=0.07); cloud\_8 — a Point \[blue\] drawn in axes (location=(0.9, 0.4), marker\_radius=0.07); cloud\_9 — a Point \[blue\] drawn in axes (location=(1.4, 2.0), marker\_radius=0.07); cloud\_10 — a Point \[blue\] drawn in axes (location=(2.0, 1.5), marker\_radius=0.07); cloud\_11 — a Point \[blue\] drawn in axes (location=(2.6, 3.0), marker\_radius=0.07); cloud\_12 — a Point \[blue\] drawn in axes (location=(3.1, 2.4), marker\_radius=0.07)

Actions:
- [08:6.152](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=486.1521041666667): heading\_lagrange is shown on the screen, written out.
- [08:12.027](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=492.0271041666667): lagrangian is shown on the screen, written out.

##### [08:17.201](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=497.2011041666667)

Narration: At a maximum, the derivative with respect to u is zero. Differentiating gives two Sigma u minus two lambda u.

Board: axes — a Figure (x\_range=(-4.2, 4.2), y\_range=(-4.2, 4.2), aspect=(1.0, 1.0)); cloud — a Point \[blue\] drawn in axes (location=(-3.1, -2.4), marker\_radius=0.07); cloud\_2 — a Point \[blue\] drawn in axes (location=(-2.7, -3.0), marker\_radius=0.07); cloud\_3 — a Point \[blue\] drawn in axes (location=(-2.2, -1.5), marker\_radius=0.07); cloud\_4 — a Point \[blue\] drawn in axes (location=(-1.6, -2.1), marker\_radius=0.07); cloud\_5 — a Point \[blue\] drawn in axes (location=(-1.1, -0.5), marker\_radius=0.07); cloud\_6 — a Point \[blue\] drawn in axes (location=(-0.5, -1.0), marker\_radius=0.07); cloud\_7 — a Point \[blue\] drawn in axes (location=(0.4, 0.9), marker\_radius=0.07); cloud\_8 — a Point \[blue\] drawn in axes (location=(0.9, 0.4), marker\_radius=0.07); cloud\_9 — a Point \[blue\] drawn in axes (location=(1.4, 2.0), marker\_radius=0.07); cloud\_10 — a Point \[blue\] drawn in axes (location=(2.0, 1.5), marker\_radius=0.07); cloud\_11 — a Point \[blue\] drawn in axes (location=(2.6, 3.0), marker\_radius=0.07); cloud\_12 — a Point \[blue\] drawn in axes (location=(3.1, 2.4), marker\_radius=0.07); lagrangian — a Math \[text\] that says "$L(u, lambda) = u^T Sigma u - lambda (u^T u - 1)$"; heading\_lagrange — a Heading that says "The Maximizer Must Be an Eigenvector"

Actions:
- [08:21.601](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=501.6011041666667): gradient is shown on the screen, written out.

##### [08:26.276](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=506.2761041666667)

Narration: Cancel the common factor of two and rearrange. Sigma u equals lambda u. The maximizing direction must therefore be an eigenvector of the covariance matrix.

Board: axes — a Figure (x\_range=(-4.2, 4.2), y\_range=(-4.2, 4.2), aspect=(1.0, 1.0)); cloud — a Point \[blue\] drawn in axes (location=(-3.1, -2.4), marker\_radius=0.07); cloud\_2 — a Point \[blue\] drawn in axes (location=(-2.7, -3.0), marker\_radius=0.07); cloud\_3 — a Point \[blue\] drawn in axes (location=(-2.2, -1.5), marker\_radius=0.07); cloud\_4 — a Point \[blue\] drawn in axes (location=(-1.6, -2.1), marker\_radius=0.07); cloud\_5 — a Point \[blue\] drawn in axes (location=(-1.1, -0.5), marker\_radius=0.07); cloud\_6 — a Point \[blue\] drawn in axes (location=(-0.5, -1.0), marker\_radius=0.07); cloud\_7 — a Point \[blue\] drawn in axes (location=(0.4, 0.9), marker\_radius=0.07); cloud\_8 — a Point \[blue\] drawn in axes (location=(0.9, 0.4), marker\_radius=0.07); cloud\_9 — a Point \[blue\] drawn in axes (location=(1.4, 2.0), marker\_radius=0.07); cloud\_10 — a Point \[blue\] drawn in axes (location=(2.0, 1.5), marker\_radius=0.07); cloud\_11 — a Point \[blue\] drawn in axes (location=(2.6, 3.0), marker\_radius=0.07); cloud\_12 — a Point \[blue\] drawn in axes (location=(3.1, 2.4), marker\_radius=0.07); lagrangian — a Math \[text\] that says "$L(u, lambda) = u^T Sigma u - lambda (u^T u - 1)$"; gradient — a Math \[text\] that says "$frac(partial L, partial u) = 2 Sigma u - 2 lambda u = 0$"; heading\_lagrange — a Heading that says "The Maximizer Must Be an Eigenvector"

Actions:
- [08:30.27](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=510.27010416666667): eigenproblem is shown on the screen, written out.
- [08:35.773](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=515.7731041666667): A box is drawn around eigenproblem.

##### [08:39.229](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=519.2291041666667)

Narration: This is the promised restatement. The geometry asked for a direction that keeps its identity while covariance assigns it a variance. The algebra calls that direction an eigenvector.

Board: axes — a Figure (x\_range=(-4.2, 4.2), y\_range=(-4.2, 4.2), aspect=(1.0, 1.0)); cloud — a Point \[blue\] drawn in axes (location=(-3.1, -2.4), marker\_radius=0.07); cloud\_2 — a Point \[blue\] drawn in axes (location=(-2.7, -3.0), marker\_radius=0.07); cloud\_3 — a Point \[blue\] drawn in axes (location=(-2.2, -1.5), marker\_radius=0.07); cloud\_4 — a Point \[blue\] drawn in axes (location=(-1.6, -2.1), marker\_radius=0.07); cloud\_5 — a Point \[blue\] drawn in axes (location=(-1.1, -0.5), marker\_radius=0.07); cloud\_6 — a Point \[blue\] drawn in axes (location=(-0.5, -1.0), marker\_radius=0.07); cloud\_7 — a Point \[blue\] drawn in axes (location=(0.4, 0.9), marker\_radius=0.07); cloud\_8 — a Point \[blue\] drawn in axes (location=(0.9, 0.4), marker\_radius=0.07); cloud\_9 — a Point \[blue\] drawn in axes (location=(1.4, 2.0), marker\_radius=0.07); cloud\_10 — a Point \[blue\] drawn in axes (location=(2.0, 1.5), marker\_radius=0.07); cloud\_11 — a Point \[blue\] drawn in axes (location=(2.6, 3.0), marker\_radius=0.07); cloud\_12 — a Point \[blue\] drawn in axes (location=(3.1, 2.4), marker\_radius=0.07); lagrangian — a Math \[text\] that says "$L(u, lambda) = u^T Sigma u - lambda (u^T u - 1)$"; gradient — a Math \[text\] that says "$frac(partial L, partial u) = 2 Sigma u - 2 lambda u = 0$"; eigenproblem — a Math \[text\] that says "$Sigma u = lambda u$"; heading\_lagrange — a Heading that says "The Maximizer Must Be an Eigenvector"

Actions:
- [08:43.316](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=523.3161041666667): v1\_line is shown on the screen, written out.

##### [08:52.229](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=532.2291041666667)

Narration: Multiply the eigenvalue equation on the left by u transpose. Because u has unit length, u transpose Sigma u equals lambda. The eigenvalue is the projected variance along its eigenvector.

Board: axes — a Figure (x\_range=(-4.2, 4.2), y\_range=(-4.2, 4.2), aspect=(1.0, 1.0)); cloud — a Point \[blue\] drawn in axes (location=(-3.1, -2.4), marker\_radius=0.07); cloud\_2 — a Point \[blue\] drawn in axes (location=(-2.7, -3.0), marker\_radius=0.07); cloud\_3 — a Point \[blue\] drawn in axes (location=(-2.2, -1.5), marker\_radius=0.07); cloud\_4 — a Point \[blue\] drawn in axes (location=(-1.6, -2.1), marker\_radius=0.07); cloud\_5 — a Point \[blue\] drawn in axes (location=(-1.1, -0.5), marker\_radius=0.07); cloud\_6 — a Point \[blue\] drawn in axes (location=(-0.5, -1.0), marker\_radius=0.07); cloud\_7 — a Point \[blue\] drawn in axes (location=(0.4, 0.9), marker\_radius=0.07); cloud\_8 — a Point \[blue\] drawn in axes (location=(0.9, 0.4), marker\_radius=0.07); cloud\_9 — a Point \[blue\] drawn in axes (location=(1.4, 2.0), marker\_radius=0.07); cloud\_10 — a Point \[blue\] drawn in axes (location=(2.0, 1.5), marker\_radius=0.07); cloud\_11 — a Point \[blue\] drawn in axes (location=(2.6, 3.0), marker\_radius=0.07); cloud\_12 — a Point \[blue\] drawn in axes (location=(3.1, 2.4), marker\_radius=0.07); lagrangian — a Math \[text\] that says "$L(u, lambda) = u^T Sigma u - lambda (u^T u - 1)$"; gradient — a Math \[text\] that says "$frac(partial L, partial u) = 2 Sigma u - 2 lambda u = 0$"; eigenproblem — a Math \[text\] that says "$Sigma u = lambda u$"; heading\_lagrange — a Heading that says "The Maximizer Must Be an Eigenvector"; v1\_line — a Line \[red\] labelled "v\_1" drawn in axes (start=(-3.7, -3.7), end=(3.7, 3.7))

Actions:
- [08:53.32](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=533.3201041666666): meaning is shown on the screen, written out.
- [09:4.699](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=544.6991041666666): meaning (the "lambda" part) is emphasized.
- [09:6.812](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=546.8116041666667): meaning (the "lambda" part) is no longer emphasized.

##### [09:7.412](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=547.4116041666666)

Narration: Therefore the first component is the eigenvector with the largest eigenvalue. The second is the perpendicular eigenvector with the next eigenvalue, and so on in descending order.

Board: axes — a Figure (x\_range=(-4.2, 4.2), y\_range=(-4.2, 4.2), aspect=(1.0, 1.0)); cloud — a Point \[blue\] drawn in axes (location=(-3.1, -2.4), marker\_radius=0.07); cloud\_2 — a Point \[blue\] drawn in axes (location=(-2.7, -3.0), marker\_radius=0.07); cloud\_3 — a Point \[blue\] drawn in axes (location=(-2.2, -1.5), marker\_radius=0.07); cloud\_4 — a Point \[blue\] drawn in axes (location=(-1.6, -2.1), marker\_radius=0.07); cloud\_5 — a Point \[blue\] drawn in axes (location=(-1.1, -0.5), marker\_radius=0.07); cloud\_6 — a Point \[blue\] drawn in axes (location=(-0.5, -1.0), marker\_radius=0.07); cloud\_7 — a Point \[blue\] drawn in axes (location=(0.4, 0.9), marker\_radius=0.07); cloud\_8 — a Point \[blue\] drawn in axes (location=(0.9, 0.4), marker\_radius=0.07); cloud\_9 — a Point \[blue\] drawn in axes (location=(1.4, 2.0), marker\_radius=0.07); cloud\_10 — a Point \[blue\] drawn in axes (location=(2.0, 1.5), marker\_radius=0.07); cloud\_11 — a Point \[blue\] drawn in axes (location=(2.6, 3.0), marker\_radius=0.07); cloud\_12 — a Point \[blue\] drawn in axes (location=(3.1, 2.4), marker\_radius=0.07); meaning — a Math \[text\] that says "$u^T Sigma u = lambda$"; lagrangian — a Math \[text\] that says "$L(u, lambda) = u^T Sigma u - lambda (u^T u - 1)$"; gradient — a Math \[text\] that says "$frac(partial L, partial u) = 2 Sigma u - 2 lambda u = 0$"; eigenproblem — a Math \[text\] that says "$Sigma u = lambda u$"; heading\_lagrange — a Heading that says "The Maximizer Must Be an Eigenvector"; v1\_line — a Line \[red\] labelled "v\_1" drawn in axes (start=(-3.7, -3.7), end=(3.7, 3.7))

Actions:
- [09:12.172](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=552.1721041666667): v2\_line is shown on the screen, written out.
- [09:16.525](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=556.5251041666667): ranking is shown on the screen, written out.

##### [09:18.612](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=558.6116041666667)

Narration: Covariance matrices are symmetric and positive semidefinite. Their eigenvectors can be chosen orthonormal, and their eigenvalues are nonnegative. Those facts are exactly what the geometric construction required.

Board: axes — a Figure (x\_range=(-4.2, 4.2), y\_range=(-4.2, 4.2), aspect=(1.0, 1.0)); cloud — a Point \[blue\] drawn in axes (location=(-3.1, -2.4), marker\_radius=0.07); cloud\_2 — a Point \[blue\] drawn in axes (location=(-2.7, -3.0), marker\_radius=0.07); cloud\_3 — a Point \[blue\] drawn in axes (location=(-2.2, -1.5), marker\_radius=0.07); cloud\_4 — a Point \[blue\] drawn in axes (location=(-1.6, -2.1), marker\_radius=0.07); cloud\_5 — a Point \[blue\] drawn in axes (location=(-1.1, -0.5), marker\_radius=0.07); cloud\_6 — a Point \[blue\] drawn in axes (location=(-0.5, -1.0), marker\_radius=0.07); cloud\_7 — a Point \[blue\] drawn in axes (location=(0.4, 0.9), marker\_radius=0.07); cloud\_8 — a Point \[blue\] drawn in axes (location=(0.9, 0.4), marker\_radius=0.07); cloud\_9 — a Point \[blue\] drawn in axes (location=(1.4, 2.0), marker\_radius=0.07); cloud\_10 — a Point \[blue\] drawn in axes (location=(2.0, 1.5), marker\_radius=0.07); cloud\_11 — a Point \[blue\] drawn in axes (location=(2.6, 3.0), marker\_radius=0.07); cloud\_12 — a Point \[blue\] drawn in axes (location=(3.1, 2.4), marker\_radius=0.07); meaning — a Math \[text\] that says "$u^T Sigma u = lambda$"; lagrangian — a Math \[text\] that says "$L(u, lambda) = u^T Sigma u - lambda (u^T u - 1)$"; gradient — a Math \[text\] that says "$frac(partial L, partial u) = 2 Sigma u - 2 lambda u = 0$"; eigenproblem — a Math \[text\] that says "$Sigma u = lambda u$"; ranking — a Math \[text\] that says "$lambda\_1 \>= lambda\_2 \>= dots \>= 0$"; heading\_lagrange — a Heading that says "The Maximizer Must Be an Eigenvector"; v1\_line — a Line \[red\] labelled "v\_1" drawn in axes (start=(-3.7, -3.7), end=(3.7, 3.7)); v2\_line — a Line \[green\] labelled "v\_2" drawn in axes (start=(3.7, -3.7), end=(-3.7, 3.7))

Actions:
- [09:25.264](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=565.2641041666666): v1\_line is indicated — a transient flash.
- [09:25.464](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=565.4641041666666): v2\_line is indicated — a transient flash.
- [09:32.625](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=572.6246041666667): axes is hidden from the screen — left the board.
- [09:32.625](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=572.6246041666667): cloud is hidden from the screen — axes left the board.
- [09:32.625](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=572.6246041666667): cloud\_2 is hidden from the screen — axes left the board.
- [09:32.625](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=572.6246041666667): cloud\_3 is hidden from the screen — axes left the board.
- [09:32.625](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=572.6246041666667): cloud\_4 is hidden from the screen — axes left the board.
- [09:32.625](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=572.6246041666667): cloud\_5 is hidden from the screen — axes left the board.
- [09:32.625](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=572.6246041666667): cloud\_6 is hidden from the screen — axes left the board.
- [09:32.625](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=572.6246041666667): cloud\_7 is hidden from the screen — axes left the board.
- [09:32.625](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=572.6246041666667): cloud\_8 is hidden from the screen — axes left the board.
- [09:32.625](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=572.6246041666667): cloud\_9 is hidden from the screen — axes left the board.
- [09:32.625](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=572.6246041666667): cloud\_10 is hidden from the screen — axes left the board.
- [09:32.625](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=572.6246041666667): cloud\_11 is hidden from the screen — axes left the board.
- [09:32.625](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=572.6246041666667): cloud\_12 is hidden from the screen — axes left the board.
- [09:32.625](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=572.6246041666667): v1\_line is hidden from the screen — axes left the board.
- [09:32.625](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=572.6246041666667): v2\_line is hidden from the screen — axes left the board.
- [09:32.625](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=572.6246041666667): eigenproblem is hidden from the screen — left the board.
- [09:32.625](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=572.6246041666667): gradient is hidden from the screen — left the board.
- [09:32.625](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=572.6246041666667): heading\_lagrange is hidden from the screen — left the board.
- [09:32.625](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=572.6246041666667): lagrangian is hidden from the screen — left the board.
- [09:32.625](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=572.6246041666667): meaning is hidden from the screen — left the board.
- [09:32.625](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=572.6246041666667): ranking is hidden from the screen — left the board.

##### [09:33.225](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=573.2246041666667)

Narration: Now calculate the example. Use the covariance matrix with fours on the diagonal and threes off the diagonal. Its positive covariance says the two variables tend to rise and fall together.

Board: Empty.

Actions:
- [09:33.225](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=573.2246041666667): heading\_example is shown on the screen, written out.
- [09:36.534](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=576.5341041666667): sigma is shown on the screen, written out.

##### [09:46.537](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=586.5371041666667)

Narration: Eigenvalues make Sigma minus lambda I singular. Subtract lambda from both diagonal entries and set the determinant to zero.

Board: sigma — a Math \[text\] that says "$Sigma = mat(4, 3; 3, 4)$"; heading\_example — a Heading that says "The Elongated Cloud, Calculated"

Actions:
- [09:51.054](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=591.0541041666667): determinant is shown on the screen, written out.
- [09:54.49](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=594.4901041666667): determinant (the "diagonal=main" part) is struck through — it is ruled out.
- [09:54.69](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=594.6901041666666): determinant (the "diagonal=anti" part) is struck through — it is ruled out.

##### [09:56.588](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=596.5876041666667)

Narration: Multiply the main diagonal and subtract the other product. We get four minus lambda squared, minus nine, equal to zero.

Board: sigma — a Math \[text\] that says "$Sigma = mat(4, 3; 3, 4)$"; determinant — a Math \[text\] that says "$op("det")(Sigma - lambda I) = op("det") mat(4-lambda, 3; 3, 4-lambda)$"; heading\_example — a Heading that says "The Elongated Cloud, Calculated"

Actions:
- [09:59.037](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=599.0371041666667): The strike through determinant (the "diagonal=main" part) is lifted.
- [09:59.237](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=599.2371041666667): The strike through determinant (the "diagonal=anti" part) is lifted.
- [10:2.822](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=602.8221041666667): polynomial is shown on the screen, written out.

##### [10:6.708](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=606.7081041666667)

Narration: Expand the square and collect terms. The characteristic polynomial is lambda squared minus eight lambda plus seven.

Board: sigma — a Math \[text\] that says "$Sigma = mat(4, 3; 3, 4)$"; determinant — a Math \[text\] that says "$op("det")(Sigma - lambda I) = op("det") mat(4-lambda, 3; 3, 4-lambda)$"; polynomial — a Math \[text\] that says "$(4-lambda)^2 - 9 = 0$"; heading\_example — a Heading that says "The Elongated Cloud, Calculated"

Actions:
- [10:7.056](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=607.0561041666666): polynomial becomes "$lambda^2 - 8 lambda + 7 = 0$".

##### [10:15.54](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=615.5396041666667)

Narration: Factor it. Lambda minus seven times lambda minus one equals zero. The two eigenvalues are seven and one, exactly the maximum and minimum projected variances from the rotating line.

Board: Unchanged from the preceding beat in this scene.

Actions:
- [10:15.54](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=615.5396041666667): values is shown on the screen, written out.
- [10:15.888](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=615.8881041666666): polynomial becomes "$(lambda - 7)(lambda - 1) = 0$".
- [10:18.477](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=618.4771041666667): values (the "7" part) is emphasized.
- [10:20.149](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=620.1491041666666): values (the "1" part) is emphasized.
- [10:20.149](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=620.1491041666666): values (the "7" part) is no longer emphasized.
- [10:29.24](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=629.2396041666666): values (the "1" part) is no longer emphasized.

##### [10:29.84](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=629.8396041666667)

Narration: The eigenvector for seven points along one, one. The other eigenvector points along one, minus one. Normalize both by dividing by square root two.

Board: sigma — a Math \[text\] that says "$Sigma = mat(4, 3; 3, 4)$"; determinant — a Math \[text\] that says "$op("det")(Sigma - lambda I) = op("det") mat(4-lambda, 3; 3, 4-lambda)$"; polynomial — a Math \[text\] that says "$(4-lambda)^2 - 9 = 0$"; values — a Math \[text\] that says "$lambda\_1 = 7, quad lambda\_2 = 1$"; heading\_example — a Heading that says "The Elongated Cloud, Calculated"

Actions:
- [10:30.467](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=630.4671041666667): vectors is shown on the screen, written out.
- [10:31.268](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=631.2681041666667): vectors (the "frac(1, sqrt(2)) vec(1, 1)" part) is emphasized.
- [10:34.182](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=634.1821041666667): vectors (the "frac(1, sqrt(2)) vec(1, -1)" part) is emphasized.
- [10:34.182](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=634.1821041666667): vectors (the "frac(1, sqrt(2)) vec(1, 1)" part) is no longer emphasized.
- [10:40.324](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=640.3236041666667): determinant is hidden from the screen — left the board.
- [10:40.324](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=640.3236041666667): heading\_example is hidden from the screen — left the board.
- [10:40.324](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=640.3236041666667): polynomial is hidden from the screen — left the board.
- [10:40.324](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=640.3236041666667): sigma is hidden from the screen — left the board.
- [10:40.324](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=640.3236041666667): values is hidden from the screen — left the board.
- [10:40.324](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=640.3236041666667): vectors is hidden from the screen — left the board.
- [10:40.324](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=640.3236041666667): vectors (the "frac(1, sqrt(2)) vec(1, -1)" part) is no longer emphasized.

##### [10:41.524](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=641.5236041666667)

Narration: Seven of the total eight variance units lie along component one. So one component explains eighty-seven point five percent of this cloud's variance. The eigenvalue calculation has returned every fact we first discovered by rotating and projecting.

Board: Empty.

Actions:
- [10:41.524](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=641.5236041666667): heading\_share is shown on the screen, written out.
- [10:41.825](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=641.8251041666666): share is shown on the screen, written out.
- [10:42.975](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=642.9751041666666): A box is drawn around share.
- [10:56.454](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=656.4537291666667): heading\_share is hidden from the screen — left the board.
- [10:56.454](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=656.4537291666667): share is hidden from the screen — left the board.

### Scene 4: [Scree Plots and Reconstruction](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=657.4953958333333)

Span: 10:57.495–14:39.662 (657.4953958333333s–879.6621666666666s).

#### Objects

- average\_error: a Math \[text\] that says "$frac(1, n) sum\_i norm(x\_i-hat(x)\_(i,k))^2 = sum\_(j\>k) lambda\_j$"
- cumulative: a Math \[text\] that says "$C\_k = frac(sum\_(j=1)^k lambda\_j, sum\_l lambda\_l)$"
- elbow: a Point \[yellow\] labelled "upright("elbow")" drawn in scree (location=(3.0, 0.8))
- error\_one: a Line \[red\] drawn in recon\_axes (start=(2.2, 0.0, 0.0), end=(2.2, 1.2, 0.6), dashed=True)
- error\_two: a Line \[red\] drawn in recon\_axes (start=(2.2, 1.2, 0.0), end=(2.2, 1.2, 0.6), dashed=True)
- errors: a Math \[text\] that says "$norm(x-hat(x)\_1) = 1.34, quad norm(x-hat(x)\_2) = 0.60$"
- first\_contribution: a Vector \[green\] labelled "2.2 v\_1" drawn in recon\_axes (start=(0.0, 0.0, 0.0), end=(2.2, 0.0, 0.0))
- first\_reconstruction: a Point \[green\] labelled "hat(x)\_1" drawn in recon\_axes (location=(2.2, 0.0, 0.0))
- full: a Math \[text\] that says "$x = overline(x) + 2.2 v\_1 + 1.2 v\_2 + 0.6 v\_3$"
- general: a Math \[text\] that says "$hat(x)\_k = overline(x) + sum\_(j=1)^k z\_j v\_j$"
- heading\_reconstruction: a Heading that says "What Keeping Components Actually Does"
- heading\_scree: a Heading that says "How Many Components Should We Keep?"
- mean: a Point \[gray\] labelled "overline(x)" drawn in recon\_axes (location=(0.0, 0.0, 0.0))
- one: a Math \[text\] that says "$hat(x)\_1 = overline(x) + 2.2 v\_1$"
- original: a Point \[red\] labelled "x" drawn in recon\_axes (location=(2.2, 1.2, 0.6))
- ratio: a Math \[text\] that says "$upright("explained ratio")\_j = frac(lambda\_j, sum\_l lambda\_l)$"
- recon\_axes: an Axes3D (x\_range=(0.0, 2.8), y\_range=(0.0, 1.8), z\_range=(0.0, 1.2))
- scree: an Axes (x\_range=(0.5, 5.5), y\_range=(0.0, 8.0), x\_ticks\_every=1.0)
- scree\_lines: a Line \[gray\] drawn in scree (start=(1.0, 7.0), end=(2.0, 2.2))
- scree\_lines\_2: a Line \[gray\] drawn in scree (start=(2.0, 2.2), end=(3.0, 0.8))
- scree\_lines\_3: a Line \[gray\] drawn in scree (start=(3.0, 0.8), end=(4.0, 0.25))
- scree\_lines\_4: a Line \[gray\] drawn in scree (start=(4.0, 0.25), end=(5.0, 0.1))
- scree\_points: a Point \[red\] labelled "7.0" drawn in scree (location=(1.0, 7.0))
- scree\_points\_2: a Point \[green\] labelled "2.2" drawn in scree (location=(2.0, 2.2))
- scree\_points\_3: a Point \[yellow\] labelled "0.8" drawn in scree (location=(3.0, 0.8))
- scree\_points\_4: a Point \[blue\] labelled "0.25" drawn in scree (location=(4.0, 0.25))
- scree\_points\_5: a Point \[magenta\] labelled "0.10" drawn in scree (location=(5.0, 0.1))
- second\_contribution: a Vector \[yellow\] labelled "1.2 v\_2" drawn in recon\_axes (start=(2.2, 0.0, 0.0), end=(2.2, 1.2, 0.0))
- second\_reconstruction: a Point \[yellow\] labelled "hat(x)\_2" drawn in recon\_axes (location=(2.2, 1.2, 0.0))
- table: a Table \[text\] that says "Kept Individual Cumulative 1 67.6% 67.6% 2 21.3% 88.9% 3 7.7% 96.6% 4 2.4% 99.0% 5 1.0% 100%" (rows=(('Kept', 'Individual', 'Cumulative'), ('1', '67.6%', '67.6%'),…, header=True)
- third\_contribution: a Vector \[red\] labelled "0.6 v\_3" drawn in recon\_axes (start=(2.2, 1.2, 0.0), end=(2.2, 1.2, 0.6))
- two: a Math \[text\] that says "$hat(x)\_2 = overline(x) + 2.2 v\_1 + 1.2 v\_2$"

#### Beats

##### [10:57.495](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=657.4953958333333)

Narration: PCA orders components from most variance to least, but a model still needs a decision: how many components should we keep? The eigenvalues give the evidence for that decision.

Board: Empty.

Actions:
- [10:57.495](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=657.4953958333333): heading\_scree is shown on the screen, written out.
- [11:5.622](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=665.6223958333333): scree is shown on the screen, written out.

##### [11:9.067](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=669.0668958333333)

Narration: A scree plot puts the component number along the horizontal axis and its eigenvalue on the vertical axis. Here are five eigenvalues in descending order.

Board: scree — an Axes (x\_range=(0.5, 5.5), y\_range=(0.0, 8.0), x\_ticks\_every=1.0); heading\_scree — a Heading that says "How Many Components Should We Keep?"

Actions:
- [11:9.879](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=669.8793958333333): scree\_lines is shown on the screen, written out.
- [11:9.879](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=669.8793958333333): scree\_lines\_2 is shown on the screen, written out.
- [11:9.879](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=669.8793958333333): scree\_lines\_3 is shown on the screen, written out.
- [11:9.879](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=669.8793958333333): scree\_lines\_4 is shown on the screen, written out.
- [11:16.904](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=676.9043958333333): scree\_points is shown on the screen, written out.
- [11:16.904](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=676.9043958333333): scree\_points\_2 is shown on the screen, written out.
- [11:16.904](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=676.9043958333333): scree\_points\_3 is shown on the screen, written out.
- [11:16.904](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=676.9043958333333): scree\_points\_4 is shown on the screen, written out.
- [11:16.904](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=676.9043958333333): scree\_points\_5 is shown on the screen, written out.

##### [11:19.93](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=679.9303958333333)

Narration: The first point is high because component one captures a great deal of variation. The second captures less. After the third, the remaining eigenvalues are small.

Board: scree — an Axes (x\_range=(0.5, 5.5), y\_range=(0.0, 8.0), x\_ticks\_every=1.0); heading\_scree — a Heading that says "How Many Components Should We Keep?"; scree\_lines — a Line \[gray\] drawn in scree (start=(1.0, 7.0), end=(2.0, 2.2)); scree\_lines\_2 — a Line \[gray\] drawn in scree (start=(2.0, 2.2), end=(3.0, 0.8)); scree\_lines\_3 — a Line \[gray\] drawn in scree (start=(3.0, 0.8), end=(4.0, 0.25)); scree\_lines\_4 — a Line \[gray\] drawn in scree (start=(4.0, 0.25), end=(5.0, 0.1)); scree\_points — a Point \[red\] labelled "7.0" drawn in scree (location=(1.0, 7.0)); scree\_points\_2 — a Point \[green\] labelled "2.2" drawn in scree (location=(2.0, 2.2)); scree\_points\_3 — a Point \[yellow\] labelled "0.8" drawn in scree (location=(3.0, 0.8)); scree\_points\_4 — a Point \[blue\] labelled "0.25" drawn in scree (location=(4.0, 0.25)); scree\_points\_5 — a Point \[magenta\] labelled "0.10" drawn in scree (location=(5.0, 0.1))

Actions:
- [11:20.406](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=680.4063958333334): scree\_points is indicated — a transient flash.
- [11:25.503](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=685.5033958333333): scree\_points\_2 is indicated — a transient flash.
- [11:28.115](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=688.1153958333333): scree\_points\_3 is indicated — a transient flash.

##### [11:31.688](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=691.6878958333333)

Narration: An individual explained-variance ratio divides one eigenvalue by the sum of all eigenvalues. It reports the fraction of total variance assigned to that component.

Board: Unchanged from the preceding beat in this scene.

Actions:
- [11:33.824](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=693.8243958333333): scree moves to a new place on the board.
- [11:33.824](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=693.8243958333333): ratio is shown on the screen, written out.

##### [11:43.201](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=703.2013958333333)

Narration: Cumulative explained variance adds the first k eigenvalues before dividing by the total. It asks how much variation remains visible after reducing the data to k component coordinates.

Board: ratio — a Math \[text\] that says "$upright("explained ratio")\_j = frac(lambda\_j, sum\_l lambda\_l)$"; scree — an Axes (x\_range=(0.5, 5.5), y\_range=(0.0, 8.0), x\_ticks\_every=1.0); heading\_scree — a Heading that says "How Many Components Should We Keep?"; scree\_lines — a Line \[gray\] drawn in scree (start=(1.0, 7.0), end=(2.0, 2.2)); scree\_lines\_2 — a Line \[gray\] drawn in scree (start=(2.0, 2.2), end=(3.0, 0.8)); scree\_lines\_3 — a Line \[gray\] drawn in scree (start=(3.0, 0.8), end=(4.0, 0.25)); scree\_lines\_4 — a Line \[gray\] drawn in scree (start=(4.0, 0.25), end=(5.0, 0.1)); scree\_points — a Point \[red\] labelled "7.0" drawn in scree (location=(1.0, 7.0)); scree\_points\_2 — a Point \[green\] labelled "2.2" drawn in scree (location=(2.0, 2.2)); scree\_points\_3 — a Point \[yellow\] labelled "0.8" drawn in scree (location=(3.0, 0.8)); scree\_points\_4 — a Point \[blue\] labelled "0.25" drawn in scree (location=(4.0, 0.25)); scree\_points\_5 — a Point \[magenta\] labelled "0.10" drawn in scree (location=(5.0, 0.1))

Actions:
- [11:43.677](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=703.6773958333333): cumulative is shown on the screen, written out.
- [11:51.2](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=711.2003958333333): table is shown on the screen, written out.

##### [11:55.875](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=715.8753958333333)

Narration: With one component, this example retains sixty-seven point six percent. With two, the total reaches eighty-eight point nine percent.

Board: ratio — a Math \[text\] that says "$upright("explained ratio")\_j = frac(lambda\_j, sum\_l lambda\_l)$"; cumulative — a Math \[text\] that says "$C\_k = frac(sum\_(j=1)^k lambda\_j, sum\_l lambda\_l)$"; scree — an Axes (x\_range=(0.5, 5.5), y\_range=(0.0, 8.0), x\_ticks\_every=1.0); heading\_scree — a Heading that says "How Many Components Should We Keep?"; scree\_lines — a Line \[gray\] drawn in scree (start=(1.0, 7.0), end=(2.0, 2.2)); scree\_lines\_2 — a Line \[gray\] drawn in scree (start=(2.0, 2.2), end=(3.0, 0.8)); scree\_lines\_3 — a Line \[gray\] drawn in scree (start=(3.0, 0.8), end=(4.0, 0.25)); scree\_lines\_4 — a Line \[gray\] drawn in scree (start=(4.0, 0.25), end=(5.0, 0.1)); scree\_points — a Point \[red\] labelled "7.0" drawn in scree (location=(1.0, 7.0)); scree\_points\_2 — a Point \[green\] labelled "2.2" drawn in scree (location=(2.0, 2.2)); scree\_points\_3 — a Point \[yellow\] labelled "0.8" drawn in scree (location=(3.0, 0.8)); scree\_points\_4 — a Point \[blue\] labelled "0.25" drawn in scree (location=(4.0, 0.25)); scree\_points\_5 — a Point \[magenta\] labelled "0.10" drawn in scree (location=(5.0, 0.1))

Actions:
- [11:56.595](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=716.5953958333333): table is shown on the screen, written out.
- [12:1.576](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=721.5763958333333): table is shown on the screen, written out.
- [12:3.364](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=723.3643958333333): table (the "row=3" part) is emphasized.
- [12:5.164](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=725.1638958333333): table (the "row=3" part) is no longer emphasized.

##### [12:5.764](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=725.7638958333333)

Narration: With three components it reaches ninety-six point six percent. Components four and five add very little.

Board: Unchanged from the preceding beat in this scene.

Actions:
- [12:6.356](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=726.3563958333333): table is shown on the screen, written out.
- [12:10.558](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=730.5583958333333): table is shown on the screen, written out.
- [12:11.058](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=731.0583958333333): table is shown on the screen, written out.

##### [12:13.597](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=733.5968958333333)

Narration: The visible bend near component three is often called the elbow. One common heuristic keeps components before the curve settles into a shallow tail.

Board: Unchanged from the preceding beat in this scene.

Actions:
- [12:16.894](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=736.8943958333333): elbow is shown on the screen, written out.

##### [12:23.346](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=743.3458958333333)

Narration: But the elbow is not a theorem and ninety-five percent is not a universal law. Keep enough components for the downstream task. For prediction, choose k with cross-validation. For compression, choose an acceptable reconstruction error. For interpretation, check stability and domain meaning.

Board: ratio — a Math \[text\] that says "$upright("explained ratio")\_j = frac(lambda\_j, sum\_l lambda\_l)$"; cumulative — a Math \[text\] that says "$C\_k = frac(sum\_(j=1)^k lambda\_j, sum\_l lambda\_l)$"; scree — an Axes (x\_range=(0.5, 5.5), y\_range=(0.0, 8.0), x\_ticks\_every=1.0); heading\_scree — a Heading that says "How Many Components Should We Keep?"; scree\_lines — a Line \[gray\] drawn in scree (start=(1.0, 7.0), end=(2.0, 2.2)); scree\_lines\_2 — a Line \[gray\] drawn in scree (start=(2.0, 2.2), end=(3.0, 0.8)); scree\_lines\_3 — a Line \[gray\] drawn in scree (start=(3.0, 0.8), end=(4.0, 0.25)); scree\_lines\_4 — a Line \[gray\] drawn in scree (start=(4.0, 0.25), end=(5.0, 0.1)); scree\_points — a Point \[red\] labelled "7.0" drawn in scree (location=(1.0, 7.0)); scree\_points\_2 — a Point \[green\] labelled "2.2" drawn in scree (location=(2.0, 2.2)); scree\_points\_3 — a Point \[yellow\] labelled "0.8" drawn in scree (location=(3.0, 0.8)); scree\_points\_4 — a Point \[blue\] labelled "0.25" drawn in scree (location=(4.0, 0.25)); scree\_points\_5 — a Point \[magenta\] labelled "0.10" drawn in scree (location=(5.0, 0.1)); elbow — a Point \[yellow\] labelled "upright("elbow")" drawn in scree (location=(3.0, 0.8))

Actions:
- [12:25.377](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=745.3773958333333): table (the "96.6%" part) is indicated — a transient flash.

##### [12:42.244](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=762.2438958333333)

Narration: A scree plot is diagnostic evidence, not an automatic command. A weak elbow may honestly mean there is no clean low-dimensional cutoff.

Board: Unchanged from the preceding beat in this scene.

Actions:
- [12:51.334](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=771.3343958333332): cumulative is hidden from the screen — left the board.
- [12:51.334](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=771.3343958333332): heading\_scree is hidden from the screen — left the board.
- [12:51.334](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=771.3343958333332): ratio is hidden from the screen — left the board.
- [12:51.334](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=771.3343958333332): scree is hidden from the screen — left the board.
- [12:51.334](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=771.3343958333332): scree\_lines is hidden from the screen — scree left the board.
- [12:51.334](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=771.3343958333332): scree\_lines\_2 is hidden from the screen — scree left the board.
- [12:51.334](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=771.3343958333332): scree\_lines\_3 is hidden from the screen — scree left the board.
- [12:51.334](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=771.3343958333332): scree\_lines\_4 is hidden from the screen — scree left the board.
- [12:51.334](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=771.3343958333332): scree\_points is hidden from the screen — scree left the board.
- [12:51.334](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=771.3343958333332): scree\_points\_2 is hidden from the screen — scree left the board.
- [12:51.334](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=771.3343958333332): scree\_points\_3 is hidden from the screen — scree left the board.
- [12:51.334](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=771.3343958333332): scree\_points\_4 is hidden from the screen — scree left the board.
- [12:51.334](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=771.3343958333332): scree\_points\_5 is hidden from the screen — scree left the board.
- [12:51.334](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=771.3343958333332): elbow is hidden from the screen — scree left the board.
- [12:51.334](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=771.3343958333332): table is hidden from the screen — left the board.

##### [12:52.534](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=772.5343958333333)

Narration: Now make component retention concrete. In principal-component coordinates, a centered observation is built from one contribution along each component direction.

Board: Empty.

Actions:
- [12:52.534](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=772.5343958333333): heading\_reconstruction is shown on the screen, written out.
- [12:57.375](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=777.3753958333333): recon\_axes is shown on the screen, written out.
- [12:58.432](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=778.4323958333333): mean is shown on the screen, written out.

##### [13:3.677](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=783.6768958333333)

Narration: This observation has a large score on component one, a smaller score on component two, and a still smaller score on component three.

Board: recon\_axes — an Axes3D (x\_range=(0.0, 2.8), y\_range=(0.0, 1.8), z\_range=(0.0, 1.2)); heading\_reconstruction — a Heading that says "What Keeping Components Actually Does"; mean — a Point \[gray\] labelled "overline(x)" drawn in recon\_axes (location=(0.0, 0.0, 0.0))

Actions:
- [13:4.257](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=784.2573958333332): original is shown on the screen, written out.
- [13:5.592](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=785.5923958333333): full is shown on the screen, written out.
- [13:6.091](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=786.0913958333333): recon\_axes turns in its own slot.

##### [13:12.717](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=792.7173958333333)

Narration: The general reconstruction formula starts at the original mean and adds back the first k score times direction contributions.

Board: full — a Math \[text\] that says "$x = overline(x) + 2.2 v\_1 + 1.2 v\_2 + 0.6 v\_3$"; recon\_axes — an Axes3D (x\_range=(0.0, 2.8), y\_range=(0.0, 1.8), z\_range=(0.0, 1.2)); heading\_reconstruction — a Heading that says "What Keeping Components Actually Does"; mean — a Point \[gray\] labelled "overline(x)" drawn in recon\_axes (location=(0.0, 0.0, 0.0)); original — a Point \[red\] labelled "x" drawn in recon\_axes (location=(2.2, 1.2, 0.6))

Actions:
- [13:14.377](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=794.3773958333333): general is shown on the screen, written out.

##### [13:20.701](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=800.7013958333333)

Narration: Keep only component one. Starting from the mean, move along v one by the first score. The green point is the rank-one reconstruction.

Board: general — a Math \[text\] that says "$hat(x)\_k = overline(x) + sum\_(j=1)^k z\_j v\_j$"; full — a Math \[text\] that says "$x = overline(x) + 2.2 v\_1 + 1.2 v\_2 + 0.6 v\_3$"; recon\_axes — an Axes3D (x\_range=(0.0, 2.8), y\_range=(0.0, 1.8), z\_range=(0.0, 1.2)); heading\_reconstruction — a Heading that says "What Keeping Components Actually Does"; mean — a Point \[gray\] labelled "overline(x)" drawn in recon\_axes (location=(0.0, 0.0, 0.0)); original — a Point \[red\] labelled "x" drawn in recon\_axes (location=(2.2, 1.2, 0.6))

Actions:
- [13:21.734](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=801.7343958333333): one is shown on the screen, written out.
- [13:24.335](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=804.3353958333332): first\_contribution is shown on the screen, written out.
- [13:27.365](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=807.3653958333333): first\_reconstruction is shown on the screen, written out.
- [13:28.712](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=808.7123958333333): error\_one is shown on the screen, written out.

##### [13:30.381](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=810.3808958333333)

Narration: It preserves the strongest coordinate and discards everything in the omitted perpendicular subspace. The red dashed segment is the information lost for this observation.

Board: general — a Math \[text\] that says "$hat(x)\_k = overline(x) + sum\_(j=1)^k z\_j v\_j$"; full — a Math \[text\] that says "$x = overline(x) + 2.2 v\_1 + 1.2 v\_2 + 0.6 v\_3$"; one — a Math \[text\] that says "$hat(x)\_1 = overline(x) + 2.2 v\_1$"; recon\_axes — an Axes3D (x\_range=(0.0, 2.8), y\_range=(0.0, 1.8), z\_range=(0.0, 1.2)); heading\_reconstruction — a Heading that says "What Keeping Components Actually Does"; mean — a Point \[gray\] labelled "overline(x)" drawn in recon\_axes (location=(0.0, 0.0, 0.0)); original — a Point \[red\] labelled "x" drawn in recon\_axes (location=(2.2, 1.2, 0.6)); first\_contribution — a Vector \[green\] labelled "2.2 v\_1" drawn in recon\_axes (start=(0.0, 0.0, 0.0), end=(2.2, 0.0, 0.0)); first\_reconstruction — a Point \[green\] labelled "hat(x)\_1" drawn in recon\_axes (location=(2.2, 0.0, 0.0)); error\_one — a Line \[red\] drawn in recon\_axes (start=(2.2, 0.0, 0.0), end=(2.2, 1.2, 0.6), dashed=True)

Actions:
- [13:38.275](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=818.2753958333333): error\_one is indicated — a transient flash.

##### [13:40.698](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=820.6983958333333)

Narration: Now retain component two as well. Add the yellow contribution from the first reconstruction. The new point moves closer to the original.

Board: Unchanged from the preceding beat in this scene.

Actions:
- [13:41.615](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=821.6153958333333): two is shown on the screen, written out.
- [13:43.949](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=823.9493958333333): second\_contribution is shown on the screen, written out.
- [13:47.688](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=827.6883958333333): second\_reconstruction is shown on the screen, written out.
- [13:47.688](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=827.6883958333333): error\_one is hidden from the screen.
- [13:47.688](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=827.6883958333333): error\_two is shown on the screen, written out.

##### [13:49.832](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=829.8323958333333)

Narration: The remaining error lies entirely along component three. If we retain that final red contribution, the reconstruction reaches the original point exactly.

Board: general — a Math \[text\] that says "$hat(x)\_k = overline(x) + sum\_(j=1)^k z\_j v\_j$"; full — a Math \[text\] that says "$x = overline(x) + 2.2 v\_1 + 1.2 v\_2 + 0.6 v\_3$"; one — a Math \[text\] that says "$hat(x)\_1 = overline(x) + 2.2 v\_1$"; two — a Math \[text\] that says "$hat(x)\_2 = overline(x) + 2.2 v\_1 + 1.2 v\_2$"; recon\_axes — an Axes3D (x\_range=(0.0, 2.8), y\_range=(0.0, 1.8), z\_range=(0.0, 1.2)); heading\_reconstruction — a Heading that says "What Keeping Components Actually Does"; mean — a Point \[gray\] labelled "overline(x)" drawn in recon\_axes (location=(0.0, 0.0, 0.0)); original — a Point \[red\] labelled "x" drawn in recon\_axes (location=(2.2, 1.2, 0.6)); first\_contribution — a Vector \[green\] labelled "2.2 v\_1" drawn in recon\_axes (start=(0.0, 0.0, 0.0), end=(2.2, 0.0, 0.0)); first\_reconstruction — a Point \[green\] labelled "hat(x)\_1" drawn in recon\_axes (location=(2.2, 0.0, 0.0)); second\_contribution — a Vector \[yellow\] labelled "1.2 v\_2" drawn in recon\_axes (start=(2.2, 0.0, 0.0), end=(2.2, 1.2, 0.0)); second\_reconstruction — a Point \[yellow\] labelled "hat(x)\_2" drawn in recon\_axes (location=(2.2, 1.2, 0.0)); error\_two — a Line \[red\] drawn in recon\_axes (start=(2.2, 1.2, 0.0), end=(2.2, 1.2, 0.6), dashed=True)

Actions:
- [13:55.138](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=835.1383958333333): third\_contribution is shown on the screen, written out.
- [13:58.435](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=838.4353958333334): error\_two is hidden from the screen.

##### [14:0.15](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=840.1498958333333)

Narration: The displayed error falls from one point three four with one component to zero point six with two. Every additional component can only reduce squared reconstruction error on the training data.

Board: general — a Math \[text\] that says "$hat(x)\_k = overline(x) + sum\_(j=1)^k z\_j v\_j$"; full — a Math \[text\] that says "$x = overline(x) + 2.2 v\_1 + 1.2 v\_2 + 0.6 v\_3$"; one — a Math \[text\] that says "$hat(x)\_1 = overline(x) + 2.2 v\_1$"; two — a Math \[text\] that says "$hat(x)\_2 = overline(x) + 2.2 v\_1 + 1.2 v\_2$"; recon\_axes — an Axes3D (x\_range=(0.0, 2.8), y\_range=(0.0, 1.8), z\_range=(0.0, 1.2)); heading\_reconstruction — a Heading that says "What Keeping Components Actually Does"; mean — a Point \[gray\] labelled "overline(x)" drawn in recon\_axes (location=(0.0, 0.0, 0.0)); original — a Point \[red\] labelled "x" drawn in recon\_axes (location=(2.2, 1.2, 0.6)); first\_contribution — a Vector \[green\] labelled "2.2 v\_1" drawn in recon\_axes (start=(0.0, 0.0, 0.0), end=(2.2, 0.0, 0.0)); first\_reconstruction — a Point \[green\] labelled "hat(x)\_1" drawn in recon\_axes (location=(2.2, 0.0, 0.0)); second\_contribution — a Vector \[yellow\] labelled "1.2 v\_2" drawn in recon\_axes (start=(2.2, 0.0, 0.0), end=(2.2, 1.2, 0.0)); second\_reconstruction — a Point \[yellow\] labelled "hat(x)\_2" drawn in recon\_axes (location=(2.2, 1.2, 0.0)); third\_contribution — a Vector \[red\] labelled "0.6 v\_3" drawn in recon\_axes (start=(2.2, 1.2, 0.0), end=(2.2, 1.2, 0.6))

Actions:
- [14:1.183](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=841.1833958333333): errors is shown on the screen, written out.
- [14:1.972](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=841.9723958333333): errors (the "1.34" part) is emphasized.
- [14:4.306](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=844.3063958333332): errors (the "0.60" part) is emphasized.
- [14:4.306](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=844.3063958333332): errors (the "1.34" part) is no longer emphasized.
- [14:11.423](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=851.4228958333333): errors (the "0.60" part) is no longer emphasized.

##### [14:12.023](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=852.0228958333332)

Narration: Averaged over the data set, the squared error after keeping k components equals the sum of the omitted eigenvalues. The scree plot and reconstruction error are therefore two readings of the same tradeoff.

Board: general — a Math \[text\] that says "$hat(x)\_k = overline(x) + sum\_(j=1)^k z\_j v\_j$"; full — a Math \[text\] that says "$x = overline(x) + 2.2 v\_1 + 1.2 v\_2 + 0.6 v\_3$"; one — a Math \[text\] that says "$hat(x)\_1 = overline(x) + 2.2 v\_1$"; two — a Math \[text\] that says "$hat(x)\_2 = overline(x) + 2.2 v\_1 + 1.2 v\_2$"; errors — a Math \[text\] that says "$norm(x-hat(x)\_1) = 1.34, quad norm(x-hat(x)\_2) = 0.60$"; recon\_axes — an Axes3D (x\_range=(0.0, 2.8), y\_range=(0.0, 1.8), z\_range=(0.0, 1.2)); heading\_reconstruction — a Heading that says "What Keeping Components Actually Does"; mean — a Point \[gray\] labelled "overline(x)" drawn in recon\_axes (location=(0.0, 0.0, 0.0)); original — a Point \[red\] labelled "x" drawn in recon\_axes (location=(2.2, 1.2, 0.6)); first\_contribution — a Vector \[green\] labelled "2.2 v\_1" drawn in recon\_axes (start=(0.0, 0.0, 0.0), end=(2.2, 0.0, 0.0)); first\_reconstruction — a Point \[green\] labelled "hat(x)\_1" drawn in recon\_axes (location=(2.2, 0.0, 0.0)); second\_contribution — a Vector \[yellow\] labelled "1.2 v\_2" drawn in recon\_axes (start=(2.2, 0.0, 0.0), end=(2.2, 1.2, 0.0)); second\_reconstruction — a Point \[yellow\] labelled "hat(x)\_2" drawn in recon\_axes (location=(2.2, 1.2, 0.0)); third\_contribution — a Vector \[red\] labelled "0.6 v\_3" drawn in recon\_axes (start=(2.2, 1.2, 0.0), end=(2.2, 1.2, 0.6))

Actions:
- [14:12.371](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=852.3713958333333): average\_error is shown on the screen, written out.
- [14:17.653](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=857.6533958333333): average\_error (the "sum\_(j\>k) lambda\_j" part) is emphasized.
- [14:24.735](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=864.7353958333333): average\_error (the "sum\_(j\>k) lambda\_j" part) is no longer emphasized.

##### [14:25.335](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=865.3353958333332)

Narration: Keeping a component means preserving one orthogonal pattern of variation. Dropping it means replacing every observation's score on that pattern by zero, then mapping the reduced coordinates back to the original variable space.

Board: general — a Math \[text\] that says "$hat(x)\_k = overline(x) + sum\_(j=1)^k z\_j v\_j$"; full — a Math \[text\] that says "$x = overline(x) + 2.2 v\_1 + 1.2 v\_2 + 0.6 v\_3$"; one — a Math \[text\] that says "$hat(x)\_1 = overline(x) + 2.2 v\_1$"; two — a Math \[text\] that says "$hat(x)\_2 = overline(x) + 2.2 v\_1 + 1.2 v\_2$"; errors — a Math \[text\] that says "$norm(x-hat(x)\_1) = 1.34, quad norm(x-hat(x)\_2) = 0.60$"; average\_error — a Math \[text\] that says "$frac(1, n) sum\_i norm(x\_i-hat(x)\_(i,k))^2 = sum\_(j\>k) lambda\_j$"; recon\_axes — an Axes3D (x\_range=(0.0, 2.8), y\_range=(0.0, 1.8), z\_range=(0.0, 1.2)); heading\_reconstruction — a Heading that says "What Keeping Components Actually Does"; mean — a Point \[gray\] labelled "overline(x)" drawn in recon\_axes (location=(0.0, 0.0, 0.0)); original — a Point \[red\] labelled "x" drawn in recon\_axes (location=(2.2, 1.2, 0.6)); first\_contribution — a Vector \[green\] labelled "2.2 v\_1" drawn in recon\_axes (start=(0.0, 0.0, 0.0), end=(2.2, 0.0, 0.0)); first\_reconstruction — a Point \[green\] labelled "hat(x)\_1" drawn in recon\_axes (location=(2.2, 0.0, 0.0)); second\_contribution — a Vector \[yellow\] labelled "1.2 v\_2" drawn in recon\_axes (start=(2.2, 0.0, 0.0), end=(2.2, 1.2, 0.0)); second\_reconstruction — a Point \[yellow\] labelled "hat(x)\_2" drawn in recon\_axes (location=(2.2, 1.2, 0.0)); third\_contribution — a Vector \[red\] labelled "0.6 v\_3" drawn in recon\_axes (start=(2.2, 1.2, 0.0), end=(2.2, 1.2, 0.6))

Actions:
- [14:38.62](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=878.6205): average\_error is hidden from the screen — left the board.
- [14:38.62](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=878.6205): errors is hidden from the screen — left the board.
- [14:38.62](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=878.6205): full is hidden from the screen — left the board.
- [14:38.62](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=878.6205): general is hidden from the screen — left the board.
- [14:38.62](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=878.6205): heading\_reconstruction is hidden from the screen — left the board.
- [14:38.62](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=878.6205): one is hidden from the screen — left the board.
- [14:38.62](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=878.6205): recon\_axes is hidden from the screen — left the board.
- [14:38.62](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=878.6205): mean is hidden from the screen — recon\_axes left the board.
- [14:38.62](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=878.6205): original is hidden from the screen — recon\_axes left the board.
- [14:38.62](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=878.6205): first\_contribution is hidden from the screen — recon\_axes left the board.
- [14:38.62](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=878.6205): first\_reconstruction is hidden from the screen — recon\_axes left the board.
- [14:38.62](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=878.6205): second\_contribution is hidden from the screen — recon\_axes left the board.
- [14:38.62](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=878.6205): second\_reconstruction is hidden from the screen — recon\_axes left the board.
- [14:38.62](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=878.6205): third\_contribution is hidden from the screen — recon\_axes left the board.
- [14:38.62](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=878.6205): two is hidden from the screen — left the board.

### Scene 5: [Units Can Choose the Components](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=879.6621666666666)

Span: 14:39.662–17:50.983 (879.6621666666666s–1070.9832291666667s).

#### Objects

- choices: a Block \[text\] that says "Use raw covariance when original scale is meaningful. Standardize when unlike units should begin with equal variance. Validate the scaling choice against the actual downstream task."
- correlation: a Math \[text\] that says "$Sigma\_z = mat(1, 0.20; 0.20, 1)$"
- heading\_choice: a Heading that says "Units Can Change the Answer"
- heading\_compare: a Heading that says "PCA Measures Numerical Variance"
- heading\_scaling: a Heading that says "Scaling Is a Modeling Decision"
- raw\_axes: an Axes (x\_range=(-50000.0, 50000.0), y\_range=(-30.0, 30.0), x\_ticks\_every=25000.0)
- raw\_cloud: a Point \[blue\] drawn in raw\_axes (location=(-40000.0, -15.0), marker\_radius=0.07)
- raw\_cloud\_2: a Point \[blue\] drawn in raw\_axes (location=(-30000.0, 5.0), marker\_radius=0.07)
- raw\_cloud\_3: a Point \[blue\] drawn in raw\_axes (location=(-15000.0, -8.0), marker\_radius=0.07)
- raw\_cloud\_4: a Point \[blue\] drawn in raw\_axes (location=(-5000.0, 12.0), marker\_radius=0.07)
- raw\_cloud\_5: a Point \[blue\] drawn in raw\_axes (location=(8000.0, -12.0), marker\_radius=0.07)
- raw\_cloud\_6: a Point \[blue\] drawn in raw\_axes (location=(16000.0, 15.0), marker\_radius=0.07)
- raw\_cloud\_7: a Point \[blue\] drawn in raw\_axes (location=(30000.0, -3.0), marker\_radius=0.07)
- raw\_cloud\_8: a Point \[blue\] drawn in raw\_axes (location=(40000.0, 10.0), marker\_radius=0.07)
- raw\_covariance: a Math \[text\] that says "$Sigma\_(upright("raw")) = mat(400000000, 40000; 40000, 100)$"
- raw\_label: a Tex \[text\] that says "Centered, original units"
- raw\_pc: a Line \[red\] labelled "upright("PC")\_1" drawn in raw\_axes (start=(-47000.0, -1.0), end=(47000.0, 1.0))
- standard\_axes: an Axes (x\_range=(-2.6, 2.6), y\_range=(-2.6, 2.6), aspect=(1.0, 1.0))
- standard\_cloud: a Point \[blue\] drawn in standard\_axes (location=(-2.0, -1.5), marker\_radius=0.07)
- standard\_cloud\_2: a Point \[blue\] drawn in standard\_axes (location=(-1.5, 0.5), marker\_radius=0.07)
- standard\_cloud\_3: a Point \[blue\] drawn in standard\_axes (location=(-0.75, -0.8), marker\_radius=0.07)
- standard\_cloud\_4: a Point \[blue\] drawn in standard\_axes (location=(-0.25, 1.2), marker\_radius=0.07)
- standard\_cloud\_5: a Point \[blue\] drawn in standard\_axes (location=(0.4, -1.2), marker\_radius=0.07)
- standard\_cloud\_6: a Point \[blue\] drawn in standard\_axes (location=(0.8, 1.5), marker\_radius=0.07)
- standard\_cloud\_7: a Point \[blue\] drawn in standard\_axes (location=(1.5, -0.3), marker\_radius=0.07)
- standard\_cloud\_8: a Point \[blue\] drawn in standard\_axes (location=(2.0, 1.0), marker\_radius=0.07)
- standard\_label: a Tex \[text\] that says "Centered and standardized"
- standard\_pc: a Line \[green\] labelled "upright("PC")\_1" drawn in standard\_axes (start=(-2.3, -1.15), end=(2.3, 1.15))
- standardize: a Math \[text\] that says "$z\_j = frac(x\_j - overline(x)\_j, s\_j)$"
- unit\_change: a Math \[text\] that says "$upright("dollars to cents") arrow.r upright("variance") times 10000$"
- warning: a Panel that says "PCA maximizes numerical variance. It does not know whether a large number is important, noisy, or merely written in a larger unit."

#### Beats

##### [14:39.662](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=879.6621666666666)

Narration: One warning belongs in every practical explanation of PCA. PCA does not understand variables. It understands numbers, and it rewards whichever direction has the greatest numerical variance.

Board: Empty.

Actions:
- [14:39.662](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=879.6621666666666): heading\_compare is shown on the screen, written out.
- [14:45.653](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=885.6531666666666): raw\_label is shown on the screen, written out.
- [14:45.653](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=885.6531666666666): standard\_label is shown on the screen, written out.
- [14:47.801](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=887.8011666666666): raw\_axes is shown on the screen, written out.
- [14:47.801](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=887.8011666666666): standard\_axes is shown on the screen, written out.

##### [14:53.115](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=893.1146666666666)

Narration: On the left, salary deviation is measured in dollars and age deviation in years. These are centered observations, so subtracting the means has already been done.

Board: raw\_label — a Tex \[text\] that says "Centered, original units"; raw\_axes — an Axes (x\_range=(-50000.0, 50000.0), y\_range=(-30.0, 30.0), x\_ticks\_every=25000.0); standard\_label — a Tex \[text\] that says "Centered and standardized"; standard\_axes — an Axes (x\_range=(-2.6, 2.6), y\_range=(-2.6, 2.6), aspect=(1.0, 1.0)); heading\_compare — a Heading that says "PCA Measures Numerical Variance"

Actions:
- [14:59.744](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=899.7441666666666): raw\_cloud is shown on the screen, written out.
- [14:59.744](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=899.7441666666666): raw\_cloud\_2 is shown on the screen, written out.
- [14:59.744](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=899.7441666666666): raw\_cloud\_3 is shown on the screen, written out.
- [14:59.744](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=899.7441666666666): raw\_cloud\_4 is shown on the screen, written out.
- [14:59.744](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=899.7441666666666): raw\_cloud\_5 is shown on the screen, written out.
- [14:59.744](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=899.7441666666666): raw\_cloud\_6 is shown on the screen, written out.
- [14:59.744](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=899.7441666666666): raw\_cloud\_7 is shown on the screen, written out.
- [14:59.744](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=899.7441666666666): raw\_cloud\_8 is shown on the screen, written out.

##### [15:5.058](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=905.0576666666666)

Narration: The salary coordinates contain tens of thousands while the age coordinates contain tens. The raw cloud therefore looks overwhelmingly horizontal, and raw PCA chooses an almost horizontal first component.

Board: raw\_label — a Tex \[text\] that says "Centered, original units"; raw\_axes — an Axes (x\_range=(-50000.0, 50000.0), y\_range=(-30.0, 30.0), x\_ticks\_every=25000.0); standard\_label — a Tex \[text\] that says "Centered and standardized"; standard\_axes — an Axes (x\_range=(-2.6, 2.6), y\_range=(-2.6, 2.6), aspect=(1.0, 1.0)); heading\_compare — a Heading that says "PCA Measures Numerical Variance"; raw\_cloud — a Point \[blue\] drawn in raw\_axes (location=(-40000.0, -15.0), marker\_radius=0.07); raw\_cloud\_2 — a Point \[blue\] drawn in raw\_axes (location=(-30000.0, 5.0), marker\_radius=0.07); raw\_cloud\_3 — a Point \[blue\] drawn in raw\_axes (location=(-15000.0, -8.0), marker\_radius=0.07); raw\_cloud\_4 — a Point \[blue\] drawn in raw\_axes (location=(-5000.0, 12.0), marker\_radius=0.07); raw\_cloud\_5 — a Point \[blue\] drawn in raw\_axes (location=(8000.0, -12.0), marker\_radius=0.07); raw\_cloud\_6 — a Point \[blue\] drawn in raw\_axes (location=(16000.0, 15.0), marker\_radius=0.07); raw\_cloud\_7 — a Point \[blue\] drawn in raw\_axes (location=(30000.0, -3.0), marker\_radius=0.07); raw\_cloud\_8 — a Point \[blue\] drawn in raw\_axes (location=(40000.0, 10.0), marker\_radius=0.07)

Actions:
- [15:13.696](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=913.6961666666666): raw\_pc is shown on the screen, written out.
- [15:17.539](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=917.5391666666666): raw\_pc is indicated — a transient flash.

##### [15:19.405](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=919.4046666666666)

Narration: That result does not prove salary is the most important scientific variable. It proves salary has the largest variance in the numbers handed to the algorithm.

Board: raw\_label — a Tex \[text\] that says "Centered, original units"; raw\_axes — an Axes (x\_range=(-50000.0, 50000.0), y\_range=(-30.0, 30.0), x\_ticks\_every=25000.0); standard\_label — a Tex \[text\] that says "Centered and standardized"; standard\_axes — an Axes (x\_range=(-2.6, 2.6), y\_range=(-2.6, 2.6), aspect=(1.0, 1.0)); heading\_compare — a Heading that says "PCA Measures Numerical Variance"; raw\_cloud — a Point \[blue\] drawn in raw\_axes (location=(-40000.0, -15.0), marker\_radius=0.07); raw\_cloud\_2 — a Point \[blue\] drawn in raw\_axes (location=(-30000.0, 5.0), marker\_radius=0.07); raw\_cloud\_3 — a Point \[blue\] drawn in raw\_axes (location=(-15000.0, -8.0), marker\_radius=0.07); raw\_cloud\_4 — a Point \[blue\] drawn in raw\_axes (location=(-5000.0, 12.0), marker\_radius=0.07); raw\_cloud\_5 — a Point \[blue\] drawn in raw\_axes (location=(8000.0, -12.0), marker\_radius=0.07); raw\_cloud\_6 — a Point \[blue\] drawn in raw\_axes (location=(16000.0, 15.0), marker\_radius=0.07); raw\_cloud\_7 — a Point \[blue\] drawn in raw\_axes (location=(30000.0, -3.0), marker\_radius=0.07); raw\_cloud\_8 — a Point \[blue\] drawn in raw\_axes (location=(40000.0, 10.0), marker\_radius=0.07); raw\_pc — a Line \[red\] labelled "upright("PC")\_1" drawn in raw\_axes (start=(-47000.0, -1.0), end=(47000.0, 1.0))

Actions:
- [15:26.184](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=926.1841666666667): raw\_pc is indicated — a transient flash.

##### [15:29.966](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=929.9661666666666)

Narration: On the right, each variable has been centered and divided by its sample standard deviation. Both begin with variance one, so their relationship, rather than their original unit sizes, determines the component.

Board: Unchanged from the preceding beat in this scene.

Actions:
- [15:32.892](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=932.8921666666666): standard\_cloud is shown on the screen, written out.
- [15:32.892](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=932.8921666666666): standard\_cloud\_2 is shown on the screen, written out.
- [15:32.892](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=932.8921666666666): standard\_cloud\_3 is shown on the screen, written out.
- [15:32.892](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=932.8921666666666): standard\_cloud\_4 is shown on the screen, written out.
- [15:32.892](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=932.8921666666666): standard\_cloud\_5 is shown on the screen, written out.
- [15:32.892](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=932.8921666666666): standard\_cloud\_6 is shown on the screen, written out.
- [15:32.892](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=932.8921666666666): standard\_cloud\_7 is shown on the screen, written out.
- [15:32.892](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=932.8921666666666): standard\_cloud\_8 is shown on the screen, written out.
- [15:43.306](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=943.3061666666666): standard\_pc is shown on the screen, written out.

##### [15:45.102](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=945.1021666666666)

Narration: The two answers differ even though they describe the same observations. Scaling has changed the geometry, and changing the geometry changes PCA.

Board: raw\_label — a Tex \[text\] that says "Centered, original units"; raw\_axes — an Axes (x\_range=(-50000.0, 50000.0), y\_range=(-30.0, 30.0), x\_ticks\_every=25000.0); standard\_label — a Tex \[text\] that says "Centered and standardized"; standard\_axes — an Axes (x\_range=(-2.6, 2.6), y\_range=(-2.6, 2.6), aspect=(1.0, 1.0)); heading\_compare — a Heading that says "PCA Measures Numerical Variance"; raw\_cloud — a Point \[blue\] drawn in raw\_axes (location=(-40000.0, -15.0), marker\_radius=0.07); raw\_cloud\_2 — a Point \[blue\] drawn in raw\_axes (location=(-30000.0, 5.0), marker\_radius=0.07); raw\_cloud\_3 — a Point \[blue\] drawn in raw\_axes (location=(-15000.0, -8.0), marker\_radius=0.07); raw\_cloud\_4 — a Point \[blue\] drawn in raw\_axes (location=(-5000.0, 12.0), marker\_radius=0.07); raw\_cloud\_5 — a Point \[blue\] drawn in raw\_axes (location=(8000.0, -12.0), marker\_radius=0.07); raw\_cloud\_6 — a Point \[blue\] drawn in raw\_axes (location=(16000.0, 15.0), marker\_radius=0.07); raw\_cloud\_7 — a Point \[blue\] drawn in raw\_axes (location=(30000.0, -3.0), marker\_radius=0.07); raw\_cloud\_8 — a Point \[blue\] drawn in raw\_axes (location=(40000.0, 10.0), marker\_radius=0.07); raw\_pc — a Line \[red\] labelled "upright("PC")\_1" drawn in raw\_axes (start=(-47000.0, -1.0), end=(47000.0, 1.0)); standard\_cloud — a Point \[blue\] drawn in standard\_axes (location=(-2.0, -1.5), marker\_radius=0.07); standard\_cloud\_2 — a Point \[blue\] drawn in standard\_axes (location=(-1.5, 0.5), marker\_radius=0.07); standard\_cloud\_3 — a Point \[blue\] drawn in standard\_axes (location=(-0.75, -0.8), marker\_radius=0.07); standard\_cloud\_4 — a Point \[blue\] drawn in standard\_axes (location=(-0.25, 1.2), marker\_radius=0.07); standard\_cloud\_5 — a Point \[blue\] drawn in standard\_axes (location=(0.4, -1.2), marker\_radius=0.07); standard\_cloud\_6 — a Point \[blue\] drawn in standard\_axes (location=(0.8, 1.5), marker\_radius=0.07); standard\_cloud\_7 — a Point \[blue\] drawn in standard\_axes (location=(1.5, -0.3), marker\_radius=0.07); standard\_cloud\_8 — a Point \[blue\] drawn in standard\_axes (location=(2.0, 1.0), marker\_radius=0.07); standard\_pc — a Line \[green\] labelled "upright("PC")\_1" drawn in standard\_axes (start=(-2.3, -1.15), end=(2.3, 1.15))

Actions:
- [15:46.507](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=946.5071666666666): raw\_pc is indicated — a transient flash.
- [15:46.707](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=946.7071666666666): standard\_pc is indicated — a transient flash.
- [15:55.76](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=955.7601666666666): heading\_compare is hidden from the screen — left the board.
- [15:55.76](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=955.7601666666666): raw\_axes is hidden from the screen — left the board.
- [15:55.76](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=955.7601666666666): raw\_cloud is hidden from the screen — raw\_axes left the board.
- [15:55.76](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=955.7601666666666): raw\_cloud\_2 is hidden from the screen — raw\_axes left the board.
- [15:55.76](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=955.7601666666666): raw\_cloud\_3 is hidden from the screen — raw\_axes left the board.
- [15:55.76](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=955.7601666666666): raw\_cloud\_4 is hidden from the screen — raw\_axes left the board.
- [15:55.76](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=955.7601666666666): raw\_cloud\_5 is hidden from the screen — raw\_axes left the board.
- [15:55.76](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=955.7601666666666): raw\_cloud\_6 is hidden from the screen — raw\_axes left the board.
- [15:55.76](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=955.7601666666666): raw\_cloud\_7 is hidden from the screen — raw\_axes left the board.
- [15:55.76](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=955.7601666666666): raw\_cloud\_8 is hidden from the screen — raw\_axes left the board.
- [15:55.76](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=955.7601666666666): raw\_pc is hidden from the screen — raw\_axes left the board.
- [15:55.76](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=955.7601666666666): raw\_label is hidden from the screen — left the board.
- [15:55.76](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=955.7601666666666): standard\_axes is hidden from the screen — left the board.
- [15:55.76](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=955.7601666666666): standard\_cloud is hidden from the screen — standard\_axes left the board.
- [15:55.76](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=955.7601666666666): standard\_cloud\_2 is hidden from the screen — standard\_axes left the board.
- [15:55.76](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=955.7601666666666): standard\_cloud\_3 is hidden from the screen — standard\_axes left the board.
- [15:55.76](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=955.7601666666666): standard\_cloud\_4 is hidden from the screen — standard\_axes left the board.
- [15:55.76](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=955.7601666666666): standard\_cloud\_5 is hidden from the screen — standard\_axes left the board.
- [15:55.76](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=955.7601666666666): standard\_cloud\_6 is hidden from the screen — standard\_axes left the board.
- [15:55.76](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=955.7601666666666): standard\_cloud\_7 is hidden from the screen — standard\_axes left the board.
- [15:55.76](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=955.7601666666666): standard\_cloud\_8 is hidden from the screen — standard\_axes left the board.
- [15:55.76](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=955.7601666666666): standard\_pc is hidden from the screen — standard\_axes left the board.
- [15:55.76](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=955.7601666666666): standard\_label is hidden from the screen — left the board.

##### [15:56.96](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=956.9601666666666)

Narration: Here is the raw covariance matrix. Salary variance is four hundred million, while age variance is one hundred. The larger numerical scale dominates the optimization.

Board: Empty.

Actions:
- [15:56.96](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=956.9601666666666): heading\_choice is shown on the screen, written out.
- [15:56.96](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=956.9601666666666): raw\_covariance is shown on the screen, written out.
- [16:2.649](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=962.6491666666666): raw\_covariance (the "400000000" part) is emphasized.
- [16:5.018](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=965.0181666666666): raw\_covariance (the "400000000" part) is no longer emphasized.

##### [16:10.494](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=970.4941666666666)

Narration: Now perform a scientifically meaningless change: write the same salaries in cents instead of dollars. Every salary value is multiplied by one hundred, and its variance is multiplied by ten thousand. Raw PCA can rotate even though nothing about the people changed.

Board: raw\_covariance — a Math \[text\] that says "$Sigma\_(upright("raw")) = mat(400000000, 40000; 40000, 100)$"; heading\_choice — a Heading that says "Units Can Change the Answer"

Actions:
- [16:16.252](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=976.2521666666667): unit\_change is shown on the screen, written out.
- [16:23.474](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=983.4741666666666): unit\_change (the "10000" part) is emphasized.
- [16:29.442](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=989.4421666666666): unit\_change (the "10000" part) is no longer emphasized.

##### [16:30.042](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=990.0421666666666)

Narration: That is the blunt warning. PCA maximizes numerical variance. It cannot distinguish signal from noise, importance from nuisance, or measurement scale from genuine structure.

Board: raw\_covariance — a Math \[text\] that says "$Sigma\_(upright("raw")) = mat(400000000, 40000; 40000, 100)$"; unit\_change — a Math \[text\] that says "$upright("dollars to cents") arrow.r upright("variance") times 10000$"; heading\_choice — a Heading that says "Units Can Change the Answer"

Actions:
- [16:30.042](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=990.0421666666666): warning is shown on the screen, written out.
- [16:34.093](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=994.0931666666667): warning (the "numerical variance" part) is emphasized.
- [16:42.755](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=1002.7546666666666): heading\_choice is hidden from the screen — left the board.
- [16:42.755](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=1002.7546666666666): raw\_covariance is hidden from the screen — left the board.
- [16:42.755](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=1002.7546666666666): unit\_change is hidden from the screen — left the board.
- [16:42.755](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=1002.7546666666666): warning is hidden from the screen — left the board.
- [16:42.755](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=1002.7546666666666): warning (the "numerical variance" part) is no longer emphasized.

##### [16:43.955](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=1003.9546666666666)

Narration: Standardization replaces each variable by its centered value divided by its sample standard deviation. The covariance matrix of standardized variables is their correlation matrix.

Board: Empty.

Actions:
- [16:43.955](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=1003.9546666666666): heading\_scaling is shown on the screen, written out.
- [16:44.303](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=1004.3031666666666): standardize is shown on the screen, written out.
- [16:54.496](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=1014.4961666666666): correlation is shown on the screen, written out.

##### [16:56.814](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=1016.8141666666666)

Narration: Do not turn that into another automatic ritual. If a variable's original scale carries real meaning, raw covariance may be appropriate. If unlike units should receive equal starting weight, standardization is often necessary.

Board: standardize — a Math \[text\] that says "$z\_j = frac(x\_j - overline(x)\_j, s\_j)$"; correlation — a Math \[text\] that says "$Sigma\_z = mat(1, 0.20; 0.20, 1)$"; heading\_scaling — a Heading that says "Scaling Is a Modeling Decision"

Actions:
- [17:1.47](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=1021.4701666666666): choices is shown on the screen, written out.
- [17:3.908](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=1023.9081666666666): choices (the "Use raw covariance when original scale is meaningful." part) is emphasized.
- [17:9.852](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=1029.8521666666666): choices (the "Standardize when unlike units should begin with equal variance." part) is emphasized.
- [17:9.852](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=1029.8521666666666): choices (the "Use raw covariance when original scale is meaningful." part) is no longer emphasized.
- [17:12.36](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=1032.3596666666667): choices (the "Standardize when unlike units should begin with equal variance." part) is no longer emphasized.

##### [17:12.96](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=1032.9596666666666)

Narration: Heavy tails, outliers, counts, and strongly skewed variables may call for robust scaling or a domain-specific transformation. Whatever you choose, document it and validate it as part of the model.

Board: standardize — a Math \[text\] that says "$z\_j = frac(x\_j - overline(x)\_j, s\_j)$"; correlation — a Math \[text\] that says "$Sigma\_z = mat(1, 0.20; 0.20, 1)$"; choices — a Block \[text\] that says "Use raw covariance when original scale is meaningful. Standardize when unlike units should begin with equal variance. Validate the scaling choice against the actual downstream task."; heading\_scaling — a Heading that says "Scaling Is a Modeling Decision"

Actions:
- [17:24.837](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=1044.8371666666667): choices (the "Validate the scaling choice against the actual downstream task." part) is emphasized.
- [17:26.985](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=1046.9846666666667): choices (the "Validate the scaling choice against the actual downstream task." part) is no longer emphasized.

##### [17:27.585](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=1047.5846666666666)

Narration: PCA is now one connected idea. Center the observations, find the direction of maximum projected variance, continue in perpendicular directions, and compute those directions as covariance eigenvectors. Use eigenvalues to judge compression, reconstruct from retained scores, and never forget that the units help define the answer.

Board: Unchanged from the preceding beat in this scene.

Actions:
- [17:47.867](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=1067.8671666666667): choices (the "Validate the scaling choice against the actual downstream task." part) is indicated — a transient flash.
- [17:49.942](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=1069.9415625): choices is hidden from the screen — left the board.
- [17:49.942](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=1069.9415625): correlation is hidden from the screen — left the board.
- [17:49.942](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=1069.9415625): heading\_scaling is hidden from the screen — left the board.
- [17:49.942](https://academa.ai/lectures/principal-component-analysis-seen-geometrically?t=1069.9415625): standardize is hidden from the screen — left the board.