Eigenvectors and Eigenvalues: The Directions a Matrix Can't Turn

Multiply a vector by a matrix and, in general, two things happen to it. Its length changes, and its direction changes. Today I want to ask about the second of those, because for almost every matrix there are a few directions that simply refuse to be turned. So here is a piece of the plane, ruled into squares. And here is the matrix we will apply to it: two, one, one, two. And let me say what applying it means. It means multiplication. Take a point on the grid, write it as a column, put A in front, and out comes a different point: two x plus y, and x plus two y. A matrix is a map that carries one point to another. Take one point and try it. This one, at one, zero. Multiply it by A: two times one plus zero is two, and one plus two times zero is one. So A sends that point over to two, one. Now watch every point do that same sum at once. The whole thing stretches out along one diagonal, and our point lands exactly where the arithmetic said it would. Let me put it back, and follow one single arrow through the same motion. This one points straight up, at zero, one and a half. Put it through the same sum and the answer is one and a half, three. Now apply the matrix. The arrow gets longer, certainly. But look where it ends up. It has swung over to the right. The matrix turned it. Not every arrow does that. Let me put the plane back and try two others, one along this diagonal and one along the other. And this time let us do the multiplication first, before anything moves. Read those two answers. The first one is three times the vector we put in. The second one is the vector we put in, unchanged, which is one times it. Now apply the matrix and watch the arrows do exactly that. The red one triples and stays on the line it was already on. The green one does not move at all. Those two directions are the eigenvectors of this matrix. Everything else in the plane swings round toward the stretching direction. These two just sit there and get scaled, by three and by one, and those two numbers are their eigenvalues. So let us write down exactly what that means.

Look again at what just happened to that red arrow. Here is the vector one, one. We put it through the matrix, and out came another vector: three, three. And here is the observation the whole subject is built on. That answer is nothing but three times the vector we started with. One vector, one plain number, no matrix. Which tells us something about the picture. Both arrows lie on this one line through the origin. So on this line, applying A is nothing but multiplying by three. The arrow got longer, but it was not turned by so much as a degree. Now try an arrow that is not on that line. Here is w, pointing straight up. A sends it to one, two. And one, two is not a multiple of zero, one. No number times zero is ever one. So w came back pointing somewhere else entirely, like almost every arrow in the plane. The arrows that survive, the ones A puts back on their own line, are the eigenvectors of A. And the number that does the stretching, three in this case, is its eigenvalue. Now say that for any matrix and any vector. Here is a general v, drawn on a general line. A vector v is an eigenvector of A when A times v comes out as a plain multiple of v itself. Written down, that is A v equals lambda v, and that is the whole definition. That lambda is the eigenvalue. And there is one piece of fine print: v is not allowed to be the zero vector. A times zero is zero for every matrix there is, so the zero vector would qualify for everything. Read the equation geometrically now. It says that when A gets hold of v, it puts it back down somewhere on that same line. If lambda is bigger than one, A stretches the arrow out along it. If lambda is between zero and one, it pulls the arrow in toward the origin. Shorter, yes. Turned, no. And now let lambda keep going, down past zero. Watch the arrow shrink as it goes, die away to nothing at the origin, and come back out the other side, pointing backwards. That still counts. Backwards along a line is the same line. Notice what the equation does not say: it does not say A leaves v alone, it says A leaves the line alone. So an eigenvector is a direction the matrix cannot turn, and the eigenvalue says what happens to lengths along it. Which leaves the real question: given a matrix, how do you find them?

Right. We have a matrix, and we want the directions it cannot turn. So start from the definition and push the algebra around. A v equals lambda v. Move everything to one side. A v minus lambda v equals the zero vector. I would love to factor the v out, but I cannot subtract a number from a matrix, so I slip in the identity: lambda v is the same thing as lambda times I times v. And now it factors. The matrix A minus lambda I, acting on v, gives the zero vector. Look at what that line actually says. Something times something comes out zero. And one of the two somethings we already know about: v is not the zero vector. That was the fine print in the definition. So the zero on the right cannot have come from the vector. It has to have come from the matrix over here. A minus lambda I has to take a nonzero vector and crush it flat to nothing. And there is exactly one number that measures whether a matrix does that. If you want to say how much of a matrix there is, in a single number, that number is its determinant. A matrix that flattens a nonzero vector onto the origin has determinant zero. Formally, then: lambda is an eigenvalue of A exactly when the determinant of A minus lambda I comes out zero. That is one equation in one unknown, and we can actually solve it. Let us do it with our matrix. Subtract lambda from each entry on the diagonal, take the determinant of what is left, and set that equal to zero. The determinant of a two by two is read straight off the two diagonals. Multiply down this one. Then multiply down the other. And take the second product away from the first. So that is two minus lambda, all squared, take away one times one. Multiply the square out and keep every term. Lambda squared, minus four lambda, plus four, minus one, equals zero. And four minus one is three, so that line tidies itself up where it stands. Lambda squared minus four lambda plus three. This has a name: it is the characteristic polynomial of A, and its degree is always the size of the matrix. This one factors on sight: lambda minus three, times lambda minus one. So the eigenvalues are three and one. And there is our confirmation, because those are exactly the two numbers we watched on the grid. One arrow tripled. The other one stayed put.

We have the eigenvalues. Now we want the directions that go with them. Go back to the factored equation, A minus lambda I times v equals zero, and put the first eigenvalue into it. With lambda equal to three, we subtract three from each entry on the diagonal. That gives us minus one, one, one, minus one, multiplying the column x, y, and the answer has to be the zero vector. Both rows say the same thing. Minus x plus y equals zero, or in other words y equals x. So any vector whose two entries are equal will do, and the tidiest one to write down is one, one. There is the line of them, and there is the arrow. And notice that an eigenvector is never alone. Every multiple of one, one is an eigenvector too, with the very same eigenvalue three. What we have really found is a whole line the matrix keeps to itself. Now the second eigenvalue. Lambda equals one, so we subtract one from the diagonal instead, and this time the matrix comes out as four ones. Every row now says x plus y equals zero, so y is minus x, and the tidiest eigenvector to pick is one, minus one. There is the second line, and there is the second arrow. Now hold on to both of those arrows, and let the matrix loose on the whole picture again. Every grid line has been dragged off somewhere new. The red arrow ran out along its own line and stopped three times further from the origin, which is its eigenvalue. The green one did not move at all, because its eigenvalue is one. Those two directions were untouchable.

One more thing, and then a warning. Here is a matrix that rotates the plane through a quarter turn: zero, minus one, one, zero. And here are three arrows sitting in it. Every arrow in the plane gets swung round by ninety degrees. Watch these three go. Not one of them finishes on the line it started on. A rotation has no favourite direction. And the algebra says the same thing. Subtract lambda from the diagonal, take the determinant, and cross the two diagonals exactly as before. That comes out as lambda squared plus one, and over the real numbers that has no roots at all. This matrix has no eigenvectors. It is not a defect in the matrix, it is honest geometry. So keep that in your pocket. Eigenvectors are common, but they are not guaranteed. Now let me lift exactly this rotation up into three dimensions, where something rather lovely happens. Here is a rotation of space about the vertical axis, and here are two arrows lying flat in the horizontal plane. Watch them turn. Every one of them changes direction, exactly the way they did down in the plane. But now there is somewhere to stand that the rotation cannot reach. An arrow pointing straight up along the axis. The rotation carries the whole of space around it and leaves that one arrow precisely where it was. So that arrow is an eigenvector, and its eigenvalue is one. And this is a general fact about three dimensional space. Every rotation has an axis, and the axis is the eigenvector with eigenvalue one. Euler proved that, and the proof is essentially this picture. So there is the whole idea, in one line. A times v equals lambda times v. The eigenvectors are the directions the matrix cannot turn. The eigenvalue attached to each one says what happens to lengths along that direction. Find those directions, and you have found the frame in which your matrix is nothing but a list of stretches. Which is why, from vibration to stability to the ranking of the whole internet, this tends to be the first question anybody asks about a matrix.