Building the Taylor Series: Convergence and Failure

Every function a machine can actually compute is built out of adding and multiplying. That is what a polynomial is. Sine is not one, so when your calculator reports a sine it is not doing trigonometry at all. It is evaluating a polynomial, and this lecture is about which one, and how far you can trust it. So here is the question. Sine is not a polynomial. Can we build one that behaves like it near zero, and how close can we actually get? I will tell you the destination first, because the destination is not the hard part. This is it. Sine of x equals x, minus x cubed over three factorial, plus x to the fifth over five factorial, and on forever. Every hard question in this lecture is hiding in that equals sign. So where do those numbers come from? Nowhere clever. Suppose I want a polynomial that behaves like f near zero, and the only tool I have is that I can force the two of them to agree at a single point: same value, same slope, same bend. So write the polynomial down with its coefficients still unknown. a nought, plus a one x, plus a two x squared, and so on. Now set x to zero. Every term carrying an x dies, and all that is left is a nought. Now differentiate. The constant vanishes, a one is standing there on its own, and everything after it still carries an x. Set x to zero again, and out falls a one. Differentiate once more. The square brings a two down in front, so a two ends up doubled, and setting x to zero leaves a two as the second derivative at zero, over two. And you can see the pattern. Each derivative strips one more coefficient bare, and the number it picks up in front is a factorial. So the k-th coefficient is the k-th derivative of f at zero, divided by k factorial. Stack them up and that is the Taylor polynomial of f about zero. Every coefficient is one derivative at one point. Nothing else about the function is used anywhere. Now aim it at sine. The derivatives of sine go round in a circle. Differentiate and you get cosine, then minus sine, then minus cosine, and then you are back where you started. Evaluate all of them at zero. Sine of zero is nothing. Cosine of zero is one. Minus sine of zero is nothing again, and minus cosine of zero is minus one. Look what that does. Every even coefficient has a zero on top, so every even power of x is wiped out. What survives is the odd powers, one, three, five, seven, with the signs flipping each time. So here is the machine running, against the curve itself. The crudest match uses one term, a nought, which is sine of zero, which is nothing. That polynomial is the constant zero, and on this picture it is the horizontal axis. It agrees with sine at exactly one point and nowhere else. Take one more term. The next coefficient is cosine of zero, which is one, so the first polynomial is simply x. That is the tangent line at the origin, and close in it is genuinely good. But watch it leave. By x equals two the line is still climbing and sine has already turned over. So add the next term, minus x cubed over six, and look what it does. It bends the line down, exactly where the line was going wrong. That is the whole idea in one picture. Every new term is a correction. Near zero it is negligible, because a high power of a small number is tiny. It only speaks up once you are far enough out for the previous curve to have failed. Add the fifth power term and the same thing happens one hump further out. Add the seventh and it happens again. Each polynomial tracks sine a little longer before it gives up and shoots off. Let me put a dot on sine and a dot on the seventh degree polynomial and walk them out. Near zero you cannot tell them apart. Past three they are still on top of each other. And then, quite suddenly, they are not. Which raises the only question worth asking about any of this. How wrong is the polynomial at a given x, and does taking more terms really drive that wrongness to zero? That gap has a name, and it is the real content of Taylor's theorem.

So let us look at that gap directly. Here is sine again, and here is the cubic we built, x minus x cubed over six. Same picture as before, except that now I want to measure the distance between the two curves. Call that distance the remainder. R three of x is sine of x minus P three of x, and it is nothing more than the vertical gap between the blue curve and the green one at whichever x you are standing at. Walk in toward zero and it closes so fast you cannot see it happen. At a half the two curves differ by about two ten-thousandths. On this picture that is thinner than the line itself. Now walk out. At two the gap is already obvious. At three it is enormous. Sine is up near a seventh and the cubic has dived past minus one and a half. Same polynomial, same degree. All that changed is how far we walked. Raising the degree buys you distance, not safety. Here is the seventh degree polynomial on the same picture. It holds on out past four, which is much better, and then it does exactly the same thing. Every fixed polynomial eventually leaves. So here is what Taylor's theorem actually says, and notice how little of it is the series. The function equals the polynomial, plus a remainder. That is an exact statement with no approximation anywhere in it. The whole question is what the remainder is. And Lagrange gives that remainder to us in closed form. It is the n plus first derivative of f, taken at some point c between zero and x, times x to the n plus one, over n plus one factorial. Hold it next to the terms we were adding. It is exactly the shape of the next term of the series, except for one thing. The derivative is taken at c, somewhere in the middle, rather than at zero. And the theorem never tells you where c is. Now watch what that buys us for sine. Every derivative of sine is plus or minus a sine or a cosine. So whatever c turns out to be, that derivative is at most one in size, and we never need to know c at all. So the remainder is trapped below x to the n plus one, over n plus one factorial. Pure arithmetic, with no trigonometry left in it. And now look at what that quantity does as n grows. Take x equal to ten, a long way from the centre. With five terms the bound is over eight hundred, which is useless. With ten it is worse, nearly three thousand. The powers of ten are still beating the factorial. But factorials win in the end. By fifteen the bound has turned over. By twenty it is forty. By thirty it is four thousandths, and by forty it is a hundred-millionth. And ten is not special. Fix any x you like, however enormous, and x to the m over m factorial goes to zero as m grows, because eventually the factorial is multiplying by numbers larger than x at every single step. Which is the sentence we were actually after. The remainder for sine dies, everywhere. So the series really does add up to sine, at every real number. Not nearly. Exactly. And it was the remainder that proved it, not the terms.

Sine was the happy case. Here is a function that is not. g of x is one over one plus x squared. Smooth everywhere, bounded, no corners, no blow-ups. There is nothing wrong with it anywhere on the real line. Its Taylor series about zero is easy to get, because you have already seen it. A geometric series sums to one over one minus r, whenever r is smaller than one in size. Now put r equal to minus x squared. Out comes one, minus x squared, plus x to the fourth, minus x to the sixth, and onward. Every even power, with the signs alternating. That is the Taylor series of g about zero. Now draw two of its partial sums. Here is one minus x squared. Here is the one that runs out to the sixth power. And here are the two walls, at minus one and at plus one. Inside those walls the partial sums close in on the curve, and taking more terms makes them better. Outside them, look what happens. The sixth power sum has already dived past minus one, while the function it came from sits calmly at four tenths. And adding more terms out there does not help. It makes things worse. Past x equals one the powers of x stop shrinking, so the series has nothing to converge to at all. It is not a bad approximation. It is not an approximation. That number one has a name, and it is the thing this lecture assumed you had not met. It is the radius of convergence. Inside it the series converges. Outside it the series diverges, and there is no argument about a remainder to be had. And here is the strange part. Nothing on the real line explains why the wall sits at one. The function is perfectly happy there. The explanation lives off the real line entirely, and it is a story for another course. So that is failure number one. The series simply stops converging past a certain distance. It is a nuisance, but it is honest, because it tells you it has given up. The second failure is far worse, because it tells you nothing at all.

Here is the function that ends this story. For x away from zero, f of x is e to the minus one over x squared, and at zero we define it to be zero. It is a perfectly ordinary looking curve. It rises from nothing on the left, flattens right out at the origin, and rises again. And it is smooth. Infinitely differentiable, everywhere, including at zero. That takes a little work to prove, but it is true. So Taylor's recipe applies to it with no complaint whatsoever. But look how hard it presses itself against the axis near the origin. At one it is about zero point three seven. At a half it is down to eighteen thousandths. At a fifth it is ten to the minus eleven, and at a tenth, ten to the minus forty four. That is not merely small. That is flatter than any polynomial can manage. x to the tenth, at a tenth, is ten to the minus ten, and f is already thirty orders of magnitude below that. The exponential crushes every power. Now differentiate it. Away from zero the derivative is two over x cubed, times the same exponential. The two over x cubed blows up as x approaches the origin, but the exponential goes to zero faster than any power can blow up. So the limit is zero. Differentiate again and you get another rational function of one over x, times that same exponential, and the same thing happens. Every single derivative at zero comes out zero. Not small. Zero. So feed those into the recipe. Every coefficient is a derivative at zero over a factorial, and every one of those derivatives is zero. So every coefficient is zero, and the Taylor series of this function is the zero function. Draw it. There it is, in red, lying flat along the axis. And now ask the question we have been asking all along. How wrong is it? Put a dot on the function and a dot on its series, and walk them out. They come apart immediately. Out at two point four the gap is nearly the whole height of the picture. Walk back in and it shrinks, but it never actually closes until you are standing at the origin itself. And here is the thing that should bother you. Adding more terms does nothing at all, because every term is zero. The remainder is the whole function, at every degree, forever. So this series converges. It converges on the entire real line, beautifully, instantly, with infinite radius of convergence. And it converges to the wrong thing. It agrees with the function it came from at exactly one point, the point we built it at, and nowhere else. Which kills the comfortable picture of what a Taylor series is. It is not a decomposition of a function. It is a list of derivatives at one point, and derivatives at one point can be completely blind to what the function does anywhere else.

So put the two cases next to each other. For sine we could bound the remainder, and the bound went to zero for every x. That is precisely why we are allowed to write an equals sign between the function and its series. For the flat function, the series was the zero function and the remainder was the whole of f, at every degree. It never went anywhere. So there is no equals sign to be had, except at the single point x equals zero. And notice what did the deciding in both cases. Not the terms. Not how fast the coefficients shrink. Not even whether the series converges. It was the remainder that settled it, both times, and nothing else in the argument mattered at all. So, three things to carry away. First, Taylor's theorem is not an approximation. It is an equality. The function is the polynomial plus the remainder, exactly, for every x and every n. There is nothing to be wrong about until you throw the remainder away. Second, the series represents the function only where the remainder tends to zero. That is a limit statement, and it has to be proved. For sine we proved it, with the factorial. For the flat function it is plainly false. And third, the one that catches people. A series converging is not the same thing as a series converging to the right answer. Our last example converged everywhere on the line and was wrong everywhere except one point. So here is the statement, whole. Sine of x is the sum from k equals zero to infinity of minus one to the k, times x to the two k plus one, over two k plus one factorial. And that equals sign is not notation. It is a theorem, and the thing holding it up is a bound on the remainder that goes to zero for every x. The series is the part you can see. The remainder is the part that decides.