Theorem 4.1.1. Taylor’s Formula.
If \(f(x)=\sum_{n=0}^\infty a_n(x-a)^n\text{,}\) then \(a_n=\frac{f^{(n)}(a)}{n!}\text{,}\) where \(f^{(n)}(a)\) represents the \(n\)th derivative of \(f\) evaluated at \(a\text{.}\)
Chapter 4 Questions Concerning Power Series
Section 4.1 Taylor’s Formula
As we saw in Section 3.3, representing functions as power series was a fruitful strategy for mathematicans in the eighteenth century (as it still is). Differentiating and integrating power series term by term was relatively easy, seemed to work, and led to many applications. Furthermore, power series representations for all of the elementary functions could be obtained if one was clever enough. However, cleverness is an unreliable tool. It would be better to have some systematic way to find a power series for a given function that doesn’t rely on being sufficiently clever.
To be sure, there were nagging questions. For example. even if we can find a power series representation of some function, how do we know that the series we’ve created represents the function we started with? Even worse, is it possible for a function to have more than one power series representation centered at a given value \(a?\) This uniqueness issue is addressed by the following theorem.
A few comments about Theorem 4.1.1 are in order. Notice that we did not start with a function and derive its series representation. Instead we defined \(f(x)\) to be the series we wrote down. This assumes that the expression \(\sum_{n=0}^\infty a_n(x-a)^n\) actually has meaning (that it converges). At this point we have every reason to expect that it does, however expectation is not proof so we note that this is an assumption, not an established truth. We’ve also assumed that we can differentiate an infinite polynomial term-by-term as we would a finite polynomial. As before, we follow in the footsteps of our 18th century forebears in making these assumptions. For now.
Problem 4.1.2.
Prove Theorem 4.1.1.
From Theorem 4.1.1 we see that if we do start with the function \(f(x)\) then no matter how we obtain its power series, the result will always be the same. The series
\begin{align}
\sum_{n=0}^\infty\frac{f^{(n)}(a)}{n!}(x-a)^n=f(a)+\amp{}f^\prime(a)(x-a)+\frac{f^{\prime\prime}(a)}{2!}(x-a)^2\notag\\
\amp{}+\frac{f^{\prime\prime\prime}(a)}{3!}(x-a)^3+\cdots\tag{4.1.1}
\end{align}
is called the Taylor series for \(f\) expanded about (or centered at) \(a\). Although this systematic “machine” for obtaining power series for a function seems to have been known to a number of mathematicians in the early 1700s, Brook Taylor (1685–1731) was the first to publish this result in his Methodus Incrementorum (1715). The special case when \(a=0\) was included by Colin Maclaurin (1698–1746) in his Treatise of Fluxions (1742). Thus when \(a=0\text{,}\) the series \(\sum_{n=0}^\infty\frac{f^{(n)}(0)}{n!}x^n\) is often called the Maclaurin Series for \(f\text{.}\)
Problem 4.1.4.
Use Taylor’s formula to find the Taylor series of the given function expanded about the given point \(a\text{.}\)
(a)
(b)
(c)
(d)
(e)
The prime notation for the derivative was not used by Taylor, Maclaurin or their contemporaries. It was introduced by Joseph Louis Lagrange in his 1779 work Thèorie des Fonctions Analytiques. In that work, Lagrange sought to get rid of Leibniz’ infinitesimals and base Calculus on the power series idea. His idea was that by representing every function as a power series, Calculus could be done algebraically by manipulating power series and examining various aspects of the series representation instead of appealing to the controversial notion of infinitesimals. He implicitly assumed that every continuous function could be replaced with its power series representation.
That is, he wanted to think of the Taylor series as a “great big polynomial,” because polynomials are easy to work with. It was a very simple, yet exceedingly clever and far-reaching idea. Since \(e^x = 1 +x +x^2/2 +\ldots\text{,}\) for example, why not just define the exponential to be the series and work with the series. After all, the series is just a very long polynomial.
This idea did not come out of nowhere. Particular infinite series, such as the Geometric Series had been known and studied for many years. Later, in the 18th century Leonhard Euler used infinite series to solve many problems, and some of his solutions are still quite breath–taking when you first see them [16].
Taking his cue from the Taylor series
\begin{equation}
f(x) = \sum_{n=0}^\infty\frac{f^{(n)}(a)}{n!}(x-a)^n\tag{4.1.2}
\end{equation}
Lagrange observed that the coefficient of \((x-a)^n\) provides the \(n\)th derivative of \(f\) at \(a\) (divided by \(n!\)). Modifying formula (4.1.2) to suit his purpose, Lagrange supposed that every differentiable function could be represented as
\begin{equation*}
f(x) = \sum_{n=0}^\infty g_n(a)(x-a)^n\text{.}
\end{equation*}
In that case \(g_1(a)\) is the derivative of \(f\) at \(a\text{,}\) \(f^{\prime\prime}(a)=2g_2(a)\) and generally
\begin{equation*}
f^{(n)}(a)=n! g_n(a)\text{.}
\end{equation*}
Lagrange dubbed his function \(g_1\) the “fonction dérivée” from which we get the modern name derivative.
Problem 4.1.6.
Let \(a\neq 0\) be a fixed number. We saw in Problem 3.3.1 that the power series of \(\frac{1}{x}\) expanded about \(a\) is given by
\begin{equation*}
\sum^{\infty}_{n=0}{\frac{{\left(-1\right)}^n}{a^{n+1}}{\left(x-a\right)}^n}
=\frac{1}{a}-\frac{1}{a^2}\left(x-a\right)
+\frac{1}{a^3}{\left(x-a\right)}^2
-\frac{1}{a^4}{\left(x-a\right)}^3+\dots
\end{equation*}
(a)
Apply Lagrange’s idea to show that \(f^{\left(n\right)}\left(a\right)=\frac{{\left(-1\right)}^nn!}{a^{n+1}}\text{.}\)
(b)
Now compute \(f^{\left(n\right)}\left(a\right)\) by directly by differentiating. Do you get the same result?
Problem 4.1.7.
(a)
In Problem 3.3.5, you determined the power series expansion
\begin{align*}
\sin (x)=\sin (a)+\amp{}\cos\left(a\right)\left(x-a\right)\\
\amp{}-\frac{1}{2!}\sin
\left(a\right){\left(x-a\right)}^2\\
\amp{}\ \ \ -\frac{1}{3!}\cos
\left(a\right){\left(x-a\right)}^3\\
\amp{}\ \ \ \ \ \ +\frac{1}{4!}\sin
\left(a\right){\left(x-a\right)}^4\\
\amp{}\ \ \ \ \ \ \ \ \ +\frac{1}{5!}\cos
\left(a\right){\left(x-a\right)}^5+\cdots.
\end{align*}
Apply Lagrange’s idea to determine the \(n\)th derivative of \(\sin x\text{.}\) Compare with the results you get from differentiating \(\sin x\) directly.
(b)
In Problem 3.3.3 you determined the power series expansion of \(e^x\) about \(a\text{.}\) Apply Lagrange’s idea to show that every derivative of \(e^x\) is given by \(e^x\text{.}\)
(c)
In part (b) of Problem 3.3.1 you determined the power series expansion of \(\ln(x)\) about \(a>0\text{.}\) Apply Lagrange’s idea to show that the derivative of \(\ln (x)\) is \(\frac{1}{x}\text{.}\)
As we observed in Section 3.1 Leibniz and his peers would have regarded the expression \(\dfdx{z}{x}\) as a fraction (of differentials) so they would have derived the formula
\begin{equation}
\dfdx{z}{y}\dfdx{y}{x}=\dfdx{z}{x}\tag{4.1.3}
\end{equation}
by simply canceling the \(\dx{y}\) that appears in the numerator and denominator, just as we did in Section 3.1 (or sometimes by “uncancelling” them to get the left side of equation (4.1.3) from the right side). Mathematicians of that era would have regarded this operation as basic algebra.
Eighteenth and nineteenth century arithmetic primers used the phrase “chain rule” to describe the daisy–chain of symbolic cancellations that occurs when we convert units. For example, to convert yards to inches we compute
\begin{equation*}
\left(2\cancel{\text{ yards}}\right)\left(3\frac{\cancel{\text{feet}}}{\cancel{\text{yard}}}\right)\left(12\frac{\text{inches}}{\cancel{\text{foot}}}\right)=72\text{ inches}\text{.}
\end{equation*}
Since the cancellations in equation (4.1.3) appeared to be just another example of the older “chain rule” the name was adopted by 20th century Calculus textbooks even though in modern times it is better understood as a mnemonic for the deeper and more abstract operation of computing the derivative of composed functions like \(f(g(x))\text{.}\)
Letting \(z=f(y)\) and \(y=g(x)\) and using Lagrange’s prime notation we see that in modern form the chain rule for Calculus is:
\begin{equation*}
\dfdx{z}{x}=\left(f\circ
g\right)^\prime\left(x\right)=\underbrace{f^\prime\left(g\left(x\right)\right)}_{\dfdx{z}{y}}\underbrace{g^\prime(x)}_{\dfdx{y}{x}{}}\text{.}
\end{equation*}
Which obviously looks nothing like the older chain rule seen as in equation (4.1.3)
In his Théorie des fonctions analytiques (1797) Lagrange provided a derivation of the Chain Rule in these more modern terms. The following problem captures his idea.
Problem 4.1.8.
(a)
(b)
Expand \(f(y)\) as a power series expanded about \(y=g(a)\) and substitute your answer from part (a) for \(y\text{.}\)
(c)
Show that the coefficient of the linear term \((x-a)\) in part (b) is given by
\begin{equation*}
f^\prime\left(g\left(a\right)\right)g^\prime(a)
\end{equation*}
All in all, Lagrange’s idea was very clever and insightful. It’s only real flaw is that the fundamental, underlying assumption is not true. It turns out that not every differentiable function can be represented as a Taylor series. This was demonstrated very dramatically by Augustin Cauchy’s famous counter-example
\begin{equation}
f(x) = \begin{cases} e^{-\frac{1}{x^2}}\amp x\ne0\\ 0 \amp x=0 \end{cases}\text{.}\tag{4.1.4}
\end{equation}
This function is actually infinitely differentiable everywhere but its Maclaurin series (that is, its Taylor series with \(a=0\)) does not converge to \(f\) (except, trivially, at the origin) because all of its derivatives at the origin are equal to zero. That is \(f^{(n)}(0) = 0, \forall\, n \in
\NN\text{.}\)
To begin with, let’s compute a few derivatives when \(x
\neq 0\text{.}\)
\begin{align*}
f^{(0)}(x) \amp = e^{x^{-2}}\\
f^{(1)}(x) \amp = 2x^{-3}e^{-x^{-2}}\\
f^{(2)}(x) \amp = \left(4x^{-6}-6x^{-4}\right)e^{-x^{-2}}\text{.}
\end{align*}
As you can see the calculations are already getting a little complicated and we’ve only taken the second derivative.
Conceptually, it is not difficult to compute these derivatives using the tools you learned in Calculus but the formulas involved do become complicated rather quickly. Some care must be taken to avoid error. To streamline things a bit we take \(y= x^{-1}\text{,}\) and define \(p_2(x) = 4x^6-6x^4\) so that
\begin{equation*}
f^{(2)}(x) = p_2(x^{-1})e^{-x^{-2}} = p_2(y)e^{-y^2}\text{.}
\end{equation*}
Problem 4.1.9. Cauchy’s Counterexample, Part 1.
To simplify the notation we make the following definitions: \(y=x^{-1}\) and \(f^{(n)}(x) =p_n(y)e^{-y^2}\text{.}\)
(a)
(b)
Unfortunately everything we’ve done so far only gives us the derivatives we need when \(x\) is not zero, and we need the derivatives when \(x\) is zero. To find these we need to get back to very basic ideas.
Let’s assume for the moment that we know that \(f^{(n)}(0)=0\) and recall that
\begin{align*}
f^{(n+1)}(0) \amp = \limit{x}{0}{\frac{f^{(n)}(x)-f^{(n)}(0)}{x-0}}\\
f^{(n+1)}(0) \amp = \limit{x}{0}{x^{-1}p_n(x^{-1})e^{-x^{-2}}}\\
f^{(n+1)}(0) \amp = \limit{y}{\pm\infty}{\frac{yp_n(y)}{e^{y^2}}}\text{.}
\end{align*}
We can close the deal with the following problem.
Problem 4.1.10. Cauchy’s Counterexample, Part 2.
(a)
Let \(m\) be a non–negative integer. Show that \(\limit{y}{\pm\infty}{\frac{y^m}{e^{y^2}}}=0\text{.}\)
(b)
(c)
Let \(f(x)\) be as in equation (4.1.4) and show that for every non–negative integer \(n\text{,}\) \(f^{(n)}(0)=0\text{.}\)
This example showed that while it was fruitful to exploit Taylor series representations of various functions, basing the foundations of Calculus on power series was not a sound idea.
While Lagrange’s approach wasn’t totally successful, it was a major step away from infinitesimals and toward the modern approach. We still use aspects of it today. For instance we still use his prime notation to denote the derivative.
Turning Lagrange’s idea on its head it is clear that if we know how to compute derivatives, we can use this “machine” to obtain a power series when we are not clever enough to obtain the series by other (typically shorter) means. For example, consider Newton’s binomial series when \(\alpha=\frac{1}{2}\text{.}\) Originally, we obtained this series by extending the binomial theorem to non-integer exponents. Taylor’s formula provides a more systematic procedure. First we compute the derivatives of \(f\) at zero:
\begin{align*}
f(x)\amp =(1+x)^{\frac{1}{2}};\amp f(0)\amp =1\\
f^\prime(x)\amp =\frac{1}{2}(1+x)^{\frac{1}{2}-1};\amp f^\prime(0)\amp =\frac{1}{2}\\
f^{\prime\prime}(x)\amp =\frac{1}{2}\left(\frac{1}{2}-1\right)(1+x)^{\frac{1}{2}-2}\amp f^{\prime\prime}(0)\amp =\frac{1}{2}\left(\frac{1}{2}-1\right)
\end{align*}
and in general since
\begin{align*}
f^{(n)}(x)\amp =\frac{1}{2}\left(\frac{1}{2}-1\right)\cdots\left(\frac{1}{2}-(n-1)\right)(1+x)^{\frac{1}{2}-n}\\
\end{align*}
we have
\begin{align*}
f^{(n)}(0)\amp =\frac{1}{2}\left(\frac{1}{2}-1\right)\cdots\left(\frac{1}{2}-(n-1)\right)\text{.}
\end{align*}
Next we substitute \(f^{(n)}(0)\) into Taylor’s formula to obtain the series
\begin{align*}
\sum_{n=0}^\infty\frac{f^{(n)}(0)}{n!}x^n \amp{}= 1+\sum_{n=1}^\infty\frac{\frac{1}{2}\left(\frac{1}{2}-1\right)\cdots\left( \frac{1}{2}-(n-1)\right)}{n!}x^n\\
\amp{}= 1+\sum_{n=1}^\infty\frac{\prod_{j=0}^{n-1}\left(\frac{1}{2}-j\right)}{n!}x^n
\end{align*}
which agrees with equation (3.2.6) in the previous chapter.
Problem 4.1.11.
Use Taylor’s formula to obtain the general binomial series
\begin{equation*}
(1+x)^\alpha=1+\sum_{n=1}^\infty\frac{\prod_{j=0}^{n-1}\left(\alpha-j\right)}{n!}x^n.{}
\end{equation*}
As you can see, Taylor’s “machine” will produce the power series for a function (if it has one), but is tedious to perform. We will find, generally, that this tediousness can be an obstacle to understanding. In many cases it will be better, or at least quicker, to be clever if we can. However, it is comforting to have Taylor’s formula available as a last resort.
The existence of a Taylor series is addressed (to some degree) by the following.
Theorem 4.1.12. Taylor’s Theorem.
If \(f^\prime, f^{\prime\prime}, \ldots, f^{(n+1)}\) are all continuous on an interval containing \(a\) and \(x\text{,}\) then
\begin{align*}
f(x)=f(a)+\amp{}\frac{f^{\prime}(a)}{1!}(x-a)+\frac{f^{\prime \prime}(a)}{2!}(x-a)^2 +\\
\amp{}\cdots+\frac{f^{(n)}(a)}{n!}(x-a)^n + \frac{1}{n!}\int_{t=a}^xf^{(n+1)}(t)(x-t)^n\dx{t}\text{.}
\end{align*}
The last term in this expression,
\begin{equation*}
\frac{1}{n!}\int_{t=a}^xf^{(n+1)}(t)(x-t)^n\dx{t}\text{,}
\end{equation*}
is called the Integral Form of the Remainder of the Taylor series.
The proof of Theorem 4.1.12 will be by Mathematical Induction but before we address the proof, notice that the \(n\)th degree polynomial
\begin{equation}
f(a)+\frac{f^{\prime}(a)}{1!}(x-a)+\frac{f^{\,\prime\prime}(a)}{2!}(x-a)^2+\cdots+\frac{f^{(n)}(a)}{n!}(x-a)^n\tag{4.1.5}
\end{equation}
resembles the Taylor series and, in fact, is called the \(\boldsymbol n\)th degree Taylor polynomial of \(f\) about \(\boldsymbol a\). Theorem 4.1.12 says that a function can be written as the sum of this polynomial and a specific integral which we will analyze in Chapter 7.
We will get the proof started and leave the formal induction proof as an exercise. To begin the induction notice that when \(n=0\) equation (4.1.5) is a restatement of the Fundamental Theorem of Calculus:
\begin{equation*}
\int_{t=a}^xf^\prime(t)\dx{t}=f(x)-f(a)\text{.}
\end{equation*}
which we rewrite as
\begin{equation*}
f(x)=f(a)+\frac{1}{0!}\int_{t=a}^xf^\prime(t)(x-t)^0\dx{t}
\end{equation*}
in order to provide the anchor step (when \(n=0\)) for our induction.
Aside: The Fundamental Theorem of Calculus.
To derive the case where \(n=1\text{,}\) we use integration by parts. If we let
\begin{align*}
u\amp =f^\prime(t)\amp d v\amp =(x-t)^0d t\\
d u\amp =f^{\prime\prime}(t)d t\amp v\amp =-\frac{1}{1}(x-t)^1
\end{align*}
we obtain
\begin{align*}
f(x)\amp =f(a)+\frac{1}{0!}\left(-\frac{1}{1}f^\prime(t)(x-t)^1|_{t=a}^{^x}+\frac{1}{1} \int_{t=a}^xf^{\prime\prime}(t)(x-t)^1\dx{ t}\right)\\
\amp =f(a)+\frac{1}{0!}\left(-\frac{1}{1}f^\prime(x)(x-x)^1+ \frac{1}{1}f^\prime(a)(x-a)^1+\frac{1}{1}\int_{t=a}^xf^{\prime\prime}(t)(x-t)^1\dx{ t}\right)\\
\amp =f(a)+\frac{1}{1!}f^\prime(a)\left(x-a\right)^1 + \frac{1}{1!}\int_{t=a}^xf^{\prime\prime}(t)(x-t)^1\dx{ t}\text{.}
\end{align*}
Problem 4.1.13.
Provide a formal induction proof for Theorem 4.1.12.
Section 4.2 Series Anomalies
Until we began this chapter our approach to series mirrored that of the eighteenth century mathematicians. By exploiting the ideas of Calculus and power series they ingeniously derived mathematical results which were virtually unobtainable before. Mathematicans were eager to push these techniques as far as they could to obtain their results and they often showed good intuition regarding what was mathematically acceptable and what was not. However, as the envelope was pushed further and further, substantial questions about the validity of their methods surfaced.
As an illustration consider the series expansion
\begin{equation*}
\frac{1}{1+x}=1-x+x^2-x^3+\cdots\text{.}
\end{equation*}
If we substitute \(x=1\) into this equation, we obtain
\begin{equation*}
\frac{1}{2}=1-1+1-1+\cdots\text{.}
\end{equation*}
If we group the terms as follows \((1-1)+(1-1)+\cdots\text{,}\) the series would equal \(0\text{.}\) A regrouping of \(1+(-1+1)+(-1+1)+\cdots\) provides an answer of \(1\text{.}\) This violation of the associative law of addition did not escape the mathematicians of the 1700s. In his 1760 paper On Divergent Series Euler said:
Notable enough, however are the controversies over the series \(1-1+1-1+\text{etc}\text{,}\) whose sum was given by Leibniz as \(\frac{1}{2}\text{,}\) although others disagree . . . Understanding of this question is to be sought in the word “sum;” this idea, if thus conceived — namely, the sum of a series is said to be that quantity to which it is brought closer as more terms of a series are taken — has relevance only for the convergent series, and we should in general give up this idea of sum for divergent series. On the other hand, as series in analysis arise from the expansion of fractions or irrational quantities or even of transcendentals, it will, in turn, be permissible in calculation to substitute in place of such series that quantity out of whose development it is produced.
Even with this formal approach to series, an interesting question arises. The series for the antiderivative of \(\frac{1}{1+x}\) does converge for \(x=1\) while this one does not. Specifically, one antiderivative of the above series is
\begin{equation*}
\ln(1+x)=x-\frac{1}{2}x^2+\frac{1}{3}x^3-\cdots\text{.}
\end{equation*}
If we substitute \(x=1\) into this series, we obtain \(\ln
2=1-\frac{1}{2}+\frac{1}{3}-\cdots\text{.}\) It is not hard to see that such an alternating series converges. The following picture shows why. In this diagram, \(S_n\) denotes the partial sum \(1-\frac{1}{2}+\frac{1}{3}-\cdots+\frac{(-1)^{n+1}}{n}\text{.}\)
From the diagram we can see \(S_2\leq S_4\leq S_6\leq\cdots\leq\cdots\leq S_5\leq S_3\leq S_1\) and \(S_{2k+1}-S_{2k}=\frac{1}{2k+1}\text{.}\) It seems that the sequence of partial sums will converge to whatever is in the “middle.” Our diagram indicates that it is ln \(2\) in the middle but actually this is not obvious. Nonetheless it is interesting that one series converges for \(x=1\) but the other does not.
Problem 4.2.1.
Use the fact that
\begin{equation*}
1-\frac{1}{2}+\frac{1}{3}-\cdots+\frac{(-1)^{2k+1}}{2k}\leq\ln 2\leq 1-\frac{1}{2}+\frac{1}{3}-\cdots+\frac{(-1)^{2k+2}}{2k+1}
\end{equation*}
to determine how many terms of the series \(\sum_{n=1}^\infty\frac{(-1)^{n+1}}{n}\) should be added together to approximate \(\ln 2\) to within \(.0001\) without actually computing what \(\ln 2\) is.
These examples illustrate something even more perplexing. As we’ve observed the divergent infinite sum \(1-1+1-1+\cdots\) does not appear satisfy the associative law for addition. Since the sum diverges (is meaningless) this is not too surprising. On the other hand while the convergent series \(1-\frac{1}{2}+\frac{1}{3}-\cdots\) does satisfy the associative law as expected, it does not satisfy the commutative law. In fact, Theorem 4.2.3 shows that it fails, rather spectacularly, to be commutative.
Bernhard Riemann (1826–1866) stated and proved a generalization of the following theorem in 1854.
Theorem 4.2.3.
Let \(a\) be any real number. There exists a rearrangement of the Alternating Harmonic Series \(1-\frac{1}{2}+\frac{1}{3}-\cdots\) which converges to \(a\text{.}\)
Aside: Series Rearrangements.
This theorem shows that a series is most decidedly not a great big sum. It follows that power series are not just great big polynomials.
To set the stage, consider the Harmonic Series
\begin{equation}
\sum_{n=1}^\infty\frac{1}{n}=1+\frac{1}{2}+\frac{1}{3}+\cdots\text{.}\tag{4.2.1}
\end{equation}
Even though the individual terms in this series converge to \(0\text{,}\) the series still diverges (to infinity) as evidenced by the inequality
\begin{align*}
\left(1+\frac{1}{2}\right)\amp +\left(\frac{1}{3}+\frac{1}{4}\right)+\left(\frac{1}{5}+\frac{1}{6}+ \frac{1}{7}+\frac{1}{8}\right)+\left(\frac{1}{9}+\cdots+\frac{1}{16}\right)+\cdots\\
\amp >\frac{1}{2}+\left(\frac{1}{4}+\frac{1}{4}\right)+\left(\frac{1}{8}+ \frac{1}{8}+\frac{1}{8}+\frac{1}{8}\right)+\left(\frac{1}{16}+\cdots+\frac{1}{16}\right)+\cdots\\
\amp =\frac{1}{2}+\frac{1}{2}+\frac{1}{2}+\frac{1}{2}+\cdots\\
\amp = \infty\text{.}
\end{align*}
Armed with this fact, we can see why Theorem 4.2.3 is true. First note that
\begin{equation*}
-\frac{1}{2}-\frac{1}{4}-\frac{1}{6}-\cdots=-\frac{1}{2}(1+\frac{1}{2}+ \frac{1}{3}+\cdots)=-\infty
\end{equation*}
and
\begin{equation*}
1+\frac{1}{3}+\frac{1}{5}+\cdots\geq\frac{1}{2}+\frac{1}{4}+\frac{1}{6}+\ldots= \infty\text{.}
\end{equation*}
This says that if we add enough terms of \(-\frac{1}{2}-\frac{1}{4}-\frac{1}{6}-\cdots\) we can make such a sum as large as we wish (and negative) and if we add enough terms of \(1+\frac{1}{3}+\frac{1}{5}+\cdots\) we can make such a sum as large as we wish (and positive). Those facts provide us with the general outline of the proof. The trick is to add just enough positive terms until the sum is just greater than \(a\text{.}\) Then we start to add on negative terms until the sum is just less than \(a\text{.}\) Picking up where we left off with the positive terms, we add on just enough positive terms until we are just above \(a\) again. We then add on negative terms until we are below \(a\text{.}\) In essence, we are bouncing back and forth around \(a\text{.}\) If we do this carefully, then we can get this rearrangement to converge to \(a\text{.}\) The notation in the proof below gets a bit hairy, but keep this general idea in mind as you read through it.
Let \(O_1\) be the first odd integer such that \(1+\frac{1}{3}+\frac{1}{5}+\cdots+\frac{1}{O_1}>a\text{.}\) Now choose \(E_1\) to be the first even integer such that
\begin{equation*}
-\frac{1}{2}-\frac{1}{4}-\frac{1}{6}-\cdots-\frac{1}{E_1} \lt a-\left(1+\frac{1}{3}+\frac{1}{5}+\cdots+\frac{1}{O_1}\right)\text{.}
\end{equation*}
Thus
\begin{equation*}
1+\frac{1}{3}+\frac{1}{5}+\cdots+\frac{1}{O_1}-\frac{1}{2}-\frac{1}{4} - \frac{1}{6}-\cdots-\frac{1}{E_1}\lt a\text{.}
\end{equation*}
Notice that we still have \(\frac{1}{O_1+2}+\frac{1}{O_1+4}+\cdots=\infty\text{.}\) With this in mind, choose \(O_2\) to be the first odd integer with
\begin{align*}
\frac{1}{O_1+2}\amp{}+\frac{1}{O_1+4}+\cdots\frac{1}{O_2}\\
\amp{}\gt a-\left(1+\frac{1}{3}+
\frac{1}{5}+\cdots+\frac{1}{O_1}-\frac{1}{2}-\frac{1}{4}-\frac{1}{6}-\cdots-
\frac{1}{E_1}\right).
\end{align*}
Thus we have
\begin{align*}
a\lt
1+\frac{1}{3}+\frac{1}{5}+\amp{}\cdots+\frac{1}{O_1}-\frac{1}{2}-\frac{1}{4}-
\frac{1}{6}-\\
\amp{}\cdots-\frac{1}{E_1}+\frac{1}{O_1+2}+\frac{1}{O_1+4}+\cdots+
\frac{1}{O_2}.
\end{align*}
Furthermore, since
\begin{align*}
a\gt 1+\frac{1}{3}+\frac{1}{5}+\amp{}\cdots+\frac{1}{O_1}-\frac{1}{2}-\frac{1}{4}-
\frac{1}{6}-\\
\amp{}\cdots-\frac{1}{E_1}+\frac{1}{O_1+2}+\frac{1}{O_1+4}+\\
\amp{}\cdots+ \frac{1}{O_2-2}
\end{align*}
we have
\begin{align*}
\frac{1}{O_2} \gt \left|1+\frac{1}{3}+\frac{1}{5}+\right.\amp{}\cdots+\frac{1}{O_1}-\frac{1}{2}-\frac{1}{4}- \frac{1}{6}-\\
\amp\cdots \left.-\frac{1}{E_1}+\frac{1}{O_1+2}+\frac{1}{O_1+4}+\cdots+ \frac{1}{O_2}-a\right|
\end{align*}
In a similar fashion choose \(E_2\) to be the first even integer such that
\begin{align*}
a\gt 1+\frac{1}{3}+\frac{1}{5}+\cdots\amp +\frac{1}{O_1}-\frac{1}{2}- \frac{1}{4}-\frac{1}{6}-\\
\cdots\amp{} -\frac{1}{E_1}+ \frac{1}{O_1+2}+\frac{1}{O_1+4}+\\
\cdots\amp{}+\frac{1}{O_2}-\frac{1}{E_1+2}-\frac{1}{E_1+4}- \cdots{}-\frac{1}{E_2}\text{.}
\end{align*}
Since
\begin{align*}
a\lt 1+\frac{1}{3}+\frac{1}{5} +\cdots\amp{}+\frac{1}{O_1}-\frac{1}{2}- \frac{1}{4}-\frac{1}{6}-\\
\cdots\amp{}-\frac{1}{E_1}+ \frac{1}{O_1+2}+\frac{1}{O_1+4}+\\
\cdots\amp{}+\frac{1}{O_2}- \frac{1}{E_1+2}-\frac{1}{E_1+4}-\cdots-\frac{1}{E_2-2}
\end{align*}
we see that
\begin{align*}
\frac{1}{E_2} \gt{}
\left|1+\frac{1}{3}\right.+\frac{1}{5}+
\amp \cdots+\frac{1}{O_1}-\frac{1}{2}-
\frac{1}{4}-\frac{1}{6}-\\
\amp{}\cdots-\frac{1}{E_1} +\frac{1}{O_1+2}+\frac{1}{O_1+4}+\\
\amp{}\cdots+\frac{1}{O_2}-
\frac{1}{E_1+2}-\frac{1}{E_1+4}-\\
\amp{}\cdots-\left.\frac{1}{E_2}-a\right|.
\end{align*}
Again choose \(O_3\) to be the first odd integer such that
\begin{align*}
a\lt 1+\frac{1}{3} +\frac{1}{5}+
\amp{}\cdots+\frac{1}{O_1}-\frac{1}{2}-\frac{1}{4}- \frac{1}{6}-\\
\amp{}\cdots
-\frac{1}{E_1}+\frac{1}{O_1+2}+\frac{1}{O_1+4}+\\
\amp{}\cdots+\frac{1}{E_1+2}-\frac{1}{E_1+4}-\cdots -\frac{1}{E_2} +\\
\amp{} \cdots \frac{1}{O_2+2} + \frac{1}{O_2+4}\cdots+\frac{1}{O_3}
\end{align*}
and notice that
\begin{align*}
\frac{1}{O_3}\gt \left|1+\frac{1}{3}\right.
+\frac{1}{5}+\amp{}\cdots+\frac{1}{O_1}-\frac{1}{2}-\frac{1}{4}-\frac{1}{6}-\\
\amp
\cdots-\frac{1}{E_1}+\frac{1}{O_1+2}+\frac{1}{O_1+4}+\\
\amp \cdots+ \frac{1}{O_2}+
-\frac{1}{E_1+2}-\frac{1}{E_1+4}-\\
\amp{}\cdots-\frac{1}{E_2}+\frac{1}{O_2+2}+
\frac{1}{O_2+4}+\cdots+\left.\frac{1}{O_3}-a\right|\text{.}
\end{align*}
Continue defining \(O_k\) and \(E_k\) in this fashion for \(k\ge 4\text{.}\) Since \(\limit{k}{\infty}{\frac{1}{O_k}}=\limit{k}{\infty}{\frac{1}{E_k}}=0\text{,}\) it is evident that the partial sums
\begin{align*}
1+\frac{1}{3} +\frac{1}{5}+\amp{}\cdots+\frac{1}{O_1}-\frac{1}{2}-\frac{1}{4}-\frac{1}{6}-\\
\amp \cdots-\frac{1}{E_1}+\frac{1}{O_1+2}+\frac{1}{O_1+4}+\\
\amp \cdots \frac{1}{O_2}+\cdots -\frac{1}{E_{k-2}+2}-\frac{1}{E_{k-2}+4}-\\
\amp \cdots-\frac{1}{E_{k-1}}+ \frac{1}{O_{k-1}+2}+\frac{1}{O_{k-1}+4}+\cdots+\frac{1}{O_k}
\end{align*}
and
\begin{align*}
1+\frac{1}{3}\amp
+\frac{1}{5}+\cdots+\frac{1}{O_1}-\frac{1}{2}-\frac{1}{4}-
\frac{1}{6}-\\
\amp \cdots-\frac{1}{E_1}+\frac{1}{O_1+2}+\frac{1}{O_1+4}+
\cdots{} \frac{1}{O_2}+\\
\amp
\cdots-\frac{1}{E_{k-2}+2}-\frac{1}{E_{k-2}+4}-\cdots-\frac{1}{E_{k-1}}
\end{align*}
must converge to \(a\text{.}\) Furthermore, it is evident that every partial sum of the rearrangement
\begin{align*}
1+\frac{1}{3} +\frac{1}{5}+\amp\cdots+\frac{1}{O_1}-\frac{1}{2}-\frac{1}{4}- \frac{1}{6}-\\
\amp{}-\cdots-\frac{1}{E_1}+\frac{1}{O_1+2}+\frac{1}{O_1+4}+\cdots+ \frac{1}{O_2}+\cdots
\end{align*}
is trapped between two such extreme partial sums. This forces the entire rearranged series to converge to \(a\text{.}\)
The two parts of the next problem are extentions of Theorem 4.2.3 but they are a bit simpler notationally since we don’t need to worry about converging to an actual number. We only need to make the rearrangement increase or decrease without bound.
Problem 4.2.4.
Show that there is a rearrangement of the Alternating Harmonic Series,
\begin{equation*}
1-\frac{1}{2}+\frac{1}{3}-\frac{1}{4}+\cdots
\end{equation*}
which:
(a)
Diverges to \(\infty\text{.}\)
(b)
Diverges to \(-\infty\text{.}\)
It is fun to know that we can rearrange some series to make them add up to anything we like but there is a more fundamental idea at play here. That the negative terms of the Alternating Harmonic Series diverge to negative infinity and the positive terms diverge to positive infinity make the convergence of the alternating series very special.
Consider what happens when we sum the series. We start with \(1\text{.}\) This is a positive term so our sum is starting to increase without bound. Next we add \(-1/2\) which is a negative terms so our sum has turned around and is now starting to decrease without bound. Then another positive term is added: increasing without bound. Then another negative term: decreasing. And so on. The convergence of the alternating Harmonic Series is the result of a delicate balance between a tendency to run off to positive infinity and back to negative infinity. When viewed in this light it is not really too surprising that rearranging the terms can destroy this delicate balance.
Naturally, the alternating Harmonic Series is not the only such series. Any such series is said to converge conditionally — the condition being the specific arrangement of the terms.
To stir the pot a bit more, some series do satisfy the commutative property. More specifically, one can show that any rearrangement of the series \(1-\frac{1}{2^2}+\frac{1}{3^2}-\cdots\) must converge to the same value as the original series (which happens to be \(\int_{x=0}^1\frac{\ln(1+x)}{x}\dx{x}\approx.8224670334\)). Why does one series behave so nicely whereas the other does not? The answer to that question will have to wait until Section 11.2.
Anomalies like these were well known in the \(1700\)s, but they were overshadowed by the overwhelming utility of the Calculus. Indeed, foundational questions raised by the above examples as well as concerns about the validity of using the infinitely small and infinitely large, while certainly interesting and of importance, did not significantly deter the exploitation of Calculus in studying physical phenomena. However, the envelope eventually was pushed to the point that not even the most practically oriented mathematician could avoid these foundational issues.
