Hexa Scholar
MathCalculus

Which Taylor terms vanish as Δx shrinks

Halve the step Δx. By what factor does the error of a Taylor approximation shrink, and which Taylor terms can be dropped when many small steps are added up?

For the smooth functions on this page the factor settles near a power of 2 as the step shrinks, and on a plot with logarithmic axes that power shows up as a slope. Stage 1 asks you to predict the factor for three functions before the page computes anything; stage 2 adds up many small steps.

Before you start · 3 quick checks on Taylor polynomials, logarithms and higher derivatives, about 3 min
Setup: the error of a Taylor polynomial and its order

Taylor polynomials. As in What degree does a Taylor polynomial need?, \(f^{(j)}\) is the \(j\)-th derivative of \(f\), and if \(f\) is \(n\) times differentiable at the center \(a\), its Taylor polynomial of degree \(n\) is

\[T_n(x) = \sum_{j=0}^{n} \frac{f^{(j)}(a)}{j!}(x - a)^j.\]

The number \(\tfrac{f^{(j)}(a)}{j!}\) is the coefficient of degree \(j\); \(T_n\) keeps the coefficients of degree 0 to \(n\) and leaves out the rest.

The error at a step \(h\). Write \(h\) for the step \(\Delta x\) away from the center. The error of \(T_n\) at \(a + h\) is

\[E_n(h) = f(a + h) - T_n(a + h).\]

Stages 1 and 2 use steps \(h \gt 0\); stage 3 allows either sign.

Order of a small quantity. Let \(g(h)\) be defined for small \(h \gt 0\), and let \(k \gt 0\). Say that \(g\) has order \(k\) when \(g(h)/h^k\) tends to a nonzero real number \(C\) as \(h \to 0\). Then halving \(h\) divides \(g\) by a number that tends to \(2^k\): \(\tfrac{g(h)}{g(h/2)} \to 2^k\) (stage 3 shows why).

Logarithmic axes. Plot \(\log|g(h)|\) against \(\log h\), with logarithms to the same base on both axes. If halving \(h\) divides \(|g|\) by \(r\), the segment between \(h/2\) and \(h\) has slope \(\log_2 r\). So a quantity of order \(k\) gives slopes that approach \(k\) as \(h\) shrinks.

Fine print: big O and order

Writing \(g(h) = O(h^k)\) means that there are numbers \(M\) and \(\delta \gt 0\) with \(|g(h)| \le Mh^k\) for \(0 \lt h \lt \delta\). A quantity of order \(k\) is \(O(h^k)\), but not the other way round: \(g(h) = h^3\) is \(O(h^2)\), and its order is 3, not 2. The order, when it exists, is the one exponent \(k\) for which \(g(h)/h^k\) has a nonzero limit.

1 See

Predict the halving factor, then plot the errors

How the answers work

Click one answer in each of the six rows. An answer is locked once clicked, and nothing about any answer is shown until all six are in. Then the page marks each answer, explains it, and opens a plot and a table of the errors. Answer again clears all of it.

2 Try

Adding up many small steps

The explorer opens when you have answered both questions above.

3 Prove

Proof: the order of a Taylor error

A smooth trail that bends upward more and more, with two straight gold boards. Each board touches the trail at its left end and runs along it there; then the trail bends up away from the board, and a gap opens that is widest at the board's right end.
Straight boards laid on a curved trail. Each board starts on the trail and points along it, like the first-order step \(T_1(a + h) = f(a) + f'(a)h\) from its left end \(a\). The vertical gap at the right end of a board is the error \(E_1(h)\) of that step.
Theorem

Assumptions. Let \(n \ge 0\) and \(k \ge n + 1\) be whole numbers. Let \(f\) be \(k\) times differentiable on an open interval \(I\) that contains \(a\), with \(f^{(k)}\) continuous at \(a\). Suppose \(f^{(j)}(a) = 0\) for every whole number \(j\) with \(n \lt j \lt k\) (when \(k = n + 1\) there is no such \(j\)), and \(f^{(k)}(a) \ne 0\). Write \(c = \tfrac{f^{(k)}(a)}{k!}\), so \(c \ne 0\).

Conclusion. As \(h \to 0\) through values \(h \ne 0\) with \(a + h\) in \(I\),

\[\frac{E_n(h)}{h^k} \to c.\]

So \(E_n(h) \ne 0\) for every \(h \ne 0\) close enough to 0, and \(\tfrac{|E_n(h)|}{|E_n(h/2)|} \to 2^k\) as \(h \to 0\) with \(h \gt 0\): the error of \(T_n\) has order \(k\).

Facts the proof uses.

  • the Lagrange form of the remainder from the Taylor polynomials page: if \(m \ge 0\) and \(f\) is \(m + 1\) times differentiable on an open interval that contains \(a\) and \(x \ne a\), then \(f(x) - T_m(x) = \tfrac{f^{(m+1)}(\xi)}{(m+1)!}(x - a)^{m+1}\) for some \(\xi\) strictly between \(a\) and \(x\);
  • continuity of \(f^{(k)}\) at \(a\): \(f^{(k)}(t) \to f^{(k)}(a)\) as \(t \to a\);
  • the limit laws for products and quotients (a quotient with a denominator whose limit is not 0).

Each option is one sentence about the \(f\), \(a\), \(n\) and \(k\) of the theorem and the steps \(h \ne 0\) with \(a + h\) in \(I\). Pick the one that is true in every case the theorem covers, \(k = n + 1\) included.

What follows
  • Big O without a nonzero term. If \(f\) is \(n + 1\) times differentiable on \(I\) and \(|f^{(n+1)}| \le M\) there, the Lagrange form gives \(|E_n(h)| \le \tfrac{M}{(n+1)!}|h|^{n+1}\), so \(E_n(h) = O(|h|^{n+1})\) even when \(f^{(n+1)}(a) = 0\). The theorem says more when its assumptions hold: the error has order exactly \(k\).
  • When the factor is not a whole-number power of 2. The theorem needs \(k\) derivatives. For \(f(x) = |x|^{5/2}\) at \(a = 0\), the second derivative \(\tfrac{15}{4}|x|^{1/2}\) exists, the third does not exist at 0, and \(T_2 = 0\). So \(E_2(h) = |h|^{5/2}\), and halving \(h\) divides it by exactly \(2^{5/2} \approx 5.657\).
  • Computed errors at tiny steps. A computer that subtracts \(T_n(a + h)\) from \(f(a + h)\) loses digits when the error is far smaller than \(f(a + h)\). The page computes each error from the series of the terms that \(T_n\) leaves out, so the values in stage 1 do not suffer from this loss of digits.
4 Apply

Problems

Each problem has hints and a full solution. Exact answers are graded exactly and can be typed as a whole number, a decimal or a fraction; for a rounded answer the problem states the accepted error.

Key point To see how fast an error shrinks, find the first term the approximation leaves out with a nonzero coefficient. Its power \(k\) gives the halving factor \(2^k\) and the slope \(k\) on logarithmic axes; measured errors give an estimate of the order back as \(\log_2\) of the halving ratio. For a bond, the duration estimate leaves out the convexity term, so its error has order 2 in the yield change, and adding convexity leaves an error of order 3 when the third derivative of the price is not 0.
Level Intro Prerequisites What degree does a Taylor polynomial need?; logarithms and higher derivatives (refreshed in Before you start) Time ~40 min

Related: What a derivative measures when you zoom in shows that the error of the tangent line, divided by \(h\), tends to 0, and Itô's lemma uses the bookkeeping of stage 2 on Brownian paths.

Further reading