Video Lecture

Text and Additional Details

We have repeatedly claimed that one of the most useful things about calculus are the ideas on how to deal with small changes in a variable, and how that effects other variables. One of the best examples of this is linear approximation, which we will introduce in this segment geometrically before diving into the analytic mechanics involved later on.

Linear approximation is ubiquitous, and in fact it is used in some remarkably deep and complicated applications. As a few examples, by way of motivation, linear approximation is arguably the key mechanic in:

• Neural Nets (a.k.a. Machine Learning or Deep Learning)
• Computational solutions, like how computers solve equalities.
• Control Theory (getting robots to understand and react to their environment)
• Advanced Fluid Dynamics (e.g., modeling how wind travels over the wing of a plane)

Linear approximation is used in many applications because it gives a way to quickly estimate outputs of very complicated equations or formulas. It accomplishes this by using an anchor point which is easy to calculate, along with a tangent line to estimate the desired value within some level of error. This may seem complicated, but as usual, it is much easier to understand if we (literally) look at what we are doing.

Let’s say we want to estimate $\sqrt {10}$. Doing this is surprisingly difficult. As a fun fact - adults tend toward linear progression rather than logarithmic or geometric progression (interestingly, very young children tend toward processing geometric progression, which is why a young enough child will say the “middle value between 1 and 9 is 3” rather than 5). Nonetheless, we can use linear approximation here to get an estimate by picking a nearby value that we can calculate, and using a tangent line.

Before we proceed though, we need to pick a good representative formula to start from. The obvious choice here would be $\sqrt {x}$. We can calculate this function nicely at $x=9$, which is relatively close to $\sqrt {10}$. But let’s take a look at the graph of $\sqrt {x}$.

The fundamental idea of linear approximation is that we want to use a nearby point that we can calculate nicely (in this case, the point (9,3)) and then use a derivative to get a tangent line at that anchor point. Once we have the tangent line, we can use the tangent line’s value (which is just a line after all, so it’s very easy to calculate) to estimate the original function’s value at the point of interest. Let’s zoom in a bit on our graph and see what we mean.

We want to use the tangent line at our anchor point, so let’s add that part.

Looking carefully, we can see that the tangent line’s value at $x=10$ is very close to, but not quite the same as, the original function. Let’s add the corresponding value of the tangent line to our graph.

So, what are we looking at? The idea here is that the tangent line has a relatively nice formula - in fact if we calculated the formula explicitly we get: $y = \frac {1}{6}(x-9)+3$. We can plug our desired $x$ value ($x=10$) into this formula, to get our actual estimate, which came out to be $\frac {10}{6} \approx 3.166$. The actual value of $\sqrt {10}$ is about $3.1622777$, which means we had an error of just over $0.1$%, which is quite close to the real value!

But, computing a tangent line directly each time is a bit tedious, and it turns out, it’s also a bit unnecessary. We can make the formula a bit more compact, but doing so makes it hard to see what is actually happening. For this reason we will revisit this picture when we discuss the analytic viewpoint of linear approximation next.

As a taste though, it is a bit clearer to see this process unfold over time, rather than looking at static graphs, for a quick animation showing what is happening with a bit more detail and explanation, see the video above!

So we’ve seen that the process of linear approximation is essentially nothing more than using a tangent line along with some kind of anchor point (chosen because it is “close to” our estimating value and easy to calculate in our original function), and using the (linear) tangent line to estimate a value for our original function that would be otherwise difficult to calculate directly.

Despite some of the analytic ’dressing’ that will make the formula look a little weird, linear approximation is nothing more than a tangent line. It is very useful to remember this when trying to compute a linear approximation, as it will help you keep the notation and variables straight (or even recover them if you forget what they are) when working out the analytic formula.