The gradient

$\newenvironment {prompt}{}{} \newcommand {\todo }[0]{} \newcommand {\oiint }[0]{{\large \bigcirc }\kern -1.56em\iint } \newcommand {\mooculus }[0]{\textsf {\textbf {MOOC}\textnormal {\textsf {ULUS}}}} \newcommand {\npnoround }[0]{\nprounddigits {-1}} \newcommand {\npnoroundexp }[0]{\nproundexpdigits {-1}} \newcommand {\npunitcommand }[1]{\ensuremath {\mathrm {#1}}} \newcommand {\RR }[0]{\mathbb R} \newcommand {\R }[0]{\mathbb R} \newcommand {\N }[0]{\mathbb N} \newcommand {\Z }[0]{\mathbb Z} \newcommand {\sagemath }[0]{\textsf {SageMath}} \newcommand {\d }[0]{\mathop {}\!d} \newcommand {\l }[0]{\ell } \newcommand {\ddx }[0]{\frac {d}{\d x}} \newcommand {\zeroOverZero }[0]{\ensuremath {\boldsymbol {\tfrac {0}{0}}}} \newcommand {\inftyOverInfty }[0]{\ensuremath {\boldsymbol {\tfrac {\infty }{\infty }}}} \newcommand {\zeroOverInfty }[0]{\ensuremath {\boldsymbol {\tfrac {0}{\infty }}}} \newcommand {\zeroTimesInfty }[0]{\ensuremath {\small \boldsymbol {0\cdot \infty }}} \newcommand {\inftyMinusInfty }[0]{\ensuremath {\small \boldsymbol {\infty -\infty }}} \newcommand {\oneToInfty }[0]{\ensuremath {\boldsymbol {1^\infty }}} \newcommand {\zeroToZero }[0]{\ensuremath {\boldsymbol {0^0}}} \newcommand {\inftyToZero }[0]{\ensuremath {\boldsymbol {\infty ^0}}} \newcommand {\numOverZero }[0]{\ensuremath {\boldsymbol {\tfrac {\#}{0}}}} \newcommand {\dfn }[0]{\textbf } \newcommand {\unit }[0]{\mathop {}\!\mathrm } \newcommand {\eval }[1]{\bigg [ #1 \bigg ]} \newcommand {\seq }[1]{\left ( #1 \right )} \newcommand {\epsilon }[0]{\varepsilon } \newcommand {\phi }[0]{\varphi } \newcommand {\iff }[0]{\Leftrightarrow } \DeclareMathOperator {\arccot }{arccot} \DeclareMathOperator {\arcsec }{arcsec} \DeclareMathOperator {\arccsc }{arccsc} \DeclareMathOperator {\si }{Si} \DeclareMathOperator {\scal }{scal} \DeclareMathOperator {\sign }{sign} \newcommand {\arrowvec }[1]{{\overset {\rightharpoonup }{#1}}} \newcommand {\vec }[1]{{\overset {\boldsymbol {\rightharpoonup }}{\mathbf {#1}}}} \DeclareMathOperator {\proj }{\vec {proj}} \newcommand {\veci }[0]{{\boldsymbol {\hat {\imath }}}} \newcommand {\vecj }[0]{{\boldsymbol {\hat {\jmath }}}} \newcommand {\veck }[0]{{\boldsymbol {\hat {k}}}} \newcommand {\vecl }[0]{\vec {\boldsymbol {\l }}} \newcommand {\uvec }[1]{\mathbf {\hat {#1}}} \newcommand {\utan }[0]{\mathbf {\hat {t}}} \newcommand {\unormal }[0]{\mathbf {\hat {n}}} \newcommand {\ubinormal }[0]{\mathbf {\hat {b}}} \newcommand {\dotp }[0]{\bullet } \newcommand {\cross }[0]{\boldsymbol \times } \newcommand {\grad }[0]{\boldsymbol \nabla } \newcommand {\divergence }[0]{\grad \dotp } \newcommand {\curl }[0]{\grad \cross } \newcommand {\lto }[0]{\mathop {\longrightarrow \,}\limits } \newcommand {\bar }[0]{\overline } \newcommand {\surfaceColor }[0]{violet} \newcommand {\surfaceColorTwo }[0]{redyellow} \newcommand {\sliceColor }[0]{greenyellow} \newcommand {\vector }[1]{\left \langle #1\right \rangle } \newcommand {\sectionOutcomes }[0]{} \newcommand {\HyperFirstAtBeginDocument }[0]{\AtBeginDocument } \newcommand {\epstopdfsetup }[0]{\setkeys {ETE}} changed the theorem header of amsthm failed\MessageBreak }}} \newcommand {\GetTitleStringSetup }[0]{\setkeys {gettitlestring}} \newcommand {\Sectionformat }[2]{#1}$

We introduce the gradient vector.

The gradient is a useful tool in vector calculus. In some sense, it has the role that the derivative has in single-variable calculus. Let’s state the definition:

Let $F:\R ^n\to \R$ be a differentiable function, the gradient $\grad F = \vector {\pp [F]{x_1},\pp [F]{x_2},\dots ,\pp [F]{x_n}}$ is a vector-valued function of $n$ variables.

Try your hand at some casual computations.

Let $F(x,y) = \sin (x)\cos (y)$ , compute: $\grad F(x,y) \begin {prompt} = \vector {\answer {\cos (x)\cos (y)},\answer {-\sin (x)\sin (y)}} \end {prompt}$

Let $\vec {p}= \vector {\pi /3,\pi /3}$ . Compute: $\grad F(\vec {p}) \begin {prompt} =\vector {\answer {1/4},\answer {-3/4}} \end {prompt}$

And now in three variables.

Let $F(x,y,z) = ze^{-7xy}$ , compute: $\grad F(x,y,z) \begin {prompt} = \vector {\answer {z e^{-7xy}(-7)y},\answer {z e^{-7xy}(-7)x}, \answer {e^{-7xy}}} \end {prompt}$

Let $\vec {p}= \vector {1,0,1/7}$ . Compute: $\grad F(\vec {p}) \begin {prompt} =\vector {\answer {0},\answer {-1},\answer {1}} \end {prompt}$

Now that we can compute the gradient, let’s see if we can figure out what it means.

The initial greatest increase

First recall what it means for a function to be differentiable:

Given a function $F:\R ^3\to \R$ and a vector $\vec {a}$ in the domain of $F$ , if one can “zoom in” on the graph at $(\vec {a}, F(\vec {a}))$ sufficiently so that it appears to be a plane, then the function is differentiable, and that plane is the tangent plane to $F$ at the point $(\vec {a},F(\vec {a}))$ .

Now let’s imagine what the gradient is telling us about the plane. First let $\vec {p}=\vector {a,b}$ and consider the tangent plane: $z = F^{(1,0)}(\vec {p}) (x-a) + F^{(0,1)}(\vec {p})(y-b) + F(\vec {p})$

If a component of $\grad F(\vec {p})$ is positive, traveling in its direction will ensure that you are “traveling uphill,” meaning you are raising the $z$ -value (at least initially).
If a component of $\grad F(\vec {p})$ is negative, traveling in its (negative) direction will also ensure that you are “traveling uphill,” meaning you are raising the $z$ -value (at least initially).

Expressing the tangent plane in terms of vectors $\vec {x} =\vector {x,y}$ and the dot product, we see $z = \grad F(\vec {p})\dotp (\vec {x}-\vec {p}) + F(\vec {p})$ From this we see that the components of $\grad F(\vec {p})$ form a vector that simultaneously points “most uphill” in the $x$ -direction, and “most uphill” in the $y$ -direction. The upshot?

The gradient points in the initial direction of the greatest increase of the function.

You may be wondering, “Why does the gradient only point in the ‘initial’ direction of greatest increase?” It can only be “initial” because the reasoning above relies on the fact that we can be “zoomed-in” enough for the surface to look like a plane. For your viewing pleasure, we have included a graph where you can see the $x$ -component of the gradient and the $y$ -component of the gradient combining to give the initial the direction one should leave from a point, and find the initial greatest increase in the function.

Given a function $F:\R ^2\to \R$ , the vectors $\grad F(x,y)$ are only pointing in two dimensions. Consider $z = F(x,y) = 3x+4y.$ This is an explicit function that can be described as a surface in three-dimensional space, yet the gradient vector $\grad F = \vector {3, 4}$ is a vector in $\R ^2$ , meaning two-dimensional space. Similarly, if you are working with a function $F:\R \to \R ^n$ , this can be described as a “surface” in $(n+1)$ -dimensional space, but the gradient vectors $\vector {\pp [F]{x_1},\pp [F]{x_2},\dots ,\pp [F]{x_n}}$ are vectors in $n$ -dimensional space.

The gradient can help us understand how a function changes in a particular direction.

The directional derivative is computed by $D_\vec {\hat {u}}(F) = \grad F(\vec {x})\dotp \vec {\hat {u}}$ where $\vec {\hat {u}}$ is a unit vector.

The directional derivative tells us how $F$ changes if we move one direction. If we want to make $D_\vec {\hat {u}}(F)$ as large as possible, it makes sense to let $\vec {\hat {u}}$ be the direction of gradient since the dot product $\grad F(\vec {x})\dotp \vec {\hat {u}}$ is largest when $\vec {\hat {u}}$ is in the same direction as $\grad F$ .

Let $F(x,y) = -x^2+2x-y^2+2y+1.$ Compute and interpret $\grad F(1,1)$ .

Write with me $\grad F(x,y) = \vector {\answer [given]{-2x+2},\answer [given]{-2y+2}}.$ However, we see $\grad F(1,1) = \vector {\answer [given]{0},\answer [given]{0}}$ . Since there is no initial direction of greatest increase, we must be at a local maximum for the function. Indeed we are, behold:

Ah! The point $(1,1)$ lies at the top of a paraboloid. In all directions, the instantaneous rate of change is $0$ .

Stand back. We’re going to do some serious calculus now. Just read, relax and enjoy.

Consider the surface given by $F(x,y)= 20-x^2-2y^2$ :

Water is poured on the surface at $(1,1/4)$ . What path does it take as it flows downhill?

Let $\vec {w}(t) = \vector {x(t), y(t)}$ be the vector-valued function describing the path of the water in the $(x,y)$ -plane. We seek $x(t)$ and $y(t)$ . We know that water will always flow downhill in the initial steepest direction. Therefore, at any point on its path, it will be moving in the direction of $-\grad F(x,y)$ We’ll ignore the physical effects of momentum on the water. Thus $\vec {w}(t)$ will be parallel to $\grad F$ . Ah! This means there is some constant $c$ such that $c\grad F(x(t),y(t)) = \vec {w}'(t) = \vector {x'(t), y'(t)}.$ Computing the gradient, $\grad F(x(t),y(t)) = \vector {-2x(t), -4y(t)}$ Then $\begin{align*} c\cdot \grad F(x(t),y(t)) &= \vector{ x'(t), y'(t)}\\ c\cdot \vector{-2x(t),-4y(t)} &= \vector{ x'(t), y'(t)}\\ \vector{-2cx(t),-4cy(t)} &= \vector{ x'(t), y'(t)}\\ \end{align*}$ This implies $-2cx(t) = x'(t) \quad \text {and} \quad -4cy(t) =y'(t)$ so $c = -\frac {x'(t)}{2x(t)} \quad \text {and} \quad c =-\frac {y'(t)}{4y(t)}.$ Now recall that the differentials $\d x = x'(t) \d t$ , and $\d y=y'(t)\d t$ , so we may write $\begin{align*} \int \frac{1}{2x}x'(t)\d t &=\int \frac{1}{4y} y'(t)\d t \\ \int \frac{1}{2x}\d x &=\int\frac{1}{4y}\d y \\ \frac{1}{2}\ln|x| +C &= \frac{1}{4}\ln|y|\\ 2\ln|x| + C &= \ln|y|\\ \ln|x^2| + C &= \ln|y| \end{align*}$ Raising $e$ to the left-hand and right hand sides, we see $\begin{align*} e^{\ln|x^2| + C} &= e^{\ln|y|}\\ x^2\cdot e^C + C &= \ln|y|, \end{align*}$ setting $K = e^C$ , we write $K\cdot x^2 = y.$ We are so close to being done, $y=K\cdot x^2$ , this is the path described in the $(x,y)$ -plane. Since the water started at the point $(1,1/4)$ , we can solve for $K$ : $K\cdot 1^2 = \frac 14 \quad \Rightarrow \quad K = \frac 14.$ Thus the water follows the curve $y=x^2/4$ in the $(x,y)$ -plane.

Perpendicularity and the gradient

Now that we know gradient vectors point in the direction of the greatest initial increase of the function, let’s learn about their geometry. Consider again a plane. This time we’ll think about $z = F(x,y) = x+y$ This plane increase with both the $x$ -values and $y$ -values and passes through the line $y=-x$ in the $(x,y)$ -plane.

Let’s look at a contour plot:

Computing the gradient we find $\grad F(x,y) = \vector {1,1}$ This vector is perpendicular to all the level curves.

This is not an accident, rather it is a general rule. Gradient vectors point in the initial direction of greatest increase. Every point on the level curve $1 = x+y$ gives the same $z$ -value. Because of this if the gradient pointed any direction other than, “directly away,” and hence perpendicularly, from the level curve, it would not be pointing in the initial direction of greatest increase. The up-shot?

Gradient vectors are always orthogonal to level sets.

The fact that the gradient is always orthogonal to level surfaces is very powerful. In fact it gives new (easier!) solutions to old problems. Let’s use this fact to find a plane tangent to a surface.

Find an implicit equation for the tangent plane to the elliptic paraboloid $z = x^2 + y^2$

at $\vec {p} = \vector {2,3,13}$ .

Consider $F(x,y,z) =x^2 +y^2 -z$ and imagine the elliptic parabolid as the level surface $F(x,y,z) = \answer [given]{0}$ Remember, the gradient is perpendicular to level surfaces. We’ll use this fact to find a normal vector to the surface, and with this vector we’ll find the tangent plane. The gradient is: $\begin{align*} \grad F(x,y,z) &= \vector{\pp[F]{x}, \pp[F]{y},\pp[F]{z}}\\ &= \vector{\answer[given]{2x},\answer[given]{2y}, \answer[given]{-1}}. \end{align*}$ Since this vector is normal to the surface, we can use it to find an implicit formula for the tangent plane to the surface by computing $\vec {n}\dotp (\vec {x}-\vec {p}) = 0$ where $\vec {p} = \vector {2,3,13}$ and $\begin{align*} \vec{n} &= \grad F(\vec{p})\\ &=\vector{\answer[given]{4}, \answer[given]{6},\answer[given]{-1}} \end{align*}$ Thus the equation of the plane tangent to the ellipsoid at $\vec {p}$ is: $\answer [given]{4}(x-2) + 6(y-3) - (z-\answer [given]{13}) = \answer [given]{0}$

Finally, sometimes we are given discrete data that can be adequately described and approximated by differentiable functions. In this case, the concept of the gradient will lead you to success.

Let $F:\R ^2\to \R$ be a differentiable function described by the following table of values:

Estimate $\grad F(3,5)$ .

We estimate $\grad F(3,5)$ by estimating the partial derivatives. To estimate $F^{(1,0)}(3,5)$ , we examine the change in $F(x,5)$ between $x=4$ and $x=3$ : $\frac {F(4,5)-F(\answer [given]{3},5)}{\answer [given]{4}-3}= \answer [given]{-1}$ We should also examine the change in $F(x,5)$ between $x=3$ and $x=2$ : $\frac {F(3,5)-F(\answer [given]{2},5)}{\answer [given]{3}-2} =\answer [given]{-5}$ Now if we average these values together, we see: $\eval {\pp {x} F(x,y)}_{(x,y)=(3,5)} \approx \answer [given]{-3}$ On the other hand, using a similar procedure, we find that: $\eval {\pp {y} F(x,y)}_{(x,y)=(3,5)} \approx \answer [given]{4}$ Thus the gradient is $\grad F(3,5) = \vector {\answer [given]{-3},\answer [given]{4}}$ Note if you leave the point $(3,5)$ in the direction of $\grad F(3,5) = \vector {\answer [given]{-3},\answer [given]{4}}$ , you head directly toward $F(2,6)= 16$ , the greatest initial increase from $(3,5)$ .