Functions of several variables: gradients, Hessians and integrals

Partial derivatives

Definition. The partial derivative $\frac{\partial f}{\partial x}$ (also written $f_x$) is the derivative with respect to $x$, holding every other variable fixed.

Example 1. For $f(x, y) = x^3y^2 + e^{xy}$:

• $f_x = 3x^2y^2 + ye^{xy}$;
• $f_y = 2x^3y + xe^{xy}$;
• $f_{xy} = \frac{\partial}{\partial y}f_x = 6x^2y + e^{xy} + xye^{xy}$.

Computing $f_{yx}$ gives the same result. Clairaut's theorem: when the mixed partials are continuous, $f_{xy} = f_{yx}$, so the Hessian is symmetric.

Exam trap. A function can have both partial derivatives at a point without being continuous there. Take $f = \frac{xy}{x^2 + y^2}$ with $f(0, 0) = 0$. Both partials at the origin are 0, yet along the line $y = x$ the function equals $\frac12$. Differentiability in several variables needs more than the existence of partials. Continuous partials are enough.

The gradient and directional derivatives

Gradient. $\nabla f = \left(\frac{\partial f}{\partial x_1}, \dots, \frac{\partial f}{\partial x_n}\right)$.

Directional derivative. The rate of change in the direction of a unit vector $u$ is

$D_u f = \nabla f \cdot u = \lVert\nabla f\rVert\cos\theta .$

Consequences.

• $f$ increases fastest in the direction of $\nabla f$, at the rate $\lVert\nabla f\rVert$.
• It decreases fastest in the direction of $-\nabla f$. This is why gradient descent works.
• Along directions perpendicular to $\nabla f$, the rate is 0.
• So the gradient is perpendicular to the level curves (or level surfaces).

Example 2. $f(x, y) = x^2y + 3y$ at $(1, 2)$.

1. $\nabla f = (2xy,\ x^2 + 3) = (4, 4)$.
2. In the direction of $(3, 4)$, the unit vector is $u = (0.6, 0.8)$, so $D_u f = 2.4 + 3.2 = 5.6$.
3. The fastest increase is $\lVert(4, 4)\rVert = 4\sqrt2 \approx 5.66$, in the direction $(1, 1)/\sqrt2$.

Exam trap. Always normalise the direction first. Using $(3, 4)$ itself would give 28, which is wrong.

Tangent plane. At $a$, the graph $z = f(x)$ is best approximated by $z = f(a) + \nabla f(a)\cdot(x - a)$.

Example 3. For $f = x^2 + y^2$ at $(1, 2)$: $f = 5$ and $\nabla f = (2, 4)$. The tangent plane is $z = 5 + 2(x - 1) + 4(y - 2)$. At $(1.1, 2.1)$ it predicts $5.6$; the true value is $5.62$.

The chain rule in several variables

If $z = f(x, y)$ with $x = x(t)$ and $y = y(t)$, then

$\frac{dz}{dt} = \frac{\partial f}{\partial x}\frac{dx}{dt} + \frac{\partial f}{\partial y}\frac{dy}{dt}.$

Example 4. $z = xy$ with $x = t^2$ and $y = e^t$.

• The chain rule gives $\frac{dz}{dt} = y \cdot 2t + x \cdot e^t = 2te^t + t^2e^t$.
• Check. Substituting first, $z = t^2 e^t$, and differentiating gives the same result.