Taming the Beast: Conquer Differential Calculus with Confidence (Part one)

Posted on | Categories: Mathematical

Differential Calculus, what’s it About?

Differential calculus can be thought of as the study of how a small change (or perturbation) in the input affects the output.

For instance, suppose a factory is set up to manufacture coins with radius $1$ cm, but due to a machine’s imprecision, it actually produces coins of radius $1 + \varepsilon$ cm.

What is the effect of this perturbation on the area of the coin? The calculation

\[
\pi(1 + \varepsilon)^2 = \pi + 2\pi\varepsilon + \pi\varepsilon^2
\]

tells us that an error of $\varepsilon$ in the radius causes an error of $2\pi\varepsilon + \pi\varepsilon^2$ in the area.

From One Variable to Several Variables

For a function $f : I \to \mathbb{R}$, we study how it changes around a point $x_0$ using the ratio

\[
\frac{\Delta y}{\Delta x} = \frac{\text{Change in output}}{\text{Change in input}},
\]

and then examine the limit:

\[
\lim_{\Delta x \to 0} \frac{\Delta y}{\Delta x}
\]

If this limit exists, we say that $f$ is differentiable at the point $x_0$. In our earlier example, at $x_0 = 1$, the limit equals $2\pi$. However, when $f$ takes a vector input, say:

\[
f : \mathbb{R}^2 \to \mathbb{R},
\]

then the ratio $\frac{\Delta y}{\Delta x}$ is no longer defined, since $\Delta x$ is a vector, and we cannot divide by a vector!

Variation in Multiple Directions

Let us fix a function $f : U \to \mathbb{R}$, where $U \subset \mathbb{R}^2$ is open. A natural idea is to study how $f$ changes along the $x$-axis and the $y$-axis:

\[
\lim_{\Delta x \to 0} \frac{f(x_0 + \Delta x, y_0) – f(x_0, y_0)}{\Delta x}
\quad \text{and} \quad
\lim_{\Delta y \to 0} \frac{f(x_0, y_0 + \Delta y) – f(x_0, y_0)}{\Delta y}
\]

These are partial derivatives, a return to the univariate case. We can go further and compute the change of $f$ along any direction $v = (v_x, v_y) \in \mathbb{R}^2$:

\[
\lim_{t \to 0} \frac{f(x_0 + tv_x, y_0 + tv_y) – f(x_0, y_0)}{t}
\]

These are called directional derivatives, and denoted by $D_v(f)(x_0,y_0)$. They describe the slope of the surface in the direction of the vector $v$.

In this figure, when you approach the point $(x_0,y_0)$ through straight paths, then the point $f(x,y)$ will move on a curve contained on the surface (the colored ones in the picture).

A More General View: Linear Approximation

Directional derivatives work well, but they are limited to approaching a point along straight lines. Yet, there are many more paths leading to a point!

To generalize, we step away from the ratio-based view and use the first-order Taylor expansion. For a function $f : \mathbb{R} \to \mathbb{R}$, we write:

\[
f(x_0 + h) = f(x_0) + f'(x_0)h + o(h)
\]

where $o(h)$ denotes a term negligible compared to $h$. This approximation involves only multiplication and addition. Note that the map $h \mapsto f'(x_0)h$ is linear.

Differentiability in Higher Dimensions

In order to define differentiability of a multivariate function, we will extend the idea first order approximation.

Definition. A function $f : \mathbb{R}^n \to \mathbb{R}$ is differentiable at a point $\mathbf{x_0}$ if there exists a linear map $L_{\mathbf{x_0}} : \mathbb{R}^n \to \mathbb{R}$ such that:

\[
f(\mathbf{x_0} + h) = f(\mathbf{x_0}) + L_{\mathbf{x_0}}(h) + o(h)
\]

as $h \to 0$, with $o(h)$ being negligible compared to $h$.

This definition aligns with directional derivatives. In fact, if $f$ is differentiable at $\mathbf{x_0}$, then for any vector $v$, the directional derivative satisfies:

\[
D_v f(\mathbf{x_0}) = L_{\mathbf{x_0}}(v)
\]

The linear operator $L_{\mathbf{x_0}}$ is often denoted by the differential $df_{\mathbf{x_0}}$.

Conclusion

Differential calculus begins with small variations in one variable, then extends to multiple variables using partial and directional derivatives. Ultimately, the most natural and general definition comes from linear approximations via the differential operator.

I would like to thank my dear friend Slim Rekik for the fruitful discussion.