Chapter 1: The history of differential equations

<blockquote John Bernoulli to his brother, James.>My brother must be extremely conceited since he believes that I am incapable of solving the problems he has solved; but if I was in the mood to do the same to him, I could come up with questions so subtle and so unusual that he would spend his whole life on them to no avail, and yet I have solved them very easily. </blockquote>

Introduction

There are many ways we can begin our study of differential equations. We can, for example, simply define an ordinary differential equation (ODE) of the $n^\text{th}$ order according to the form \begin{equation} \label{eq:intro_G} G\Bigl( x, y, y', y'', y''', \ldots, y^{(n)} \Bigr) = 0, \end{equation}

to be solved for some unknown function $y(x)$. Then we can jump straight into learning about the available techniques for solving ODEs and their theory. Do solutions of (\ref{eq:intro_G}) exist and if so, are they unique? Do they oscillate like sinusoidals, are they polynomial, or do they grow and decay like exponentials? This is a very mathematical perspective—focus on a unified theory and work towards the practical.

Alternatively, we can begin from a practical perspective. Beginning from physical principles (such as Newton's Law), we can derive the relevant differential equations for modeling a physical problem. A typical goal would be to derive the equation governing the motion of a pendulum and to solve it. Any mathematical techniques we require will then be introduced in order to fit the situation.

In this lecture, we shall start learning about ODEs from a historical perspective. In this way, we will cover both the development of differential equations from the physics, and also see how the mathematics emerged in conjunction. Most of this lecture follows from Chap. 21 in [1].

First-order differential equations

Perhaps the first well-documented problem which initiated the study of differential equations was the famous tautochrone problem, posed by Huygens in 1693: what is the curve along which it takes the same amount of time for a particle moving under the influence of gravity to fall to the bottom, irrespective of its initial position. (Though Huygens had originally posed the question in a different context: what is the curve along which a pendulum must swing so that it performs a complete oscillation in the same amount of time, regardless of amplitude).

In 1690, James Bernoulli published what is arguably the first paper on differential equations in order to solve the tautochrone problem (in addition, he mentions the word `integral' for the first time in history). The differential equation he proposed, written in a simplified form was \begin{equation} \label{eq:intro_iso} \frac{dy}{dx} = \sqrt{\frac{y}{a-y}}, \end{equation}

for some constant, $a$. The solution of this ODE produces curves known as cycloids (or isochrones). Let us take a moment's pause and solve our first differential equation!

Example (Isochrone problem as an ODE)

We solve the isochrone problem (\ref{eq:intro_iso}) by rearranging the equation: \[ \sqrt{\frac{a-y}{y}} \, \frac{dy}{dx} = 1. \]

We have functions of $x$ on the left and on the right. We thus reason that, since the functions are equal, their integrals (area under their curves) must also be equal: \begin{equation} x = \int \sqrt{\frac{a-y}{y}} \, \frac{dy}{dx} \, dx = \int \sqrt{\frac{a-y}{y}} \, dy. \end{equation}

The integral on the right requires making the trigonometric substitution of $y = a \sin^2 (\theta/2)$, and once substituted, we get \[ x = \frac{a}{2} (\theta + \sin\theta) + C, \]

for some constant $C$. Different values of $C$ will give different curves, only shifted in the horizontal direction. We shall set $C = 0$ to examine the solution which has $y(0) = 0$. This gives the cycloid (or isochrone) as described by a parametric equation \[ x = \frac{a}{2}(\theta + \sin\theta) \qquad \text{and} \qquad y = \frac{a}{2} (1-\cos \theta). \]

The method we just used is called separation of variables, and we will go over the method in more detail in the next lecture.


In the same paper, James Bernouilli posted the problem of finding the curve assumed by an inelastic cord hung freely between two fixed points (a catenary). In 1691, Leibniz, Huygens, and John Bernoulli all published solutions to this problem. It is given by the differential equation \begin{equation} \frac{dy}{dx} = \frac{1}{c} \int_0^x \sqrt{1 + (y')^2} \ dt. \end{equation}

James and John Bernoulli would go on to solve various other `hanging-cord' problems in 1691 and 1692, and in 1696, the famous brachistochrone problem was posed by John: <quote> I, Johann Bernoulli, address the most brilliant mathematicians in the world. Nothing is more attractive to intelligent people than an honest, challenging problem, whose possible solution will bestow fame and remain as a lasting monument. Following the example set by Pascal, Fermat, etc., I hope to gain the gratitude of the whole scientific community by placing before the finest mathematicians of our time a problem which will test their methods and the strength of their intellect. If someone communicates to me the solution of the proposed problem, I shall publicly declare him worthy of praise. </quote>

The problem he posed was the following: <quote> Given two points A and B in a vertical plane, what is the curve traced out by a point acted on only by gravity, which starts at A and reaches B in the shortest time. </quote>

Five mathematicians from four countries would send in solutions: Newton (England), Jacob Bernoulli (Switzerland), Leibniz and Von Tschirnhaus (Germany) and de L'Hopital (France). The solution, it turns out, is simply a particular cycloid (except with particular constants related to the physical constants).

In the late 1690s, the mathematicians had turned their interest to the subject of orthogonal trajectories: finding the curve or family of curves that cut a given family of curves at a right angle. This problem has wide applicability to problems in physics (e.g. light rays cut through a changing medium). Here is an example of the problem considered by Liebniz:

Example (Orthogonal trajectories)

Let $\mathcal{F}_1$ be the family of curves generated by $y^2 = 2bx$, where $b \in \mathbb{R}$. What is the family, $\mathcal{F}_2$, of curves such that each element of $\mathcal{F}_2$ is orthogonal to each element of $\mathcal{F}_1$ at an intersection point?

First, we note that the slope of the curves in $\mathcal{F}_1$ is \[ \frac{dy}{dx} = \frac{b}{y} = \frac{y}{2x}. \]

Thus, if $\overline{y}(x)$ denotes those curves in $\mathcal{F}_2$, we need \[ \frac{d\overline{y}}{dx} = -\frac{2x}{\overline{y}}. \]

We shall use the same technique as before (separation of variables) and integrate both sides of the equation \[ \int \overline{y} \, \frac{d\overline{y}}{dx} = \int -2x \, dx \quad \Rightarrow \quad \frac{1}{2}\overline{y}^2 = -x^2 + C, \]

for some constant $C$. Different choices of $C$ generate different ellipses, forming the family $\mathcal{F}_2$. An example of one of these orthogonal trajectories is shown in the figure. There, curves in $\mathcal{F}_1$ are shown solid and curves in $\mathcal{F}_2$ are shown dashed. Any given element of $\mathcal{F}_2$ intersects any curve in $\mathcal{F}_1$ at a right angle.

Orthogonal curves

The next important topic of the early 1700s concerned exact equations and integrating factors. The principal investigators of these problems were Euler (1734-35) and Clairaut (1739-40). An example of such an equation is \begin{equation} 2x + y^2 + 2xy \frac{dy}{dx} = 0. \end{equation}

There are other subtleties of first-order differential equations we shall not discuss (one is the nature of singular equations such as $y = xy' +f(y')$—known as Clairaut equations). However, by 1740, all the elementary methods of solving first-order ODEs had been discovered, and so we will go onwards to the subject of second-order equations.

Second-order differential equation and series solutions

Second-order ODEs had already arisen earlier (for example, James Bernoulli treated a second-order equation in his modeling of the velaria, that is, shape of a sail under the pressure of the wind), but the great majority of work in the 18th century towards the theory was devoted to understanding pendulums and oscillatory motions.

In introductory physics and engineering classes, you are taught about two principal systems of mechanical vibrations. The first is a mass-spring-damper system, where a mass $m$ is connected to a spring with constant $k$ and forced by $F(t)$. The spring is additionally damped by $\zeta$ and its displacement, $x(t)$ follows \begin{equation} m \frac{d^2 x}{dt^2} + \zeta \frac{dx}{dt} + kx = F(t). \end{equation}

The second example is the motion of a pendulum with mass $m$ and length $\ell$, with air damping $\zeta$. The angular displacement, $\theta(t)$ is then \begin{equation} m\ell \frac{d^2\theta}{dt^2} + \zeta \ell \frac{d\theta}{dt} + mg\sin\theta = 0. \end{equation}

These are both examples of second-order ODEs, and certainly, the work of the Bernoullis, Euler, Ricatti, and others were all intrinsically tied to similar equations. However, in actuality, the study of second-order ODEs proceeded in much `messier' lines than for first-order ODEs, partly due to the fact that very few elementary solutions exist, except for the simplest classes of equations.

Consider as an example, the Bessel equation, for which a version had already been introduced by Daniel Bernoulli in 1733 to describe the oscillations of a weighted chain set into vibration: \begin{equation} \label{eq:intro_bessel} \frac{d^2 y}{dx^2} + \frac{1}{x} \frac{dy}{dx} + \left(\alpha^2 - \frac{\beta^2}{x^2} \right) y = 0, \end{equation}

for constants $\alpha$ and $\beta$. In order to study these equations, the idea of series solutions were developed (largely spurred by Euler). Let us go through a quick example of how series expansions are done.

Example (Bessel equation)

Consider the Bessel equation (\ref{eq:intro_bessel}). We assume that $y$ can be expanded into a series about $x = 0$: \[ y(x) = Ax^n + Bx^m + \ldots, \]

where $0 < n < m$ (why?). Substitution into the equation gives \[ \biggl[ n(n-1) Ax^{n-2} + \ldots \biggr] + \frac{1}{x} \biggl[ n Ax^{n-1} + \ldots \biggr] + \biggl[ -\frac{\beta^2}{x^2} + \ldots \biggr] \biggl[ Ax^n + \ldots \biggr] = 0. \]

The dots represent terms which are smaller around $x = 0$. If we drop the smaller terms, we get \[ n(n-1) + n - \beta^2 = 0, \]

which is solved to give $n = \pm \beta$. Thus, we would say that the solution of (\ref{eq:intro_bessel}) behaves like \[ y(x) = Ax^\beta + \ldots \qquad \text{or} \qquad y(x) = Ax^{-\beta} + \ldots \]

Astronomy and the three-body problem

One of the great physical motivations towards developing the theory of differential equations was in order to better understand the motion of two or more bodies, each moving under the gravitational attraction of the other.

Consider two point unit masses, located at coordinates $\textbf{x}_1 = [x_1, y_1, z_1]$ and $\textbf{x}_2 = [x_2, y_2, z_2]$. If $r$ is the distance between the two masses, then by Newton's law, we have \begin{equation} \frac{d^2 \textbf{x}_1}{dt} = \frac{\textbf{x}_1 - \textbf{x}_2}{r^3} \qquad \frac{d^2 \textbf{x}_2}{dt} = \frac{\textbf{x}_2 - \textbf{x}_1}{r^3}. \end{equation}

This gives a system of six second-order equations, which can be solved once we supply the initial position and velocities of both bodies (yielding twelve initial conditions). In fact, Daniel Bernoulli had solved this two-body problem in 1734, and had shown that the bides move in a conic section with respect to the common center of mass.

The problem of $n$ bodies, and most regrettably, for three bodies, cannot be solved exactly. Thus to make progress, one is required to proceed in two different ones: the first way is to derive general mathematical results which can shed light on important properties of (restricted classes) of solutions; this endeavor was most taken to heart by the work of Lagrange in around 1772.

The second approach is to derive approximate solutions, and this idea applied in problems in celestial mechanics led to the birth of what is today known as `perturbation theory'. The idea behind this theory is to introduce small perturbations to a solution that you already know.

Summary

Date Problem Description Mathematician
1690 Isochrone problem Finding a curve along which a body will fall with uniform vertical velocity James Bernoulli
1696 Brachistochrone problem Finding the path down which a particular will fall from one point to another in the shortest time John Bernoulli
1698 Orthogonal trajectories & Finding the curve(s) which cuts a family of curves at right angles John Bernoulli
1713-1733 Vibrations of strings Deriving the shape and fundamental frequencies of a string (possibly loaded with a distribution of weights) Brook Taylor
John Bernoulli
Daniel Bernoulli
Euler
1739 Harmonic oscillators Modeling the motion of forced pendulums and springs Euler et al.
1743, 1750 Linear homogeneous and inhomogeneous ODEs of $n^\text{th}$ order Finding a unified methodology which can be used to solve the class of equations Euler
1764-69 Series solutions (Hypergeometric {\amper} Bessel series) Solving differential equations by writing the solution as an infinite series Euler
(1734) Two-body problem Predicting the motion of two bodies subjected to mutual gravitational attraction Daniel Bernoulli
1772 Three-body problem
(General theorems)
Producing general theorems which can explain the motion of three bodies subjected to gravitational attraction Lagrange
1770s Three-body problem
(Approximations)
Producing approximations to solutions of the three-body problem Clairaut
Euler
Laplace

As you have seen, part of the difficulty in summarizing a history of differential equations is that history proceeds in a pedagogically nonlinear fashion. The subject did not evolve moving from simple ODEs to more difficult ODEs, but rather, in response to fashionable trends and problems. We will `clean-up' this treatment beginning in our next lecture when we look at systematic techniques for solving first-order differential equations.