The Fourier Series via Linear Algebra

I didn't post last week because I was on vacation. But on vacation I decided to write about something a little out of my comfort zone: Fourier series. (Yeah. Try being my wife.)

Fourier series (and the related Fourier transform) made some sense to me for, but I never really learned how to derive them so they always seemed a bit magical. As I was going through Arthur Mattuck's excellent differential equations course at MIT's Open Courseware, the Fourier series clicked for me, so I thought I'd distill this out.

I'm tempted to categorize this post under "things Isaac should have learned in school (had he been paying attention), but I don't think I ever had a course that taught this. If you're still paying attention, I will assume that you recall your basic linear algebra and have some idea (and interest in) what a Fourier series is. But given that, this isn't hard.

It's All Linear Algebra

Recall that the Fourier series represents periodic functions as a sum of sines and cosines. In this post, we'll deal with functions with a period that lies in the interval $ [-\pi, \pi]$. The generalization to arbitrary periods is straightforward, but this interval illustrates the scheme well.

The fundamental "click" for me was that this was all linear algebra. The Fourier series:

1. Looks at functions over an interval as a vector space with an inner product;
2. Picks an orthonormal basis for the space; and
3. Represents an arbitrary function in this basis by projecting it out on the basis.

Given that we're working over the interval $[-\pi, \pi]$, we'll define a vector space $V$ where the scalars are taken from $ \mathbb{R}$ and the vectors are functions over the interval $ [-\pi,\pi].$ In addition, we'll define an inner product by:

$\langle f,g\rangle = \frac{1}{2\pi}\displaystyle\int_{-\pi}^\pi f(x)g(x) dx$

I'll leave it to you to check that these meet the requirements of a vector space and an inner product. This is worth doing, particularly if using functions as vectors seems odd to you.

In any case, that's all for step 1.

Step 2: Choose an Orthonormal Basis

We're going to choose an orthonormal basis $ S$ for our vector space. The vectors of $ V$ are functions, so our basis will be a set of functions. The basis will be infinite, meaning that $ V$ is infinite dimensional.

To build our basis, we're going to use the constant function $ 1(x)$ as well as sines and cosines of a positive integral multiple of $ x$. In particular, our basis is:

$ S = \begin{Bmatrix}\vphantom{\displaystyle\int}\sqrt{2}\sin(x), \sqrt{2}\sin(2x), \sqrt{2}\sin(3x), ...,
\\ \vphantom{\displaystyle\int}\sqrt{2}\cos(x), \sqrt{2}\cos(2x), \sqrt{2}\cos(3x), ...,
\\ \vphantom{\displaystyle\int}1(x)\end{Bmatrix}$

For these to be an orthonormal basis, we first have to show that any two of these are orthogonal. I.e., that for all $ f,g \in S$ with $ f\ne g, \int_{-\pi}^\pi f(x)g(x) dx = 0$. There are a few cases to check:

• If one of $ f(x)$ and $ g(x)$ is $ 1(x)$ then the inner product is just an integral of a sine or cosine function over a whole number of periods, which is zero.
• If $ f = \sin(kx)$ and $ g = \cos(nx)$, then $ f(x)g(x)$ is an odd function, so the integral is zero.
• If both $ f(x)$ and $ g(x)$ are both sines (or cosines) with different coefficients, then this is also zero. As Mattuck suggests, you can work this out via complex exponentials or trig identities; he does it in his lecture via differential equations and a nice symmetry argument.

We don't need to show linear independence separately because it's implied by orthogonality. But we do need to check that all of the vectors have unit length:

• For $ 1(x)$, we find that $|1(x)|^2 = \langle 1(x),1(x)\rangle = \frac{1}{2\pi}\int_{-\pi}^\pi 1 dx = 1$.
• For $ \sqrt{2}\sin(kx)$, we find that $ |\sqrt{2}\sin(kx)|^2 = \frac{1}{2\pi}\int_{-\pi}^\pi 2\sin^2(kx) dx = 1$. So our sine vectors are normalized.
• This same argument holds for the cosines.

So $ S$ is an orthonormal basis. We haven't shown that $ S$ actually spans the space $ S$ of functions. I'll mention this again, but for now, it's sufficient to know that it spans some space.

Step 3: Represent a Function in this Basis

Now that we have a basis, we can take an arbitrary vector $ F(x)$ and write it as a linear combination of the basis vectors:

$ F(x) = \displaystyle\sum_{s\in S} t_s s$

All we need to do is figure out the constants $ t_s$. But since $ V$ is an inner product space, we can find the coefficient $ t_s$ for each $ s\in S$ as:

$ t_s = \langle F(x),s\rangle = \frac{1}{2\pi}\displaystyle\int_{-\pi}^{\pi}F(x)s(x)dx$

Instead of using $ t_s$, we'll use $ a_k$ as the constant term for the $ \cos(kx)$ term, and $ b_k$ for the $ \sin(kx)$ term. For now, we'll use $ c$ as the constant for the $ 1(x)$ term.

Now:

$ \begin{array}{rcl}
F(x) &=& \displaystyle\sum_{s\in S} t_s s = \displaystyle\sum_{s\in S} \langle F(x),s\rangle s\\
&&\\
&=&c*1(x) + \displaystyle\sum_{k=1}^\infty a_k \cos(kt) + b_k \sin(kt)
\end{array}$

Where:

$ \begin{array}{rcl}\vphantom{\displaystyle\int}a_k & = & \frac{1}{\pi}\int_{-\pi}^{\pi}F(x)\cos(kx)dx \\
\vphantom{\displaystyle\int}b_k & = & \frac{1}{\pi}\int_{-\pi}^{\pi}F(x)\sin(kx)dx  \\
\vphantom{\displaystyle\int}c & = & \frac{1}{2\pi}\int_{-\pi}^{\pi}F(x)dx \end{array}$

And that's the Fourier series. Note that we've folded in two $ \sqrt{2}$ factors into the the $ a$ and $ b$ terms, which is why they are missing the leading $ \frac{1}{2}$.

This isn't quite the form you usually see. We can tweak this slightly by noticing that $ 1(x) = \cos(0x)$, so we can replace that constant term and its associated constant with a cosine term—we just have to watch out for that $ \frac{1}{2}$. Doing so, we get the usual formulation:

$ F(x) = \frac{1}{2}a_0 + \displaystyle\sum_{k=1}^\infty a_k \cos(kt) + b_k \sin(kt)$

Where the $ a$ and $ b$ terms are as above.

A Few Loose Ends

One thing I glossed over is the space spanned by our basis. A quick counting argument shows that there are

$ |V| = \mathfrak{c}^{\mathfrak{c}} = 2^{2^{\aleph_0}}$

functions in our vector space, but only

$ |\text{span}(S)| = \mathfrak{c}^{\aleph_0} = (2^{\aleph_0})^{\aleph_0} = 2^{\aleph_0}$

functions in the span of our "basis". Since $ 2^{\aleph_0} < 2^{2^{\aleph_0}} $, our basis doesn't actually span the space of functions. We're missing a lot of functions—but which ones? It turns out that our basis actually spans the set $ L^2$ of square integrable functions, but I'm afraid that showing this is still beyond me.

Also still beyond me is the extension of this to the full real line: the Fourier transform. Hopefully I'll be back sometime to explain that one, too.