The Fourier Series via Linear Algebra

I didn’t post last week because I was on vacation. But on vacation I decided to write about something a little out of my comfort zone: Fourier series. (Yeah. Try being my wife.)

Fourier series (and the related Fourier transform) made some sense to me for, but I never really learned how to derive them so they always seemed a bit magical. As I was going through Arthur Mattuck’s excellent differential equations course at MIT’s Open Courseware, the Fourier series clicked for me, so I thought I’d distill this out.

I’m tempted to categorize this post under “things Isaac should have learned in school (had he been paying attention), but I don’t think I ever had a course that taught this. If you’re still paying attention, I will assume that you recall your basic linear algebra and have some idea (and interest in) what a Fourier series is. But given that, this isn’t hard.

It’s All Linear Algebra

Recall that the Fourier series represents periodic functions as a sum of sines and cosines. In this post, we’ll deal with functions with a period that lies in the interval $[-\pi, \pi]$ . The generalization to arbitrary periods is straightforward, but this interval illustrates the scheme well.

The fundamental “click” for me was that this was all linear algebra. The Fourier series:

Looks at functions over an interval as a vector space with an inner product;
Picks an orthonormal basis for the space; and
Represents an arbitrary function in this basis by projecting it out on the basis.

Given that we’re working over the interval $[-\pi, \pi]$ , we’ll define a vector space $V$ where the scalars are taken from $\mathbb{R}$ and the vectors are functions over the interval $[-\pi,\pi].$ In addition, we’ll define an inner product by:

$\langle f,g\rangle = \frac{1}{2\pi}\displaystyle\int_{-\pi}^\pi f(x)g(x) dx$

I’ll leave it to you to check that these meet the requirements of a vector space and an inner product. This is worth doing, particularly if using functions as vectors seems odd to you.

In any case, that’s all for step 1.

Step 2: Choose an Orthonormal Basis

We’re going to choose an orthonormal basis $S$ for our vector space. The vectors of $V$ are functions, so our basis will be a set of functions. The basis will be infinite, meaning that $V$ is infinite dimensional.

To build our basis, we’re going to use the constant function $1(x)$ as well as sines and cosines of a positive integral multiple of $x$ . In particular, our basis is:

$S = \begin{Bmatrix}\vphantom{\displaystyle\int}\sqrt{2}\sin(x), \sqrt{2}\sin(2x), \sqrt{2}\sin(3x), ..., \\ \vphantom{\displaystyle\int}\sqrt{2}\cos(x), \sqrt{2}\cos(2x), \sqrt{2}\cos(3x), ..., \\ \vphantom{\displaystyle\int}1(x)\end{Bmatrix}$

For these to be an orthonormal basis, we first have to show that any two of these are orthogonal. I.e., that for all $f,g \in S$ with $f\ne g, \int_{-\pi}^\pi f(x)g(x) dx = 0$ . There are a few cases to check:

If one of $f(x)$ and $g(x)$ is $1(x)$ then the inner product is just an integral of a sine or cosine function over a whole number of periods, which is zero.
If $f = \sin(kx)$ and $g = \cos(nx)$ , then $f(x)g(x)$ is an odd function, so the integral is zero.
If both $f(x)$ and $g(x)$ are both sines (or cosines) with different coefficients, then this is also zero. As Mattuck suggests, you can work this out via complex exponentials or trig identities; he does it in his lecture via differential equations and a nice symmetry argument.

We don’t need to show linear independence separately because it’s implied by orthogonality. But we do need to check that all of the vectors have unit length:

For $1(x)$ , we find that $latex|1(x)|^2 = \langle 1(x),1(x)\rangle = \frac{1}{2\pi}\int_{-\pi}^\pi 1 dx = 1$.
For $\sqrt{2}\sin(kx)$ , we find that $|\sqrt{2}\sin(kx)|^2 = \frac{1}{2\pi}\int_{-\pi}^\pi 2\sin^2(kx) dx = 1$ . So our sine vectors are normalized.
This same argument holds for the cosines.

So $S$ is an orthonormal basis. We haven’t shown that $S$ actually spans the space $S$ of functions. I’ll mention this again, but for now, it’s sufficient to know that it spans some space.

Step 3: Represent a Function in this Basis

Now that we have a basis, we can take an arbitrary vector $F(x)$ and write it as a linear combination of the basis vectors:

$F(x) = \displaystyle\sum_{s\in S} t_s s$

All we need to do is figure out the constants $t_s$ . But since $V$ is an inner product space, we can find the coefficient $t_s$ for each $s\in S$ as:

$t_s = \langle F(x),s\rangle = \frac{1}{2\pi}\displaystyle\int_{-\pi}^{\pi}F(x)s(x)dx$

Instead of using $t_s$ , we’ll use $a_k$ as the constant term for the $\cos(kx)$ term, and $b_k$ for the $\sin(kx)$ term. For now, we’ll use $c$ as the constant for the $1(x)$ term.

Now:

$\begin{array}{rcl} F(x) &=& \displaystyle\sum_{s\in S} t_s s = \displaystyle\sum_{s\in S} \langle F(x),s\rangle s\\ &&\\ &=&c*1(x) + \displaystyle\sum_{k=1}^\infty a_k \cos(kt) + b_k \sin(kt) \end{array}$

Where:

$\begin{array}{rcl}\vphantom{\displaystyle\int}a_k & = & \frac{1}{\pi}\int_{-\pi}^{\pi}F(x)\cos(kx)dx \\ \vphantom{\displaystyle\int}b_k & = & \frac{1}{\pi}\int_{-\pi}^{\pi}F(x)\sin(kx)dx \\ \vphantom{\displaystyle\int}c & = & \frac{1}{2\pi}\int_{-\pi}^{\pi}F(x)dx \end{array}$

And that’s the Fourier series. Note that we’ve folded in two $\sqrt{2}$ factors into the the $a$ and $b$ terms, which is why they are missing the leading $\frac{1}{2}$ .

This isn’t quite the form you usually see. We can tweak this slightly by noticing that $1(x) = \cos(0x)$ , so we can replace that constant term and its associated constant with a cosine term—we just have to watch out for that $\frac{1}{2}$ . Doing so, we get the usual formulation:

$F(x) = \frac{1}{2}a_0 + \displaystyle\sum_{k=1}^\infty a_k \cos(kt) + b_k \sin(kt)$

Where the $a$ and $b$ terms are as above.

A Few Loose Ends

One thing I glossed over is the space spanned by our basis. A quick counting argument shows that there are

$|V| = \mathfrak{c}^{\mathfrak{c}} = 2^{2^{\aleph_0}}$

functions in our vector space, but only

$|\text{span}(S)| = \mathfrak{c}^{\aleph_0} = (2^{\aleph_0})^{\aleph_0} = 2^{\aleph_0}$

functions in the span of our “basis”. Since $2^{\aleph_0} < 2^{2^{\aleph_0}}$ , our basis doesn’t actually span the space of functions. We’re missing a lot of functions—but which ones? It turns out that our basis actually spans the set $L^2$ of square integrable functions, but I’m afraid that showing this is still beyond me.

Also still beyond me is the extension of this to the full real line: the Fourier transform. Hopefully I’ll be back sometime to explain that one, too.

stdin

On Data — And Not

The Fourier Series via Linear Algebra

It’s All Linear Algebra

Step 2: Choose an Orthonormal Basis

Step 3: Represent a Function in this Basis

A Few Loose Ends

Leave a Reply Cancel reply

It’s All Linear Algebra

Step 2: Choose an Orthonormal Basis

Step 3: Represent a Function in this Basis

A Few Loose Ends

Share this:

Like this:

Leave a Reply Cancel reply