Isaac Kunen

Professionally, I’ve built Snowflake’s Snowpark, Tableau Prep, SQL Server’s geospatial support, and more. Recently, I’ve been learning physics at the UW and teaching at North Seattle College.

Personally, I’m a father, a husband, a cellist, a photographer, and a tinkerer.

All opinions expressed here are mine alone: you can’t have them.

7 minute read

I didn't post last week because I was on vacation. But on vacation I decided to write about something a little out of my comfort zone: Fourier series. (Yeah. Try being my wife.)

Fourier series (and the related Fourier transform) made some sense to me for, but I never really learned how to derive them so they always seemed a bit magical. As I was going through Arthur Mattuck's excellent differential equations course at MIT's Open Courseware, the Fourier series clicked for me, so I thought I'd distill this out.

I'm tempted to categorize this post under "things Isaac should have learned in school (had he been paying attention), but I don't think I ever had a course that taught this. If you're still paying attention, I will assume that you recall your basic linear algebra and have some idea (and interest in) what a Fourier series is. But given that, this isn't hard.

It's All Linear Algebra

Recall that the Fourier series represents periodic functions as a sum of sines and cosines. In this post, we'll deal with functions with a period that lies in the interval $ [-\pi, \pi]$. The generalization to arbitrary periods is straightforward, but this interval illustrates the scheme well.

The fundamental "click" for me was that this was all linear algebra. The Fourier series:

  1. Looks at functions over an interval as a vector space with an inner product;
  2. Picks an orthonormal basis for the space; and
  3. Represents an arbitrary function in this basis by projecting it out on the basis.

Given that we're working over the interval $[-\pi, \pi]$, we'll define a vector space $V$ where the scalars are taken from $ \mathbb{R}$ and the vectors are functions over the interval $ [-\pi,\pi].$ In addition, we'll define an inner product by:

$\langle f,g\rangle = \frac{1}{2\pi}\displaystyle\int_{-\pi}^\pi f(x)g(x) dx$

I'll leave it to you to check that these meet the requirements of a vector space and an inner product. This is worth doing, particularly if using functions as vectors seems odd to you.

In any case, that's all for step 1.

Step 2: Choose an Orthonormal Basis

We're going to choose an orthonormal basis $ S$ for our vector space. The vectors of $ V$ are functions, so our basis will be a set of functions. The basis will be infinite, meaning that $ V$ is infinite dimensional.

To build our basis, we're going to use the constant function $ 1(x)$ as well as sines and cosines of a positive integral multiple of $ x$. In particular, our basis is:

$ S = \begin{Bmatrix}\vphantom{\displaystyle\int}\sqrt{2}\sin(x), \sqrt{2}\sin(2x), \sqrt{2}\sin(3x), ...,
\\ \vphantom{\displaystyle\int}\sqrt{2}\cos(x), \sqrt{2}\cos(2x), \sqrt{2}\cos(3x), ...,
\\ \vphantom{\displaystyle\int}1(x)\end{Bmatrix}$

For these to be an orthonormal basis, we first have to show that any two of these are orthogonal. I.e., that for all $ f,g \in S$ with $ f\ne g, \int_{-\pi}^\pi f(x)g(x) dx = 0$. There are a few cases to check:

  • If one of $ f(x)$ and $ g(x)$ is $ 1(x)$ then the inner product is just an integral of a sine or cosine function over a whole number of periods, which is zero.
  • If $ f = \sin(kx)$ and $ g = \cos(nx)$, then $ f(x)g(x)$ is an odd function, so the integral is zero.
  • If both $ f(x)$ and $ g(x)$ are both sines (or cosines) with different coefficients, then this is also zero. As Mattuck suggests, you can work this out via complex exponentials or trig identities; he does it in his lecture via differential equations and a nice symmetry argument.

We don't need to show linear independence separately because it's implied by orthogonality. But we do need to check that all of the vectors have unit length:

  • For $ 1(x)$, we find that $|1(x)|^2 = \langle 1(x),1(x)\rangle = \frac{1}{2\pi}\int_{-\pi}^\pi 1 dx = 1$.
  • For $ \sqrt{2}\sin(kx)$, we find that $ |\sqrt{2}\sin(kx)|^2 = \frac{1}{2\pi}\int_{-\pi}^\pi 2\sin^2(kx) dx = 1$. So our sine vectors are normalized.
  • This same argument holds for the cosines.

So $ S$ is an orthonormal basis. We haven't shown that $ S$ actually spans the space $ S$ of functions. I'll mention this again, but for now, it's sufficient to know that it spans some space.

Step 3: Represent a Function in this Basis

Now that we have a basis, we can take an arbitrary vector $ F(x)$ and write it as a linear combination of the basis vectors:

$ F(x) = \displaystyle\sum_{s\in S} t_s s$

All we need to do is figure out the constants $ t_s$. But since $ V$ is an inner product space, we can find the coefficient $ t_s$ for each $ s\in S$ as:

$ t_s = \langle F(x),s\rangle = \frac{1}{2\pi}\displaystyle\int_{-\pi}^{\pi}F(x)s(x)dx$

Instead of using $ t_s$, we'll use $ a_k$ as the constant term for the $ \cos(kx)$ term, and $ b_k$ for the $ \sin(kx)$ term. For now, we'll use $ c$ as the constant for the $ 1(x)$ term.

Now:

$ \begin{array}{rcl}
F(x) &=& \displaystyle\sum_{s\in S} t_s s = \displaystyle\sum_{s\in S} \langle F(x),s\rangle s\\
&&\\
&=&c*1(x) + \displaystyle\sum_{k=1}^\infty a_k \cos(kt) + b_k \sin(kt)
\end{array}$

Where:

$ \begin{array}{rcl}\vphantom{\displaystyle\int}a_k & = & \frac{1}{\pi}\int_{-\pi}^{\pi}F(x)\cos(kx)dx \\
\vphantom{\displaystyle\int}b_k & = & \frac{1}{\pi}\int_{-\pi}^{\pi}F(x)\sin(kx)dx  \\
\vphantom{\displaystyle\int}c & = & \frac{1}{2\pi}\int_{-\pi}^{\pi}F(x)dx \end{array}$

And that's the Fourier series. Note that we've folded in two $ \sqrt{2}$ factors into the the $ a$ and $ b$ terms, which is why they are missing the leading $ \frac{1}{2}$.

This isn't quite the form you usually see. We can tweak this slightly by noticing that $ 1(x) = \cos(0x)$, so we can replace that constant term and its associated constant with a cosine term—we just have to watch out for that $ \frac{1}{2}$. Doing so, we get the usual formulation:

$ F(x) = \frac{1}{2}a_0 + \displaystyle\sum_{k=1}^\infty a_k \cos(kt) + b_k \sin(kt)$

Where the $ a$ and $ b$ terms are as above.

A Few Loose Ends

One thing I glossed over is the space spanned by our basis. A quick counting argument shows that there are

$ |V| = \mathfrak{c}^{\mathfrak{c}} = 2^{2^{\aleph_0}}$

functions in our vector space, but only

$ |\text{span}(S)| = \mathfrak{c}^{\aleph_0} = (2^{\aleph_0})^{\aleph_0} = 2^{\aleph_0}$

functions in the span of our "basis". Since $ 2^{\aleph_0} < 2^{2^{\aleph_0}} $, our basis doesn't actually span the space of functions. We're missing a lot of functions—but which ones? It turns out that our basis actually spans the set $ L^2$ of square integrable functions, but I'm afraid that showing this is still beyond me.

Also still beyond me is the extension of this to the full real line: the Fourier transform. Hopefully I'll be back sometime to explain that one, too.

You May Also Enjoy

Lessons From the Signal Leak

4 minute read

I’ve found a lot of the coverage about the Trump administration’s of Signal messages to The Atlantic frustrating. To read most of the coverage, the major mistake was Mike Waltz’s inclusion of Jeffery Goldberg in the conversation, and the big questions are about the impact of this particular leak. I have two alternative takeaways, one about the general security attitude, and one about Signal itself.
(Continued...)

Five Eights

2 minute read

With the recent news that Adam Selipsky is leaving AWS, I thought I’d share my funny Adam story.
(Continued...)

Two Unequal Products

4 minute read

Here’s a reasonably elegant solution to a fun problem I stole from Timothy Gowers: prove that for every positive integer n, there do not exist positive integers a, b, c, d with ad=bc and n^2 < a < b < c < d < (n+1)^2.
(Continued...)

Au Revoir, Snowflake!

2 minute read

Just reading this blog, you might guess that all I do is leave jobs. First leaving Tableau, and now, four years later, departing Snowflake. Here are some thoughts on the future.
(Continued...)