Homework: convolution product

The convolution product of functions #

Background #

In class we defined the convolution of two functions \(f,g: \mathbb{R} \rightarrow \mathbb{R}\). In order to simplify things, we will assume that both function have compact support, that one of them is continuous and the other one Riemann-integrable. Then convolution defines a new function \(f * g \) via: \[ f*g(x) = \int_{-\infty}^{\infty} f(y)g(x-y) \; dy\]

Exercise: Try your hand at computing the convolution product for pairs of functions \(f\) and \(g\). This notebook gives instructions on how to do this in Python. Follow its lead. If this will be your first time using Google’s Colab, please work through the following quick introduction and a notebook on linear algebra with Python.

Properties of the convolution product #

Homework exercise: We can think of convolution as a slightly exotic product of functions. Show that this product is commutative; that is, that \[f*g = g*f.\] Hint: Use change of variables from calculus.

Let us define a sequence of step functions by the following formula: \[ h_n(x) = \begin{cases} 0 & \text{if } |x| > \tfrac{1}{n}\\ \tfrac{n}{2} & \text{otherwise}.\\ \end{cases} \] Each has compact support and this support diminishes as \(n\) grows. It is an interesting sequence of functions with which to take a convolution product.

Homework exercise: Choose your favorite continuous function \(f\), but choose wisely as you will have to work with it. Use the Python notebook above to graph the sequence of functions \(\{f * h_n\}_{n \in \mathbb{N}}\) together with the original function \(f\). While you can’t really do this for all \(n\), do enough so that you have a sense of what is going on. Formulate a conjecture and support it with images that you create. We will address it in class.

Aside: an application in probability: #

The convolution product appears in a number of seemingly unrelated settings. One is probability theory. A continuous probability density function, or pdf, is a non-negative function \(f:\mathbb{R} \rightarrow \mathbb{R}\) which describes the distribution of a statistic among a population.

A probability density function for a normal distribution.

The meaning of a pdf is derived by integrating it. The value of \(\int_a^b f \) represents the percentage of individuals in the underlying population with statistic between \(a\) and \(b\).

Convolutions appear when the statistic we are studying is a sum of simpler ones. For instance, the annual (absolute) return of a portfolio of two equities is the sum of their individual returns. Let us assume that these are random and that we know that the annual return of the first stock has pdf \(f\) and the second follows the pdf \(g\). A natural question is whether one can compute the pdf of the returns of the entire portfolio. Or more mathematically,

Question: If the random variable \(X\) has pdf \(f\) and the random variable \(Y\) has pdf \(g\), then what is the pdf of the sum \(X+Y\)?

The answer is of course the convolution of the two functions, otherwise you would not be reading about this result on a page dedicated to the topic.

Theorem: If \(X\) and \(Y\) are independent random variables distributed according to probability density functions \(f\) and \(g\), then the sum \(X + Y\) is distributed with pdf \(f * g\).

Challenge question: Consider a pdf \(f\) and define the iterated convolution product: \[f^{(n)} = f * f * \ldots * f \] What happens as \(n\rightarrow \infty\)? If you have seen the Central Limit Theorem, explain your answer.