Data Science Part 20: Frames/Wavelets

In the frame, we consider some set of vectors [v1, ..., vN], which spans V. But [v1, ... vN] might be linearly dependent (N > dim(V) where N is greater than the dimensionality of V). It might be a good thing if [v1, ..., vN] have explanatory value, or sparse coefficients. 

A frame is a set of vectors such that ∃v 0 < A <= B < ∝ such that 

A||v||^2 <= ∑i|<v, vi>|^2 <=  B||v||^2, ∀v∈V. So, the intuition |<v, vi>| ^ 2 = Θ(||𝑣||^2). So we have a tight frame here. 

Supports that Va  = [Va1, ..., VaN] and Vb = [Vb1, ..., VbN] are both orthonormal bases. Both Va and Vb are tight frames with A = B = 1. Consider their concatenation Vc = [Va Vb] and Vc has a tight frame with A = B = 2. If you take the norm of a signal and compare it to the energy in the output then there is a gain between the two and bounded both from below and above by 2 numbers A and B. There is also a dual frame, h, that allows you to go in the reverse direction. 

Wavelets are a multi-scale tool for time frequency analysis. The same wavelet structure is happening at different scales. There is a high frequency and short time scale on a fine scale. There is a low frequency and long time scales on a coarse scales. We can construct an orthonormal basis with wavelets. Wavelets can provide sparse representation for piecewise smooth signals. 

Which leads to the topic of continuous time wavelets.  The continuous time function is Ψ(t). Zero mean is ∫Ψ(𝑡)𝑑t = 0. A unit norm function is  ∫|Ψ(𝑡)|^2 𝑑t = 1. We refer to Ψ(𝑡) as a wavelet, typically concentrated around t = 0. 

A Mexican Hat wavelet is the Second derivative of a Gaussian. 

We consider a translation u ∈(-∝,+∝) and scale s ∈(0,+∝). 

We can define a continuous wavelet transform of f ∈L2(ℝ), where was have a finite energy function f over a real line and we can define W, the wavelet transform, as the inner product of the function f(t) with the particular wavelet:

and provides insights about time-frequency behavior. Here is an example transform:

There are large coefficients around discontinuities, and coefficients decay in smooth regions. 

A continuous wavelet transform resemble an orthogonal expansion, but it is highly redundant. The solution to this is to sample a continuous transform and obtain the frame. 

At each scale, sample translations uniformly, and the resolution is proportional to scale. 

In a fine resolution/scale, each term covers little spatial width but more frequency and scale. A coarse resolution covers more spatially but less frequency/scale. 

A wavelet can be dfeined partially as a super-charged fourier transform. Sines and cosines form an orthogonal basis of what we want to represent. We can generalize and provide better representation for certain types of functions. There's this observation that low frequencies tend to last a long time and don't change that much over time. We can break things up into many different frequency resolutions. The higher frequencies need to be sampled more and need more temporal accuracy. 

We get smaller and smaller orthogonal functions in time and space. We can have a mother wavelet, and from this mother wavelet we can derive all of the smaller wavelets.