1
M7777 Applied Functional Data Analysis 2. From Data to Functions — basis systems
Jan Koláček (kolacek@math.muni.cz)
Dept. of Mathematics and Statistics, Faculty of Science, Masaryk University, Brno
n Koláček (SCI MUNI)
M7777 Applied FDA
Fall 2019 1/36
From Data to Functions
How do we go from
St. Johns
10H
3
o o. E ,0
Ol
data
A
to
functions?
100
200
Days
300
St. Johns
15
10
O
a. 5 E
-5
100
200
Days
Jan Koláček (SCI MUNI)
M7777 Applied FDA
Basis Expansions
We consider
and
Let us denote
• 0*(t) = (<Di(t),.
• c = (ci,...,ck)' We write
y, = x(ti) + ei,      e; ~ i.i.d
K
j'=i
• •, ^(0) ... a basis system for x(t) ... basis coefficients
x(r) = 0*(t)c.
Jan Koláček (SCI MUNI)
M7777 Applied FDA
Fall 2019 3/36
The Monomial Basis
Monomial Basis
4.72
4.70
co
CQ
4.68
4.66
4.64
25
0*(t) = (1)
St. Johns
50
Days
75
100
10
cd
H—'
co
CD Q.
E
.cd
Days
							
							
		<					
	I						
* • 0	J*					\	
	k 1(	)0	2(	)0	3(	)0	
Jan Koláček (SCI MUNI)
M7777 Applied FDA
Fall 2019 4/36
The Monomial Basis
♦*(t)
Monomial Basis
2
0
									
									
									
									
									
									
									
									
1 0		25		50		75		100	
Days
(i,0
St. Johns
							
					"\		
		<				i	
						Sc \	
* • 0	V 4*						&
	k 1C	)0	2(	)0	3(	)0	
Days
Jan Koláček (SCI MUNI)
M7777 Applied FDA
Fall 2019 5/36
The Monomial Basis
Monomial Basis
20
10
co
CQ
-10
25
0*(t) = (l,i,t2)
St. Johns
50
Days
75
100
10
cd
H—'
co
CD Q.
E 0
.cd
-10
				Jf				
								
			/J*					
		/s						
J		f						%
								
(	)	u	)0	2(	)0	3C	)0	
Days
Jan Koláček (SCI MUNI)
M7777 Applied FDA
Fall 2019 6/36
The Monomial Basis
Jan Koláček (SCI MUNI)
M7777 Applied FDA
Fall 2019 7/36
The Monomial Basis
Jan Koláček (SCI MUNI)
M7777 Applied FDA
Fall 2019 8/36
The Monomial Basis
Monomial Basis
10
co
CQ
-10
**(t) = (1, t, t2, Í3, t4, t5)
St. Johns
									
									
									
■									
									
									
									
									
(	) 25			50		75		100	
10
cd
H—'
co
CD Q.
E
.cd
						
						
						
						
	I*					%
						
0	100	200		300		
Days
Days
Jan Koláček (SCI MUNI)
M7777 Applied FDA
Fall 2019 9/36
The Monomial Basis
Summary • Formula
K
numerically difficult for more than six terms problem with derivatives estimation
K
Dx(t) = 5]c^-1
7=1
monomial derivatives get simpler, whereas the opposite happens in most real-world data (oversmoothing)
Jan Koláček (SCI MUNI)
M7777 Applied FDA
Fall 2019 10/36
The Monomial Basis
First derivative
St. Johns
0.1
9- o.o
								
								
		/						
—								
								
								
								
c	) 100			200		300		
Days
Estimate
St. Johns
0.1
cd
H—'
co
c5 0.0
Q_
E ß Q
-0.1
-0.2
								
								
								
								
								
								
								
								
1 0		100		200		300		
Days
Jan Koláček (SCI MUNI)
M7777 Applied FDA
Fall 2019 11/36
The Monomial Basis
Second derivative Estimate
St. Johns St. Johns
Days Days
Jan Koláček (SCI MUNI)
M7777 Applied FDA
Fall 2019 12/36
The Fourier Basis
The Fourier Basis
• Basis functions
0*(t) = (1, sin(cjt), cos(cjt), sin(cj2t), cos(cj2t),... sin(cj/Wt),cos(cj/Wt))
u ... defines the period of oscillation, i.e. uj — 2tv/P, P is the period
K = 2M + 1 where M is the largest number of oscillations required in a period of length P
Jan Koláček (SCI MUNI)
M7777 Applied FDA
Fall 2019 13/36
The Fourier Basis
Fourier Basis
4.72
4.70
co
CQ
4.68
4.66
4.64
100
0*(t) = (1)
St. Johns
200
Days
300
10
cd
H—'
co
CD Q.
E
.cd
							
							
		<					
	I						
* • 0	J*						
	k 1(	)0	2(	)0	3(	)0	
Days
Jan Koláček (SCI MUNI)
M7777 Applied FDA
Fall 2019 14/36
The Fourier Basis
0*(t) = (l,sin(cj£),cos(cj£))
Fourier Basis St. Johns
Days Days
Jan Koláček (SCI MUNI)
M7777 Applied FDA
Fall 2019 15/36
The Fourier Basis
0*(t) = (1, sin(cjt), cos(cjt), sin(cj2t), cos(cj2ř))
Fourier Basis
St. Johns
co u CÜ
-5
								
								
							\	
							\	
								
								
								
								
(	) 100			200		300		
10
cd
H—'
co
CD Q.
E
.cd
100
Days
200
Days
Jan Koláček (SCI MUNI)
M7777 Applied FDA
The Fourier Basis
0*(t) = (1, sin(cjt), cos(cjt), sin(cj2t), cos(cj2ř), sin(cj3t), cos(cj3t))
Fourier Basis
St. Johns
co
CQ
-5
								
								
							\	
■								
								
								
								
								
(	) 100			200		300		
10
cd
H—'
co
cd q.
E
,cd
				
				
				
				
				%
				
0	100	200 300		
Days
Days
Jan Koláček (SCI MUNI)
M7777 Applied FDA
Fall 2019 17/36
The Fourier Basis
0*(t) = (1, sin(cjt), cos(cjt),..., sin(cj5t), cos(cj5t))
Fourier Basis
St. Johns
co u CÜ
-5
								
								
								
								
								
								
								
								
(	) 100			200		300		
10
cd
H—'
co
CD Q.
E
.cd
100
Days
200
Days
300
Jan Koláček (SCI MUNI)
M7777 Applied FDA
Fall 2019 18/36
The Fourier Basis
Summary • Formula
x(t)
• Excellent computational properties, especially if the observations are equally spaced
• Natural for describing periodic data x inappropriate for special types of data (e.g. growth curves)
• Derivatives retain complexity, easy to compute
D sin(cjt)
oj cos(cj t), D cos(cj t)
Jan Koláček (SCI MUNI)
M7777 Applied FDA
Fall 2019 19/36
The Fourier Basis
First derivative & Estimate
St. Johns
0.1
9- o.o
				\\				
				\\ H \l				
		J		u 1	i			
—	// / /							
	A* ✓/ t /				n A A			'1
					\\		' S*\ 1 / \\ I	/ 1 f
							V	
c	)	100		200		300		
Days
Second derivative & Estimate
St. Johns
CM Q
-0.01
								
—	\    ' / \ ' / \/	\\ \\ \\ \\	J' ^		T	if If \ /	ft Ii V /'	
								
								
								
1 0		100		200		300		
Days
Jan Koláček (SCI MUNI)
M7777 Applied FDA
Fall 2019       20 / 36
The Bsplines Basis
Splines
• Splines are polynomial segments joined end-to-end.
• Segments are constrained to be smooth at the joins.
• The points at which the segments join are called knots.
• System is defined by
• The order m (order = degree+1) of the polynomial,
• the location of the knots.
Thus K — ^interior knots + m
• Bsplines are a particularly useful means of incorporating the constraints.
See de Boor, 2001, "A Practical Guide to Splines", Springer.
Jan Koláček (SCI MUNI)
M7777 Applied FDA
Fall 2019 21/36
The Bsplines Basis
3 knots, local constants (m = 1) =>• K = 2
0*(t) = (Bspll.l(t), Bspll.2(t))
Bsplines Basis
St. Johns
co
CQ
										
										
										
										
										
										
										
										
										
0 1C			)0                    200 3C					)0		
10
cd
H—'
co
CD q.
E
.cd
		4		
				
				
	if			
				V
	k			
100
200
Days
300
Jan Koláček (SCI MUNI)
M7777 Applied FDA
Fall 2019 22/36
The Bsplines Basis
Knots monthly (nknots = 13), local constants (m = 1) =>- K = 12
0*(t) = (ßsp/l.l(i), Bspll.2(t),Bspll.l2(t))
Bsplines Basis
15
10
co
CÜ
100
St. Johns
10
cd
H—'
co
cd q.
E
.cd
200
Days
300
100
200
Days
300
Jan Koláček (SCI MUNI)
M7777 Applied FDA
Fall 2019 23/36
The Bsplines Basis
Knots monthly (nknots = 13), local linear (m = 2) =>- K = 13
**(t) = (Bspl2.1(t), Bspl2.2(t),..., Ssp/2.13(t))
Bsplines Basis St. Johns
The Bsplines Basis
Knots monthly (nknots = 13), local quadratic (m = 3) =>- K = 14
0*(t) = (ßsp/3.1(i), Bspl3.2(t),Bspl3.U(t))
Bsplines Basis
St. Johns
10
CO
cc CQ
100
10
cd
H—'
co
CD q.
E
,cd
200
Days
300
100
200
Days
300
Jan Koláček (SCI MUNI)
M7777 Applied FDA
Fall 2019       25 / 36
The Bsplines Basis
Knots monthly (nknots = 13), local cubic (m = 4) =>- K = 15
0*(t) = (ßsp/4.1(i), BsplA.2(t),ßsp/4.15(i))
Bsplines Basis
St. Johns
10
w 5
co
CÜ
100
10
cd
H—'
co
CD q.
E
.cd
200
Days
300
100
200
Days
Jan Koláček (SCI MUNI)
M7777 Applied FDA
Fall 2019 26/36
The Bsplines Basis
Knots monthly (nknots = 13), splines of order m = 6 =>- K = 17
0*(t) = (ßsp/6.1(r), Bspl6.2(t),ßsp/6.17(r))
Bsplines Basis
St. Johns
co
CÜ
-4
																		
																		
																		
																		
																		
-																		
																		
																		
																		
c			100 200										300					
10
cd
H—'
co
CD q.
E
,cd
100
200
Days
Jan Koláček (SCI MUNI)
M7777 Applied FDA
Fall 2019 27/36
The Bsplines Basis
Summary
• Number of basis functions:
#/interior knots + order
• Derivatives up to m — 2 are continuous.
• B-spline basis functions are positive over at most m adjacent intervals fast computation for even thousands of basis functions.
• Sum of all B-splines in a basis is always 1; can fit any polynomial of order m.
• Most popular choice is order 4, implying continuous second derivatives. Second derivatives have straight-line segments.
Jan Koláček (SCI MUNI)
M7777 Applied FDA
Fall 2019 28/36
The Bsplines Basis
Choosing Knots and Order
• The order of the spline should be at least k + 2 if you are interested in k derivatives.
• Knots are often equally spaced (a useful default) But there are two important rules:
• Place more knots where you know there is strong curvature, and fewer where the function changes slowly
• Be sure there is at least one data point in every interval.
• Later, we'll discuss placing a knot at each point of observation.
• Co-incident knots reduce the number of continuous derivatives at each point. This can be useful (more later).
Jan Koláček (SCI MUNI)
M7777 Applied FDA
Fall 2019 29/36
The Bsplines Basis
Other
The f da library in R also allows the following bases:
Constant   d>*(t) = 1, the simplest of all.
Power   0*(t) = (tAl, tA2, £As,..., tXK), powers are distinct but not necessarily integers or positive.
Exponential   0*(t) = (eAlt, eA2t, eAst,..., eXf<t)
Other possible bases include
Wavelets   especially for sharp, local features (not in f da) Empirical   functional Principal Components (special topics)
Jan Koláček (SCI MUNI)
M7777 Applied FDA
Fall 2019       30 / 36
Problems to solve
A Ca nadian Weather Data
• Load the variable Canadian Weather from the fda package.
• Set the time of each measurement to the half of the day, i.e. t = (0.5,1.5, 2.5,..., 364.5)
• Create a cubic B-spline basis with knots at each point of season change and plot it (see Figure 1).
• Create a cubic B-spline basis with knots at each fifth day and plot it.
• Smooth data observed in Edmonton, Halifax, Montreal and Ottawa with using the created basis and plot the results (see Figure 2)
• Do previous step with Fourier basis (see Figure 3). How many basis functions would be appropriate?
• Plot the first derivatives of the Fourier-basis-smoothed functions (see Figure 4).
© Refinery Data
• Load the variable refinery from the fda package and plot it.
• Create a cubic B-spline basis with knots (0, 33, 67, 98,130,162,193), smooth the data and plot the result. Then double (triple) the value 67 in knots and do the same (see all in the Figure 5).
Jan Koláček (SCI MUNI)	M7777 Applied FDA		Fall 2019 31/36
Problems to solve
Jan Koláček (SCI MUNI)
M7777 Applied FDA
Fall 2019       32 / 36
Problems to solve
Jan Koláček (SCI MUNI)
M7777 Applied FDA
Fall 2019       33 / 36
Problems to solve
Jan Koláček (SCI MUNI)
M7777 Applied FDA
Fall 2019       34 / 36
Problems to solve
Jan Koláček (SCI MUNI)
M7777 Applied FDA
Fall 2019       35 / 36
Problems to solve
Time
Figure 5.
Jan Koláček (SCI MUNI)	M7777 Applied FDA		Fall 2019       36 / 36