the norm of every fi is that ish fi fi for all the fi are orthogonalthat ish fi for orthonormal basis it follows then that the fi are linearly independentthat isthe only linear combination (xthat is equal to fi (xfor all is the one where ai and for an orthonormal system fi is called an orthonormal basis if there is no other that is orthogonal to all the fi (other than the zero functionalthough the general theory allows for uncountable basesin practice the set is taken to be countable example (trigonometric orthonormal basislet be the hilbert space (( ) )where (dxw(xdx and is the constant function ( alternativelytake and the indicator function on ( pthe trigonometric functions ( gk (xcos(kx) hk (xsin(kx) form countable infinite-dimensional orthonormal basis of hilbert space with an orthonormal basis behaves very similarly to the familiar euclidean vector space in particularevery element ( functionf can be written as unique linear combination of the basis vectorsx ffi fi ( fourier expansion exactly as in theorem the right-hand side of ( is called (generalizedfourier expansion of note that such fourier expansion does not require trigonometric basisany orthonormal basis will do example (example (cont )consider the indicator function ( { pas the trigonometric functions {gk and {hk form basis for (( ) dx)we can write (xa ak cos(kxbk sin(kx)( = = rp rp rp where dx / ak cos(kx) dx and bk sin(kx) dxk this means that ak for all kbk for even kand bk /( pfor odd consequently sin(kxf ( ( = figure shows several fourier approximations obtained by truncating the infinite sum in ( the function spaces typically encountered in machine learning and data science are usually separable spaceswhich allows for the set to be considered countableseee [