In TD learning, is the weight function used as a linear filter?
They are the same in the two equations.  See equation 9.3:  v = w*u.  w and u are bolded in that equation to indicate that they are vectors.  In equation 9.3, they are vectors of simultaneously presented stimuli, but you can apply the same equation if u is a vector of a single stimulus over time and w are the weights of each time step of that stimulus.  Indeed, a linear filter is simply a way to weight different time steps of a time-varying input.  Note that v in equation 9.3 is a single value, whereas v in equation 9.6 is a function of t, i.e. it has a different value at each time point depending on what part of u is currently occurring.