MATLAB Implementation of Karplus Strong algorithm with filter function?
The filter(b,a,x) function of MATLAB does not operate on symbolic variables. Your line of code 

syms z


creates a symbolic variable z, in MATLAB environment, and later
you feed this variable into the filter() function through its second argument in this line:

H = filter(1,1-alpha*z^(-M),x);


where the filter(b,a,x) function, in this mode is expecting a numerical coefficient vector $a[k]$ associated with the DT system that's defined by the coefficient vectors $a[k]$ and $b[k]$.

In order to get the filter() function working, replace the symbolic expression 

1-alpha*z^(-M) 


with the numerical one, given integer M and real alpha

a = [1 zeros(1,M-1) -alpha]