Error plot between known and estimated data The channel is an FIR model with input $u$. The input takes in values which are symbols from some constellation. Using an equalizer such as the Least Mean Squares (LMS), I estimate the input to the channel from the known noisy observations and the channel coefficients. Let the estimated input to the channel be denoted by $\hat{u}[n]$.

The coefficients of the channel are denoted by $c$ of length $L$. The noisy output of the channel is $y$. Therefore we have, $y[n] = \mathbf{c}^H*\mathbf{u} + v[n] \tag{1}$ for $n = 1,2,\ldots,N$ where $v[n] \sim N(0,\sigma^2)$ is a Gaussian noise. The weights of the equalizer are given by $w$ and the equalizer is of length $2L+1$. The input to the equalizer is $y$. The output of the equalizer is given by $\hat{u}[n] = y*\mathbf{w}$ where $*$ denotes the convolution. My questions are:

1) Mean Square error between the estimated input and known / desired input to the channel for different values of Signal-To-Noise ratio (SNR) --The result of equalization will give us the estimates of the input to the channel. The estimate of the input to the channel is $\hat{u}[n]$. If I need to calculate the error between the estimated input and the known input, I should probably use the formula: $MSE = \frac{1}{T}\frac{\sum_{n=1}^{N}\hat{u}[n] - {u}[n]}{N}$ and $T$ denotes the number of independent trails. As an example, the input can take values from the set $\{1,2,3,4\}$. The first 10 symbols of the data can be $2,1,3,1,1,2,4,1,2,3$. These are the known values. Then to validate the model after equalization,let the estimates input symbols to the FIR channel be $\hat{u} = 1,3,1,1,1,2,4,1,2,2$. The error, $e = (u[1]-\hat{u}[1])^2+(u[2]-\hat{u}[2])^2+(u[3]-\hat{u}[3])^2 + (u[4]-\hat{u}[4])^2 +(u[5]-\hat{u}[5])^2 + ...+(u[10]-\hat{u}[10])^2$
Is this the correct way?
I am not using the communication tool-box in Matlab, and building my own code

2) Symbol error rate -- In communications, I have seen the symbol error plot (please refer :http://www.dsplog.com/2008/03/29/comparing-16psk-vs-16qam-for-symbol-error-rate/).

How is Using the same data which is used to obtain the MSE plot between the estimated symbols and known symbols, how can I obtain the SER plot? An example in Matlab will be very helpful.

Thank you.
Your formula/method for computing MSE between estimated and known inputs looks good to me. For symbol error rate you could use something like a Hamming distance which simply counts the number of times the estimated symbol is different from the actual symbol. In your 10 symbol example the error rate is 4/10 i.e 40%.

In Matlab you can do something like:

symbol_error_rate_percent = sum(u_hat ~= u)/length(u) * 100


You can compute this for $T$ trials and average them, like you did for MSE. Or you could just make a really long sequence of length $10 \times T$ and do it once.