Is transformation from one state representation to other state representation changes the transfer function of the system
The similarity transformation is only a change of basis (co-ordinates) and doesn't change the input to output relations, viz, the transfer function (TF).
The similarity transformation preserves the eigen values; i.e. poles of the system; i.e. denominator of the TF. However, to show that the TF doesn't change, we need to show that the zeroes do not change either.
Assuming that \$P\$ invertible,
$$
\begin{align}
\frac{Y(s)}{U(s)} ={} &amp; CP\ (sI - P^{-1}AP)^{-1}\ P^{-1} B\\
{}={} &amp; CP\ (sP^{-1}IP - P^{-1}AP)^{-1}\ P^{-1} B\\
{}={} &amp; CP\ (P^{-1}\ (sI - A)\ P)^{-1} P^{-1} B\\
{}={} &amp; CPP^{-1}\ (sI - A)^{-1}\ P P^{-1} B
      &amp; \tiny{(ABC)^{-1} = C^{-1}B^{-1}A^{-1}}\\
{}={} &amp; C (sI - A)^{-1} B\ \ \ \blacksquare\\
\end{align}
$$