Where to get transtion matrix for Kalman filter?
Simple: if you do not have a model, you cannot apply the Kalman filter. Or you could and make up a model, but you cannot expect any of the optimality properties of the filter to hold. Based on that paper, it seems they assume that $\mathbf{x}_k$ is known or at least directly measureable...