In Wigner's classification, one observes that the full automorphism group of the Lorentzian manifold $\mathbb{R}^{1,3}$ is precisely the well-known Poincare group. Motivated by this and basic principles of quantum mechanics, its irreducible projective representations are then studied (by passing to unitary representations of the cover). 

My question is this: how should I think of the relation between (projective) Hilbert space $\mathcal{H}$ and the actual space-time manifold? If I am in a frame $X$ and someone else is in a different frame $Y$ related to mine via the transformation $\Lambda$, how does this induce a transformation on the Hilbert space? If I am correct, I should not think of both observers having different Hilbert spaces associated to them, but somehow $\Lambda$ gives rise to a projective automorphism of $\mathcal{H}$. I do not see how this happens (except maybe just via the regular representation if the Hilbert space happens to be one of certain functions on $\mathbb{R}^{1,3}$ but this need not be the case if we just consider spin for example. 

Maybe this is a silly question, but I hope I managed to convey my confusion.