Does USB connector do any thing special like impedance matching or it's just a way to connect two devices to each other?
Yes, it is possible to connect a host and device on the same PCB simply by running a differential pair of the appropriate impedance between them.  In practice you can get away with some mismatch, so you can do this on FR4 with no problem even without requiring special controlled-impedance fabrication.  Many embedded or laptop computers do this.

I've never had any difficulty with USB 2.0 high speed (480 Mbps) just by following basic high speed design practices; USB 3.0 would require more care and probably simulation.