is it possible to design high-speed PCBs in a modular way?
Certainly is feasible - usually sold as "computer-on-module", like the Raspberry Pi compute module. In that case the core board interfaces to a mainboard over a SODIMM connector.

You do have to take care of signal integrity over daughterboard connectors. What you may find is that having lots of connectors isn't cost-effective; good connectors are surprisingly expensive.