How to make perfboard circuits more compact
The ultimate answer is to use SMD components on a PCB - the parts are smaller, and using traces instead of wires will reduce stray capacitance; this can have a large effect in sensitive circuits.  I understand your reluctance to use a PCB because it is more work and more expensive initially - however, in the long run it sounds like the only way you will get what you want.

You can use perfboard and wire everything together manually, but debugging those circuits is very difficult.