How to tinker with CNN architectures?
CNN Architectures: These are proven configurations of the combination of CNN/FC/POOL etc layers, that seem to work in commonly defined problems.

Now from your question what I understand is you want to "Tinker" with the architecture and not to come up with an entirely new configuration from scratch. One thing that can help you in this scenario is transfer learning.

Transfer Learning: It is a process of using pre-learnt weights from a similar task on altogether different data, to your current problem and data.

In transfer learning what we do is we take some layers usually 80-90 % of the layers keep the weights the for these layers and delete the weights for remaining layers, and initialise these layers with some kind of initialisation batch norm or something like that. And train this newly created model with the data you have.

This have proven to be effective in various problems.

You can come up with innovative ways to make changes to arbitrary layers of this network train them and check for error metrics. If you follow this approach whatever you are claiming in your question can be achieved.