How to use 5 by 5 or 7 by 7 kernel size for a deep learning network with 3 by 3 kernels?
The main think you need to be mindful about is the size of the output of your convolutional layer, remember that it's going to be
$$
\frac{W-K+2P}{S} + 1
$$
with your K the kernel size, so if you increase your kernel size the size of your output will decrease faster. You might have to reduce the number of layers. If you're using a framework for which you need to specify input size for each layer then in that case you would need to change that as well.