Smaller test data set than training data set in machine learning
It's normal (and expected even) to have a Test Set that is smaller than your Training Set. 

In general, the more training data you have, the better your performance should be. That is, there'll be less variation in your model parameters if trained on more examples.

This is especially true for Deep Learning approaches (your Convolutional Neural Network, CNN, you mentioned).

Below is a graph that illustrates the above. Where 'Classical Learning' refers to Machine Learning models (SVM, Random Forest, etc.)



From this explanation and graph, it should now make sense why you get a lower test accuracy when you have a smaller training set. There simply isn't enough variation in the training samples, leading to overfitting.

There are numerous ways to avoid overfitting, through data augmentation, adding Dropout layers (in the case of CNN), or regularizing your model. Going into the details is beyond this answer.

However, one of the best and easiest ways to help avoid overfitting is through increasing the the training set.

By increasing the training set, you follow the graph above and decrease variation in parameters but more importantly expose your model to more samples and variation in the dataset.

This helps explain, in part, why you see a higher test accuracy when using a larger test set.

Should you reduce your test set to maintain a ratio? Probably not, in a similar way to your parameters experiencing variation with a small training set the same is true with your test set. You'll want a big enough test set to cover enough variation in samples so the accuracy returned can be trusted.

This StackOverflow answer provides good detail and rationale on choosing the training/test split. 

Let me know if you have any questions! I can't comment yet, hence a relatively general answer.