Should I use GPU or CPU for inference?
It is true that for training a lot of the parallalization can be exploited by the GPU's, resulting in much faster training. For Inference, this parallalization can be way less, however CNN's will still get an advantage from this resulting in faster inference. Now you just have to ask yourself: is faster inference important? Do I want this extra dependencies (a good GPU, the right files installed etc)?
If speed is not an issue, go for CPU. However note that GPU's can make it an order of magnitude faster in my experience.