Powers of 2 for batch_size in model fit in deep learning
It's not just on GPU, you want to have a small power of two as submultiple for CPU as well.

The reason is to allocate the processors on the GPU (or the SIMD registers) as best as possibles. This is directly linked to the warp sizes and what the GPU vendors offer for maximum block sizes,

See also this thread on SO which what the warp size is and its relation to best occupancy.