Multiple source Smartphone charger pack
No, your design seems fine in theory. All grounds should be connected at a single point on your board. And make sure you properly size the wires or pcb traces for the current you are expecting. As well as making sure your rotary switch can handle the current as well.