Converting 5v to 3.3V for GPS purpose while keeping noise at the lowest
On the USB side, the specification is for 10 \$ \mu\$F maximum. In my experience, higher values are usually fine but that's anecdote, so stick with the specification. It's followed by a linear regulator anyway. 

You almost certainly don't need that amount of capacitance on the 3V3 line either, and just adding capacitors does not lower the noise (and you should think about what you need as well). Stick to the datasheet or reference implementation, unless you need something radically different. For modules like this, a good starting point is a 47 \$ \mu\$F electrolytic cap with a parallel 10-100 nF ceramic cap.

Smaller capacitors should go closer to the point they're decoupling, so your 100 nF cap should be as close as possible to the microcontroller to keep the current return path short.