PIC 32 interface to Logitech force feedback USB joystick
Microchip have various examples that use their USB stack (comes with all demos and documentation) and work with various drivers that you can adapt. I think HID would probably work for your joystick.

So the answer is, no you (almost certainly) don't have to write your own driver, but you will have to adapt the firmware a bit.

Here is a screen snip of (some of) the host examples I have on my laptop: