Adapt fatFS library to PIC32
Microchip Harmony and MLA both contain good fast file access libraries, supporting multiple backing stores including SD, MMC, CF-II, USB MSD, etc.  I suggest you give them a go since they are already tailored to the PIC32 chips.