Use UART PIC24FJ64GB002
A diagram would help convey what you're doing. I think this is what you have implemented:

"a device" (USB device) &lt;-> USB &lt;-> PIC24FJ64BG002 (USB host) &lt;-> UART &lt;-> Arduino

Let's take a look at the clock options for this part:



Let's also look more closely at the USB-specific clock stuff:



CONFIG2 contains your oscillator-related stuff:

_CONFIG2(
POSCMOD_HS &amp;                      
I2C1SEL_PRI &amp;
IOL1WAY_OFF &amp;
OSCIOFNC_ON &amp;
FCKSM_CSDCMD &amp;
FNOSC_PRIPLL &amp;
PLL96MHZ_ON &amp;
PLLDIV_DIV2 &amp;
IESO_OFF
)


Recapping what we know:


Your source crystal is 8 MHz (input from POSC)
POSCMOD is set to HS Oscillator Mode
FNOSC is set to Primary Oscillator + PLL
PLLDIV is set to divide-by-2


Also, from the datasheet:


The default setting for CPDIV is 0x00, which is divide-by-1 (32 MHz)


So, what's our conclusion? I would say that unless there's other code messing with the bits somewhere, the situation is as is shown in figure 8.2 - HSPLL is running at 32 MHz (96 MHz USB PLL output, divided by three, then divided by one).

And, because we know that

\$ F_{CY} = \dfrac {F_{OSC}}{2} \$

the answer should be 16 MHz.

Now, to your baud rate formula:

baud = (FCY/(16*baud))-1;


(It's bad practice to have the same variable name on both sides of the equation: the left side should be renamed to BRG or something more like what it is - it's the baud rate generator value, not the baud rate itself.)

If you look up the PIC24 Family Reference Manual for UART, you'll find the values pre-computed for you:




By default, BRGH is zero so unless code is messing with it, your baud rate generator should be set to 103 decimal. (Your formula is for BRGH - 0 and with a 16 MHz clock rate agrees with this table.)

I would dare a guess and say that your UART issues, if they persist, aren't clock related...