PIC24FJ64GA002 not running at speed
According to the data sheet:


  The processor clock source is divided by two to pro-
  duce the internal instruction cycle clock, FCY. In this
  document, the instruction cycle clock is also denoted
  by FOSC/2. The internal instruction cycle clock, FOSC/2,
  can be provided on the OSCO I/O pin for some
  operating modes of the primary oscillator.


So, if you are getting 4MHz out from the OSCO pin then the internal oscillator must be running at 8MHz.

8MHz is the base frequency of the internal Fast RC oscillator.

To get to 32MHz the 8MHz must go through the PLL to multiply it by 4.

In theory, with a clock of 32Mhz you should see 16MHz on the OSCO pin.

This you are supposedly already doing, both through the configuration settings and the OSCCON register.

All your postscalers are set to 1:1, which is good.

Things to try and/or clarify:


Is the core actually running at only 8MHz, or is the OSCO pin giving the pre-PLL frequency (unlikely)? Try running a simple program which toggles an IO pin HIGH and then LOW in the space of \$2 T_{CY}\$ and measure the width of the pulse.
Try running the chip with an external crystal at 8MHz and use the PLL to try and get 32MHz.  This can just be a temporary arrangement to confirm the PLL is being enabled.
Investigate looking at the state of the PLL to ensure it has settled as the first thing your program does.  I couldn't see exactly which flag to check for this during my cursory glance through the datasheet, but if using the PLL it should always be the first thing your program does.