PWM to analog RC based filter with minimum analog voltage level
Start with the simplest solution and go from there. All you need is 3 resistors. See the schematic below.

 

If you needed additional DC accuracy, you can re-clock your PWM signal with a low voltage buffer/inverter (A 74AHC1G04 is one option) powered by a dedicated 2.5V reference. Pull-down resistor \$R_2\$ would not be required in this case.