Fentons Reagent tips: H₂O₂ + FeSO₄
I'm not an expert in Fenton chemistry but it appears that difficulty in getting the Fenton reagent to reduce COD to below 500 ppm is not unusual.  


Does your particular wastewater have  chelants such as phosphates, EDTA, formaldehyde, and/or citric/oxalic acids?
Perhaps at long reaction times all the iron you add is being precipitated as iron (hydr)oxides.  Could you lower the pH further?



  Reaction rates with Fenton’s Reagent are generally limited by the rate
  of $\ce{.OH}$ generation (i.e., concentration of iron catalyst) and less so
  by the specific wastewater being treated. Typical $\ce{Fe:H2O2}$ ratios are
  1:5-10 wt/wt, though iron levels &lt; 25-50 mg/L can require excessive
  reaction times (10-24 hours). This is particularly true where the
  oxidation products (organic acids) sequester the iron and remove it
  from the catalytic cycle. Fenton’s Reagent is most effective as a
  pretreatment tool, where CODs are >500 mg/L. This is due to the loss
  in selectivity as pollutant levels decrease...In addition to free radical scavengers,
  the process is inhibited by (iron) chelants such as phosphates, EDTA,
  formaldehyde, and citric/oxalic acids. Because of the sensitivity of
  Fenton’s Reagent to different wastewaters, it is recommended that the
  reaction always be characterized through laboratory treatability tests
  before proceeding to plant scale.


My tip would be to increase the concentration of both $\ce{H2O2}$ and $\ce{FeSO4}$, and, if possible, to make sure the pH is 3 (or maybe even lower).  If that doesn't work, do you think iron chelators could be present in your particular wastewater sample(s)?