How to set radio register of cc2500 and cc2511 to maximum data rate (500 Kbps)?
Does your hardware works with a lower data rate?
Can you connect the two units with a coax cable and two 10dB attenuators (One on each unit) and verify a link at 500kbps? 
Do you get CRC or framing errors from the devices? 
Do you have a wifi network on channel 1, bluetooth tranceiver, other 2.4GHz devices near your units?

Edit: Did you check the crystals? They should be within 80ppm of each other. For 2 nodes with 27MHz crystals of the CC2500 it means a maximum of 200Hz difference between the 2 units. The CC25xx requires a maximum 40ppm error from its timing source (Crystal or oscillator). Try generating a CW signal and measure the difference with a spectrum analyzer.

To make measurement of the crystal easier, you can output its frequency/192 to GDO0.