Which cable to use from iMac 5k to 30 inch Dell monitor?
I'd start with a dual link dvi adapter if you brought both to me. 


Mini DisplayPort to Dual-Link DVI Adapter


If you have bought the display, dell support should have a KB article / manual explaining if dual link is needed for the resolution you want. If you haven't bought, make sure the pre-sales support can answer that. 

That iMac can drive far higher resolution at 60 Hz:


  single-stream 4K (3840x2160) - https://support.apple.com/en-us/HT206587


You might even get by with single link dvi - it will certainly work, just possibly at lower resolution if your overriding concern is saving $$