Get correct row version based on combination of dates - query simplification
If I've understood what you are trying to do correctly then

WITH T AS
(
SELECT *, 
       RN = RANK() OVER (PARTITION BY  iud, id, date ORDER BY insert_date DESC, update_date DESC)
FROM #temp
)
SELECT *
FROM T 
WHERE RN=1;


May well perform better - especially if there is a covering index on (iud, id, date, insert_date DESC, update_date DESC)