Is it a good idea to whiten a colored noise in order to apply formulas for AWGN?
The short answer is yes, whitening noise is usually recomended (if you know the spectrum of the noise) but whitening also whitens your signal.  Detecting random signals in colored noise is a well known problem is sonar signal processing.  There are a number of derivation that I like particularly:

Nielsen, Richard O. Sonar signal processing. Artech House, Inc., 1991.

and Burdic

@book{RN17,
   author = {Burdic, William S.},
   title = {Underwater acoustic system analysis},
   publisher = {Prentice Hall},
   ISBN = {0139476075},
   year = {1991},
   type = {Book}
}


Also the updated 

@book{van2004detection,
  title={Detection, estimation, and modulation theory, part I: detection, estimation, and linear modulation theory},
  author={Van Trees, Harry L},
  year={2004},
  publisher={John Wiley \&amp; Sons}
}


in section 4.3

and if you can find it, in the old volume 3

If you look, a copy of Eckart's  classic paper can be found at Scripts.