Is it possible to downgrade oracle 11.1.0.6.0 to 10.2.0.4.0?
Yes, it is possible, as stated in the 11.1 documentation:

Supported Releases for Downgrading

Oracle mentions 10.2.0, because ideally you need not to set anything more specific (more digits). This is explained in:

How To Change The COMPATIBLE Parameter And What Is The Significance? (Doc ID 733987.1)


  Use only the first 3 digits for the compatible parameter unless there
  would be some very specific instructions to do otherwise.  Patch sets
  are normally not intended to add/change actual functionality, hence
  there is no need to increase the compatible parameter and set the 4th
  digit.


If you set it to 10.2.0.4.0, the downgrade should be still successful to version 10.2.0.4, but not anything lower.