How IC encoder ΤΧ-2, encodes signals?
According to bit format (page 6), there are two frequencies. In case
  of RF transmission, which is/are the frequency/ies of the encoded
  signal?


The encoded signal (pin SO) comprises two frequencies in three parts. Part A is a marker to indicate that data is about to start it is 500 Hz on a 75% mark space ratio. Part B is the data and it appears to be "n" cycles of 1 kHz. Part C is the same as part A and marks the end of the data. The data is a repeated 1kHz cycle (n times) where n varies between 4 and 64. This is decoded by the receiver.

Please bear in mind that these frequencies are not the RF transmission frequency - this is set by an unspecified circuit shown on page 9.

For infrared you use a different output pin (SC) and this has the same data format but the data is modulated by a carrier of either 38kHz or 57kHz. It also appears to me that although the data format is the same it will be a slower baud rate because the oscillator is meant to run slower for standard IR applications.


  In case of IR transmission when oscillator generates 114 KHz or 76 KHz? It has to do with electronic components around oscillator?


Both infrared and rf applications require a variable resistor (or set of fixed resistors) connected that set the oscillator to run at either 114 kHz or 76 kHz (IR) or 128kHz (RF).