Video Transmitter/Receiver Books&Schematics
I would recommend looking at commercial transmitters and the ICs involved. Then read the datasheets/application notes of those ICs. Usually you can learn a lot just from application notes.