How is the control of airspace over disputed territories determined?
International air traffic control is divided into Flight Information Regions (FIR) by ICAO through international agreement. Normally this works fairly well. Even countries that are antagonistic to each other wish to maintain safe air traffic. 

Currently there is one situation that causes some real consternation, though. That's the dispute between Greece and Turkey over northern Cyprus. ICAO officially assigns the region to Greek ATC, but the northern part of Cyprus, which aligns with Turkey feels that the airspace is theirs and operates a separate ATC that  is not in cooperation with the official ATC, causing confusion for pilots.

They often create long and ridiculous notams to state their point.