Paper in oil (PIO) capacitors: what is special?
Oil submersion has been commonly used for high voltage applications such as power engineering, for capacitors as well as circuit breakers / relays up until the late 70s, and is still used today, albeit rarely, apparently.The benefits cited by the previous generation of engineers include:


Suppression of arcing to a higher voltage than air
Improved cooling of local heat spots due to circulation of the fluid
Improved insulation between the contact electrodes, having a higher dielectric constant than air
Elimination of bubbles within the paper or film, or between electrodes, thus avoiding sporadic arcing and variation in characteristics 
In specific high or low pressure deployments, oils were preferred due to being less compressible (almost not at all) compared to air. Compression or rarefaction would unpredictably change the distance between, and thus capacitance between, the electrodes.


Vintage audio amplifiers were vacuum-tube based, and often operated at high voltages, thus justifying the use of such high voltage friendly components. However, modern semiconductor based amplifiers work at (typically) much lower voltages, thus ought not have any strong engineering justification for continuing to use oil-filled capacitors. There's no accounting for the spiritual beliefs of audiophiles, though.

Footnote:
Oil was later largely replaced in circuit breakers by SF6 gas, which is not flammable, hence eliminating the safety hazard of oil. Vacuum based circuit breakers are also now in use in high voltage deployments. These two options do not appear to be prevalent in capacitors, though.