How to get pickled mushrooms to be less sour?
Draining the juice, refilling the jar with oil and then storing to reduce the acidity won't work because:


Acid and lye can be diluted only with water or liquids like water (if I remember correctly, ethanol will also work) but not oil. 
You might run into the risk of botulism. (Maybe not since the mushrooms are sour. If you don't measure the pH you can't be sure). 


The only thing you can do is to soak the portion of mushrooms you intend to eat in water before eating. Replacing the brine with water of the whole jar beforehand to lower the pH will shorten the shelf life significantly.