Are stale blocks retained forever or are they "pruned" automatically at some depth in the chain? In theory the chain could be re-mined from a stale block, and their existence seems superfluous and just use up data storage, are they automatically pruned after x height, and if so where in the code does this happen?
In Bitcoin Core (as of 0.19.1) they are not pruned unless pruning is enabled in general (in which case they're pruned along with all other blocks).

They are indeed superfluous, but they're also cheap to keep. Stale blocks are assumed to be rare (if they're not, the network has bigger problems), and if disk space is an issue in general, pruning should already be enabled.

Note that only blocks with verified PoW and a chance to be part of the best chain are actually downloaded and stored. This means that exploiting stale block storage as a DoS attack is incredibly expensive.