Why is the default image format for saving renders is in PNG?
TL;DR: It's a well supported file type that is a pretty good mix of file size, detail preservation, and alpha support.

I'm not a developer, so I can't say for sure, but I think there are a number of good reasons why PNG is the default format. It's by no means perfect, there is no perfect image format, they all have some benefits and disadvantages.

One of the  most likely reasons is that PNGs are widely used and well supported. You won't run into another program that doesn't support PNGs. Being a software that can be used with other programs, it would make sense for the default file type to be a widely accepted one.

PNGs also have better compression that JPEGs for the average blender render. JPEGs are more commonly used for photos with soft transitions, as the lossy compression is much less obvious. Once you add an sharp details though, such as edges or text, the compression becomes an issue. That is most likely why JPEG is not the default.

While EXR has a number of advantages that make it great for compositing and serious rendering, for the average blender user it's just not necessary. The downsides of EXR is the file size and the potential conflicts with other software or websites. I would guess that most blender users never even touch EXR, and thus the benefits would be lost, yes some disadvantages would be noticed.

Overall, I think PNG was chosen as its pretty good at everything. It has an alpha channel, lossless compression, decent color range, and relatively small file sizes. While you could find a file type better and several of these criteria, you won't find a well supported one that is better than PNG at everything.