EXR color difference to PNG
Explanation first:

PNGs can't handle pixels that are emissive and transparent at the same time because PNG does not support associated alpha transparency.

You are on the right track using EXR, what you need to understand is that EXRs are exported as linear information (meaning no "gamma curves" are applied to make them display nicely on a monitor). The images on EXR format are not constricted by the limits of Display Referred, with none of the color transforms set by the color management. (visit this link for a deeper explanation). Think of EXR images as exact copies of the render layer with all of the brightness and color, which might exceed the limits of what the monitor can display.

An attempt to answer:

EXR have to undergo some color transform before they can be displayed nicely on the screen. If you import them onto other software (like photoshop) they need to be interpreted as linear before they can be displayed,and probably scaled so that they fit within the values used by the display device. If you do the compositing in blender, the EXRs should have no problem. Please add information on how you are viewing them and what you want to do with the image.