is there a way to blur the edges of the holdout shader
From your screens it seems to me like you want to smooth part of an edge that is (in reality) optically smoothed by Bloom Effect. 
If you can’t use bloom effect in post-production directly, try this … take your mask object, blur its alpha, invert and subtract it from alpha of fire object.

There are two issues to solve ...


ad First - your example material setup use for Volume - Emission shader, that is visible in viewport render, but invisible in render (F12). Material is rendered, but alpha for this material is not generated. From developer side the bug is viewport vs render (F12) inconsistency, but F12 is as designed = emission without alpha is correct. This can be solved by adding Volume Scatter shader.






ad Second - in blender wasn’t a way to get ID pass (or Cryptomatte) of object with volumetric material. From 2.82 beta build, blender use AOV that can output any state of material node tree. So I used AOV to separate specific alpha for Blurring.
(Properties Editor > Layers > Passes > Shader AOV, create one. Go to Shader Editor, add AOV Output node with proper name and set white colour. For more about AOV see Blender Manual or Blender Today #86.)


Note: I had to add Transparent shader to Surface (feeding only Volume didn’t object pass as well). Pass of this volumetric object get object shape, not volumetric texture. 

Since your scene consist only from fire I blurred mask object (cube).
To get "fire" pass you can use Alpha socket ...


Show before/after. Sorry for poor render (6 samples only).


... or convert render with ColorRamp into black&amp;white image. That is not mathematically accurate, but gave me nicer result. 






(I created also pass for orange fire, but in this comp was not needed.)