How to build an operator that corrects StringProperty("subtype = 'DIR_PATH') from relative path?
With a setter and getter.
Instead of writing an operator, here is a suggestion re using a boolean property to toggle between the two, and set the path accordingly.
IIRC, (and I didn't) had asked a similar question re globbing string properties with a file or folder subtype, in particular to glob a file.  Have asked a new question.
For the case shown in question, consider adding your own &quot;use relative&quot; property
Have set up a generic setter / getter on path property, and on boolean property the getter simply looks at if the path starts with &quot;//&quot; it is relative, and toggle using the setter.  Will also use the option available in filebrowser of setting relative_path of the invoked and active in the file browser,  operator bpy.ops.buttons.directory_browse()

Example, file saved in tmp folder, draw method added to text editor footer for testing
import bpy

def update(self, context):
    print(&quot;update&quot;)
 
def set(self, value):
    print(&quot;setter&quot;) 
    self[&quot;foo&quot;] = value
         
def get(self):
    return self.get(&quot;foo&quot;, &quot;//&quot;)

bpy.types.Scene.foo = bpy.props.StringProperty(
    subtype='DIR_PATH',
    default=&quot;//&quot;, 
    get=get,
    set=set,
    update=update,
    )
    
def get(self):
    return self.foo.startswith(&quot;//&quot;)

def set(self, value):
    print(&quot;set relative&quot;, value)
    if value:
        self[&quot;foo&quot;] = bpy.path.relpath(self.foo)
    else:
        self[&quot;foo&quot;] = bpy.path.abspath(self.foo)    

bpy.types.Scene.foo_relative_path = bpy.props.BoolProperty(
    name=&quot;Use Relative&quot;,
    get=get,
    set=set,
    )


def draw(self, context):
    layout = self.layout
    row = layout.row(align=True)
    row.prop(context.scene, &quot;foo&quot;)    
    row.prop(context.scene, &quot;foo_relative_path&quot;)



classes = []


def register():
    for cls in classes:
        bpy.utils.register_class(cls)
    bpy.types.TEXT_HT_footer.append(draw)

def unregister():
    for cls in classes:
        bpy.utils.unregister_class(cls)
    bpy.types.TEXT_HT_footer.remove(draw)

if __name__ == &quot;__main__&quot;:
    register()