Cannot set "is_overridable_library" option via python ["_RNA_UI"]
The value behind is_overridable_library can be set through the method property_overridable_library_set() of an object. The following example demonstrates how to set a custom property named "prop" to be overridable.

import bpy


obj = bpy.context.object
obj.property_overridable_library_set('["prop"]', True)


The use_soft_limits is dynamically evaluated and it's not necessary to set it when using the Python API. If the soft_max or soft_min are different than the max or min, it will automatically show as enabled.

Calling bpy.ops.wm.properties_edit() results in an error message, because it expects that self._last_prop is set. This only happens in the invoke() function, not when calling execute() directly. Therefore the execution context has to be set by passing INVOKE_DEFAULT.

bpy.ops.wm.properties_edit("INVOKE_DEFAULT", data_path="object", property="prop", value="1.0", default="1.0", min=0, max=1, use_soft_limits=False, is_overridable_library=True, soft_min=0, soft_max=1, description="", subtype="NONE")


The downside of this is that the UI popup appears and has to be confirmed. Therefore the operator cannot be used without user interaction.