Remove from CollectionProperty
As I was about to click the button to post this question, I realized maybe the CollectionProperty class extends/has the same methods as a dictionary or list. Anyway, with a little trial and error I found that it does (sort of).

To remove a single element:

context.world.my_collection_prop.remove(2)


To clear the whole collection:

context.world.my_collection_prop.clear()


I really don't understand how details like this get left out of the Blender API documentation.