Entity forms bypass the access handler
In your delete route add _entity_access: yourentity.delete to the route requirements.


As Ivan pointed out in the comment, this is a matter of the route not automatically requiring entity access when it points at an entity form. The entity access requirement must be added to the route explicitly.