How can I alter next value of membership id (which would be auto incremented)?
This is more of mysql than CiviCRM. You will need to reset the AUTO_INCREMENT value for that table.

ALTER TABLE civicrm_membership AUTO_INCREMENT= 10001


Thanks
Pradeep