since the huguenots essentially seized control of the colony in 1720 and sent his soldiers packing , the governor had been relegated to ceremonial duties , living off of what silver they would give him .