There were very few native people living in the Cape . Because of this , slaves were brought from Indonesia , Madagascar and India to work at the colony . The Dutch transported slaves from Indonesia , Madagascar , and India as labor for the colonists in Cape Town .