By the early 17th century , the island and its smaller neighbors ( above all , the Tortuga island ) became places often visited by Caribbean pirates . By the early 17th century , the island and its smaller neighbors ( notably Tortuga ) became regular stopping points for Caribbean pirates .