How to Grow an Olive Tree from a Pit. The olives, as known as Olea, is a genus with about 20 species belonging to the Oleaceae family. Olives are native to warm temperate and tropical regions in Europe, Australasia, Africa, and southern Asia.