Rhea Silvia was the daughter of Numitor Silvius , king of Alba Longa , a legendary town founded by Ascanius , son of Aeneas , prince of Troy . Their maternal grandfather was Numitor , rightful king of Alba Longa , a descendant of the Trojan prince , Aeneas and father to Rhea Silvia ( also known as Ilia ) .