The manor passed to Edouard de Provana, of a northern Italian family, who in 1386 married Marguérite de Donneloye, daughter of Othenin, and in 1440 Jacques de Glâne, lord of Villardin and de la Molière, and his descendants possessed it until 1528, when Catherine de Glâne married Aubert Loys.
On the death of Jean Loys in 1739 the castle passed to his son Georges, lord of Orzens, who died without issue in 1753, and then to his brother Paul (died 1784), subsequently to Paul's son Etienne-Charles, the last lord of Villardin, who died without issue in 1800.