Named Petit Palais to distinguish it from the Palais des Papes, the original structure was built during the period of the Avignon Papacy by Cardinal Bérenger Fredoli the Elder in around 1318-20.
The building suffered during its use from 1396 as a fortified citadel during the Western Schism, and was a wreck by the time the war ended in 1411.