Philip belonged to the Wetterau Association of Imperial Counts, in which Albert took a leading role.
In 1572, Philip had Wanborn Castle, a 12th-century structure in the vicinity of Saarbrücken, torn down and a Renaissance style hunting lodge with four wings named constructed on the spot.