Is 香气袭人 only ever used for a good smell?
袭 in 袭人 is referring to 'to rush '  

"袭人" means "rush upon people"

Example:

一阵阵恶臭袭人而來 - waves of horrible stench come rushing upon people

一阵阵花香袭人而來 - waves of floral fragrant come rushing upon people

強烈硫黃味袭人鼻腔 - a strong sulfur smell rush upon people's nasal cavities

花香 is a popular topic for poets and "花香袭人" (floral fragrant rush upon people)sound poetic but not any other smell. Phrases like "肉香袭人，酒气袭人，屍臭袭人" all sound unnatural because these smell are not popular topic for poets

Meanwhile,  "扑鼻" (rush into nose) is as literary as "袭人" but less poetic sounding. "花香扑鼻，肉香扑鼻，酒气扑鼻，尸臭扑鼻" are all natural sounding literary phrases.