Little is known about al-Mada'ini's life.
In Baghdad, al-Mada'ini was a pupil of the Mu'tazili scholar Mu'ammar ibn Abbad al-Sulami (died 830), and came under the patronage of the musician Ishaq ibn Ibrahim al-Mawsili (died 850), with whom he also formed a close and lasting friendship-it is said that he died in al-Mawsili's house.