Banksia rosserae is known only from an area south-south-east of Mount Magnet in inland Western Australia.
Although most of the land around Mount Magnet has the heavy clay soil and Acacia aneura (Mulga) scrub common throughout semi-arid Western Australia, the area on which B. rosserae occurs is unusual in having deep yellow sand and laterite.