Of the 4,150 full-time firefighters across Wales' three fire brigades, only 40 are black and minority ethnic (BME).
Five of north Wales' 880 firefighters are BME, in mid and west Wales it is 15 of 1,390 and in south Wales it is 20 of 1,880. BME people make up 4% of Wales' population, according to the latest census.
Fire minister Mike Penning said diversity was "not an optional extra".
"Firefighters, like police officers and other local emergency services personnel, should reflect the communities they serve and we expect fire and rescue authorities to do much more to improve BME... representation," he added.
The Fire Brigades Union (FBU) said the fact just 4.4% of firefighters in England and Wales were BME was "totally unacceptable".
The figures, released by the Department for Communities and Local Government, show 10 firefighters, all in south Wales said "don't know" when asked about their ethnicity.
Although there are no statistics for a breakdown of Welsh cities, in places like Greater Manchester, just 2.8% of firefighters are from an ethnic minority compared to 16.2% of the population as a whole.
Mick Nicholas, the most senior black member of the FBU, told BBC Radio 5 live the fire service had been "inconsistent" at recruiting from non-white communities.
Just 0.9% of Wales' firefighters are non-white, official figures have shown.