The former HSBC bank building on the corner of Bute Street and James Street is due to house 16 flats including a four-bedroom penthouse apartment.
Cardiff planners said the proposals would help regenerate the area.
The Victorian Society opposed the plan, saying it "would cause a high degree of unjustified harm" to the building.
The four-storey building was constructed in 1874 for coal owners and exporters Cory Bros and was altered and extended in 1914.
The building has been empty since 2014 and planners, in their report recommending approval to councillors, said the proposals were "welcomed in principle as the development would revitalise a large long-term vacant listed building".
They said the changes to the roof showed "appropriate regard to the special interest of the building".
But The Victorian Society had objected to the plans - particularly the penthouse - saying it would "cause a high degree of unjustified harm to the listed building and surrounding Mount Stuart Square Conservation Area".
It said the building "proudly and splendidly evokes the extraordinary wealth Cardiff enjoyed at the end of the 19th Century".
Cardiff council's planning committee passed the proposals on Wednesday subject to conditions and the development starting within 18 months.
Plans to turn a Grade II-listed building in Cardiff Bay into a restaurant and flats have been given the go-ahead by the council.