Two bombs exploded in the Mulberry Bush and the Tavern in the Town in November 1974, killing 21 people and injuring 182 others.
Six men were jailed in 1975 but their convictions were quashed in 1991.
The service, at the city's St Philips Cathedral, paid tribute to other terrorism victims.
As part of the service, Birmingham Library was lit up in blue, white and red to also remember the victims of the recent Paris terrorist attacks, the council said.
Justice 4 the 21 campaign group, set up by victims' relatives, is calling for another memorial to remember the victims.
There is already a monument off St Philip's Place in the city.
But the University of Birmingham and victims' families are discussing plans for a more prominent tribute - possibly in New Street near to where one of the pubs used to be.
The families are also lobbying the government for inquests into the 21 deaths.
Relatives met Home Secretary Theresa May and Northern Ireland Secretary Theresa Villiers earlier this year to make a fresh call for the hearings.
An inquest was opened days after the bombings - at the time the worst ever terrorist attack on English soil - but it closed without hearing evidence in 1975 after the guilty verdicts.
After the convictions were quashed, the former Director of Public Prosecutions, Barbara Mills, placed a 75-year embargo on files relating to a Devon and Cornwall Police inquiry into the West Midlands Police investigation, meaning those files are not due to be released until 2069.
Relatives of the victims of the IRA Birmingham pub bombings have attended a memorial service to mark the 41st anniversary of the attacks.