Haki Africa said the figure could be higher but some families were reluctant to come forward for fear of victimisation by state security agents.
The police oversight body said 52 officers were on trial in connection with such killings countrywide.
The coastal region has suffered several terror attacks in recent years.
The Somali Islamist militant al-Shabab group has been targeting Kenya over the deployment of Kenyan forces to Somalia.
Most of the victims were youths including returnees from Somalia who had surrendered after the government promised them amnesty, Haki Africa's report said.
Others were allegedly killed as a result of excessive use of force during police crackdowns on protests and other religious groups that had been accused of radicalising young people.
Saada Suleiman, who attended the report's launch in the capital, Nairobi, said her husband had been missing for two years since his arrest in a raid on Masjid Musa mosque in Mombasa known for its radical sermons.
Haki Africa Executive Director Hussein Khalid said some of those killed were terror suspects, including several radical Muslim clerics, but this was no excuse for the state to execute them without due process.
The BBC's Abdinoor Aden in the capital, Nairobi, says officers from the anti-terror unit have been heavily implicated in the report.
Murshid Nassir, who heads the National Police Service Commission which vets officers, told the BBC that so-called "death squads" did not exist within the force and extrajudicial killings were not condoned.
While the chairman of the Independent Policing Oversight Authority promised to take action against officers found guilty of engaging in the killings.
Macharia Njeru said 52 officers had been charged and more than 300 officers were being probed for gross human rights violations.
Kenya's anti-terror police have carried out at least 81 extrajudicial killings in country's mainly Muslim coastal region since 2012, a rights group says.