The protesters made the 12km (7-mile) journey to highlight high levels of bacteria in the bay.
They said they were concerned about the health of the athletes taking part in Olympic sailing competitions.
Trials for the events are due to start in Rio waters next Saturday.
The  authorities in Rio initially promised that 80% of the bay would be cleaned up in time for the Games, but they admitted more recently that they would not meet these targets.
Only one of eight water treatment plants has been built so far.
The Brazilian sailor and Olympic medallist Isabel Swan was one of the organisers of the protest.
She  competed in the Beijing Olympics in 2008 and says the area used for sailing in China had problems with a large algae bloom but this was dealt with two months ahead of the competition.
She told the BBC the failure by Rio de Janeiro city and state to meet their commitments made her feel "very bad".
"It's important that we can leave a legacy for our children and for Brazilian people," she said.
"It was very important to bring the Olympic Games to Rio, but we weren't able to clean the bay.
I want to create a movement that will inspire people to defend Guanabara Bay and help the government make a real commitment to cleaning it up. "
Trials for the Olympic sailing competitions due to start next Saturday have been hampered by concerns about the impact on the health of the athletes.
A recent investigation by the Associated Press showed that the waterways in Rio to be used in the Olympics are carrying high counts of disease-causing viruses directly linked to human sewage.
Sergio Ricardo, an ecologist and founder of the Bahia Viva (Living Bay) movement, which organised the protest, said the pollution which affects the bay is caused by a number of different factors.
"The main challenges are sewage treatment and industrial control. The oil industry is encroaching on the fishermen in the bay. We need public policy for fishing, reforestation and sewage treatment so that the bay can live."
A flotilla of more than 30 boats and water craft has cruised across Rio de Janeiro's Guanabara Bay to protest against water pollution with a year to go before the 2016 Olympics.