The visitors' centre was one of the few elements of the County Antrim prison praised in a recent damning report.
But the bus that took people between the centre and the prison stopped after the start of a new contract.
The Northern Ireland Prison Service said it has arrangements in place to "assist people with mobility issues".
Just before Christmas, a private company took over the running of the visitors' centre from the Quaker Service, a family support charity, and the bus service ended.
Malcolm McCormick said that has made it difficult for people to visit the jail as it is too far to walk.
He had been visiting a friend regularly in the prison, but because of his disabilities he could not drive and had to use public transport.
But when he arrived at the visitors' centre at the start of December he was told he would have to make his own way to the prison entrance.
"[The prison] is not the right access level for visiting," he said.
"They really don't care about people who've got disabilities, or people who have to go visit their loved ones inside."
Until December, Mr McCormick chose a train closest to his visit, and then waited at the visitors' centre until his time slot, like many other prison visitors.
They were then taken to the main prison in a minibus, which, like the visitors' centre, was run by the Quaker Service.
But when the contract for the visitors' centre was awarded to a private company, the provision of a bus was not included in the tender.
It is about a quarter of a mile from the centre to the prison, which Mr McCormick said was too far for him and others, "especially people with mobility problems, people with bad sight, or carrying children".
He said although a taxi was able to drop him at the main building from the train station, he was not able to wait in the prison building itself and that the visitors' centre was the only place people could wait.
Mr McCormick felt his only option was to get a taxi all the way from Belfast, costing him "about forty quid on each visit".
The recent report on Maghaberry Prison was damning on almost all fronts, but it did give one area a glowing review.
It said: "The welcoming visitors' centre continued to offer good support, including to first-time visitors who were identified.
"The bus transporting visitors between the visitors' centre, the searching area and the prison also now provided a service to and from the local Moira train station."
Mr McCormick said the prison service needs to do more to help visitors.
Raymond McCartney, the deputy chair of the Northern Ireland Assembly's justice committee, said there should be "compassion" shown to prisoners' families and friends visiting the jail.
"I think it's an important service - the distance from the visitors' centre to the front gate is considerable," he said.
"Recently, when I was in the prison and spoke to the senior managers, they believed a solution could be brought forward.
"I would be hopeful that, perhaps in the coming weeks, that we could see a resolution to this."
In a statement, the Northern Ireland Prison Service said: "The contract for the visitors' centre at Maghaberry was awarded through an open and transparent procurement competition.
"The prison service will continue to work with the provider to ensure that services are in place to support relationships between families and prisoners."
A disabled man has said his visits to Maghaberry Prison have become more difficult since changes were made to the visitors' bus service.