Napo, which represents probation workers, said it understood the Stanley knife went missing at HMP Northumberland two weeks ago, but no lock-down or search was ordered.
It comes a week after concerns were raised about safety in the prison.
Sodexo Justice Services, which took over the facility in 2013, said all appropriate security procedures were followed.
Mike Quinn, from Napo's Northumberland branch, said: "The report of a knife potentially in the hands of a prisoner is deeply concerning.
"If true, it just confirms the dire state of the prison, and has major safety implications for staff, visitors and prisoners.
"We would repeat calls made last week for an immediate inspection of the prison."
The Ministry of Justice said that as it was a privatised prison, it was up to Sodexo to comment.
A Sodexo spokesman said: "All the appropriate security procedures were followed - to suggest otherwise is simply untrue."
A knife remains "unaccounted for" at a privatised prison, a union has claimed.