Jamal Mahmoud, 21, died at the London prison on Tuesday. Two prisoners, aged 34 and 26, have been arrested on suspicion of murder.
BBC News understands the weapon used was a "hunting-style knife".
The motive is not yet clear, although claims about a dispute between gangs or drugs debts are being investigated.
It is understood the stabbings happened in a prison wing and inmates were then moved into the jail's exercise yard, according to the BBC's Home Affairs Correspondent Danny Shaw.
He said a prison source had told the BBC the knife used in the attack had been recovered, and described it as "bigger than a Bowie knife but smaller than a machete".
The weapon was not improvised, the source said.
Mahmoud was attacked on the fifth floor landing of one of the prison wings, before being thrown over the railings, falling about 30ft (nine metres).
Mahmoud, from Enfield in north London, was jailed in July for helping to hide a machine gun.
A friend of the dead man, Danny Rynne, a scaffolder from Enfield, described him as "small" and "placid".
"He's got a girlfriend who's just had a baby that was born two weeks ago. It's a shame what happened. I don't know too much about what it's about but it's a shock to everyone," Mr Rynne said.
How dangerous are our prisons?
The Prison Governors Association (PGA) said inmates at Pentonville were living in "squalid and brutal conditions".
"If a society is judged by how it treats those it locks up, then we are in a very dark place," it said.
22,195
Assaults
31%
Rise compared to the same period of the previous year
2,953 Serious assaults
5,423 Attacks on prison staff
5 Apparent homicides (in 12 months to June 2016)
The association has renewed calls for a public inquiry amid what it said was an "unprecedented" rise in prison violence and suicides.
In response to those comments, a Ministry of Justice (MoJ) spokesperson said the justice secretary had made clear that safety in prisons was "fundamental to the proper functioning of our justice system and a vital part of our reform plans".
"We are fully committed to addressing the significant increase in violence, self-harm and self-inflicted deaths in our prisons," the spokesperson added.
The MoJ said measures would include spending an extra Â£14m in 10 prisons and increasing staffing levels by more than 400 prison officers.
Analysis: BBC Home Affairs Correspondent Danny Shaw
Pentonville's Governor Kevin Reilly has noted that in recent weeks people had "felt anxious" about violence at the jail.
Writing in an internal newsletter, seen by BBC News, Mr Reilly said reducing violence was one of his "top five priorities".
He said although there had been a slight reduction in violent incidents during the month, people remained concerned.
In July the Pentonville Independent Monitoring Board said the government should "knock down" or "urgently upgrade" the 174-year-old prison.
The board said the jail was "decrepit" and blamed the former legal high Spice for driving an illicit economy which in turn had led to violence, self-harm and bullying.
In its annual report on Pentonville, the board said there were 16 violent incidents each week but the number had fallen year on year.
The Pentonville death is the third killing this year in jails in England and Wales, and the 11th since the start of 2015.
Visitors queuing to see friends and relatives inside the prison spoke of their safety concerns.
"My son is 21. I haven't felt the same since I heard the news," one woman said.
"As a mother everyone tells you at least you know where he is and he is safe - but actually he's not safe, is he?
"No matter their age or what they have done, no-one deserves to die in prison."
"What has happened is horrendous," former prisons minister Andrew Selous told BBC Radio 4's Today programme.
"I have met the parents of prisoners who have been murdered and it is horrific and my deepest sympathy goes out to all those affected.
"The government absolutely gets the seriousness of the violence issues, there is a huge amount of work being done to reduce violence."
The director of the Prison Reform Trust, Peter Dawson, called for fewer people to be sent to jail to "reduce the pressure" on prisons.
"The Victorians thought [Pentonville] could accommodate 900 prisoners... we say it can accommodate up to 1,300.
"That means in practice that almost everybody in a prison like Pentonville can expect to share a cell which the Victorians thought was fit for one person."
No prison officers are understood to have been injured in the attack, the Prison Officers' Association said.
In a statement, the union said "the unprecedented rise in violence in all of our prisons must not be underestimated".
It added: "We now ask for the Ministry of Justice to fully investigate this matter and the underlying problems within the prison estate."
In July 2015, the then Justice Secretary Michael Gove said: "Pentonville is the most dramatic example of failure within the prison estate, but its problems, while more acute than anywhere else, are very far from unique,"
A Prison Service spokesman said: "Police are investigating an incident at HMP Pentonville. It would be inappropriate to comment further at this stage."
Pentonville Prison
A prisoner has died and two others were critically injured in a stabbing at HMP Pentonville, leading to claims the prison system is in a "dark place".