The 16-year-old went to Michael Harrington's Grangetown home, in Cardiff, after suspecting him of sexually assaulting a schoolgirl, and fatally stabbed the 67-year-old.
He was convicted of manslaughter after being cleared of murder at Cardiff Crown Court.
Mrs Justice Nicola Davies said the case was a "tragedy".
There was never a formal complaint made against Mr Harrington, the court heard.
Sentencing the boy on Friday, the judge told him: "This was a tragedy for you and a tragedy for your family and is something you will live with for the rest of your life.
"You are of positive good character and you genuinely believed that this man had sexually abused (the schoolgirl).
"But you took the law into your own hands and no one has the authority to do that.
"The offence is so serious that only a custodial sentence is justified."
The teenager, who cannot be named, went to Mr Harrington's home on 3 August, stabbing him in his neck and ear with a steak knife.
Mr Harrington died four days later.
The boy claimed he never meant to kill the man, but stabbed him during a struggle with the knife.
Despite his denials, he was found guilty of manslaughter by a majority verdict of 11-1 by the jury.
A schoolboy who killed a pensioner for allegedly abusing a girl has been given a three-year custodial sentence.