Mark Jones, 45, of Cwmbran, Torfaen, fractured Amelia Jones's skull when he was supposed to be looking after her.
He had denied murder but was convicted at Newport Crown Court on Thursday and jailed for life.
He was also jailed for four years after admitting perverting the course of justice which will be served concurrently.
Mr Justice Wyn Williams said Jones had shown no remorse for his "wicked and despicable" actions.
He said: "It may well be that you are never released from prison or if you are, you will be a very old man."
The court heard he had "totally destroyed" the life of his daughter, Sarah Jones, and given her surviving children nightmares.
A victim impact statement from Ms Jones, said: "It really upsets and saddens me now, looking back, I spent such little time with her.
"It seems like she didn't really exist, like she was some kind of dream."
The court was told the rest of her children were taken from her for about 18 months, between November 2012 and April 2014.
One child in particular was "afraid his grandfather would kill him if he got out of prison", the statement said.
"There have been times when I blamed myself for what happened to Amelia, because I was the person who invited him back into my life," Ms Jones added.
"As far as I'm concerned he is no longer my father."
Amelia died in November 2012 after suffering a "catastrophic" bleed to the brain.
During his trial, Jones claimed he fell with Amelia in his arms on one occasion and passed out while holding her on another.
But the court heard she had suffered a skull fracture, brain swelling and multiple rib fractures and had been injured at least three different times.
The prosecution said the injuries "could not have been accidental" and were all deliberately inflicted by her grandfather, who frequently had sole charge of Amelia.
The prosecution said he hurt her because he disliked her father, Ian Skillern.
A man who murdered his five-week-old granddaughter has been jailed for life and will serve a minimum of 25 years.