Fire crews were called to the property in Lon Tanyrallt, Alltwen, near Pontardawe, at 01:40 BST on Wednesday.
Jac Davies was rescued from an upstairs bedroom but died at the scene.
His three-year-old brother was also rescued and taken to Swansea's Morriston Hospital along with his sister, six, an 11-month-old brother and their mother who had all managed to escape the blaze.
They were all suffering from suspected smoke inhalation which is not thought to be life-threatening.
In a tribute, his mother, Jennifer Davies, said in a statement: "My big, handsome blue-eyed boy - you're my life, my world, my everything and will never, ever forget you.
"Go ride your train up to heaven and wait for me.
"Kelsey, Riley and Andrew will always remember you."
Mid and West Wales Fire and Rescue Service said Ms Davies, called for help and neighbours tried to get in to help the stranded children.
The cause of the blaze is being investigated by the police and fire service.
Chris Margetts, senior operations manager for the fire service, described the blaze as "very severe" but said smoke detectors may have raised the alarm.
He added: "It is a very close community, and specialist teams from the fire and rescue service will be working with the community in the coming days."
Outside the family home, visibly shocked neighbours stood in the street with their hands over their mouths, trying to make sense of the tragedy.
One woman was consoled by a police officer as she laid a floral tribute at the scene along with two teddy bears.
A card left with some flowers said: "To Jack. So sad to hear you are sadly gone. Harry will miss you, seeing you smile and playing with you at school. Gone too young."
South Wales Police Det Ch Insp Esyr Jones, of Swansea and Neath Port Talbot CID, said: "I would like any witnesses or anyone involved at the scene who have not yet spoken to police to come forward with any information they may have."
A four-year-old boy has died following a fire at a house in Neath Port Talbot.