Media playback is not supported on this device
Elis Ligtlee of the Netherlands led from the front to win gold, with Australian Anna Meares in third.
James, 24, missed the London Games and then had the best part of 18 months out with injury and health issues.
"I was so desperate for that medal," the Welsh star told BBC Sport. "I can't believe it happened."
She was last with one lap to go but found a way around the outside to force her way into second on the line to win Britain's fourth medal on the track.
It was a second medal in two Olympic kierins for GB after Victoria Pendleton won the event in London, where it made its debut.
The silver medal came just minutes after Laura Trott became the first British woman to win three gold medals as she won team pursuit gold with Joanna Rowsell-Shand, Elinor Barker and Katie Archibald in a new world record time.
Abergavenny-born James, who was watched by her boyfriend George North, the Wales and British and Irish Lions rugby player, told BBC One: "I came in with good form and I love the keirin and I love racing.
"Once I got through the first round, I just wanted to enjoy the day.
"It was so special, I have 10 of my family here. My mum has never been on a long-haul flight before.
"I have had so much support from British Cycling, it has been a tough two years and I couldn't have done it without everyone's help."
North later tweeted: "Words can't describe how proud I am! Everything she's been through. What an absolute hero."
James underwent an operation in 2014 to remove potentially cancerous cells following a routine smear test.
That scare was followed by keyhole surgery on a shoulder injury and then months of rehabiltation for a long-term knee injury.
Britain's Becky James bounced back from a serious knee injury and a cancer scare to take silver with a late surge in the Olympic women's keirin in Rio.