Media playback is not supported on this device
Richard and Peter Chambers, Rob Williams and Chris Bartley won Britain's third rowing medal, with defending champions Denmark taking bronze and Australia fourth.
Media playback is not supported on this device
The Chambers duo became the first British brothers to win an Olympic medal since Greg and Jonny Searle took bronze in the coxless four at the 1996 Atlanta Olympics.
Richard Chambers said: "That was really, really brutal. We were just fighting, fighting through the whole thing to get ourselves back into contention.
"We struggled to keep with the pace of the other crews in the first quarter of the race but we dug our heels in and fought really hard."
Peter Chambers said: "We are gutted. We were unlucky to get a silver with those conditions. We're delighted with silver but we wanted to get gold.
"Fair play to the South Africans, they won that fair and square."
Williams said: "We wanted to win. We have to be happy to get a medal at our home Olympics. It's a shame we didn't win but what can you do? It's a shame."
The British quartet, who took bronze at last year's World Championships, arrived having won at the
They followed up an  where they overhauled Australia in the final 250m, with the fastest time in the semi-finals to progress to the final.
They were slow out of the blocks and were down in fifth as Denmark took an early lead at 500m.
The British boat fought back and were close to level with Denmark and a resurgent South African crew as they entered the final 500m.
But they just missed out on the final push, getting pipped on the line by South Africa.
It was the first Olympic rowing gold for South Africa and the first medal of any kind for the Chambers brothers, Williams and Bartley.
Great Britain's lightweight men's four won silver as South Africa beat them to gold by just a quarter of a second in a thrilling final at Eton Dorney.