Beavon smashed home just before the half-hour mark to inflict a second successive defeat on Swindon, following their loss at Barnsley last weekend.
The striker punished hesitancy among the Swindon defence with a clinical volley, the Brewers' first ever goal at the County Ground.
Wes Thomas dragged wide from inside the penalty area for the home side.
Jon Obika and Louis Thompson also missed the target while Yaser Kasim had a free-kick pushed over by Jon McLaughlin as Burton held out for the win.
Burton Albion boss Jimmy Floyd Hasselbaink told BBC Radio Derby:
Media playback is not supported on this device
"I'm very happy. I think when you're coming to Swindon, you know what they're about. They can keep the ball for fun and they can test your nerves and get overloads in certain areas, but we made it very uncomfortable for them. We didn't let them get into their stride, we didn't let them play their game and get into their rhythm.
"For us, it was all about winning the ball back in their half and trying to penetrate in the last third and I think we did that really well.
"We should have scored more goals and that's something we need to work at, but I must say I'm extremely happy with how the team has worked today. We were very solid and we were quite creative with the ball at times, but the belief needs to become greater because they can do a lot more than they think."
Stuart Beavon's first-half strike moved Burton to the top of League One with their first ever win at Swindon.