Getting rid of rough edges after removing old paint
Yeah, that's totally fine.  You do that all the time with restorative surface prep, alternating layers of epoxy primer, "bondo" (Abatron, West System ultralight filler etc.), sanding, primer, bondo, rinse wash repeat. 

Though honestly, the best thing you can possibly do with a rickety old garage is stabilize it.  Why? Because stabilizing an old building is vastly cheaper than building a new one.  A new building, I swear, is 1/3 entitlements - fighting with the government for your right to build the building at all, all the extra junk you have to add to the new building's specifications to get approved - concrete foundation, AFCI breakers, trench new utilities to meet new code, requiring a bathroom for Pete's sake which means you must now plumb water and gas or much larger electrical so you can heat the building all year, sweet Nelly,  there's really no end to it.  It's a nightmare.  So yeah, if you ever foresee wanting a building this size, save this one.