How to tell which freehub body is a compatible replacement for the one on an old Shimano FH-HG50 rear hub?
Most of the time, the problems you mention can be fixed by removing the freehub completely, pulling the seal in back (note orientation) and running some combination of degreaser/solvent/oil/air/etc through it, depending on its condition and what you have to work with. There are some symptoms that denote something is broken (crunching, truly erratic operation/sound/feel) and others that suggest things are whole and in place but the lubrication is gummed up and so the pawls are moving reluctantly in both directions, which is common and can cause what you describe.
The parameters are fit to the shell, length, and bearing cup profile.
Most 7/8/9/10 Shimano freehubs use the same fit to the shell. There are a few oddballs but not many. That part is easy and it's easy to find a freehub that will bolt on, either cannibalized or new.
7-speed freehub bodies are shorter, and so for a given OLD, the hubs they came on have more symmetrical bracing angles than an 8/9/10, or in other words the center to right flange number is bigger. Stick an 8/9/10 on and arrange the right side axle hardware on such that the new freehub body wants it to be (this you don't have a choice about, see below) and then when you go to put the left back together with the stock FH-HG50 hardware, you'll have a mutant that wants to be too long, 138mmish I believe. Take away spacer from the left to make it 135 again and what you are doing is creating a hub where the left flange is closer to the left dropout than an 8/9/10 hub would have put it. The reason they wouldn't put it there is it increases the bracing angle disparity between the left and right side spokes past what is generally considered good, such that the tension disparity between left and right side is too great, or in other words your left side spoke tension needs to get too low. Some amount of this effect is acceptable when hacking hubs, like nudging it around by a few percent. Spocalc.xls and some of the better online spoke calculators show you the change in tension ratios so you can play with it in the abstract before doing it. I believe that putting an 8/9/10 freehub on an HG50 shell will probably push it too far, but I would check first off measurements from the hub shell in front of me, so that's what I recommend you do. If it was the same shell that was also used on some 130mm 7spd hubs, it might be fine.
You always need to at least check your final spacing and dish when doing a hack type (non-stock) freehub replacement, and usually will need to make some adjustments to both. Don't try to piece together random advice from the internet about the left side spacer configuration you will wind up needing. It can only be done reliably by measuring.
I always do this by using whatever right side cone and axle hardware arrangement the new freehub body wants. That means I use its native dust shield and cone rather than screw around with transplating the old ones into it. Then use the center to right flange dimensions from its native stock hub to build out your right side spacer/locknut stack. In other words, if it's a 135mm hub and that dimension is 21mm, the outer locknut face needs to be (135/2-21) away from the right flange. There are probably some situations where it's fine to do it the other way and transplant the cone and dustshield in from the old hub, and the bearing cup profile is close enough to identical that it doesn't matter, but I don't do it that way so can't really advise either way. One reason I don't do it that way is most of the time it's a relatively recent 8/9/10 body and the cone is obtainable from either Wheels Mfg or Shimano, so it just feels simpler and more likely to work out well.