Are the Goons un-fun?
Personally, I like that goons opens up the possibility of winning without buying green cards as well as the possibility to have monstrous turns when you've stacked a few goons.

But fun is subjective, if it's not fun for a subset of the players in your group, it is simply not fun for them.

While attacks usually get meaner in Dominion games with more players, Goons isn't really one of the attacks that gets nastier in larger Dominion games.  Since being hit by 10 goons is no worse than being hit by 1, the number of players doesn't matter so much.  Contrast this with a card like witch, where the damage done scales in proportion to how many are played.

Messing with all 5 opponents isn't necessarily better than messing with a single opponent.  You still have to beat all of your opponents.  Imagine you could either get 5 points, or cost every opponent in the game 3 points.  In a 6 player game, you could theoretically cost each opponent 3 points, for a total of 3 * 5 = 15 points, but you'd still only have a relative advantage on each opponent of 3 points.  If instead you took the five point gain, you would have gained 5 points on each of your opponents.  Even though the "net damage" increases with the number of players, the "relative advantage per player" is really the important metric, which is often number of players invariant in Dominion.