Reducing the effect of down voters with rating system
You should look into other estimators of location.

What you want is a robust estimator, with a high break-down point.

The extreme approach would be the median.

But you may get more numerically interesting results with a trimmed mean.

You define a threshold, say 2%. Then you remove the top 2% of votes, and the bottom 2% of votes, and take the mean only of the remaining entries. An app with 98% 5 stars will still get a 5.0

But to prevent manipulation, I would look into other signals. Such as clustered votes from a single region, for example.