Does a builder count as a follower when scoring?
Builders (and Pigs) are not followers and are not counted when calculating a majority.

In your case, the city would be scored for the owner of the big follower.

Originally, builders and pigs were followers but this was changed a while back.  Rio Grande Games adopted this rule upon the release of the Big Box Edition.