Here, the answer would be 'you should follow her/him', which means that the 'whom' in the question is referring to the object in the answer. That makes 'whom to follow' correct, since 'whom' should be used in objective cases and 'who' in subjective.