Previous research has demonstrated that people preferentially remember reputational information that is emotionally incongruent to their expectations, but it has left open the question of the generality of this effect. Three conflicting hypotheses were proposed: (a) The effect is restricted to information relevant to reciprocal social exchange. (b) The effect is most pronounced for emotional (approach-and-avoidance-relevant) information. (c) The effect is due to a general tendency of the cognitive system to attend to unexpected and novel information regardless of its (emotional) content. Here, we varied the type of to-be-remembered person information across experiments. To stimulate expectations, we selected faces whose facial appearance was rated as pleasant or disgusting (Experiments 1 and 2), as intelligent or unintelligent (Experiment 3), or as being that of a lawyer or a farmer (Experiment 4). These faces were paired with behavior descriptions that violated or confirmed these appearance-based 1st impressions. Source memory for the association between the faces and the descriptions was assessed with surprise memory tests. The results show that people are willing to form various social expectations based on facial appearance alone, and they support the hypothesis that the classification of the faces in the memory test is biased by schema-congruent guessing. Source memory was generally enhanced for information violating appearance-based social expectations. In sum, the results show that person memory is consistently affected by different kinds of social expectations, supporting the idea that the mechanisms determining memory performance generalize beyond exchange-relevant reputational and emotional information.