text: BERT struggles with representations of num- 
  bers. Addition and number decoding tasks showed 
  that BERT does not form good representations for 
  ﬂoating point numbers and fails to generalize away 
  from the training data (Wallace et al., 2019b). A 
  part of the problem is BERT’s wordpiece tokeniza- 
  tion, since numbers of similar values can be divided 
  up into substantially different word chunks. 
  
  However, BERT cannot reason based on its 
  world knowledge. Forbes et al. (2019) show that 
  BERT can "guess" the affordances and properties of 
  many objects, but can not reason about the relation- 
  ship between properties and affordances. For ex- 
  ample, it “knows" that people can walk into houses, 
  and that houses are big, but it cannot infer that 
  houses are bigger than people. Zhou et al. (2020) 
  and Richardson and Sabharwal (2019) also show 
  that the performance drops with the number of nec- 
  essary inference steps. Some of BERT’s world 
  knowledge success comes from learning stereotypi- 
  cal associations (Poerner et al., 2019), e.g., a person 
  with an Italian-sounding name is predicted to be 
  Italian, even when it is incorrect. 