How to Avoid rarely used discrete feature values in a dataset On Google's ML crash course it states:


  Good feature values should appear more than 5 or so times in a data
  set. Doing so enables a model to learn how this feature value relates
  to the label. That is, having many examples with the same discrete
  value gives the model a chance to see the feature in different
  settings, and in turn, determine when it's a good predictor for the
  label.


This make sense, but if that happens what should we do? For example, consider a dataset with street_name as feature and out 2000 rows, only 4 of them have street_name equal to "Learning St.".

Should we remove those rows containing rare feature values? Or what?

Any insights would be greatly appreciated.
Working with Categorical Features can raise a few challenges. For instance, you might encounter features with high cardinality in certain values or the other case (your case), features with rare categories.

The first thing you can consider is, what you will lose if you drop those rows. What extra info do you get with keeping the rare categories. Here you probably need the domain knowledge of the project to identify the impact of the specific data. If your dataset is small or you don't feel safe to remove those rows, then you should look for alternatives.

The second approach is to group all the rare categories in one new called "Rare" or "Other". So, whenever you have a record with that value, you replace it with the new one. 

Finally, you can try either a Dimentionality Reduction such as PCA or a Feature Hashing where you will map multiple categories in one new value.