How to handle features which are not always available?
If the feature doesn't make sense in a subset of the samples, doesn't this mean that this is (or should be) a separate dataset, that needs a second model? That's one approach I'd think about.

The second would be to work with the data (feature) itself. It's probably best to use neutral value.


In case of numerical value:


try using a mean or median value, calculated on all entries
try using an extreme value, e.g. -1 if your feature has only positive values. This should indicate that the feature is missing and the network should be able to handle it.

In case of textual value, e.g. word embeddings, replace the value with a placeholder like N/A that doesn't have an embedding