turn off features during training in deep network
A simple solution which might work would be to declare a feature as non relevant. So you simply specify a value, which makes the model learn that it should not be used in this iteration. So when you set the values to zero, the model wll just think that its a zero, so the meaning is ambiguus. However if you set it to -1 e.G. but -1 does not occur in your data at all, the model might be able to learn that -1 represents a non existent feature and puts more attention on the others.
I'm looking for a solution at the following problem:
I have my neural network with 8 features and 6 outputs.
The network predicts some info on a graph, where a record is related to a node in the graph.
It works perfectly when I have all the examples (nodes) with the same degree of neighbours.
Now I would need it to work also for nodes with less neighbours, but on a general form, meaning that I do not know a priori how many they are.
Thus I tried putting a zero when the the neighbour does not exist, but the predictions where all strange, not correct at all.
So my questions is: is there a way such that during training I can switch off some input neurons looking at how many effective features I have on that example?
To be more precise: the first example taken by the NN has the full neighbour, thus I have 8 values; the second has 6 values, so I would like to switch off 2 input neurons and so on.
Any ideas ?