Is there a way to nullify a specific feature in test set while evaluating a tensorflow model?
For deep models there's no general input-independent way to do this kind of feature ablation you want (take a pretrained model and just change that feature's representation on the test set).
Instead I recommend you do training time ablation: train different variations of your model with different feature combinations and compare their validation set performances. This will actually tell you how much each feature helps.
Idea behind nullifying/ignoring a feature from test set is to understand how important is it considered by the model to predict the target variable (by comparing the evaluation metric's value). For numerical variables, I thought of setting them to 0, assuming the multiplication (with weights) would be 0 and thus it would get eliminated from the set. Is this approach right, else what should be done?
I am using tensorflow's DNNRegressor for modelling.