Can neural networks learn to ignore an input datum? Disclaimer: I'm not a student in computer science and most of my knowledge about ML/NN comes from YouTube, so please bear with me!



Let's say we have a classification neural network, that takes some input data $w, x, y, z$, and has some number of output neurons. I like to think about a classifier that decides how expensive a house would be, so its output neurons are bins of the approximate price of the house.

Determining house prices is something humans have done for a while, so let's say we know a priori that data $x, y, z$ are important to the price of the house (square footage, number of bedrooms, number of bathrooms, for example), and datum $w$ has no strong effect on the price of the house (color of the front door, for example). As an experimentalist, I might determine this by finding sets of houses with the same $x, y, z$ and varying $w$, and show that the house prices do not differ significantly.

Now, let's say our neural network has been trained for a little while on some random houses. Later on in the data set. it will encounter sets of houses whose $x, y, z$ and price are all the same, but whose $w$ are different. I would naively expect that at the end of the training session, the weights from $w$ to the first layer of neurons would go to zero, effectively decoupling the input datum $w$ from the output neuron. I have two questions:


Is it certain, or even likely, that $w$ will become decoupled from the layer of output neurons?
Where, mathematically, would this happen? What in the backpropagation step would govern this effect happening, and how quickly would it happen?


For a classical neural network, the network has no "memory," so it might be very difficult for the network to realize that $w$ is a worthless input parameter.

Any information is much appreciated, and if there are any papers that might give me insight into this topic, I'd be happy to read them.
This is a learnable behavior, given enough data. 
We would expect the an error to backpropagate to $w$ whenever its use harmed classification accuracy. In this case, that would be whenever $|w|&gt;0$. In general, I'm not sure how long this would take.

However, the speed of $w$'s convergence to zero would benefit from regularization, which is often basically a penalty on the magnitude of your network weights added to the loss function you're optimizing. If $w$ truly doesn't matter to classification, then regularization will definitely drive it to zero.