Minimize correlation between input and output of black box system
Please allow me to paraphrase your question to make sure I get your question right.
Suppose you have 1 million data entries. Each entry consists of three inputs $X_1$, $X_2$, $X_3$ and one output $Y$. One of the three inputs, $X_1$, is a noise. You want to find the relation between $X_1$ and $Y$. So you can remove the impact of $X_1$ on $Y$.
Let's use a simplified example, $Y = 2X_1 + 3X_2 + 4X_3$
If you update each of the 1 million entries with:
$Y_{new} = Y - 2X_1$
Now you can examine the updated 1 million entries to find the relation between input $X_2$, $X_3$ and $Y_{new}$:
$Y_{new} = 3X_2 + 4X_3$
The relation between $X_1$ and $Y$ can be computed with different algorithms, such as regression, decision tree. How to measure which algorithm generates the best result, i.e. truly reflects the relation between $X_1$ and $Y$ hence removes the impact of $X_1$ on $Y$ as much as possible?
The problem you're solving is an interesting one. I would use a free tool such as Knime to find out which algorithm generates the best result in modeling the relation between $X_1$ and $Y$. It's faster than coding in Python or Matlab.
Details: to put the problem intuitively, suppose the price of a house is determined by location, size of the house, condition of the house. Assume these three factors are independent from each other. You want to remove the impact of &quot;condition of the house&quot; on &quot;price&quot; as much as possible. In essence, you want to find the relation between &quot;condition of the house&quot; and &quot;price&quot; which may not be linear.
You can let Knime read the 1 million entries of &quot;condition of the house&quot; and &quot;price&quot;, try Decision Tree, SVM, or even Ensemble Learning, and see which algorithm generates the best model of $X_1$ and $Y$. In other words, try and error :)