Find inputs that give highest variance in output space I have a relatively simple problem that I think should have a known solution, but that I can't figure out. Any help would be greatly appreciated.
Basically, I have a function $f : \mathbb{R}^d \rightarrow \mathbb{R}^p$ which has vector input of features and vector output of images. Given a training set of combinations of the input variables and their corresponding outputs $\{x_i, y_i\}_{i=1}^N$, I would like to find the input feature (i.e. element of the inputs) that leads to the highest variability in the outputs, i.e. highest pixel-wise variance. Broadly speaking, I'm hoping to quantify the relationship between each feature and its effect on the variance of the output, to get an idea of how &quot;sensitive&quot; that feature is.
Methods like PCA wouldn't work because I am working in a supervised setting. Whereas PCA tries to find the directions of maximum variance of an unsupervised dataset, I am trying, in a sense, to find the directions of maximum variance in the label (output) space as it relates to the input space. Does an answer to this exist? Thanks in advance.
Very interesting question! I didn't get if the out put of one input vector is just one image or a set of images but this does not change the solution much.
I propose you directly learn the relation between input and the variance of output images. For simplicity, I assume you target the pixle-wise variance within an output image. Then your training data would become $\{x_i, Var(y_i)\}_{i=1}^N$ instead of $\{x_i, y_i\}_{i=1}^N$!
Then your problem can be solved as:

Classification: You threshold the &quot;high variance&quot; and discritize your output into high variance and low variance classes
Regression: Even better! you directly learn the value of variance and have more flexibility on interpreting the results at the end (e.g. the thresholding mentioned above can be applied afterwards).

I hope I understood your question right, I was clear and it could help!
Good Luck!