Title: Recovering Pairwise Interactions Using Neural Networks,
Abstract: Recovering pairwise interactions, i.e. pairs of input features whose joint
effect on an output is different from the sum of their marginal effects, is
central in many scientific applications. We conceptualize a solution to this
problem as a two-stage procedure: first, we model the relationship between the
features and the output using a flexible hybrid neural network; second, we
detect feature interactions from the trained model. For the second step we
propose a simple and intuitive interaction measure (IM), which has no specific
requirements on the machine learning model used in the first step, only that it
defines a mapping from an input to an output. And in a special case it reduces
to the averaged Hessian of the input-output mapping. Importantly, our method
upper bounds the interaction recovery error with the error of the learning
model, which ensures that we can improve the recovered interactions by training
a more accurate model. We present analyses of simulated and real-world data
which demonstrate the benefits of our method compared to available
alternatives, and theoretically analyse its properties and relation to other
methods.