Reluctant Interaction Modeling

Including pairwise interactions between the predictors of a regression model can produce better predicting models. However, to fit such interaction models on typical data sets in biology and other fields can often require solving enormous variable selection problems with billions of interactions. The scale of such problems demands methods that are computationally cheap (both in time and memory) yet still have sound statistical properties.

Motivated by these large-scale problem sizes, we adopt a very simple guiding principle: One should prefer a main effect over an interaction if all else is equal. This "reluctance" to interactions, while reminiscent of the hierarchy principle for interactions, is much less restrictive. We design a computationally efficient method built upon this principle and provide theoretical results indicating favorable statistical properties. Empirical results show dramatic computational improvement without sacrificing statistical properties. For example, the proposed method can solve a problem with 10 billion interactions with 5-fold cross-validation in under 7 hours on a single CPU. 

This is joint work with Guo Yu and Ryan Tibshirani.

Please register for the event.

About Jacob Bien

Jacob Bien is an associate professor in Data Sciences and Operations in the Marshall School of Business at the University of Southern California (USC). He received a B.S. in physics and a Ph.D. in statistics from Stanford University. Before joining USC, he was an assistant professor at Cornell University in the Department of Biological Statistics and Computational Biology and in the Department of Statistical Science. His research focuses on statistical machine learning and in particular the development of novel methods that balance flexibility and interpretability for analyzing complex data.  His work is supported by the US's National Science Foundation, National Institutes of Health, and the Simons Foundation.