We further show that SHAP analysis can succinctly elucidate the impacts of input features, which erodes the paradigm that ML models are black boxes . SHAP values can be calculated for nearly all supervised ML models, and we encourage future chemical and polymer informatics studies to take advantage of explainability in ML . A recent study also used coloring of substructures when training a graph neural network for interpretable ML , which suggests that feature-importance analysis of ML models can be extended beyond fixed representations to learned chemical representations.