In this work, we describe a ML model for predicting IRIS spectra from chemical structure. This model is based on the Graphormer architecture, which extends graph neural networks (GNNs) via transformers. Graphormer has already been applied successfully to chemical systems, having won the 2021 Open Catalyst challenge and achieving state-of-the-art predictions across a variety of (bio)chemical domains. Using a multi-staged transfer learning scheme inspired by the psychology of human learning and natural language processing (NLP), we show how different spectral libraries offer distinct contributions (e.g., anharmonic scaling, denoising) to the final model's understanding of experimental IRIS measurements. Further, we utilize a flexible description of molecular charge (i.e., (de)protonation, sodiation) and explainability techniques to show that Graphormer-IRIS successfully derives "chemical intuition" from transfer learning and a highly contextual understanding of molecular charge and bulk thermodynamic behavior.