Typically, experimental chemists manually annotate substructures (e.g., C NMR, H NMR, IR, MS) or sublattices (e.g., XRD) based on error-prone table lookups, intuition, and deductive reasoning. With advances in computational techniques, researchers also employ simulation software for validation, checking, for instance, whether the computational predictions for their manually obtained structural model match the experimental evidence. Given that high-fidelity simulations of spectra can be very time-consuming, there has been mounting attention to predicting spectra for a given structure using machine learning (ML) approaches. For instance, there have been efforts to predict NMR shifts using ML approaches. However, those approaches are most valuable in validating a proposed structure assignment, but they cannot directly generate a list of possible structural candidates based on measured spectra. To propose a list of structures for a given measurement, database lookups are frequently used. While there has been some initial work to build spectral databases and optimized techniques to retrieve structures for them given an experimental spectrum, these methods are limited as they fail to combine multiple spectroscopic techniques that are typically required to fully characterize a compound. More importantly, they only work on structures for which spectra have been deposited in a database and hence cannot be used to predict novel structures.