In this study, we developed the MultiModalTransformer (MMT) model, an innovative architecture for molecular structure elucidation that integrates multiple spectroscopic modalities including NMR, IR, and MS data. Our research demonstrates enhanced molecular structure prediction accuracy through the integration of diverse spectral data, achieving up to 94% correct identifications for experimental samples. We implemented a robust simulated data generation pipeline and an iterative improvement cycle, enabling effective performance on experimental data despite training only on simulations. The model's robustness is evidenced by its ability to overcome initial structural inaccuracies, maintaining substantial predictive power even with incorrect starting guesses. Additionally, the MMT provides explainable predictions through token-based analysis, offering insights into its decision-making process. While there's potential for further enhancement through increased experimental data training and automated peak-picking, the current model already serves as a powerful tool for chemists, effectively bridging the gap between simulated and experimental data. Its adaptability makes it particularly valuable for real-world applications where perfect initial structural information may not be available, representing a significant advancement in automated structure elucidation.