We here showed how CASSCF and CASPT2 dynamics can differ quite drastically in the ultrafast time scale. Small variations in how the electronic energies are computed can also induce relevant differences that might affect the computation of spectroscopic signals, quantum yields, or kinetic profiles. Our details analysis, easily reproducible thanks to the user-friendly module available in PySpawn, explains at the same time the reason why these types of studies are required and why an interface like PySpawn/OpenMolcas is needed and essential to have an accurate and balanced description of nuclear dynamics electronic structure to disclose the ultrafast dynamics of molecules unambiguously. We hope that this investigation together with the free availability of our interface will push even more research efforts in this direction.