1. A non-transitory computer readable storage medium storing instructions comprising instructions for a simulation environment that when executed causes one or more processors to: notify one or more peers and one or more user modules on a start of a checkpoint in a simulation; receive notification of being ready for checkpointing from the peers after the peers close inter-process communication channels; receive notification from the user modules of being ready for checkpointing after the user modules shutdown operating system dependent resources; and save a state of the simulation in response to the received notifications from the peers and user modules.