1. A method of checkpointing an application in a parallel computing system of the type that includes a plurality of hybrid nodes, each hybrid node including a host element and a plurality of accelerator elements, each host element including at least one multithreaded processor, the method comprising, in a first hybrid node among the plurality of hybrid nodes: executing at least a portion of the application in the host element and at least one accelerator element; and in response to receiving a command to checkpoint the application, checkpointing the host element separately from the at least one accelerator element.