Every 500 steps, the full training state is saved in a subfolder in the output_dir. The checkpoint has the format checkpoint- followed by the number of steps trained so far. For example, checkpoint-1500 is a checkpoint saved after 1500 training steps.