What should a student do, when a professor delays submitting his letter of recommendation?
Politely and very gently remind the professor that the letter of recommendation is due. That's really all you can do. If your professors says they will submit it on time, try to believe them.

As you have found out after posting the question, to the extent that it doesn't hold up other parts of their process, people on admissions committees understand that your flakey letter writers don't mean that you are flakey and will usually do what they can to accept or consider late letters. Often, departments will remind students or letter writers that their letters are missing, at, or even after, a deadline.