Is there any regulation that force(or recommend) to change flight number in case of flight delay?
An airline's flight number for commercial purposes is not necessarily the same as the identifier the aircraft uses for air traffic control purposes. For example, British Airways flight may show up on a flight tracking web service as BA### but air traffic control identifies them as "Speedbird ####" where the number may or may not be the same as the airline's commercial flight number.

So, to answer your question, there is no FAA regulation that forces the airline to change commercial identifiers. The ATC identifiers are set when the dispatcher files the flight plan, and they must be unique to ATC. The system will reject it if the identifier is in use, or will be during any period of flight.