I moved my website from one VPS to another on 16th April.The A record of the domain name has a TTL of 86400. I made the IP address change at 19:00 GMT 16th April.I do not want the site to face downtime so I am keeping the old VPS alive.Today is the third day and the access log on the old server shows hits from Baidu spider.It has been more than 60 hours but why is this crawler still hitting the old IP address?I am using AWS Router53 for DNS.
Baidu spider does not obey DNS TTL
dns;web crawlers;baidu
Baidu spider also does not properly obey robots.txt in some cases.   They seem to have a history of doing things their own (wrong) way.Unless you have a Chinese website that gets a significant amount of traffic from the Baidu search engine, why would you keep old server up just to make the Baidu spider happy?