Is this XKCD comic an accurate timeline of the earth's average temperature?
Yes, it's accurate, see e.g. this picture taken from the Wikipedia Geologic temperature record page:



You could dig up the source if you wanted from the wikipedia article (or the graph description page), but any graph will be generally the same. Since there was an ice age, you'd expect global temperature to be low ~20k years ago and so on.