PhD: Accepted into another PhD program - should I finish the current semester in my old program?
You got a high GPA so you might have a better chance of getting into a Phd program. 

You already got accepted into the program of your choice, so at the moment your GPA is useless. 

Your only consideration should be when you want to start, if you like the courses so much that you'd take them for general interest, and if you want to do something after your Phd where GPA suddenly counts again (instead of publications / references).