Possible UI panel memory leak?
Blender allocates memory for the undo stack when you perform undo-able actions. There is a limit to this allocation determined by the number of undo steps you have specified. See "User Preferences->Editing->Undo->Steps".