Is it self-plagiarism to reuse my own unpublished material?
No. Unpublished work is unpublished. It's almost as if it doesn't exist. Given that it is your work, there is no case of plagiarism. I don't even think you need to cite yourself --- what would you actually be citing? 

Indeed, how would you prove that you plagiarised yourself?

If your unpublished work has been circulated among other authors and has been cited by them, maybe you can cite it too. But that seems a bit odd.