Uninstall/upgrade tensorflow failed: __init__.cpython-35.pyc not found
Posting my own answer (alternative) here in case someone overlooked comment:
I forced to delete the package in the python/lib/site-packages/ and reinstalled the tensorflow-gpu, and it seems working well. 
Though I solve this problem via such alternate I would still like to know the root cause and long-term fix for this.
I previously installed tensorflow-gpu 0.12.0rc0 with Winpython-3.5.2, and when I tried to upgrade or uninstall it to install the newer version using both the Winpython Control Panel and pip, I got the following error:
FileNotFoundError: [WinError 3] The system cannot find the path specified:
'c:\\users\\moliang\\downloads\\winpython-64bit-3.5.2.3qt5\\python-3.5.2.amd64\\lib\\site-packages\\tensorflow\\contrib\\session_bundle\\testdata\\saved_model_half_plus_two\\variables\\__pycache__\\__init__.cpython-35.pyc'
I installed the tensorflow-gpu 0.12.0rc0 through Winpython-3.5.2 pip, and the __init__.cpython-35.pyc does exist at the correct directory. So I don't understand how this error could happen? And it prevents me from getting the new version.