Sharing > Remote Management and network security
You shouldn't be, no. 

But that may depend on whether you are then also internet connected on your Winbox at the time you're VNCing into the Mac, especially if you're using IE, even more so if you're using IE with .net framework applications, or even more exposed, Active X as well.

You could potentially end up exposing the Mac to attack via the VNC connection if your Win box were compromised.