Mac Pro 3,1 Non-Apple GPU upgrade but
If your desire is to boot ONLY Windows on the machine, you can do this based on the setup you described by installing a Boot Camp partition and just setting the boot drive to that.

Since you have OSX installed on the machine, you should be able to find a utility on the machine called "Boot Camp Assistant". This tends to be in the Utilities folder of the Applications list. You should have a DVD of Windows 7 to get going with this since that Mac doesn't technically support Windows 8+. You'll want to start with 7 if possible since that way you'll get Apple's software support on the Windows installation. After that, you may be able to upgrade to Windows 8.1 or 10 but I've never personally tried it.

Normally installation of Windows is not a big deal, but without a proper "Apple Supported" GPU you might run into problems where the machine attempts to reboot into Windows and either goes back to the Mac or gets stuck at a black screen. In general, try to keep the Windows 7 disc in the drive and give the Mac some time to boot into Windows before powering off; for some reason, it takes maybe 20 seconds or more for the Mac to boot into non-OSX and you can get anxious that nothing is happening. If you're adventerous, there are some guides on the internet which should be able to guide you through a disc-less installation process that leverages VirtualBox.

Once you've got Windows installed, telling the Mac to boot by default into Windows is trivial: go to the System Preferences, Startup Disk and select that you want the BOOTCAMP partition to be the boot disk.

Going back to OSX without an Apple GPU can be done one of two ways:


Have the Apple Boot Camp software for Windows installed. There is a tray icon which has a selector to allow you to boot back into OSX.
During the Mac's startup sound, hold down Command-PR (the P and R keys simultaneously). Wait for the startup sound to happen a second time. This zaps the PRAM which will cause the Mac to boot the first OSX partition it finds.


both of these methods will probably require you to do the "Startup Drive" -> "BOOTCAMP" thing again in preferences.



Got it working? Great! Now the one downside of all this: your GPU will always run at PCIe 1.1 speeds unless you do something to the card. With some specific models, there are people on the internet who have figured out how to physically modify the card to run at 2.0 speeds (The MacPro3,1 is PCIe 2.0 for the 16x slots). I think this may involve cutting a resistor or something; you're on your own here. The other way is that you (ta da!) flash the card with an Apple EFI.

All said and done, you can make your Mac boot into Windows by default and have a Windows machine in a Mac Pro. However, if you can find an Apple EFI for your GPU, I would strongly suggest doing it since it'll just make life easier.