Remove "Windows" Entry from Mac Boot Loader
When you don't use the Boot Camp utility to remove Windows, the Windows boot image on your EFI partition is not removed. This is what you see when you hold down ⌥ during boot.

The procedure to solve this is not for the faint of heart. You need to mount the EFI partition in OS X (normally, it only gets mounted for system updates).

Mounting the EFI partition

List the partitions on your Mac:

diskutil list


You should see something like this:

/dev/disk0
 #: TYPE                     NAME          SIZE       IDENTIFIER
 0: GUID_partition_scheme                  *251.0 GB  disk0
 1: EFI                                    209.7 MB   disk0s1
 2: Apple_HFS                Macintosh HD  250.1 GB   disk0s2
 3: Apple_Boot               Recovery HD   650.0 MB   disk0s3


Take note of /dev/disk0 and 1: EFI. The numbers 0 and 1 have significance later.

Now mount the EFI partition (replace the numbers 0 and 1 if they are different in your output):

sudo mkdir /Volumes/efi
sudo mount -t msdos /dev/disk0s1 /Volumes/efi


Remove the Windows boot image

Now navigate to /Volumes/efi/EFI. This directory is supposed to only contain an APPLE directory. On a machine where I had Windows installed, it also contained a Microsoft and Boot directory. These can be safely removed, but for extra safety I would recommend renaming them to something different instead.

Then umount and delete the directory you created earlier.

sudo umount /Volumes/efi
sudo rmdir /Volumes/efi