Boot Camp requiring disk on High Sierra
Below are answers to the questions in your post.

I did a quick check with the internet and it would appears 64 bit Windows 7 is the last version of Windows that Apple officially supports on your model Mac. Therefore, installing a Windows 8, 8.1 or 10 usually requires more than the Boot Camp Assistant. 

I have a 2011 iMac with a working optical and High Sierra, so I doubt High Sierra is the reason the optical drive fails to work.

The Boot Camp Assistant does require installing Windows from a DVD on your model Mac. Although, many users here at Ask Different have modified the Boot Camp Assistant in a attempt to install Windows using a flash drive.

Officially, installing from an ISO file did not start with Macs until the 2015 model year. Apple does not support installing from an ISO with any 2009 model year Macs.

Using an external DVD drive may or may not work. There is no actual way to tell other than trying.

Personally, I would try installing the current version of Windows 10. It has been reported that a Windows 7 or 8.1 product key will work with the current Windows 10 download. I tested this earlier this year and was successful using a 2018 Mac mini.

When I installed High Sierra on my 2011 iMac, firmware upgrades were installed that allowed the Mac to EFI boot the Windows 10 installer from an ExFAT formatted internal or USB drive. Originally, I was required to use an optical drive to install Windows, but now I can also use a flash drive or install directly from the ISO file. I do not know if the same would be true for your 2009 iMac.

The answer that closest matches your question can be found at Any solution to bootcamp windows 8.1 with macbook pro 15" 2010?. This is closely related to the two questions given below.

No bootable device USB 2.0 MacBook Pro mid 2014  

Boot camp install of Windows 7 issue, no bootable devices  

After High Sierra was released, the use of a virtual optical drive could be eliminated on certain model Macs. This led to the following.

How to install Windows 10 into a 2011 iMac without using the Boot Camp Assistant, an optical (DVD) drive or third party tools?

However, the above answer involved using the graphical user interface (GUI) which was reported to cause problems. The latest answer is below, but has not been tested on an actual Mac.

When trying to install Windows 10 on 2011 iMac keep getting error 0x8007000D windows cannot open the required file D:\sources\install.wim

The problem, with the newer answers, is that you want Windows 8.1 and the instructions are for Windows 10. The solution may simply require substituting 8.1 or 10 in the instructions. Otherwise, the instructions could be modified where you would EFI boot from Window 10, but install Windows 8.1 to BIOS boot. This would be similar to the question Dual Boot Win 7 on MBR with Win 10 on GPT.