Mojave can't be installed but still boots to Installer partition
You would have to get another Mac with an ethernet port, boot your Mac in Target Disk mode (Command T) and connect it via ethernet to the other Mac. You can also use the Thunderbolt 2 ports to network the two Macs. 

Once it's up, you can access your Mac's HD as if it was an external HD, and you can remove the file from there.

Additionally, you could try booting into recovery Mode (Command R) and fix your HD's permissions with Disk Utility.

The thing is, the App store would have prevented you from even downloading Mojave, since it checks your hardware for compatibility. Mojave also lists its system requirements as well a the compatible systems on the website, as well as the App store.

So, I suppose you must have downloaded Mojave from somewhere else. The reason why we can't install Mojave in older systems is because of compatibility. Mojave also needs a lot of resources that only newer systems can handle. The oldest systems that can still handle Mojave are those released in 2012, whereas those from 2011 and below can only handle up to High Sierra.