RG6 vs RG11 - conductor size and connections
You could use better connectors - just had a quick look and found (there are probably others) Belden's (no affiliation) FS11 F-Conn for RG11, which uses a pin that connects to the central wire, so that the pin is the expected/standard size for F-connectors. No cramming the wrong size thing in...

Mind you, for 40 feet at typical Over The Air Broadcast Frequencies there should be minimal difference between RG11 &amp; RG6 - The highest channels in UHF should be where the most difference is, and for 50 feet of cable there should only be about 0.4 dB difference between them, while at VHF frequencies 50 feet should be 0.25 - 0.3 dB - not clear from your question what the overall length is - if it's 100 feet rather than 40 feet, double those numbers, but still a small difference at broadcast frequencies (54-806 MHz) unless you are actually dealing in several hundred feet, or higher frequencies not seen in broadcast TV but common in satellite TV.