Any process that stops, prevents, or reduces the frequency, rate or extent of the process preceding formation of the peptide bond between the first two amino acids of a protein in a mitochondrion.
negative regulation of mitochondrial translation initiation