Does the formation of water inside the mitochondrial matrix help contribute to the proton gradient during the electron transport chain?
Your description of it is largely correct, but the electron transport chain does not simply "dump" charged oxygen ions in the mitochondrial matrix. Instead, cytochrome C oxidase (complex IV) binds the O$_2$ molecule to one of its heme groups, and the reduction O$_2$ + 4 H$^+$ + 4 e$^-$ $\rightarrow$ 2 H$_2$O occurs at the heme group before water is released. Some more details on the enzyme is found here and here (or in any biochemistry book).

In theory you're right that producing water in the mitochondrial matrix would dilute the H$^+$ concentration there a little (along with the concentration of everything else). But this dilution is negligible: the mitochondrial matrix has pH $\approx$ 7.8, which is the same as a H$^+$ concentration of about 15 x 10$^{-9}$M, while the concentration of water (yes, there is such a thing ;) is 55 M. So adding a water molecule has virtually no effect compared to removing a proton.