Why don't the products of combustion stop combustion?
You are exactly correct that in a closed system, the buildup of products would inhibit the reaction. As the concentrations of $\ce{CO2}$ and water vapor increase, the concentration of the oxygen will effectively decreases until the reaction slows and eventually stops.  

Fortunately we rarely have to worry about this because we rarely try to do such a combustion in a closed system. Rather, as in the case of a car engine for example, there is a system in place to actively vent out the exhaust waste products ($\ce{CO2}$ and water) while pumping in more oxidant (oxygen) and fuel (hydrocarbon gasoline).