Is there a limit to the number of inputs that can be summed with an op-amp summing circuit?
A summing amplifier: -



A summing amplifier showing the internal noise source of the op-amp: -



Now let's show that circuit with all the summed inputs connected to ground: -



The noise gain of this circuit is: -

$$1 + \dfrac{R2}{R1/3}$$

So, as you add more inputs (more R1 resistors connected to ground) the R1/3 term becomes R1/4 (four inputs) then R1/5 for 5 inputs. This means that the output noise caused by the grounding of inputs gets bigger as you add more inputs.

But you might say that the inputs are connected to voltage sources and not grounded. And I would say it matters not one bit because whether inputs are connected to ground or a real voltage source then noise becomes progressively amplified as you add more inputs. Sure, an unconnected input won't add more noise gain.

But also, as you incorporate more summing inputs the non-inverting input node acquires more parasitic capacitance to ground and it eventually dominates the input resistors at high frequencies (hence why I've removed the input resistors): -



And the noise gain clearly becomes very massive at higher frequencies because Xc becomes very large.


  are there any other factors that would limit you from adding inputs
  indefinitely?



The inability of the op-amp to provide the current to all the inputs and thus not maintain the virtual earth condition required for proper numerical summing.