Low Pass RC Filter for Load Cell
This is a broad question. The short answer is, it depends on what range of frequencies you want to filter out, and you will need external resistors. The long answer is, it depends on the sensitivity of your strain gage, how precise and fast you need your output to be, and how much time you want to devote to understanding the answer to this question.

As for changing the output voltage, yes and no; yes, you would likely change your output voltage if you were to modify the circuit below, but no, because it's not applicable here. (TL;DR- if you don't want a more step-by-step answer, just click the link to the UC Santa Barbara laboratory exercise at the bottom.)

I've inserted a schematic below of the simplest possible low-pass filter with what you have described.







simulate this circuit &ndash; Schematic created using CircuitLab

This circuit is relatively stupid, and will probably not do what you need, but it's a good starting point. The cutoff frequency for a low-pass filter, as described by $$ f_c = \frac{1}{2 \pi R C}$$

can be easily calculated as 455 Hz, or roughly 2 ms. Another (more simplistic) way to think about it is that anything faster than 2 ms will not be detectable. 

Ah, if only electrical engineering were so simple as just a resistor and capacitor in parallel.

One popular (i.e., precise) implementation of a strain gage is using a Wheatstone bridge, which I've found via an online PDF of a laboratory exercise: Strain Gage Sensors Laboratory Exercise - UC Santa Barbara

This lab was written by people far more experienced than I, so I'll let them take over from me here. 

I hope I've given you some of the tools necessary for acquiring more of your own answers. Please let me know if this has helped, or if you have other questions!