In a JPEG file, how much is noise and how much is the image?
Your problem here is twofold:


Image Noise / low  SNR due to darkness
problems to average things out due to JPEG compression


The first point is something we cannot really change per se; your averaging approach surely is a good one. 

The second problem now is due to how the JPEG compression works in your camera:


Gamma correction is applied to the RGB image first. That is a nonlinear operation, "pushing together" high values and stretching the lower values. That way, you don't get "pitch black" everywhere for dark images. Sadly, this also makes our quantization noise problem harder.
The resulting R'G'B' pixel data, is converted to the Y'CbCr colorspace, meaning you convert the threedimensional red,green,blue signal to the threedimensional brightness,Blue-difference,Red-difference signal. 
The Cb/r channels are low-pass filtered and downsampled
The original-resolution Y', and the reduced-resolution Cb and Cr images are split into 8x8 pixel blocks separately
These blocks are separately DCT'ed
A quantization matrix is chosen according to pick a quality/compression trade-off. This results in different DCT coefficients being omitted, saved with fewer or more bits.


As shown in the other answer, averaging on the reconstructed RGB image is a bad idea, since the random noise must lead to you basically seeing an effect of the IDCT of the quantization matrix in each block – meh.

So, first of all, notice that it's pretty certain that the for the DCT coefficient that defines the constant intensity offset ("DC component", an EE would say), the highest quantization level was chosen.

Conversely, at least for most  higher-frequency components in the 8x8 blocks, quantization error energy is higher.

Thus, inherently, the DCT's constant bin is the most reliable – also, it happens to be the average of the 64 pixels in the block, and thus reduce noise already (if noise's frequency distribution was white, that would be the case for all components, but neither is it reliably white in cameras nor is the gamma correction linear...). 

So, what you probably want to do as a first step is


work on the  Y'CbCr, not the RGB reconstruction
only look at the Y component first
ignore everything that is not the DC component in each 8x8 block


You'd get an image of $\frac18$ the width and height of your original photo. It's probably still going to be noisy – with the shot noise that is probably relevant in a low-light setting contributing to the average in each frame. However, averaging out that over time will reduce that effect.

You can probably get an noise reduction, too, by summing up the non-zero-frequency coefficient, weigh them with a factor &lt; 1, and subtracting them from the DC component. The idea behind that is that when you set a single pixel in a 8x8 frame, you  get energy in all coefficients (the cosine transform of a dirac has energy at all frequencies, if you think about it). Since you do not get all coefficients, most of them being thrown away or quantized heavily, you'll never be able to fully remove the effect of shot noise from the average coefficient – but you can, at least on average, reduce the effect. Choice of that factor would depend on camera sensor, gamma factor, quantization matrix, the way the DCT is implemented (truncating fixed point?) and thus, too many unknowns, but you could probably just empirically try a couple of values and minimize the variance of the result over multiple frames.