Sampled offset voltage data and its uncertainty
If we have the sampled data in time series of this offset how can we calculate the uncertainty of the mean value of the offset?


Assuming that the reading is equal to the offset plus Gaussian noise that is independent, sample to sample, then you would first calculate the standard deviation of the noise, then you would calculate the standard deviation of the error after averaging \$n\$ samples of data.

That's a big assumption.


  I mean what is the uncertainty of the mean value of the offset due to random noise? Is it the standard deviation or standard error? I want to be able to say “This is the mean value of the offset with this standard uncertainty”


The mean value of the estimate of the offset, assuming the conditions set out above, is a homework problem for a basic statistics class.  It's more or less (estimated standard deviation) / sqrt(n), where n is the number of samples.