Picking the correct filter for accelerometer data
As pointed out by @JohnRobertson in Bag of Tricks for Denoising Signals While Maintaining Sharp Transitions, Total Variaton (TV) denoising is another good alternative if your signal is piece-wise constant. This may be the case for the accelerometer data, if your signal keeps varying between different plateaux.

Below is a Matlab code that performs TV denoising in such a signal. The code is based on the paper An Augmented Lagrangian Method for Total Variation Video Restoration. The parameters $\mu$ and $\rho$ have to be adjusted according to the noise level and signal characteristics.

If $y$ is the noisy signal and $x$ is the signal to be estimated, the function to be minimized is $\mu\|{x-y}\|^2+\|{Dx}\|_1$, where $D$ is the finite differences operator.

function denoise()

f = [-1*ones(1000,1);3*ones(100,1);1*ones(500,1);-2*ones(800,1);0*ones(900,1)];
plot(f);
axis([1 length(f) -4 4]);
title('Original');
g = f + .25*randn(length(f),1);
figure;
plot(g,'r');
title('Noisy');
axis([1 length(f) -4 4]);
fc = denoisetv(g,.5);
figure;
plot(fc,'g');
title('De-noised');
axis([1 length(f) -4 4]);

function f = denoisetv(g,mu)
I = length(g);
u = zeros(I,1);
y = zeros(I,1);
rho = 10;

eigD = abs(fftn([-1;1],[I 1])).^2;
for k=1:100
    f = real(ifft(fft(mu*g+rho*Dt(u)-Dt(y))./(mu+rho*eigD)));
    v = D(f)+(1/rho)*y;
    u = max(abs(v)-1/rho,0).*sign(v);
    y = y - rho*(u-D(f));
end

function y = D(x)
y = [diff(x);x(1)-x(end)];

function y = Dt(x)
y = [x(end)-x(1);-diff(x)];


Results: