PSD - Is this formula correct?
My question is: Should the output be this, and, if so, should I therefore apply a filtering algorithm even though matlab does not?


Without seeing your code I can't determine what's the reason for a difference in the two plots. There's multiple things that come into play like window type, overlap, and FFT size.

I can't say for sure if you should filter your data. It depends on the application. If you're just going to plot it, it may not be necessary. If the out of band energy or noise is really high and leaking into the signal band, then maybe. If this is going to be the input to some other process and you think the out of band energy is degrading some results, then sure, go ahead and filter. Without knowing your application it's tough to say what's right or wrong.

Anyway, in order to filter your signal to remove out of band noise, there's two approaches, but the easiest would be to do it in the time-domain.

Filter the data in the time-domain with either an FIR or IIR filter. Have a look at functions fir1 and fir2 in MATLAB. Once you design your filter, use freqz to quickly plot the frequency response to make sure it's correct. Once you have your filter 'b', compute:

% design filter 'b' using fir1 or fir2
y = filter(b, 1, x); % Filter input x with filter b
% compute PSD of y


Depending on your filter you should see suppression of energy in the stop bands.