Signal conditioning of the acceleration data from the ADXL 354
Summary:
I think you can get useful data out of the ADXL354.  DC couple it to your ADC, and take care of the 1g offset in software.

Reasoning:
The first thing is to figure out if your sensor can even cover the needed range of frequencies.
From the IEEE paper that tlfong01 linked to in the comments:

The frequency range for acceleration measurement is between 0.1 and 12600 Hz at temperatures from −74 to 250 deg C. The breakout accelerometer sensor is used to measure the vibrations of noise leakage in a plastic water pipeline. The following sensors were used for the accelerometer: ADXL203 [62],  ADXL335[93], MiniSense 100 Piezo[93],condenser microphone [92], MEMs (Hitachi − metal H34C) [93] and MMA7361[94]. These sensors have different sensitivities for measuring the vibration signals (acceleration) from leaks.

That says you need a sensor sensitive from 0.1Hz to 12.6kHz.
From your description, the ADXL354 is DC coupled.  From the datasheet you linked to, the ADXL354 has a built in low pass filter with a cutoff of 1.5kHz
That sort of indicates that the ADXL354 won't deliver comparable results to what's described in the paper.  The bandwidth just isn't there.
However, the data samples shown in the paper appear to be averages over one second.  That seems to be usual for this kind of leak detection (I looked over a couple of other papers about leak detection, and the data samples were all averages over 1 second.)
That sort of says the bandwidth isn't that important.  Other than the one quote above, there's really no other discussion of bandwidth.  None of the equations in the papers seems to involve the frequency of the vibrations at all.
In comparison, there's this paper which does incorporate the frequency content into the leak detection algorithm - and says that this allows detection of vibrations that the typical accelerometer methods can't detect.
From that paper, it seems that the vibration frequency depends on the length of the vibrating pipe segment.
The authors used a pipe segment of about 2 meters length, and had vibration frequencies of less than 200 Hz.  Longer pipes (rather, pipes with a longer distance between the mounting points) would vibrate at lower rates, shorter pipes at higher rates.  The pipe diameter will also play a role.
The low pass filter in your ADXL354 will have an effect on the shortest length of pipe you can detect a vibration on.  I'd say getting to within a meter or two would be sufficient, so the bandwidth of the ADXL364 should be enough.
I wouldn't worry about the constant 1g acceleration from gravity.  I wouldn't use an analog filter to remove it, either.
I'd collect the data, then average all data points and then subtract the average from each data point.  That is in effect a high pass filter, but it will always have a cutoff lower than any signal in your measurements.
In doing the FFT, that will remove the spike at zero Hz (and any bleedover into the lower frequencies.)
If you are not using the spectrum based method, then I expect you will be using some kind of RMS calculation to get your 1 second average sample values.  Removing the DC will also be useful there.

If I were trying it, I'd go with the spectrum based method and long sampling times.  The more points you use in the FFT, the clearer the peaks will be and the lower the noise.  This will allow you to detect vibrations that are at or below the rated sensitivity of the accelerometer.
That allows you to detect smaller leaks with a (somewhat) cheaper accelerometer.
Alternatively, you can get more accurate location data for larger leaks.