How can I collect data from a hydrophone?
You need three things: an amplifier (sorry, unavoidable), an ADC, and some digital storage.  With the quality of modern op-amps, you totally do not need to be concerned about noise and distortion; you can easily achieve >100dB SNR.  Nonlinearities in the hydrophone itself will be a far worse source of distortion.

If you don't amplify the signal, you will be using only 100mV (of 2500mV total) dynamic range, which means you get about 11 bits of precision from a 16-bit ADC.  Also, the ADC does NOT have a very high input impedance and it may load down the piezo output; you might get absolutely nothing.  The amplifier stage is therefore critical to match the (high impedance, low level) source signal to the (somewhat low impedance, higher-level) input of the ADC.

Audio ADCs are commonly available and support 16-24 bits of precision.  8 bits is available too but I suspect would be insufficient dynamic range.  Your 5kHz bandwidth tells me these are long-range signals and you need enough dynamic range to ensure that nearby noises do not saturate your input stage.

An audio ADC will expect an input of a couple of volts, hence the need for the amplifier.  It sounds like you need probably about 20x gain, which is very achievable with a high impedance op-amp, with low noise and distortion.

You could consider an automatic gain control, but that will introduce distortions and make the output much harder to process.

You take the streaming ADC output (I2S, usually) from your ADC and stream it through a microcontroller into some flash storage, probably an SD card.  16 bits at 22kHz is 3.6GB/day; a 64GB card is pretty affordable and will give you 17 days of recording.