Acoustic record and store devices
There are plenty of devices that can do this. Simply Google portable recorders. They generally all have a microphone built in. Some have line level inputs, and some can also take XLR inputs and provide phantom power for condenser microphones.

If you don't need the XLR inputs, I can recommend the Zoom H1. It has a stereo microphone, can also take a line level input and is fairly cheap. I have used this to record various foley sounds including bird songs. The sounds save onto an SD card in high quality 24-bit/96 kHz WAV files for analysis on a computer later.

It's best to get a little tripod for it too to stop any handling noise.