Which kind of AA battery should I use to power up my ESP-32
It looks like the esp8266 has an input voltage range of 2.5v to 3.3v, and it is fairly power hungry, especially with the radios and peripherals all turned on. There are a couple of things you can do to extend the runtime of the device, namely:


Turn off the peripherals when you aren't using them. From the data sheet, the wifi uses between 50 and 200mA when it's on, which is quite a lot of power. You didn't mention what you're doing with the device, but if for instance you're making a data logging application which uploads some data to a server once a minute, then there's no reason to have the radio powered up all of the time. 
Similar to point 1, if the micro controller isn't doing anything, there's no reason to keep it on. According to the data sheet, the micro controller uses 15mA by itself, but putting it into a light sleep uses 0.9mA, and putting it into a deep sleep uses \$20\mu A\$. Again taking the example above, there's not much of a reason to have the microcontroller powered on in between uploads of data, so if you can shut it off, do so.
Use a larger battery. Pretty self explanatory, using a larger battery (such as C or D cells) in place of the AA cells will give you more runtime. Additionally, you may want to consider using a lithium battery (with protection!), so when the battery does die you can recharge it instead of throwing it away.
Use a voltage regulator. Part of the problem you're having is that fresh alkaline batteries provide 3.0V, and by the time you've discharged them to 2.5V  the device shuts off. However, the batteries still have a lot of energy left, which can't be used by your device. To fix this, you can use a buck (step-down) converter, and use a higher voltage battery pack. For example, if you instead use a 6V battery pack (4 cells), and a buck converter that brings the voltage down to 3V for the micro, you can run the cells down to 3V before the microcontroller gives out. Alternatively, you could use a buck/boost converter which can convert the input voltage to a lower or higher voltage. Using this, you may even be able to run your 6V battery down to 1.5V, or run your 3V pack down to 1.5V, giving you more runtime.