MOSFET driver is running hot
at first:  let's look at TC4451/2 datasheet:
http://ww1.microchip.com/downloads/en/DeviceDoc/20001987C.pdf
The   TC4451/TC4452   are   single-output   MOSFETdrivers. These devices are high-current buffers/drivers capable of driving large MOSFETs and insulated gate bipolar transistors (IGBTs). 

Then look at Functional Block Diagram,  an output already are powerful mosfet. Drop your Q1 and all around, and connect led+resistor to output 6 OR 7 and + OR - accordingly. don't connect 6 and 7 together.

Second: if it just LED, ESP32 has enough 1ma current on output to drive led.

and finally: are you sure your Q1 not burnt?  take it separately, connect to as on your diagram, and using  var. resistor slowly increase voltage on gate from 0 to 5, at  "0"  it will completely shut.