Making an ESP32 based flight controller
If you can accept STM32, you should definitely look at Betaflight, it's a full opensource firmware stack for a quadcopter.
It has a whole bunch of libs for the different sensors.
It is considered as the reference for quadcopters.
Note that many many manufacturers make boards for this purpose at a fairly low cost, if you don't want to reinvent the wheel.