Do we need to burn bootloader in new Atmega328p before uploading the arduino code
Tools > Burn Bootloader in the Arduino IDE does two things:


Set fuses according to the settings in boards.txt for the Tools > Board selection
Install the bootloader.


The latter is pointless since when you do an Upload Using Programmer the bootloader will be overwritten but the former is very important to do the first time you use a new chip(or anytime you want to change the fuses) to be sure that the correct fuses are set, which likely will be different from the default settings of the chip when you get it. Upload Using Programmer does not set fuses, the only way to do this via the Arduino IDE is Tools > Burn Bootloader.

So yes, you should Burn Bootloader.