Power adapter for handheld sewing machine
Yes. Not only will the adapter heat but also the device may not work properly.

This is because the power needed by the device is not provided. you gave it 6V 300mA instead of 6V 800mA. IF the device wants to draw 800mA, the adapter will be in danger.