How to fill in hinge and door handle mortises?
I'd put in a piece of matched wood, then use an appropriate filler to smooth the junction between the patch and the frame.

If it's painted, you can get away with other materials.