How can I insulate the door to my attic?
Well, if there's a lack of insulation under the plywood floors, taking up the plywood, insulating, and replacing the plywood would be the correct method.
If there's insulation there but it's inadequate, that's a different issue.
Hard to know about door insulation without any idea what sort of door you have. If it's a trap door you can probably put some insulation over the top of it in some way, but the way will depend on how it opens/operates.