Would insulating a door between my closet and attic help keep the closet warmer?
Better sealing is usually step one. You say a portion of the back wall can be lifted out of the way to access the storage behind. This makes me suspect the sealing isn't as good as it could be. (If you have a non-contact thermometer, you could point it at the edges of this removable portion to confirm, and compare to other areas of the wall. Otherwise, you could use you hand, and see if you can feel a draft; this will likely work better in the winter.)

As to how to fix this, you'll basically want some form of weatherstripping. This will probably need to be combined with a latch system to hold the access door shut against the weatherstripping, in order to provide a proper seal.