How can I remove the caulk from this window?
It could be construction adhesive, which would be hard to completely remove. Use a sharp utility knife to open up the crack. Don't worry about removing material that is still providing a seal, just remove the rough stuff in the opening so the new caulk will be able to bond to solid material. Fill the enlarged opening with caulk and you're done, unless you feel the need to paint it.

BTW, you can't paint conventional silicone caulk.