Is it ok to delete the Amazon .apk files?
You can redownload them any time you want or just install them using a file manager, no need to put them back there.  

If you backup them to your PC it's a good idea to use something like this to keep track of the various apks.