The U.S. Virgin Islands and Puerto Rico are both located in the Caribbean, making them popular hot spots for U.S. travelers. The other three U.S. territories that you can visit without a passport are all located in the Pacific Ocean; American Samoa, Guam, and the latest addition, the Northern Mariana Islands.