Why are portrait photos sometimes sideways?
I found the answer to my own question.   It seems cameras take the picture and then store an flag, the Exif orientation, rather than just storing the rotated data, because the latter would be too time-consuming or it would be too expensive to give the camera custom hardware. (https://www.impulseadventure.com/photo/exif-orientation.html)

But not all software supports these image rotation flags, so it's best to process images when importing them to convert them to an image with no rotation flag.  Some tools for doing this are here:

http://how-to.wikia.com/wiki/How_to_auto-rotate_digital_photos_to_their_proper_orientation

The exifautotran script seems to work for me, e.g.:

exifautotran *.jpg