Revert "Avoid overflows when reading pgm files with maxval != 255 and != 65535."

The patch worked on little endian because pgm decoding was broken
but it was not correct in any way.

This reverts commit 7651c0e49b1f6e66120ef32570a87e3f09224574.
