Is it possible to change the password for a WiFi network when it is not in range? Long-pressing on a WiFi network that is in range brings up a menu that includes "Modify network", which can be used to change the password for the network. Networks that are not in range do not show up on this screen. 

Under "Saved networks", long-pressing is the same a simple press. This brings only the options "Forget" and "Done". 

Is it possible to modify the password for a WiFi network that is not in range? Must it appear under current WiFi networks? 

I am trying to do this on both unrooted Android 5.1 and 6.0 devices.
You can delete it ('Forget') and create it new again, here you can define the password. Write down the SSID (Routers name) and encryption way (e.g. WPA) before you delete it ;)

Without root you cannot view or edit the /data/misc/wifi/wpa_supplicant.conf file (this is the path on Android CM13), where the network configs are saved. That means ssid, passwords (psk) etc.