Unable to set Photos system library
This might be simpler using EasyFind from the App Store [freeware]

A search [including invisibles] for apple.photos immediately after launching Photos holding  Opt ⌥ , then quitting, shows several modified files, including 

~/Library/Containers/com.apple.Photos/Data/Library/Saved Application State/com.apple.Photos.savedState
/private/var/folders/nc/c83zpk8x7kd9qrxrd3nnbfym0000gn/T/com.apple.Photos.librarychooserservice

It might be useful [after making sure you have a good backup] to test out some of those. SavedState is always a prime candidate for this type of thing, I'm less sure about the chooserservice, but it sounds apropos.