SQL Server connection from iis website: "error: 26 - Error Locating Server/Instance Specified"
Some of the mystery is solved - The reason that some sites could connect and some couldn't is that there was an additional connection string stored in the machine.config file that two of the sites were using.  The site that worked when a port number was specified did not use this connection string, so worked.  The other sites didn't work when port number was specified because I hadn't added the port number to this connection string in machine.config.  Adding the port number to that connection string as well as the ones stored in web.config allowed all the sites to work.

Additionally, the reason that iisreset had an effect is that once IIS knows the port number that SQL is using, it seems to 'remember' the port number, and so it no longer has to rely on using port 1434.  Once I did an iisreset it 'forgot' what port number was used, so had to make a request on port 1434, which for some reason doesn't work.

So my question has gone from a very confusing 'works sometimes' situation to a basic 'cannot communicate on port 1434' situation.