SQL Server Report Manager 404 using HTTPS
The problem has to do with the certifcate being used with SSRS. If your certificate is created with hostname.whatever.com, you can only access the Report URLs through that path, https: //hostname.whatever.com/Reports. If you were to try https:// 10.xxx.xxx.xxx/Reports (where the IP is the one on the server) you will receive the 404 errors.

So the fix for my situation:
I found the friendly name used to generate the certifcate on the server was dorked up. I am in the process of getting that fixed to be the correct fully qualifed DNS name of the server in use. I found this out by installing SQL Server Report Server on a server that had the certificate created with the correct name of the server being used.

Once I have that certificate corrected and verify will simply mark this as the correct answer.

EDIT
Well another problem came up in trying to change the certificate. The admins of the server removed the certificate before I could properly remove it from RSCM. So now it will not bind to the new certificate because it already sees one binded. There is really no clean, easy way of fixing this since you don't have access to IIS (SSRS uses HTTP.sys). So following this KB, you have to go into NETSH to remove the binding(s). Which you will note in that KB article the link to NETSH does not provide much info, so use this one.
I had to use the two following commands to delete the bindings on Window Server 2008 R2:
WARNING: I am on a single, standalone server. If you are in a server farm or have multiple instances of SSRS running, be very careful with which SSL binding you delete.


This is after getting to the "netsh http>" prompt:
delete sslcert ipport=0.0.0.0.:443
delete sslcert ipport=[::]:443


I did that and then went into RSCM and selected the new certificate that was installed on teh server. That is not the end though, for some reason it kept removing the binding and showing "unknown" in the link. I stopped SSRS and then opened up the rsreportserver.config file and found the old ("bad") binding still was showing in the config file. I removed that went back into RSCM and started the service and it seemed to fix it.