Should Amazon RDS SQL Server 2017 Web Edition Max Memory settings be different from that of a locally installed instance?
No, I wouldn't treat this any differently than you would an on-prem instance with regards to max server memory.  The Windows Server 2012 box is almost certainly a VM, but hopefully Amazon is a managing that well to avoid VM problems.

I would definitely reserve 3 GB at minimum for the OS and other overhead.