Why isn't my S3 Cross Region Replication working for some files?
So I verified it was definitely public access causing the issue somehow; uploading an identical file but with the public read acl on totally broke it.

I worked with AWS support and they noticed I had "Block public access" enabled in the destination bucket.  Apparently rather than replicating the object and then making it private, it just fails replication.

tl;dr if you want to replicate objects that are public read, don't block public read at the destination.