Are Availability Group Databases on a Secondary replica automatically set to read-only by the AG?
I would give the docs a good read. Particularly the section on availability replicas.


  Every availability replica is assigned an initial role-either the primary role or the secondary role, which is inherited by the availability databases of that replica. The role of a given replica determines whether it hosts read-write databases or read-only databases. One replica, known as the primary replica, is assigned the primary role and hosts read-write databases, which are known as primary databases. At least one other replica, known as a secondary replica, is assigned the secondary role. A secondary replica hosts read-only databases, known as secondary databases.


As can be seen above, each replica can either be read-only or read-write (secondary or primary respectively). In addition there can only be one primary (read-write) replica. When we refer to replica in this respect, we are referring to the database level. Each server can host multiple availability groups, with potentially mixed primary-secondary configuration.

The situation you are referring to with multiple read-write replicas is not supported by AlwaysOn Availability Groups, which are designed to meet High Availability and Read-Scale needs, not multi-master setups. You should look at two-way replication technologies to support this need.

As for the read-only setting itself, this is automatically set by the availability group, although it's worth saying that this is not the same setting as the "Database Read-Only" setting found in the database properties (or is_read_only in sys.databases), nor is it the same setting as the file group level read-only option.