When should I think about upgrading our RDS MySQL instance based on memory usage?
Before you switch server models, you need to know what differences there in configuration

I sat down and carefully looked over all default MySQL settings that would normally be written in a my.cnf. All are values are the same with the exception of two


max_connections
innodb_buffer_pool_size


Amazon set the following values based on server model

MODEL      max_connections innodb_buffer_pool_size
---------  --------------- -----------------------
t1.micro   34                326107136 (  311M)
m1-small   125              1179648000 ( 1125M,  1.097G)
m1-large   623              5882511360 ( 5610M,  5.479G)
m1-xlarge  1263            11922309120 (11370M, 11.103G)
m2-xlarge  1441            13605273600 (12975M, 12.671G)
m2-2xlarge 2900            27367833600 (26100M, 25.488G)
m2-4xlarge 5816            54892953600 (52350M, 51.123G)


For more information see my past posts on MySQL in Amazon RDS


Sep 21, 2012 : MySQL 5.5 Runs Out of Memory, Drops All Connections When Creating Many Databases
Sep 18, 2012 : How do I reduce the database size on AWS RDS MySQL?
Sep 10, 2012 : Does the Amazon RDS backup/snapshot service lock tables?
Aug 02, 2012 : Local database vs Amazon RDS
Jul 25, 2012 : Scaling Percona datacenters: setup and replication