title: How to diagnose extremely slow AWS RDS MySQL Performance?
body: <p>My DB has around 15 tables, each with 40 columns, with 10.000 rows each.
  Most of it with VARCHAR, some indexes and foreign keys.</p>
  
  <p>Sometime I need to reconstruct my database (design flaw, working on it), which takes about 40 seconds locally. Now I'm trying to do the same to a AWS RDS MySQL 5.75 instance, but it takes forever, something like 40-50 minutes. The last time I had to do this same process it took no more than 5 minutes, still way more than the local 40 seconds, but I'm happy with it.</p>
  
  <p>My internet speed is at about 35 Mbps Download / 5 Mbps Upload.
  I know it's not fast, but it's consistent, and it hasn't changed since my last rebuilt.</p>
  
  <p>I enabled General Logs, but all I can see are the INSERT queries, occasionally some "SELECT 1".
  I do have same space for improvements on my code, but still, from 00:40:00 to 50:00:00, it seems that there's something else going on.</p>
  
  <p>Any ideas on how to diagnose and find the bottleneck?
  Thanks</p>
  
  <p>--</p>
  
  <p>Additional relevant information:
  It is a Micro instance from AWS, all of the relevant monitoring indicators are basically flat: CPU at 4%, Free Storage Space at 20.000 MB, Freeable Memory at 200 MB, Write IOPS at around 2,5, the server runs a 5.7.25 MySQL, 1vCPU, 1Gb of RAM and 20GB of SSD. This is the same as 3 months ago when I last rebuilt the database.</p>
  
  <p><code>SHOW GLOBAL STATUS</code>: <a href="https://pastebin.com/jSrAzYZP" rel="nofollow noreferrer">https://pastebin.com/jSrAzYZP</a>
  <code>SHOW GLOBAL VARIABLES</code>: <a href="https://pastebin.com/YxD7dVhR" rel="nofollow noreferrer">https://pastebin.com/YxD7dVhR</a>
  <code>SHOW ENGINE INNODB STATUS</code>: <a href="https://pastebin.com/r5wffB5t" rel="nofollow noreferrer">https://pastebin.com/r5wffB5t</a>
  <code>SHOW PROCESS LIST</code>: <a href="https://pastebin.com/kWwiyGwf" rel="nofollow noreferrer">https://pastebin.com/kWwiyGwf</a>
  <code>SELECT * FROM information_schema...</code>: <a href="https://pastebin.com/eXGBmetP" rel="nofollow noreferrer">https://pastebin.com/eXGBmetP</a></p>
  
  <p>I haven't made any big changes to the server configuration, except enabling logs, e maxing out <code>max_allowed_packets</code> and saving logs to file.</p>
  
  <p>In my backend I have a Flask app running, when it receives the API call, it takes a bunch of pickled objects and adds them all to the database (appending the Flask SQLAlchemy class to a list) and then running <code>db.session.add_all(entries)</code>, trying to run a bulk operation. The code is the same, both for <code>localhost</code> and my remote server.</p>
  
  <p>It does get slower in three specific tables, most of them with VARCHAR columns, but nothing different from my last inserts - it seems odd that the problem would be data, or the way the code is structured, or at least doesn't seem reasonable that this would result in a 20 second (localhost) to 40 minutes (hosted server) time, specially when the rest of the tables work mostly the same.</p>