title: mySQL to GCP Cloud-storage slowness
body: <p>I made this code to copy all tables from mySQL to json files in GCP Cloud-Storage.</p>
  <pre><code>#!/usr/bin/env python3
  #mysql to bigquery (all tables)
  
  
  import os
  import pandas as pd
  import pandas_gbq as pd_gbq
  import modin.pandas as pd_mod
  from sqlalchemy import create_engine
  from google.oauth2 import service_account
  import mysql.connector
  import pandas as pd
  from google.cloud import storage
  from google.oauth2 import service_account
  import csv
  # Service account file for GCP connection
  credentials = service_account.Credentials.from_service_account_file('key.json')
  #  BigQuery Variables
  os.environ[&quot;GOOGLE_APPLICATION_CREDENTIALS&quot;]=&quot;/home/python2/key.json&quot;
  #  MySql Variables
  MYSQL_USERNAME = 'root'
  MYSQL_PASSWORD = 'password!'
  MYSQL_HOST = 'localhost'
  MYSQL_DATABASE = 'test'
  # Path for the dump directory
  DIRECTORY = 'dump'
  
    
  def main():
  
      con_uri = 'mysql+pymysql://{}:{}@{}/{}'.format(
          MYSQL_USERNAME,
          MYSQL_PASSWORD,
          MYSQL_HOST,
          MYSQL_DATABASE
      )
      print('Connection url {}'.format(con_uri))
  
      try:
          engine = create_engine(con_uri, pool_recycle=3600).connect()
      except Exception as e:
          print('Error {}'.format(e))
  
      tables_query = 'SELECT table_name ' \
                     'FROM information_schema.tables ' \
                     'WHERE TABLE_TYPE = &quot;BASE TABLE&quot; ' \
                     'AND TABLE_SCHEMA = &quot;{}&quot;;'.format(MYSQL_DATABASE)
  
      list_tables = pd.read_sql(tables_query, con_uri)
  
      for index, row in list_tables.iterrows():
          table_id = '{}.{}'.format(DATASET_ID, row['table_name'])
          print(&quot;*******************************&quot;)
  
          print('Loading Table {}'.format(table_id))
          df = pd_mod.read_sql_table(row['table_name'], engine)
          storage_client = storage.Client()
          bucket = storage_client.get_bucket(&quot;my-buckets-123&quot;)
          blob = bucket.blob(&quot;{}.json&quot;.format(row['table_name']))
          df = pd.DataFrame(data=df).to_json(orient='records')
          #df = pd.DataFrame(data=myresult).to_csv(sep=&quot;;&quot;, index=False, quotechar='&quot;', quoting=csv.QUOTE_ALL, encoding=&quot;UTF-8&quot;)
          blob.upload_from_string(data=df)
  
  if __name__ == &quot;__main__&quot;:
      main()
  </code></pre>
  <p>It works but really slow, currently I have only 2 tables with size of 30-40MB but it doesn't seems like it's working in parallel to do it faster.
  I also get those warnings when running the script:</p>
  <pre><code>UserWarning: Dask execution environment not yet initialized. Initializing...
  To remove this warning, run the following python code before doing dataframe operations:
  
      from distributed import Client
  
      client = Client()
  UserWarning: `read_sql_table` defaulting to pandas implementation.
  
  </code></pre>
  <p>What am i doing wrong? how can I fix this code to work in parallel / faster? is it possible to do it in chunks?</p>