title: Best Practices to run and schedule Python Scripts on Google Cloud (no Cloud Function)
body: <p>I'm building an ETL process that use Firestore and BigQuery. I also write 2 Python scripts: the first one get data from external service and write them to BigQuery, the second one write data from BigQuery to Firestore. I cannot deploy this scripts as Cloud Function (or Cloud Run) due to the timeout limit (9 and 60 minute).
  My goal is deploy the two scripts separatley (keeping the Cloud Function logic), schedule the first one daily and start the second one after the first has ran and written data to BigQuery.</p>
  <p>Following
  <a href="https://towardsdatascience.com/the-easiest-way-to-run-python-in-google-cloud-illustrated-d307c9e1651c" rel="nofollow noreferrer">https://towardsdatascience.com/the-easiest-way-to-run-python-in-google-cloud-illustrated-d307c9e1651c</a> I deploy a containerized python script to Compute Engine, but is not easy schedule and syncronize the 2 scripts. I could Scheduling a VM instance to start and stop (<a href="https://cloud.google.com/compute/docs/instances/schedule-instance-start-stop" rel="nofollow noreferrer">https://cloud.google.com/compute/docs/instances/schedule-instance-start-stop</a>) but I don't know when the first script finish. I could write some cloud functions to start and stops instance based on pub/sub messages (<a href="https://cloud.google.com/scheduler/docs/start-and-stop-compute-engine-instances-on-a-schedule" rel="nofollow noreferrer">https://cloud.google.com/scheduler/docs/start-and-stop-compute-engine-instances-on-a-schedule</a>) but it doesn't seem like the cleanest way.</p>
  <p>Cloud Scheduler in addition with App Engine or Kubernetes Engine should be the best way to do this works, but I have to start from scratch with GAE or GKE and they seem suitable for application more complex then mine.</p>
  <p>Do you have a solution that combine speed of development, clean of process and cost reduction?</p>