App Engine Standard connection to Cloud SQL Latency Randomly
It's hard to tell where exactly the problem is because there isn't enough information. I'm not sure how your application implements connecting, so it's hard to judge if that's the problem or not. Additionally, it's impossible to separate network latency vs query latency without more information.  

Here some advice that might potentially help:


Make sure you are using the /cloudsql/&lt;INSTANCE_CONNECTION_NAME&gt; unix socket to connect. You can follow the instructions here. This will likely provide the lowest latency. 
Make sure you are opening/closing connections correctly. You should be opening a connection, performing your queries, then closing the connection. Look up how best to do this in PHP given your stack. 
It looks like you are using a db-g1-small instance. This is a shared-core instance, and isn't recommended for production use (this is a particularly strange decision since you are also using a failover). Try upgrading to a machine size that is covered under the SLA.