AWS CodeDeploy in GCP Style
AWS CodeDeploy in GCP style - Rimantas Ragainis - Medium
The IssueAWS CodeDeploy is software deployment service, which handles the complexity of updating your applications and eliminating manual operations. Which is true. But when your business requirements suddenly shift the tide towards hybrid-cloud engagement and GCP is in your spotlight, you’re in trouble.GCP doesn’t currently have alternative to AWS CodeDeploy service. It’s worth to note though that the most similar one is Cloud Build, which provides ability to split custom deployment step-by-step with services (aka builders, e.g. curl, docker, gsutil etc.) or custom scripts to launch, but everything is being ran inside new Docker container. In that case, you would have to dockerize current AWS infrastructure and come up with idea how to make communication between containers (unable to mount persistent OS volume).Does it sound like a lot of paperwork? It does for me. My expectation is to spend as less time & effort as possible and ideally do not touch AWS side at all. So.. I came up with idea of CodeDeploy service in GCP style - that’s where my journey began.The codeAs my intention is to use the same AppSpec file - just for getting the whole picture - let’s take a peek into it:I will not go through the file line-by-line as Amazon has done it already and in more sophisticated way — you can find all the meanings in their specs. Next we need is to push our deployable content (including AppSpec file) to remote/cloud storage which would be accessible by deployment managers — AWS (awscli) and GCP (gsutil & gcloud) respectively:# AWS via awscli toolaws deploy push \  --application-name app \  --s3-location s3://codedeploy/automated-app.zip \  --source ~/app_source# GCP via gsutil tooltar -cf - -C ~/app_source . | gsutil cp - gs://codedeploy/automated-app.zipOnce deployment is ready, next step is to initiate actual deployment process:# AWS via awscli toolaws deploy create-deployment \  --application-name app \  --deployment-group-name=app \  --s3-location bucket=codedeploy,key=automated-app.zip,bundleType=zip# GCP Pub/Sub topic creation (run once)gcloud alpha pubsub topics create codedeploy \   --project <GCP_PROJECT_ID> \   --labels=storage_bucket=codedeploy,storage_key_extension=zip,storage_key_prefix=automated-# GCP via gcloud toolgcloud pubsub topics publish \  projects/<GCP_PROJECT_ID>/topics/codedeploy \  --attribute application=app,storage-location=gs://codedeploy/automated-app.zipAt this point, AWS CodeDeploy will handle the process itself as defined via UI console, but let’s look at the GCP case more closely. I’m using Pub/Sub messaging service to deliver codedeploy event within active pool of app servers. To be precise, a message body itself is empty, but extra attributes (or headers) are set — in my opinion it’s just more scalable and easier to handle event data on other side.Finally, I’m missing last part of a puzzle — it’s a custom deployment agent which should be able to listen to broadcasting events, handle them properly and run deployment process. Google gives you list of client libraries to choose from with robust examples as a kick-starters based on your flavor. I chose a python way:And in order to keep script up & running, let’s enable systemd service:All done!2019-07-04 17:21:30,674702 | app-10.x.x.x | Listening for messages on "projects/xxx/subscriptions/app-10.x.x.x"...2019-07-04 17:35:44,861695 | app-jobs-10.x.x.x | Skipped deploying as current "app-jobs" doesn't match against "app".2019-07-04 17:55:50,073332 | app-10.x.x.x | Deploying application "app" with {u'storage-location': u'gs://codedeploy/automated-app.zip', u'application': u'app'}...