Why is it so hard to package and deploy software? I want to understand what exactly is the inherent complexity of creating self-contained binaries? Currently it seems that golang is the only game in town when it comes to almost completely solving the deployment problem but even go has some issues with versioning dependencies. Why aren't more languages solving this problem at the compiler/language level?Note that I'm not talking about library code. I'm talking about an actual application or a service that needs to get to a production environment as quickly and as reliably as possible.
Deployments, another unsolved problem in software engineering
deployment;packages;production
Dependencies. Take a very simple Python application. I can distribute it as a script. It will work on some machines, and won't work on others, because the script is using Python 3 and those machines have only Python 2 installed, or no Python at all.So now, I need to make sure, during deployment, that a specific version of Python is either available or will be installed during deployment.How would you solve that at compiler/language level?...See also: Has anyone nailed dependency management?Configuration. Take a very simple service which receives calls from the outside. It works on my machine. I deploy it on the server, and it stops working. Nothing happens: the service receives no calls, no matter what I do.This is because the server has a firewall which blocks the port used by the service.Which means that my deployment script should open the specific port. If I have total control over the servers and know that every server uses UFW as the only firewall, the task is relatively easy. If I don't have such control and has to support multiple firewalls, the task becomes more complex.There is nothing a compiler/language can do for me there.Infrastructure. Take an example of a web app which uses MongoDB. If there is no MongoDB server, the app won't work. This means that database should be deployed as well, probably on a different server. Thus, I should create a dedicated virtual machine for the database, deploy MongoDB, and configure the deployed web app by setting the proper connection string.But then, it appears that I need a failover, so now I need two database servers, and I need to change the connection string accordingly. More importantly, I need to configure the failover so the data replicates as expected after one of the database machines is deployed.Then, I don't want to lose all the data if the VMs which host the database are destroyed, so I make backups. Now, the deployment script should also handle the recover from the latest backup.Those three points are just a small subset of all the challenges you may have during deployment.In all those points, neither your compiler, nor the programming language can help. This is because most of the deployment challenges are not inherent to the language/compiler/framework you use.But wait, there are instead a bunch of tools which are specific to deployments:Package managers. Have you used npm in Node.js, pip in Python or NuGet in C#/Visual Basic? Their handling of dependencies is particularly exciting, and they make it quite easy to deploy a small software product. Have you used apt-get? The sole fact that you can simply type apt-get install something and let the tool do all the job, no questions asked, is nearly magic.Configuration managers/infrastructure automation tools. Tools such as Chef, Puppet or OpenStack do a great job of making it easy to automate deployments. It doesn't mean that you can say Make me a MongoDB failover and watch the result (although, it does exactly that for simple cases), but at least, you can have a reproducible, automated process which makes your life much easier.