Jenkins Is Getting Old - <AUTHOR> https://itnext.io/jenkins-is-getting-old-2c98b3422f79 <comment><USER4> Disclaimer: I'm pretty biased towards Gitlab -- write about the things you can do it from time to time and they gave me some free swag once.

Best CI I've ever used is Gitlab CI[0]. The runner is completely open source[1] and you can use your own runner with your gitlab.com (or local instance) projects -- set it up in an autoscaling group[2] for savings.

I run [https://runnerrental.club](https://runnerrental.club) but Gitlab also recently released the ability to pay for minutes in 11.8 [3], so my product is more-or-less dead in the water but I don't mind since Gitlab is such an excellent tool, I'm glad to see them fill the need.

But back to Gitlab CI -- the YAML configuration documentation[4] is pretty fantastic -- Most easy things are easy and hard things are possible. I suspect that one could run an entire startup like circleci/travis/drone based on _just_ the software that Gitlab has open sourced and made available already.

[0]: [https://docs.gitlab.com/ee/ci/](https://docs.gitlab.com/ee/ci/)

[1]: [https://gitlab.com/gitlab-org/gitlab-runner](https://gitlab.com/gitlab- org/gitlab-runner)

[2]: [https://docs.gitlab.com/runner/configuration/runner_autoscal...](https://docs.gitlab.com/runner/configuration/runner_autoscale_aws)

[3]: [https://about.gitlab.com/2019/04/22/gitlab-11-10-released/#p...](https://about.gitlab.com/2019/04/22/gitlab-11-10-released/#purchase- add-on-ci-runner-minutes)

[4]: [https://docs.gitlab.com/ee/ci/yaml/](https://docs.gitlab.com/ee/ci/yaml/)

<comment><USER5> Multirepo pipelines behind a paywall is a good reason to say "No, thanks".

<comment><USER6> TBH, its easily done in shell via REST API. We use custom pipeline runner from simple Powershell scripts that works even better for us then default style.

We set all jobs to manual and then our script triggers them depending on commit message, person, moon phase etc.

But really, this should be in core. Its very hard to do multirepository stuff. Its not that easy to do mono repo stuff too - I really need a pipeline withint sub-project, a mother pipeline, option to run whatever one I want etc... Gitlb pipelines could be a lot better.

<comment><USER4> I find it very easy to make multi-repo stuff, I've also wrote about it: [https://vadosware.io/post/fun-with-gitlab-ci](https://vadosware.io/post/fun- with-gitlab-ci)

Also, running pipelines in different projects and whenever you want is accessible from the web interface... It could be cleaner, but IMO adding even more syntax to the YAML file can also be a rabbit hole -- CURL from the script section seems like not a bad middle ground.

Am I misunderstanding what is being referenced?

<comment><USER7> This post seems very focused on projects with just a single developer. The "recognizing and tagging" versions for example completely breaks down when you have multiple people commit and merging things to master and multiple pipelines running that are trying to recognize and tag versions.

<comment><USER4> The post is very much not complete -- there is much more a team would have to standardize to fully take a set up like this into production, bu I'm not sure that it "completely breaks down" with more people contributing.

As long as there is _some_ release coordination, a system like this can work. In particular, I've found that the lightest way to get a system like this working for x > 1 developers is to have a release-vX.X.X branch for releases that are going out and vX.X.X tags for releases once they have actually landed.

You mention the recognizing and tagging versions being an issue -- are you imagining a world where two people are releasing something at the same time? I'm not exactly sure

If we assume that what that posts suggests _isn 't_ feasible, regardless of how you deploy and do your CI, if your release process can be done by a human, it can likely be automated. Gitlab CI is most robust and yet easiest to understand automated CI system I've seen for making that happen (at least that was my intended point).

</comment></comment></comment></comment></comment></comment><comment><USER8> Hello, I see a lot of great feedback in this post. I am a product manager working at CloudBees, the primary corporate sponsor of Jenkins. Jenkins is now in the Continuous Delivery Foundation as well.

While it is easy to bash on an inanimate object, there are some very dedicated and empathetic people who care deeply about the project. Some of those people do this work in their off-hours and some to this work as part of their daily work activities AND also in their off hours.

In that spirit, we want to make Jenkins better and created a separate group at CloudBees in the Product and Engineering teams late last year. They focus on open source work for Jenkins and on some proprietary things for CloudBees Core (built on Jenkins). We also have a dedicated user experience/product designer who started working on the project a few months ago. One of the first things he and I worked on was creating a curated, tailored version of Jenkins via the CloudBees Jenkins Distribution. This distribution will focus more and more over time on a guided workflow for continuous integration and delivery with Jenkins. These patterns will also be shared with open source Jenkins - <USER2> through direct contributions and others through suggestions (better plugin categorization, documentation, etc.).

Please use this comment thread to share your constructive, honest feedback about how we can improve Jenkins.

<comment><USER9> I think its already mentioned about Jenkins Configuration as Code. But in general stick to a configuration method, I have had to migrate from JJB > groovy scripts > init.d groovy > Jenkinsfile.

And can we PLEASE open up the issues tab on the Github repos especially for plugins. There is currently no way to provide any feedback/report issues on these plugins because the "issues" tab is disabled. Our current options are \- Put a comment on the plugin wiki page \- Hunt down the the relevant support forum for the plugin \- Find the original source repo and post an issue

All of which are not ideal and does nothing to directly help the development of the plugins.

<comment><USER10> Jenkins has used Jira for bug tracking since before GitHub existed, so although the project's code and all plugins are now managed on GH, bug tracking continues to live in Jira. Anyone can create an account and add issues here: [https://issues.jenkins- ci.org/secure/Dashboard.jspa](https://issues.jenkins- ci.org/secure/Dashboard.jspa) All plugins track issues there, so you just put the appropriate plugin, or "core" for Jenkins itself, in the 'component' field and the issue should be assigned to the right person.

</comment></comment></comment><comment><USER11> I sincerely wish I could move away from Jenkins for the reasons stated in TFA (GUI-oriented, slow, hard to backup/config, test-in-production mentality and boundless plugins) but I've never found something that fits the bill.

The much-touted repo integrations (travis, circle...) all have an exclusive focus on build-test-deploy CI of single repos.

But when you have many similar repos (modules) with similar build steps you want to manage, and want to have a couple of pipelines around those, and manage the odd Windows build target, these just give up (it's docker or bust). Sadly, only Jenkins is generic enough, much as it pains me to admit.

Anyone got a sane alternative to jenkins for us poor souls?

<comment><USER12> TeamCity from JetBrains is the same thing as jenkins, except the core features are working core features instead of broken plugins. It's paid software though, you get what you pay for. [https://www.jetbrains.com/teamcity/](https://www.jetbrains.com/teamcity/)

On the other hand there is Bamboo from Atlassian. [https://www.atlassian.com/software/bamboo](https://www.atlassian.com/software/bamboo)

I really don't understand this mentality of there is no better tools when there are better tools than jenkins and they've been around for a while.

<comment><USER13> Fine, when you are one specific vendor shop, like Jetbrains or Atlassian stack and you have got plenty of financial power, then there is always cool features, what can bring benefit. But in the end CI and CD systems are glorious semi-smart cron runners. Are these tools 10x better than Jenkins. Not so much, CI/CD is from one of the standpoint most important and in the same time less important tool, delivery should suck very much to migrate to new platform just because. Jenkins shines here, it's not perfect, but it works. More or less it's for free from licensing standpoint, you don't have to go thru Corporate procurement hell. It's not for free from workforce perspective, but none of these tools are with zero configuration. Just x,y,z, still some yaml or some other crazy configuration needs to be done (like Bamboo dsl).

<comment><USER12> Of course it can be 10 times better. It's so trivial to be 10 times better.

First you checkout the project from the repo and it just works, doesn't matter GIT, SVN or whatever. How many plugins does it take to checkout a project in jenkins? Is there even a git plugin working nowadays?

Then, you build the project. If it's any of C# or Java for example, the ant/maven/sln/nuget files are detected automatically, just click next and it's built. Does jenkins even understand what is a requirements.txt? Hint: It's not a bash script.

The JVM and the Visual Studio are detected automatically on all the build slaves and the project is already building in the right places. If you want to aim for specific tool versions, there are presets variables on all hosts to filter where to build and to use in build scripts so paths are always right. How is the build matrix plugin in Jenkins lately? Broken as usual?

The project is built, or is it building? It's easy to tell because there is a clear colored status icon and the estimated time to completion is displayed. Teamcity offers that out of the box for maybe 15 years now. Well, jenkins finally got a progress bar too a couple years ago. I guess I'm defeated, Jenkins caught up on basic core functionality only a decade late, I can't justify to pay for working and polished tools anymore. Well, I hope our sysadmin will install the Extra Status Icon Plugin or we'll have to live without the big colored circles next to the build.

<comment><USER14> TeamCity is indeed quite good.

</comment></comment></comment></comment></comment><comment><USER15> OP really needs to try Concourse. Same container-based workflow as Drone that is touted as a solution, but more mature, and much more testable than Drone.

Concourse really hits his requirements for ops-friendliness and testability. It's easy to upgrade because the web host and CI workers are completely stateless, and the work you do with Concourse is easy to test because the jobs themselves are all completely self-contained. Because Concourse forces you to factor your builds into inputs, tasks, and outputs, it becomes somewhat straightforward to test your outputs by replacing them with mock outputs.

The main issue with Concourse is that it has a huge learning curve to use effectively. When you have many pipelines, you end up learning the undocumented meta pipeline pattern to help manage them. You end up implementing stuff like mock outputs by yourself, since it's not really a first-class concept. Concepts like per-branch versioning that have been in products like Jenkins for years are only now entering development as "spaces". All of the configuration can be version controlled, but it's all YAML, so to wrangle everything together, you end up using something which will compile down to YAML instead. RBAC much improved in v5 but still leaves much to be desired. There are no manual steps, only manual jobs. Job triggering is relatively slow due to a fundamental design tradeoff where webhooks trigger resource checks which trigger jobs instead of triggering jobs directly, to make resources easier to maintain. Nobody really tells you this on the front page.

It has its flaws. But if you delve into it you see very quickly that it's an incredibly solid product. The ability to run one-off jobs from developer workstations on CI workers, and to easily SSH into those CI worker containers from developer workstations, is incredibly helpful for debugging issues. Because everything is stateless, everything tends to "just work". If you go in with your eyes open about its limitations, you'll have a good time.

<comment><USER16> The thing that turned me off concourse last time I checked it out is that their documentation assumes (assumed?) you're going to use BOSH. I don't want to have to learn and maintain yet another infrastructure as code tool, just for my build server. I know you can run concourse without it, but all their examples seemed to use it and I didn't want to hit edge cases that they didn't account for. So I gave up before too long.

<comment><USER17> There's a helm chart available for Concourse, I haven't tried it yet though. Definitely agree about BOSH making it harder to get started.

[https://github.com/helm/charts/tree/master/stable/concourse](https://github.com/helm/charts/tree/master/stable/concourse)

</comment></comment></comment><comment><USER18> Our setup runs Jenkins master and slaves as Kubernetes pods, with plugins limited to only the very few required to get GitHub integration and slaves working.

Jobs are configured by adding an entire GitHub organization. All repositories with corresponding branches, pull requests and tags are automatically discovered and built based on the existence of a Jenkinsfile.

Everything is built by slaves using Docker, either with Dockerfile or using builder images.

Job history and artifacts are purged after a few weeks, since everything of importance is deployed to Bintray or Docker repositories.

By keeping Jenkins constrained in this fashion, we have no performance issues.

<comment><USER19> That is exactly how we're doing it as well, though I am interested in checking out Cloudbees'Jenkins. We've recently incorporated Zalenium (selenium grid which autoscales nicely natively in kubernetes) - just had to work a little magic with automatic service creation during builds.

</comment></comment><comment><USER20> I receive vulnerability notifications for Jenkins, pretty much regularly... mostly XSS and RCE.

[https://www.cvedetails.com/vulnerability- list/vendor_id-1586...](https://www.cvedetails.com/vulnerability- list/vendor_id-15865/product_id-34004/Jenkins-Jenkins.html)

I'm just waiting for Apache to adopt it, and then it'll sit and fester like everything else in the Apache graveyard, full of vulnerabilities and slowly decaying.

Those are just Jenkins core exploits too... there are so many many more for Jenkins plugins.... [https://www.cvedetails.com/vulnerability- list/vendor_id-1586...](https://www.cvedetails.com/vulnerability- list/vendor_id-15865/Jenkins.html)

<comment><USER21> Jenkins is now part of the CD Foundation ([https://cd.foundation/](https://cd.foundation/)) which is one of the linux foundation sub-foundations. Don't expect it to show up in the apache foundation.

<comment><USER22> I don't think tinix was excpeting it to literally become a apache project - <USER3> was just saying its in a state of decay that apache is infamous for.

</comment></comment></comment><comment><USER23> Jenkins's plugin manager is absolutely terrible.

If you're stuck on an older version of Jenkins, you better not click the "refresh" button in the plugin management page, cause otherwise the page is just filled with red warnings saying that the latest version of each plugin is incompatible or has dependencies that are incompatible with your current Jenkins version.

There is afaik no way to install the last plugin that was compatible with your version of Jenkins.

<comment><USER24> > There is afaik no way to install the last plugin that was compatible with > your version of Jenkins.

Probably not directly, but if you know the version, you can download the HPI and install it manually. Jenkin's Docker build also contains install- plugins.sh ([https://github.com/jenkinsci/docker/blob/7b4153f20a61d9c579b...](https://github.com/jenkinsci/docker/blob/7b4153f20a61d9c579b5f653a4a99fb543f26c41/install- plugins.sh)) that you can use to install specific version of plugin via command line.

<comment><USER23> Thanks! That script definitely gives some pointers on how to update things.

</comment></comment></comment><comment><USER25> We use Jenkins at work and have found a pretty damned sweet spot.

Let me start by saying that we used to use GitLab, a lot of it was because of the CI but I didn't have a great experience trying to manage it on top of Kubernetes, they've since introduced a Kube native package and I've been told it's much easier, but with the deployed omnibus we ran into a lot of issues with runners randomly disconnecting, it became frustrating to the point where I had developers not wanting to use GitLab and finding interesting ways to work around it.

So I set up Jenkins on a dedicated EC2 instance with a large EBS volume for workplace storage and installed the Jenkins plugin then I wrote a Jenkins library package that exposes a function to read a gitlab yaml file and generates the appropriate stages with parallel steps that execute as pods in khbernetes - took about a week to get the things we actually used from Gitlab CI and their YAML DSL working correctly.

Now we very happily use Jenkins, mostly through YAML but in the occasions where things are much easier going directly to Groovy to use interface with plugins, developers can.

<comment><USER25> I want to specify this is my own experience and I think a lot of our own issues may have been from mismanaging our self-deployed setup. I've had a lot more experience managing Jenkins.

GitLab and their approach to CI (easy to use yaml) really facilitated developers writing CI, which increased our software quality overall.

<comment><USER26> Have you used the Jenkins Job DSL plugin?

I'm just getting started using it, but it seems like the solution to scaling up to a lot of Jenkins jobs. There's a good talk about it, and since you're one of only two people in the thread who used the word DSL and you are having a good experience with Jenkins, I thought I'd ask.

My config is similar except my single EC2 node is actually running Kubernetes via kubeadm, it's a single node Kubernetes cluster and has enough room for about 3 of my worker pods to execute concurrently before the rest have to wait.

(But that's just my setup and has nothing to do with Job DSL.)

For me, managing Jenkins via the helm chart has been the best part of the deal, but I'm a pretty big fan of Helm already...

<comment><USER25> I haven't used it, we use the GitHub Org Folder scan to automatically add a job for every Repo that has a Jenkinsfile- so I don't have to do anything, it's pretty great.

<comment><USER26> Awesome, that also sounds good

The big advantage of the job DSL plugin is that if you have many similar repos, you don't just treat them all the same with multiple copies of the same Jenkinsfile, you actually can build the jobs from the same source, inheritance-style.

There could be some reasons not to do this, but if you have 100 jobs and 20 or more of them are mostly identical in form except for some parameter, it's a better strategy than multiple Jenkinsfile.

But then again if your Jenkinsfile isn't changing, it might not matter.

</comment></comment></comment></comment></comment><comment><USER27> I'm not at all ashamed, nor a single bit remorseful to comment on the fact that it took a catastrophic data loss for a team I once worked with to finally sit down and look at our CI/CD pipeline before deciding "Maybe jenkins is overkill for what we need".

Which was something I had been kvetching about for _months_ and expressly warned, multiple times to our release manager as a point of concern given how quickly plugin vulnerabilities are reported (as someone comment on elsewhere in the thread).

The day finally came when someone from one of our other offices went to update some infrastructure as code repos, poof. Jenkins server gone. They didn't have a roll back plan, and to complete the trifecta, they somehow also killed all of the instance volume backups. An entire sprint was summarily dedicated to creating a new build pipeline, I had resumes out the door the next day.

This article hits so many of our pain points I joked to a current coworker who followed me out of that place that I wanted to print it and mail it to our former RM.

<comment> Iwan-Zotow WHat do they/you use now?

<comment><USER27> No clue. Didn't stick around long enough to find out, that moment was officially the last straw for me. There was a lot of hemming and hawing, I left and product, project and dev still hadn't made their minds up. Ops (where I lived) would give the team ideas and proposals, dev gatekept _everything_ from tickets to the VCS.

This was one of those shops that hired people and gave them devops job titles, but demanded they maintain very monolithic status quos with everything from tickets and stand-ups to release management.

</comment></comment></comment><comment><USER28> Jenkins is terrible but I've not found something better yet.

For me, the great thing is that it is an automation platform where we can integrate builds, scheduled tasks (cron), and very basic end-user UIs with Plugins that let me customize the whole experience.

Azure Pipelines comes close in capabilities. If they made it easier to manually start a job and for them to accept their parameters from the user, then it'd probably have everything i need.

</comment><comment><USER29> There are lots of alternatives for Jenkins as a CI builder but I haven't found many for Jenkins as a web-based cron with a nice UI that keeps track of history, has tons of options for notifications on failures, and quickly allows re-building (even with user-supplied parameters!)

We use Jenkins for running backup jobs, periodically updating data, and building quick little jobs for support staff to run--infrequently enough that they don't warrant adding to our admin app but frequently enough that bothering at developer adds up.

<comment><USER30> For periodic admin tasks, you might consider giving Ansible AWX a try ([https://github.com/ansible/awx](https://github.com/ansible/awx)). It is a web interface for managing Ansible playbooks, and lets you configure jobs that another user can then execute and provide parameters. It keeps track of all run results, has a fair number of supported integrations for notifications, and can schedule runs.

I work for Red Hat, and AWX is the upstream community project for Ansible Tower, which we provide support for. AWX is one of our newer open source projects (we open sourced it after acquiring Ansible), so you'll sometimes have better luck searching for information on "Ansible Tower".

</comment></comment><comment><USER31> I spent quite a while grappling with Jenkins pipelines, and while it definitely has its warts, once you get over the hump and get one pipeline the way you want it, it's quite easy to use it as a starter for other projects.

Personally I go to great lengths to minimise the plugin dependencies and push the logic into bash scripts or Maven/Gradle. The Jenkinsfile just calls the bash scripts, so I can run it before I change it. Where I use plugins in the pipeline I document what they are and what they do in case we ever need to go from scratch.

It is hard to test but the linter picks up quite a lot of errors before push, and if you're doing a multi-branch pipeline you can just test your changes by pushing to your own branch. It's a long cycle, but the variation should go down the more you have them.

It has also been indispensable as an ad-hoc task scheduler. For example, I have a job that cleans up old branch artifacts every Saturday night. It was easy to alert on failures and see the results and history of past runs. I don't know anything else that would have fit the bill.

When you make software over 10 years old in such a fast-changing environment, the legacy of the software can weigh you down. I've worked on similar products that were tied to their legacy counterparts (sharing a database), and every change meant carrying the baggage of backwards compatibility with the legacy platform. If I could do it all again, I'd have done it the Basecamp way: Get a clean break from the legacy version, and have a migration path to the new system.

I would like to see CloudBees take all they've learned over the years, take the great work they've done on Blue Ocean and their core job scheduling system and put it all together without carrying the history with it.

</comment><comment><USER32> Another container based option is Screwdriver, which started as a Yahoo in- house solution built on top of Jenkins. But it is now fully open source and no longer makes use of Jenkins in any way:

[http://screwdriver.cd/](http://screwdriver.cd/)

<comment><USER33> This looks really good! For a long time Jenkins has seemed like the only option for flexible jobs. Tools like Drone and Travis and the many other that use a "config file" approach just aren't able to scale against your needs. They're fine for a run-of-the-mill build but that generally just isn't enough.

<comment><USER34> Drone is working on support for Starlark, a python-inspired configuration language used by the Bazel Build system [1]. For complex pipelines, this should be more analogous to Jenkins scripting and may improve scaling Drone for larger projects (time will tell).

[1] [https://github.com/bazelbuild/starlark](https://github.com/bazelbuild/starlark)

</comment></comment></comment><comment><USER35> I work at a big enterprise that uses Jenkins. We support a large group of legacy applications and the waterfall type development processes that lately have been papered over with agile-lite. I work about 80% of the time administering (Cloudbees) Jenkins. I have lived through Freestyle projects, migrating to Job DSL, and now we are moving to declarative pipelines in shared libraries. I agree with 95% of the article's pain points. Still, I'm a fan, perhaps because each iteration is significantly less painful from the previous one.

Still Jenkins works well for us in our environment. Our waterfall style development processes make for ops-friendly, cookie-cutter type builds: one deployable asset per repository, one deploy target, all projects laid out the same way... Jenkins shared libraries provide powerful ways to capture the commonalities among the projects and provide a way for us to manage them centrally, while providing some (small) flexibility for customizations by the developement teams. Shared libraries also allow the ability to test changes using selected versions (branches) of the pipeline for specific builds. Looking forward to what's next (I still have a few mortgage payments to make...).

</comment><comment><USER36> Jenkins also routinely has massive security holes.

Last major exploit I heard about - the matrix.org exploit, was from privilege escalation through a Jenkins vulnerability [1]

[1] [https://www.zdnet.com/article/matrix-hack-forces-servers- off...](https://www.zdnet.com/article/matrix-hack-forces-servers-offline-user- credentials-leaked/)

\--

This isn't nec a knock against Jenkins itself, but certainly a knock against thousands of orgs running their own unpatched Jenkins servers, often on the same machine as their other apps

</comment><comment><USER37> It's true that Jenkins is old, and has systemic architecture problems which mean that I'm sure we haven't seen the last RCE for it. RCEs in build systems are a nightmare because build systems necessarily end up knowing secret tokens.

But drone.io is no kind of replacement. It only works with Docker containers! Even small Jenkins installs can quickly end up with Windows targets, mobile builds, etc.

<comment><USER34> > It only works with Docker containers

This will only be true for another week or two. We have devised a framework for alternate runtimes, including running pipelines directly on the host machine or in traditional virtual machines, that will be released end of week. Reference issue [https://github.com/drone/drone/issues/2680](https://github.com/drone/drone/issues/2680)

<comment><USER38> Does it support running on macOS and Windows, too? CI for native apps is IMHO the most common lock-in for Jenkins at the moment. Anything web or container related can almost always be implemented using pretty much any other CI/CD solution out there.

<comment><USER34> Yes, it supports windows, macos, and linux. I expect it will also support bsd although I have not tested this yet. This should land in master some time this week.

</comment></comment></comment></comment><comment><USER39> Unrelated to the content, but the stock photos throughout this article felt completely needless to me. We're not 5 year olds, we don't need pictures unless they add to the content.

</comment><comment><USER40> Jenkins-X is really cool. I love how you can group services together into an environment and deploy that to a single Kubernetes namespace, and how well it uses github releases.

I really hope Gitlab's Auto-Devops team is looking at it closely and stealing all the great ideas it has because that's what our team is using.

<comment><USER41> Jenkins X requires Kubernetes and Helm. Not all companies are deploying like this.

(I work for Codefresh, a competitor of Jenkins X)

</comment></comment><comment><USER42> The most interesting part of this is the very last paragraph.

There really _isn 't_ a good self-hosted solution for build metadata/metrics, unless you write something yourself. It would be interesting to see a GUI dashboard for Drone/whatever builds and pipelines.

<comment><USER43> I never understood the purpose of dashboards. In the context of CI, I see three use cases and there is always a better solution than a dashboard.

1\. Detect broken builds. It is better to actively notify somebody by mail, chat, whatever.

2\. Prevent broken master. It is better to reject the pull request/patch before it becomes a problem.

3\. Analyse the system. It is better to download the data and enable the use of whatever analysis tools are suitable.

<comment><USER44> Direct emails can't replace the collaborative "hey why are these still red" moments without spamming the whole team.

They can show non-event information such as build time trends at a glance.

</comment></comment></comment><comment><USER45> Anyone who agrees with the OP should give Zuul-ci.org a look.

I run Zuul in production with GitHub and AWS, and it's been really useful for us. It scales to the moon, parallelizes everything, and having _all_ job/pipeline configuration in git means you can test things like CI job changes _before_ they're committed.

Cross-repo pre-merge dependency management means I can build whole speculative futures in PR's _before_ it lands and breaks the build.

Funny story, the Zuul build status badge is just a static image, because Zuul does not allow code to merge if it fails tests: [https://zuul- ci.org/docs/zuul/user/badges.html](https://zuul- ci.org/docs/zuul/user/badges.html)

Full disclosure: I worked on a defunct zuul based service for a little while, and still am a core developer with lots of code in Zuul. A few presentations I've given are here:

[http://fewbar.com/the-build-is-never-broken/](http://fewbar.com/the-build-is- never-broken/) [http://fewbar.com/zuul-ci-crossing- streams/](http://fewbar.com/zuul-ci-crossing-streams/)

</comment><comment><USER46> AWS CodePipeline with CodeBuild supports GitHub webhooks, export artifacts between stages, many types of caching (S3 artifact caching, Docker caching, file/volume caching), Secrets Manager environment variable integration, queueing between stages, pipeline retries, stage retries, manual approval steps, integration with CloudWatch and many of services like Lambda, and provisioning all of the above via API, CLI, SDK, or CloudFormation.

<comment><USER47> > queueing between stages

Doesn't it only support a queue depth of 1 at each stage? Last I checked, if you have a change that is queued up waiting to get into a stage, and a newer change comes along, that newer change will supersede the old one. That makes CodePipeline only good for workflows where you only care about deploying the 'newest' of something.

I think the worst thing about CodePipeline is how hard they make it to run custom code from your pipeline. Your options are Lambda (limited to 15 minutes per run, or you have to rewrite your Lambda function to be called once every 30 seconds, essentially using the first run to start your deployment and every subsequent run to check to see if it's done yet or not) or CodePipeline Custom Actions (where you have to write the AWS SDK code for interacting with CodePipeline).

The AWS Developer Tools team could learn a thing or two from Azure Pipelines. They did it "right", IMO: you can create a 'bash' stage in your pipeline which runs whatever script you want from the build agent (which can either be hosted my Microsoft or hosted yourself). That's all I really want. CodePipeline could support that with a custom action but it's more stuff that I would have to set up.

</comment></comment><comment><USER48> I'd never used Jenkins before, a few weeks ago I needed to do some CI that was more complicated than just pytest or whatever. Installed jenkins, put together a dockerfile, connected the github hooks and It Just Works. I'm a happy customer

<comment><USER49> The problem is people want to do really complex things with Jenkins. A few hundred repos, each with dozens of builds; Multiple operating systems... There is a reason a team of 10 works full time to administer our CI system: it is complex (the CI system is more than Jenkins)

</comment></comment><comment><USER50> It is good to support open source software vs closed source. Gitlab is a modern good open source alternative to Jenkins which is also open source. Gitlab has its configuration files default in yaml files which is easy to read for humans. Jenkins uses XML which is hard for humans to read.

One should not use close source solutions just because it is easy to use. Just because it is easy does not mean it is good for you in the long run.

</comment><comment><USER51> "cripple-wear" made me cringe. Shouldn't it be "-ware", anyways?

</comment><comment><USER52> I recommend [https://concourse-ci.org/](https://concourse-ci.org/)

<comment><USER53> I really love the ideas/architecture behind Concourse, but there's a few things that disqualified it in favour of Jenkins for a new CI pipeline during prototyping:

\- no hooks to manage worker scaling ([https://github.com/concourse/concourse/issues/993](https://github.com/concourse/concourse/issues/993)). Our builds are _heavy_ and we'd run up an enormous AWS bill without something like ([https://wiki.jenkins.io/display/JENKINS/Amazon+EC2+Plugin](https://wiki.jenkins.io/display/JENKINS/Amazon+EC2+Plugin))

\- no way to restart a build, resorting to 'empty commits', which is a huge red flag for useability ([https://github.com/concourse/concourse/issues/413](https://github.com/concourse/concourse/issues/413))

\- limited documentation/examples (network effect)

<comment> tedk-42 Wow I never knew Jenkins could scale its workers like that. We definitely have performance issues with our Concourse cluster running in K8s. It's got it's own set of dedicated nodes but we need to scale the workers better as they're ofter under heavy load throughout the day when the devs are pushing code and a bunch of tests are running in Concourse (and PR checks as well)!

Concourse definitely feels more refined than Jenkins. Like other commenters have said, it's a steep learning curve to grasp how things move between Tasks/Jobs.

</comment></comment></comment><comment><USER54> Co-founder of cloudbees here, just some corrections:

1) there are (since 1 year) restartable builds/stages in open source (a bit over a year IIRC) - when the article mentions that they are not in open source.

2) Jenkins X is something VERY DIFFERENT from Jenkins (despite the name), it is "master less", yes requires a kube cluster to power it, but it has no Jenkins instances as you know it, in a lot of ways quite a bit different as it uses Tekton as the engine for running pipelines (which has a bunch of advantages). So I wouldn't group it in with the same challenges and constraints. It is something new that shares a name and some of the ideas.

Jenkins, along with Spinnaker and Tekton and Jenkins X are now part of [https://cd.foundation](https://cd.foundation) \- worth a look to see how they are evolving (expect some changes).

<comment><USER55> Unfortunately people are looking for simplicity and this is going in the opposite direction. A kubernetes cluster is just offloading a large part to yet another component you have to run. Spinnaker is another completely separate system.

This is the opposite of what teams want. Drone, Gitlab, Teamcity as mentioned in these comments is a far better approach for 99% of companies who want a solid working solution.

<comment><USER56> Soon, "running on Kubernetes" will be like "running on Linux", i.e., it won't add any operational complexity because you anyway have a Kubernetes cluster running.

So maybe you are not there yet, but for a future-oriented CI/CD platform with self-hosting option, using Kubernetes as basis is a good approach.

</comment></comment></comment><comment><USER57> Tangentally I've had great success with Drone. Holding similar frustrations that other commenters are expressing here it was a breath of fresh air to see how dead simple making custom steps in Drone was compared to actually needing to write code for Jenkins.

Every step in Drone is just a container, so if I want a Golang build container I can just set the step image to Golang:latest and start running build commands. And if you're wanting to encapsulate some logic into a "plugin" you can just pre-make your own docker image and from there any settings you pass in with the steps are converted to environment variables that can be accessed inside the container. Many of the default drone plugins like docker registry build, etc. Are just a couple line bash scripts on GitHub.

</comment><comment><USER58> Is buildbot old too? I sure enjoyed the flexibility. [https://buildbot.net/](https://buildbot.net/)

<comment><USER59> One complaint about buildbot is that if you get too creative, your buildmaster.cfg gets very hard to maintain, but if you stay very diligent, just having python (and being able to print or log whatevery is happening) makes debugging and having complex setups be very easy.

Also, if you ever need to schedule jobs/tasks (not just ci builds) across multiple machines, buildbot is great because all you need is a master, and slave python processes which just need a network connection to the master.

</comment></comment><comment><USER60> I think a better headline would have been something more like "Jenkins doesn't have the features I require". There are plenty of examples of older pieces of software or older languages that still work perfectly fine today.

<comment><USER61> You should really read the article before you comment on it.

</comment></comment><comment><USER62> What I'd like to see is a CI/CD engine backend. Most CI/CD solutions have relatively the same feature set:

\- Code builds

\- Code deploys

\- Git Hooks

\- Parallel builds

\- Matrix builds

\- Build as configuration

\- Etc

If there were an engine that supported these features, the community could create a number of frontends that target their specific needs.

Jenkins was held back, in my opinion, for a number of years by its antiquated UI. And while Jenkins does have an API of sorts, writing a new UI for it is not a simple task.

I've used Drone for a while, and it gets a lot of these things right, but at the end of the day it is a holistic, integrated solution.

I'd like to see a composable, modular CI/CD framework.

<comment><USER63> What is a "matrix build" in the context of CI/CD?

<comment><USER62> When your software needs to build on, or run on multiple targets, it's useful to define it in a matrix. Here's an example: [https://0-8-0.docs.drone.io/matrix- builds/](https://0-8-0.docs.drone.io/matrix-builds/)

</comment></comment></comment><comment><USER64> [https://outline.com/VvVhg6](https://outline.com/VvVhg6)

<comment><USER65> Yours is not the first post that contains nothing more than an outline version of the link. Why is this becoming a thing? The page seems perfectly readable as it is...

<comment><USER66> The page is actually a medium post which immediately pops up an annoying modal trying to get you to give them your personal information and phones home who knows what kind of analytics.

I'm not sure about the ethics of outline.com, but I can totally see why people are using it.

<comment><USER67> I'm not sure about the ethics of taking other people's content and putting it behind a paywall.

<comment><USER68> But it's Medium's content. You gave it away when you posted it here. In exchange, you got exposure[1]

[1]: [https://theoatmeal.com/comics/exposure](https://theoatmeal.com/comics/exposure)

<comment><USER69> As far as I know, Medium writers opt-in to this, in exchange for money if a paying user reads the article.

</comment></comment></comment></comment></comment></comment><comment><USER70> They should definitely dig more into Jenkins X: it evolves a lot and it's pretty new but they're gonna have a stable version soon!

</comment><comment><USER71> I'm still pretty new to devops and I personally love Jenkins but man is it a PITA at times. My biggest gripe is that every plug-in seems to be no longer actively maintained. We really on a huge list of plug-ins and in some cases upgrading Jenkins will break plug-ins. It's gotten to the point where I'm afraid to update to even minor or patch versions.

</comment><comment><USER72> I'm a very strong Concourse proponent. We've been using it exclusively for three years, and it's better than anything we've experienced that our customers use (I run a consultancy).

We run Concourse London User Group, and the first few instances were like some kind of Jenkins survivor's support group.

</comment><comment><USER73> Trying to integrate Jenkins w/ Github Enterprise proved to be a real pain for me. It wasn't clear what git* plugin is the best to use for how I wanted to setup builds. It's literally impossible to deploy Jenkins in a repeatable manner with just configuration files, you must get it online and use the api to configure such simple things as API keys (eg, specify the ID of the key so your jobs actually reference the key and don't break).

Another big pain point, for me was, no obvious openstack integration. I wanted my jobs to run on ephemeral instances so all my dependencies could be defined in the job themselves, not having to rely that the Jenkins worker is setup _just right_.

I think Gitlab runners get this right, but I have not investigated too far.

<comment><USER74> You definitely can configure Jenkins in a repeatable manor. See [https://wiki.jenkins.io/display/JENKINS/Groovy+Hook+Script](https://wiki.jenkins.io/display/JENKINS/Groovy+Hook+Script)

You can write all the configuration in groovy and it will execute when the Jenkins process starts up.

There is also "Jenkins Configuration as Code" plugin which allows you to have a single yaml file to configure most of the Jenkins system.

[https://github.com/jenkinsci/configuration-as-code- plugin/bl...](https://github.com/jenkinsci/configuration-as-code- plugin/blob/master/README.md)

<comment><USER73> I managed to accomplish the task at hand ~2.5 years ago, so yes, it's possible to deploy jenkins in a repeatable manner.

I guess what I'm driving at is that it's not very declarative and it's cumbersome.

</comment></comment></comment><comment><USER75> Jenkins is to CI,

as Nagios is to monitoring.

<comment><USER76> Both work ?

<comment><USER77> No, the key difference is that Nagios has perfectly viable alternatives (Zabbix admin here and it's not the only one). Jenkins however has no alternatives.

Bamboo or whatever it is called now is typical Atlassian crapware. Expensive as fuck, eats more resources than the stuff it builds, and did I mention it is yet another half assed product that got shoddily integrated into the usual Atlassian lineup?

Gitlab CI is great for anything that is _code_ (think build, test, deploy), but it is not suited for abstracting "non-development" jobs which can perfectly be automated in Jenkins (e.g. creation of a dev environment with fresh data from production). Plus it is Docker and the runners are polling - which means at minimum 10s _startup time_ compared to milliseconds for a Jenkins shellscript job running on a ssh connected slave!

Github and friends are cloud which is a big no-no. We're placing too much power in the hands of AWS, GCE and Azure already, no way in hell it is a good idea to put private source code to a cloud provider.

<comment><USER78> GoCD?

</comment></comment></comment></comment><comment><USER79> Our org is happy with buildkite. We moved to that after fighting Jenkins for a number of years.

</comment><comment><USER80> See [https://build.gocd.org/](https://build.gocd.org/) for the live instance of GoCD used to build Gocd (apache licensed CD orchestration tool backed by Thoughtworks).

</comment><comment><USER81> I agree with this article on all points. Currently we have a massive jenkins pipeline sprawl that's difficult to maintain. It is also difficult to create new jobs in Jenkins itself, specially if you are using pipelines. My average is around 100 test builds before I can get a full pipeline success for anything of modest complexity.

If all you are doing is using jenkins to run simple bash scripts, you may be able to get away with it. The problems start when you want to add some logic to the pipeline – which you are doing, otherwise why bother with a pipeline?

First things first: are you going to use the scripting pipeline, or the declarative pipeline? The declarative pipeline is a bit better, but it lacks examples, has lots of bugs (I've littered my code with references to JENKINS- XXXXX) and is very restrictive (arguably by design). Of course, you can have 'script' blocks inside your pipeline->stages->stage->steps blocks.

Then you want to take advantage of parallelization or conditional steps, and to visualize that you want Blue Ocean. Problem is, not all plugins are compatible with Blue Ocean, it also doesn't have all features, so you drop down to 'old' jenkins often.

People will want to have a whole bunch of tools with incompatible versions in their builders. Not all are supported natively, so you need to figure out your versioning.

Once you figure all that, congratulations. Next guy to automate something will either find a similar pipeline to copy from, or will endure all the pain again. At this point you may want to use Groovy.

Groovy was absolutely the wrong tool for the job. Yes, I get it that it works with Java, which Jenkins is based on. Still it is the wrong choice. You see, the kind of things you want to automate often involve passing commands around, be them bash, ansible, SQL statements, what have you. Groovy's string escaping rules will ensure your life will be pretty miserable ([https://gist.github.com/Faheetah/e11bd0315c34ed32e681616e412...](https://gist.github.com/Faheetah/e11bd0315c34ed32e681616e41279ef4))

You could get around most of these by perhaps moving most of the logic to containers and then running those. There again you'll run into problems with declarative pipelines, random things won't work and you'll be scratching your head.

However, if you are going to do that anyway, you're better off using a more modern system for CI, any system. Drone was already mentioned, there's also Concourse and a bunch of others. For CD, you can use Spinnaker as well.

Or maybe keep jenkins around but forget all the fancy stuff. Delegate all the 'thinking' to scripts and pretend the more recent development has never happened. You'll be saner that way.

<comment><USER82> If it takes you 100 tries for a moderately complex jenkins pipeline, you have other problems that are not jenkin's fault.

<comment><USER81> I wish I could bring you here to see you do better.

Or do you mean systemic corporate problems? In that case, I agree.

It still doesn't change the fact that Jenkins does not make my job any easier. I'll spend a day worrying about Jenkins idiosyncrasies ("why can't I use a pipe in sh", "why did my bash escaping disappear completely", "why 'dir' doesn't work with a container build agent?! (JENKINS-33510)", "why this input plugin won't work with blue ocean", "why can't I use a for loop in this piece of code in particular but it works elsewhere" (JENKINS-27421)).

Whereas with concourse or other newer build systems I can write a simple YAML description, which is modular and uses an existing standard, and test that in isolation. And then provide it as a building block for other tasks.

<comment><USER82> I feel you, but why are your jenkins pipelines so complicated? I feel like your workplace's deployable artifacts should follow a familiar pattern and there should not be much guessing/re-inventing the wheel with jenkins scripts. I feel like complicated builds are usually the result of an application that is not very well thought out in the first place.

<comment><USER77> > I feel like complicated builds are usually the result of an application that > is not very well thought out in the first place.

Welcome to the world of enterprise Java or .net programming. Loads upon loads of crap. Best served with multiple frontends (e.g. web + mobile) which need different npm versions to compile and all of it out of a single fucking pom.xml which is a nightmare in itself!

<comment><USER82> If you're mixing npm with pom files you are asking for trouble. Jenkin's shortcomings have nothing to do with npm's crappy package management. (not saying you said that, just pointing it out)

</comment></comment></comment></comment></comment></comment><comment><USER83> </read-article>

Um, so doesn't his workflow buy an essentially infinite maintenance problem with all his containers? This sounds like tomorrow's Node left-pad debacle in a different form.

</comment><comment><USER84> Of all the undifferentiated heavy lifting you can offload to your cloud provider, CI makes a lot of sense:

\- You probably want to store artifacts in its object store anyway.

\- Rent exactly the right amount of compute for the concurrent builds running in any given minute.

\- If your projects have sufficiently simple build-and-test entrypoints i.e. "make" then you're highly portable and not too worried about lock-in. You don't even need to be deploying to the cloud provider you're building on.

</comment><comment><USER85> We ran Jenkins in-house and it was a miserable experience. Always breaking. Any time we needed to update it we knew it was going to break, and it did. Debugging startup failures was always a pain if you had a moderately complex setup. There were times where it was quicker to wipe the entire configuration and start over than trying to debug it.

We switched to CircleCI and it did everything we needed and we never had to lose half a day to CI problems again.

</comment><comment><USER86> Interesting, not a single mention of Vespene, or more relevantly: [https://medium.com/@michaeldehaan/discontinuing- vespene-17fd...](https://medium.com/@michaeldehaan/discontinuing- vespene-17fd9c0f3079)

Michael DeHaan worked on it after Ansible, but he got zero traction, and so shuttered the project. I think he should have stuck it out longer, because people hate Jenkins.

</comment><comment><USER87> No mention of Jenkins Shared Libraries? When Jenkinsfile pipelines start getting complex, or need to make them more generic, you can create groovy lib-s in a separate repository. Single line to load the lib dynamically at start of Jenkinsfile. Then you can call library functions/tasks passing appropriate params within your pipeline. But as anything Jenkins, documentation lacking or out of date.

</comment><comment><USER88> Normally I have no issues getting things to work but I bounced off trying to get modern Jenkins to build a github gradle project three times now. I even used to have one, way back, that built maven projects from SVN and deployed into artifactory but I can't seem to manage the same now. I thought it was me but maybe jenkins just became really awful in the meantime.

</comment><comment><USER89> This completely fails to mention the security issues with Jenkins, as well. [https://www.cyberark.com/threat-research-blog/tripping- the-j...](https://www.cyberark.com/threat-research-blog/tripping-the-jenkins- main-security-circuit-breaker-an-inside-look-at-two-jenkins-security- vulnerabilities/)

</comment><comment><USER90> Not at all.

In what concerns Java and .NET projects, Jenkins is just a mature tool that gets the job done.

We have no reason to consider anything else.

</comment><comment><USER91> Anyone looking for a more back-to-basics self-hosted CI might be interested in Laminar[1] - this article reminds me why I was inspired to write it

[1] [https://laminar.ohwg.net/](https://laminar.ohwg.net/)

<comment><USER92> It's not distributed, from what I see. I don't see stuff about agents.

Oh, it does, via DYI SSH, sort of. Could work, I guess.

<comment><USER91> Right, the philosophy is more about making it easy to DIY stuff and getting out of your way, rather than endlessly tacking on features and integrations.

</comment></comment></comment><comment><USER93> We currently use Appveyor (good) and TeamCity (okish) and feed into Octopus Deploy. We have ~50ish projects (web, backend systems, embedded firmware) that all go through these systems. Works really well!

</comment><comment><USER94> _Getting_ old? It's ancient. Nobody should use Jenkins for new projects. I even started writing my own piece about it on a plane back from a recent vacation.

</comment><comment><USER95> I want pause on breakpoint <x> in cloud worker <y> from my IDE — tooling which does not assist me to bring that about is working against my needs.

</comment><comment><USER96> I have been with jenkins since about 2010.

At its peak I had to shepherd ~50 masters of varying ages, plugins and levels of customisations.

Yes, it allowed repeatable builds, and it can be, in certain circumstances a very good replacement for cron jobs where you need to administer >5000 machines.

However, it has an offensive UI, the plugin system appears to be designed to deliberately cause things to break, versioning the flows is difficult, unless you rely on plugins (which break) or want to learn groovy.

Automating and making jenkins it's self repeatable is a soul sapping task.

In short, jenkins needs to disappear.

Yes, cirle is crap, bamboo suck massive genitals.

However I have had great success with gitlab runners. Now that gitlab is actually reliable, they are a brilliant first class integration into a git workflow.

<comment><USER97> GitLab CI is pretty good.

I hate, on a fundamental level this whole "bash in YAML" trend. Let's take a powerful, turing complete language and put it in a _config file_.

What? I don't even...

And the fact that you only get _one_ .gitlab-ci.yml file. That is legitimate madness. Want modularized CI/CD configuration? Not in these parts, buster. Have submodule build dependencies? We laugh at your use case.

Sure as shit better than Jenkins though.

<comment> core-questions > I hate, on a fundamental level this whole "bash in YAML" trend.

This is why I hate Ansible. Looks simple until you need to do something complex, and then it becomes horrible. Chef's use of an actual language is far better, though on the flip side it's much harder to provide any kind of interface to configure it outside of a text editor.

If only there were some language where config data and code were the same data structure. (hmmmmmm)

<comment><USER98> > If only there were some language where config data and code were the same > data structure. (hmmmmmm)

Do you perhaps mean syntax rather than structure? (Or a subset of the syntax.)

If I've guessed correctly, that language has already been wedged into lots of places that it's a poor fit, so I guess there's precedent there.

<comment><USER99> I am pretty sure Lisp was wedged into some places where it is a poor fit, but "lots"? Did you guess Javascript by any chance?

<comment><USER98> I did guess Javascript, as it happens.

Was I wrong? If it was meant to be a reference to Lisp, I take much of it back.

<comment> ISO-morphism The parent was most likely referring to lisp, but note that there are other homoiconic [1] languages.

[1] [https://en.m.wikipedia.org/wiki/Homoiconicity](https://en.m.wikipedia.org/wiki/Homoiconicity)

</comment></comment></comment></comment></comment></comment></comment><comment><USER100> It seems like KNative wants to move into that space.

Regular turnover in the Programming world. You have to re-learn everything once or twice a decade.

</comment><comment><USER101> groovy pipelines solve a lot but cost a lot. Finding where to draw the line is really really hard. Anyway jenkins works, is free, is easy to setup, and easy to use for everyone except the one writing the groovy :)

groovy sucks, jenkins works (high praise, really), but you get what you get

</comment><comment><USER102> How does sourcehut (sr.ht)'s build service compare to the alternatives mentioned here?

</comment><comment><USER103> drone is like google cloud build and probably and whatever amazon has

</comment><comment><USER104> Classic product promo by poo pooing on someone else.

</comment><comment><USER105> gitlab CI is awesome. too bad it is not a standalone thing.

<comment> jl-gitlab You can actually use it standalone, for example with GitHub repos: [https://docs.gitlab.com/ee/ci/ci_cd_for_external_repos/githu...](https://docs.gitlab.com/ee/ci/ci_cd_for_external_repos/github_integration.html)

(Disclaimer: GitLab employee here)</comment></comment>


Ask HN: Google or Facebook? - <AUTHOR> Have very similar offers from both companies to join as an engineering manager. Who will you choose to work at? considering (random order):<p>- impact<p>- work-life-balance<p>- growth potential<p>- culture & fun <comment><USER2> Facebook, in a few months the perks are going to be even higher if you are at the senior level position.

Facebook is launching dating service.

Their senior employees will most likely get a blessed account which is gonna match with more ladies.

This is something Google can't provide you. But yea, your cash compensation might go down a bit.

</comment><comment><USER3> No opinion, but you definitely need to read Exhibit B from the Damore lawsuit if you're considering Google.</comment>


Cheap and compact medical testing - <AUTHOR> http://news.harvard.edu/gazette/story/2014/08/cheap-and-compact-medical-testing/

<comment><USER2> Url changed from [http://www.gizmag.com/harvard-multifunctional- electrochemica...](http://www.gizmag.com/harvard-multifunctional- electrochemical-detector/33263/), which points to this.</comment>


Tom's Obvious, Minimal Language (like INI, only better) - <AUTHOR> https://github.com/mojombo/toml

<comment><USER2> Possible improvements occur to me:

1\. If indentation is significant and required, dotted notation should be optional:

        [foo]
            [bar]
                [car]

The parser should be able to understand that this results in a foo.bar.car structure without explicitly using dotted notation in the keys.

2\. Support include files. With #1 implemented, includes would be extremely powerful and flexible. Since indentation is required, no special syntax is needed to relatively position the included files. And without requiring dotted notation in keys, include files can be reused in a configuration (such as common settings in the [servers] section of the example).

</comment><comment><USER3> [https://github.com/mojombo/toml/commit/e54978f05197d16fde3ef...](https://github.com/mojombo/toml/commit/e54978f05197d16fde3ef4716fcd348d981b16fe)

<http://news.ycombinator.com/item?id=5272634>

<comment><AUTHOR> Whoops. I tend to rely on the site itself to block (recent) dupes. I guess the different title threw it, though I would have expected the identical URL to get caught.

</comment></comment><comment><USER4> I don't know if this is the answer, but I'm a fan of someone trying.

I personally never jumped on YAML or JSON. I liked that they weren't XML, but they each felt like XML++ too much for me. There was still too much, I don't know, weirdness/typing/oops! Don't get me wrong, I think they were better than XML, but only marginally. It would be nice to see more of a phase change type of approach. I'd LOVE to see something like this (or something else) that involves much less typing/special characters/etc than YAML and JSON.

Personal opinion.

<comment><USER5> How many less characters could there be while still maintaining meaning? Could you show an example of what you'd prefer?

</comment></comment><comment><USER6> The format could be very simple, but... allowing comments after values make it unnecessarily complex to parse. For example:

         value = "example # no that's not a comment" # but here's one

If inline comments where not allowed we could just check the first and last character for a quote and we know we have a string, same for arrays (check for [ and ]). But with inline comments, we need to parse the whole string.

As far as I can see both the C# and Python parsers don't handle this kind of comment correctly at the moment, and probably many implementations will have troubles with that. They should just drop this option in my opinion.

<comment><USER7> WAT?. Python can parse the above very nicely, thank you.

        >>> value = "example # no that's not a comment" # but here's one
        >>> value
        "example # no that's not a comment"

<comment><USER6> At the time I wrote that, the only Python parser was removing the comments by splitting the line on "#", which mean it wouldn't work with the example I mentioned.

It seems there are now several other Python parsers, so some of them are probably getting it right.

</comment></comment></comment><comment><USER8> So the objection to JSON comes down to a couple of things: no comments and fragility in face of a trailing comma. I wonder if the right thing though would be to fix JSON (maybe even call it something else) rather than throw the baby out with the bathwater.

Personally, of course, I think we should just use a slightly modified version of s-exprs which would be easier to parse.

(:keyword value)

where value can be an atom, list of atoms or a list of keyword value pairs. Comments could follow the Lisp convention and start with semi-colon and end with newline. Only 5 characters to escape, namely, '(', ')', ':', ' ' and ';'.

</comment><comment><USER9> INI, JSON and YAML all have problems. None of them are serious enough for me to consider this for a small project with one or two config files required. Good to be able to point to a config file syntax done right though.

<comment><USER10> Do tell about the problems with JSON for use as a configuration object? Almost every language has a library that can parse JSON.

<comment><USER9> It's hard to author by hand. Ever added a comma after the last element in a list? It's not very forgiving.

<comment><USER11> This is actually one of the worst things about it to me, other than the lack of comments. A language that has comma-separated lists and doesn't allow trailing commas on the list as a whole is frustrating to use in a version controlled context because adding an item requires modifying the item before, which leads to more (and less obviously solvable) merge conflicts.

<comment><USER12> I've seen a style of list initialization that sort of addresses this issue. I like it, but I don't see it used very often. Darn you standards!

        kittyBreeds = ["Himalayan"
                      ,"Bengal"
                      ,"Siamese"
                      ,"Burmese"
                      ,"Cornish Rex" //rawr
                      ,"LaPerm"
                      ,"Manx"

        //This is not a real cat! >:(
        //              ,"lolcat"
                      ,"Munchkin"
                      ,"Ocicat"
                      ];

In this way the issue with the commas is relegated to the first element, rather than the last. Since appending is more common, this seems a better style.

<comment><USER11> Interesting approach. This strikes me as kind of like the classic "put the constant on the left" advice in C/C++ that, no matter how many people agree it's a good idea, no one ever does because the cognitive effort of understanding it is high enough to diminish the correctness benefit.

In other words, you know you're just gonna end up with this in your code after a few months:

kittyBreeds = ["Himalayan", ,"Bengal", "Siamese", "Cornish Rex", ,"Manx"]

and not only is it now even worse for changing lines (correcting it and adding another will be -1+3 instead of just -1+2, the optimal +1 a distant memory), but it's just plain a mess and probably people don't know why they need to do it.

</comment></comment></comment></comment></comment></comment><comment><USER13> I see your mixed use of square brackets and raise you a syntax error.

<comment><USER14> Yeah, that's gonna be a tough one.

But I think the "square bracket list" things only ever occur in value position, and the headers only ever occur in root position, so it's not _that_ hard.</comment></comment>


A quick weekend project: Text any Twitter handle to 203-403-6263 to get a vcard. - <AUTHOR> http://rogerpincombe.com/social-contact-info-203-403-name

<comment><AUTHOR> I made it for a Twilio competition: [http://www.twilio.com/contests/2011/07/new-developer- contest...](http://www.twilio.com/contests/2011/07/new-developer-contest- twilio-infochimps.html).

I'm not trying to self-promote (as this isn't a commercial service or a company), I just want to see what everyone thinks.</comment>


Ostemper – Automatic Hardening and Tweaks for OS X - <AUTHOR> https://github.com/koconder/ostemper <comment><AUTHOR> I have been working on combining the works of a number of open-source scripts for improving security and setting up mac os x for developers in one single script for 95% of environments. Would love any feedback and/or feature suggestions!</comment>


Nyxt Browser 2.0.0 Pre-release - <AUTHOR> We are happy to announce a pre-release for Nyxt version 2! This is not a stable release!
Cool new features:
+ New "auto-mode", automatically enables and disables modes depending on the website. Example: automatically turn on your proxy whenever visiting travel websites.
+ New "data profiles" help you customize and separate your data. Example: a "work" instance of Nyxt may use different cookies but the same bookmarks as the "personal" instance.
+ Built-in Lisp REPL
CHECK out our article on our website (https://nyxt.atlas.engineer/article/release-2-pre-release-1.org) for more details.
DOWNLOAD: https://nyxt.atlas.engineer/download SOURCE: https://github.com/atlas-engineer/nyxt
Thanks for reading :-) <comment><USER2> I had problems getting this to work yesterday on macOS Catalina, but it installed and ran nicely on my Ubuntu Linux laptop. I say this just to encourage people to go the Linux route as it seems easier.

<comment><AUTHOR> Things are not all roses on the packaging side of macOS :-) We are working on it, and help is appreciated!

<comment><USER2> It costs $99/year to be an Apple developer, but then self signing executables with codesign is fairly easy. Getting apps into the Apple App Store is a pain, not recommending that.

Alternatively, I think more people use brew than macports, so that supporting a brew install would be good.

<comment><AUTHOR> The hard part is getting WebKitGTK+ to compile, the rest is easy.

<comment><USER2> I am downloading the Qt dependencies, and will try a build from source on macOS using: export NYXT_RENDERER=qt

</comment></comment></comment></comment></comment><comment><USER3> It's pretty exciting to see new web browsers where the whole source is customizable without having to do a complete recompilation. I was just evaluating switching from firefox to qutebrowser. I'll look into this one too.

I see it uses webkit2gtk. I wonder how it compares to qt5-webengine, in particular with respect to the devtools. Since qt5-webengine is based on chromium, it seems preferable. Have you guys considered supporting alternate backends?

EDIT: Oh! I see you guys already have a repo for a Common Lisp binding to QtWebEngine.

[https://github.com/atlas-engineer/cl-webengine](https://github.com/atlas- engineer/cl-webengine)

</comment><comment><USER4> I went to the Download page and hit the "Get Nyxt for GNU/Linux!" button, but it hit a 404 page.

Also, just a minor suggestion, but you should style the "Download for macOS", "Download for GNU/Linux (x86-64)", and "Download Source" headers as buttons or links. It looks like they are just text headers with nothing underneath them until you click on them. The cursor doesn't even change when hovering.

<comment><AUTHOR> You are absolutely right! We accidentally named it "next" instead of "Nyxt". Link fixed! Sorry about that. Please try again.

<comment><USER4> Awesome. I am downloading it now. Thank you for the quick fix!

</comment></comment></comment><comment><USER5> These new ideas are good. However, I would want to see a browser which is not based on WebKit and such things like that, instead being one specifically designed for expert users, with many differences than existing engines.

<comment><AUTHOR> Me too. Realistically this is far beyond our budget available :-\

I hope one day we can do this.</comment></comment>


Most entrepreneurs are wasting their lives - <AUTHOR> https://medium.com/@ArshadGC/earth-is-dying-and-youre-working-on-what-d768b6538b48 <comment><USER2> The original title is better imho.</comment>


Punks with Ph.D's - <AUTHOR> http://blog.earbits.com/online_radio/punks-with-ph-ds/

<comment><USER2> I have paid my dues in the punk community. I find the similar ethos between punk and hacker culture very interesting.</comment>


Ask HN: Why and what do you use PostgreSQL for? - <AUTHOR>

How do you use it in your web apps? Do you use it as a KV store? What PostgreSQL specific feature do you use? How have you tuned it for your specific case? How have you scaled it? I would love to hear how PostgreSQL's features are used in a variety of scenarios. <comment><USER2> I often select PostgreSQL for my clients, generally strictly adhering to the relational paradigms and letting the web caching layer keeping out the performance problems. PostgreSQL installations I saw are almost never tuned, except few paramers like effective_cache_size and the use of a separate raid array for the transaction log.

Scaling I saw is done vertically : faster hardware, more memory, SSD in raid 10. One of my clients tried once an horizontal scaling (with C-JDBC) but reverted back and bought a monster hardware.

</comment><comment><USER3> Back my Django site, because it's one of the databases on Django's list and I hear better things about PostgeSQL than MySQL. I have not taken any of the other actions on your list (yet).

My non-expert impression from reading around is that Postgres has a better future than MySQL, and is just a better solution. Oracle's clouding of MySQL's future didn't help it when I made my decision.

</comment><comment><USER4> I use it extensively for the PostGIS extensions. I do a lot of work with geospatial data, and extensions to commercial databases get expensive.</comment>


Microsoft's best hope after Ballmer? A break up - <AUTHOR> http://www.washingtonpost.com/blogs/innovations/wp/2013/08/26/microsofts-best-hope-after-ballmer-a-break-up/

<comment><USER2> _" To me, Microsoft seems like the former Soviet Union—Politburo, five-year plans, and all."_

Here is the essence of a horseshit analogy. Microsoft is compared to an oligarchy - the Politburo [1]. And then by magic transformed into the entire Soviet Union. This allows the advancement of an argument for breakup based on shortcomings in the delivery of consumer goods.

 _" That's why Steve Ballmer's replacement should not be one executive but should be a number of people who have experience in different domains and who can run independent operating companies."_

Mr. Wadhwa, how did that work for ATT? The article is utterly bereft of intellectual coherence. Microsoft is far more tightly focused than Samsung or Siemens and more profitable to boot.

[1]and what corporation couldn't be?

<comment><USER3> I agree with your essential claim that it is poor reasoning to support a pre- disposed view, but sometimes folks reason backwards to see if they can justify their opinion.

Microsoft is clearly a complex place, and huge in ways that are hard to comprehend. But it is also conflicted. The 'essence' of the breakup argument are that the company's business units need to be able to execute on their objectives independently of the other company business units. And that is true for companies of this size. When it doesn't work well, which we saw in AT&T and Sun Micro, it causes internal friction and damage, when it does as IBM and GE have shown it can, it really does allow for getting more stuff done over all.

There are lots of ways that Microsoft could go, personally I think they would do well to create three 'views', Microsoft Consumer which presents the 'Surface' operating system to end users, Microsoft Developer which presents a developer focussed system to end users, and Microsoft Enterprise which presents a managed infrastructure to Enterprise customers. Each view has goals, but they grow from the same roots.

<comment><USER2> I agree that we often reason backward from our conclusions - I'll even admit that sometimes an analogy is a useful way of explaining our reasoning. However, even were the analogy to the Soviet Union presented coherently, the interpretation of the benefits of its breakup require glossing over two decades of economic turmoil, repressive governments, loose nukes, the collusion of government officials in the pillaging of national assets and ghastly breakdowns in the rule of law in both Russia and the other former republics [and satellite states]. Lots of really bad shit happened after the wall came down [yes, and before it came down].

But, for the sake of avoiding speculations over counterfactuals and arguments over what counts as better, let's suppose that the analogy has some merit and that there is a case for breaking up Microsoft. Let me further suppose that that case is not based on traditional antipathies, and is instead founded upon a business case analysis.

Under these suppositions, why is there a case for breaking up a very profitable Microsoft, but not one for breaking up the far less profitable and vastly less focused Siemens? [software, light bulbs, trains, power plants, MRI's, financial solutions]

The other thing that needs to be provided are cases where a large profitable company with minimal physical assets was successfully broken up.

Absent a well reasoned internal business case for breaking Microsoft up, I am of the opinion that the business case is external: the possibility of breakup (or even the mere suggestion thereof) creates uncertainty -> creates volatility in the share price -> creates opportunities for arbitrage. Until there's a business case based on profits put forth, that's the only money trail I can follow.

</comment></comment></comment><comment><USER4> Don't be silly. We're having a world war in the IT scene at this moment among a number of monstrous companies: Microsoft, Google, Apple, Amazon and their allies. This proposal is like suggesting of partitioning the Soviet Union in 1942.

</comment><comment><USER5> It makes no fuckin sense to break Microsoft apart. Microsoft is actually highly focused. There's synergy between every division, whether on the product side (e.g. hotmail/outlook accounts used in Windows, Office, Xbox, Internet services) or the backend/platform side (e.g Windows kernel powering PCs, Xbox, Tablets, Phones. Or the Azure cloud powering Office 365, XboxOne Cloud, and miscellaneous Windows SAAS, in addition to providing web-services AND also pushing and marketing the Windows and Xbox development platforms).

What a terrible article.

<comment><USER6> > There's synergy between every division

I disagree with this claim, and offer as proof the lack of Office on any of MS's mobile devices. Compare with Apple, who had versions of all of their iWork apps available on the iPad when that device launched. Further, Office has its own UX norms which frequently contradict the Windows norms.

I do not claim the article was high-quality, but I to take issue with the idea that there is intradivision synergy at MS. The opposite seems to be true, at least for now.

<comment><USER7> > lack of Office on any of MS's mobile devices

WP7 had Office. WP8 has Office. Hell, Office for phones started as Pocket Office in 1996 for Windows CE 1.0, and has been updated all the way until Windows Mobile 6.5.

>who had versions of all of their iWork apps available on the iPad when that device launched

Furthermore, the iPhone existed for 3 years before iWork was released for it. Since we're talking about mobile devices, it's only fair to use the original iDevice for comparison, since Office for Mobile is available for phones. Office for RT was released, and _built-in_ to the OS, for free.

Office Web Apps for SkyDrive was released 3 years ago. iCloud's iWork implementation is still in Beta.

>Office has its own UX norms which frequently contradict the Windows norms.

It's generally followed Windows UI style. Windows 7 brought in Ribbon in some built-in apps, and Windows 8 extended that even further.

You can try as hard as you want to diss Office, but their product offerings have been top notch across devices.

</comment></comment></comment><comment><USER8> Wadhwa mostly fails to address that Microsoft (prior to a couple of weeks ago) was already "broken" up and organized by business groups (or customer segments) in much the same way he is proposing (Xbox = IEB, Online Services = Bing, Windows = Windows, Office = Office, etc).

I realize he is proposing an entire break-up, but I would have liked it if he had approached the problem providing the context of the difference between his proposed plan and the business unit separation. Afterall, Microsoft is famous for teams fighting each other for power and resources and meanwhile still requiring cooperation.

</comment><comment><USER9> This is like Deja Vu. They said the same thing about IBM last century (I was an employee at IBM yorktown for a couple years). IBM was dying, the PC (clones mostly) won the day and was in offices everywhere. The plan was to split IBM up into lots of little companies.. They brought in a CEO (Gerstner) who basically said, don't split it up, being big is a benefit and lets leverage that. It worked, and the company started doing better.

oblig wiki page: [http://en.wikipedia.org/wiki/Louis_V._Gerstner,_Jr](http://en.wikipedia.org/wiki/Louis_V._Gerstner,_Jr).

<comment><USER10> Your link doesn't work - HN doesn't make the last . part of it: [http://en.wikipedia.org/wiki/Louis_Gerstner](http://en.wikipedia.org/wiki/Louis_Gerstner)

<comment><USER9> Thanks!

</comment></comment></comment><comment><USER11> HA ha ha ha. No way. Don't discount M$. They have a very solid track record of coming late to the game, failing, flailing over and over... and once a market matures they move in hard and take it over.

<comment><USER12> They haven't been able to do that for a while, and their competition is now a lot smarter and much richer than even Microsoft.

Apple's iPhone business alone is larger and more profitable than all of Microsoft combined. Take out the iPhone and Apple is still bigger.

<comment><USER13> I guess if you consider 2010 "a while". That is when Microsoft released the Kinect.

<comment><USER14> And the Xbox itself before that.

<comment><USER12> Xbox is brought up a lot in these threads although it has been a total business failure. Even after losing billions over many quarters, making a quarterly profit only recently, it's still in a virtual 3-way tie for the tiny gaming console market.

<comment><USER15> And it took how long to get this far? Console gaming is a tough industry to be in.

</comment></comment></comment></comment></comment></comment><comment><USER16> Well, this is wishful thinking. With their recent major re-org, Microsoft went the complete opposite direction, and they are going to have tightly coupled, horizontal/functionally-aligned departments rather than loosely coupled, vertical/product-aligned divisions. Will be interesting to see how that works out.

</comment><comment><USER17> I too have written less than inspired articles in the past, so, I'll assume the author was under some kind of editorial pressure and couldn't think through the core idea.

The idea of giving away RT in order to compete with Android is only viable financially with the multiple network effects between the products of different divisions. If he proposes breaking up Microsoft, he should, at least, propose a business model that would work with the broken up company.

</comment><comment><USER15> At the time, I thought that _not_ breaking up Microsoft during the monopoly trial (a very remote possibility) was the worst thing that could have happened to Microsoft. As a large company, they lost a lot of the tech edge that they had in the 90's. Had they split into three groups something like: OS/ servers/dev tools, Office applications, and Internet services/Xbox(?), they would be a much more formidable competitor in the future.

Imagine how much would be different now, had there been baby-microsofts competing during the last ten years.

</comment><comment><USER18> Not going to happen. Microsoft will continue to make gobs of money no matter what, just because so many users are afraid of change and learning new ways to do things (i.e. not using Office). Until Apple products drop dramatically in price, the average user will continue to buy PCs with Windows and other MS products.

The phone and Surface might end up failing (which is a shame, both of those products are WORLDS better than they ever should have been), but the classic Windows PC will stick around for quite awhile, and Xbox isn't going anywhere either.

<comment><USER18> It would definitely make sense, but I don't see it happening.

</comment></comment><comment><USER19> I know it is silly, but its weird to see this post in the light of Bezos as owner of this publication. He is not exactly neutral on this topic. AWS competes with Azure.

<comment><USER20> I think you mean Azure competes with AWS.

<comment><USER19> And it's not just that. The Kindle competes with the Nook and Surface RT.

<comment><USER20> The Surface RT is not competition.

What I'm saying by all this is you only have a competitor when they're a threat. Azure and Surface RT are not threats, they're merely nuisances.

</comment></comment></comment></comment><comment><USER21> This actually makes a lot of sense. A lot of articles have been talking about how there's no obvious candidate for new CEO, and that it's basically too much for any single person to take on.

But if Microsoft were split into divisions (Windows/Mobile/Tablets, Enterprise/Office, Xbox/entertainment, Bing/Hotmail somewhere?) that could freely compete, it's actually pretty easy to imagine suitable CEO's for each one.

I'm truly hard-pressed to think of any value Microsoft as a conglomerate of these divisions, actually provides, strategy- or synergy-wise -- at least that's visible to those of us outside the company.

But is there any kind of precedent for this? It's common for companies to spin off a division, but I can't think of any company voluntarily truly breaking itself up like this. It's hard to imagine a board saying, "we admit we're not providing strategic value here, let's break ourselves up".

<comment><USER20> Remember that every division with the exception of Windows + Office has a hard time turning a consistent profit.

XBox (independent) vs. Sony, Android and Apple is a losing proposition, they may as well sell the division. In a high-stakes game like that, you need a parent company with deep, deep pockets.

Windows independent of Office is worthless. There's virtually no reason to use Windows apart from the Office infrastructure built around it, or the entrenched base Visual Studio developers. It's somewhat popular as a gaming platform, but as Linux starts to be reshaped into a first-class gaming operating system, that will quickly become a non-factor.

The online division is the most troubled, basically a black-hole for money. Surely it could be _made_ profitable, or simply sold for a massive chunk of cash to someone like Yahoo! who could afford to pick it up.

It's not that the divisions would do better independently, but they're part of a highly dysfunctional family that cannot survive independently.

<comment><USER18> The average non-power user isn't going to want to learn how to use Linux, they're comfortable with Windows so they'll stick with Windows. I predict that they'll make two very distinct version of Windows next time, one for tablets like Windows 8 and one a lot more like Windows 7 for the desktops.

<comment><USER17> The non-power user will use whatever comes with the computer, as long as it can access Facebook.

<comment><USER18> But if it's too different and hard to figure it, they'll deem it as crap and that it doesn't work. That's the biggest reason why MS was so afraid to let XP die.

</comment></comment></comment></comment></comment><comment><USER22> Not smart; Outlook is driving Exchange (or the other way round), so are lots of other products; a breakup would not help the bottom line.

<comment><USER23> Outlook wasn't originally part of the Office team's domain.. it was originally part of the Exchange team. Though I do think that Exchange's integration is second to none for teams, and that Outlook as an email client is actually pretty good these days, and integration with Exchange, and bundling with Office keeps it all afloat.

That said, there's no reason Office+Exchange couldn't be split off as a separate sub-organization together, separated from the larger windows org... possibly combined with webmail services (hotmail/outlook.com/office365), simply using an azure org for their deployment/infrastructure.

Hardware could be spun off as well, with "special" deals with a core windows team to be able to build on "top" of windows core.

The Windows org could be responsible for windows core, windows desktop os, and developer tools.

There, you could then have three independant organizations within microsoft that could utilize the core resources, while still having autonomy.

The office org could then concentrate on bringing their services to a broader audience... not worrying if their Office for Android would cut into windows slate sales... and the slate/devices team could expand upon their UIs instead of having to bind into windows core.. and the core team wouldn't need to build clumbsy desktop UIs that don't fit the patterns people have been using for two generations.

</comment></comment><comment><USER24> Perhaps Microsoft should break up - but there's a lot of wishful thinking in there. Windows RT would beat Android in tablets if it were its own company? How? It couldn't do that with billions of dollars behind it.

</comment><comment><USER25> No, not a breakup. Silly. Throwing out the baby with the bathwater. Passing up a great opportunity.

Review: Windows 7 desktops remain important for people running high end applications from AutoCad, Adobe, Mathematica, Office, etc. Also developers for code to run on Windows Server or in house Windows 7 desktop applications.

There are many client devices with many more to come.

The client devices need the Internet and servers, and as concerns about security increase clients should just cache data, easy to erase quickly in case the device falls into other hands, and not store the data; servers need to be very secure; and many organizations and individuals will want their data on their servers in their physical space protected by the Fourth Amendment.

Shrink wrapped software? Okay, have a nice application and want to write it for sales, installation, support, and usage on all the different client devices, Windows XP, Vista, 7, 8, smart phones, tablets? Heck no. And, for Windows 7, etc. a user is very reluctant just to install a shrink wrapped application due to issues of system security and stability. So, the shrink wrapped business is in deep trouble except for the big applications from AutoCad, Adobe, Mathematica, etc.

So, what to do about the work of shrink wrapped applications? Sure: Make them Web apps; that is, use a Web browser for the user interface; let the browser run on whatever client; and put the rest of the code on a server. If the client can't run a good Web browser, then do the same thing by writing a client app that uses a Web server for the data, algorithms, and computation.

So, we've got it: For Microsoft, push Windows Server for the servers. Push client devices. Have really good Web browsers. And in cases where _own both ends of the wire_ , take advantage of that for more in functionality.

Fundamental point: People using devices, including mobile, want some _utility_ , _functionality_ , etc. Since a single mobile device is quite limited in what it can do, the device is mostly for user interface (UI) for services, applications, algorithms, data, etc. on servers. So, the servers remain just crucial.

Microsoft has shown that it knows how to run huge server farms, well managed, with minimal staff. That's a huge business advantage. Computing is charging on; e.g., there is a new solid state mass memory that can put a terabyte on the area of a postage stamp. It's been a while since Intel knew how to put 1000 cores on a single processor. New operating system concepts will be coming forward. All this progress will need lots of software development, e.g., for servers. Microsoft's got the people, funds, market, etc. to do that work and lead in it.

Mistake: Dunkin Donuts sells more donuts than Windows sells copies of Windows 8. Similarly for hamburgers at McDonald's. Similarly for smart phones from Apple. And, for all three cases, so what for Microsoft? There may be a lot of new client devices, but that does not mean that Microsoft has to dominate in all of them.

All the smart phones in the world won't mean that a high end, 64 bit Windows 7 desktop system will be of no interest; a smartphone and a high end Windows 7 desktop just are not the same thing and are not really in direct competition; even more the case for Windows Server.

Breakup? Windows Server can't exist alone and, instead, must _serve_ the many client devices. So, Microsoft should stay in the business of soft/hardware for some client devices. So, don't breakup.</comment>


VINEBOX (YC W16) Lets You Try New Premium Wines, by the Glass - <AUTHOR> https://blog.ycombinator.com/vinebox-yc-w16-lets-you-try-new-premium-wines-by-the-glass <comment><AUTHOR> It's interesting to see YC investing in a subscription-based food/drink business years after that wave has peaked in popularity. It would seem there must be something special that I'm overlooking here.

Perhaps technology involved in the glass tubes or IP related to single servings of wine. Maybe it's just the relationships and the idea of cutting out middleman from vineyard to glass.</comment>