Concourse CI - <AUTHOR> https://concourse.ci <comment><USER2> Have been working with it for a few months, and I am not positive about it.

UI is pretty but often breaks. Is unusable when you have a pipeline with dozens of concurrent jobs.

Concept of teams is fine, but when you have to switch between them, even with oauth authentication, it is a pain.

Job concurrency control is binary.

We never have been able to have worker pool scale down without a hiccup, always some darn worker hogging containers and atc not removing it, leading to stalled pipelines.

That overlay network it comes with, garden thingy, creates so many problems and solves just one...

Oh and not having BUILD_ vars available in tasks is rude, thank you very much, but there are cases when it is just mandatory and concourse makes it impossible to do.

At least new version has better secrets handling, previously it was a joke.

<comment><USER3> > UI is pretty but often breaks. Is unusable when you have a pipeline with > dozens of concurrent jobs.

Find this hard to believe - we have some pretty spectacular pipelines which render without a hiccup.

<comment><USER4> 100+ jobs will break your browser. RabbitMQ has been complaining about it for a while.

<comment><USER5> 100+ jobs in one pipeline?

<comment><USER2> Yes, some systems we work with have over 100 items ( jobs and resources) in one pipeline ... because microservices...

</comment></comment></comment></comment></comment><comment><USER6> Not sure when it came about, but when I first evaluated Concourse, not being able to trigger jobs manually was a primary blocker. Glad this showed up on HN again, because they've added it in at some point. My favourite thing about Concourse was really its ideas around "Resources". This always felt so much better than this idea of plugins due to how unified the experience was. Also, the ability to implement resources for yourself is extremely easy. So if I had an internal software running, being able to build something for it meant defining three bash scripts.

That said, the docs around implementing resources still needs some improvements. Whatever I learnt was from cloning and modifying existing resources.

<comment><USER7> This is also close to my experience. Improving, but not there yet. Unfortunately, it will retrigger the same job with the updated version of the resources. With other CI tools, it was possible to replay the same job, with whatever inputs they took by the time they ran, with concourse it does not seem t be possible.

A practical impact of this issue is that we cannot simply reprocess a previous deploy step, with the old artifacts and inputs in a middle of an outage.

<comment><USER8> You can achieve this by clicking on a resource and hitting toggling the "power button" to off for each of the resources you don't want to include in the build.

<comment><USER7> While this may even work, it is not practical and it is a disappointing experience for a pipeline centric type of CI such as concourse.

In other tools, hitting re-run would simply replay the job, with the same state it was executed in the first run. I would expect concourse to behave similarly, but no.

<comment><USER9> > _In other tools, hitting re-run would simply replay the job, with the same > state it was executed in the first run._

If the job failed, you get this. Otherwise I figure that the idea is that you're satisfied with those versions of the resources acted on by the job.

Disclosure: I work for Pivotal.

</comment></comment></comment></comment></comment><comment><USER10> I know everyone hates on Jenkins these days, but (at least for the workloads I am building) Concourse feels like a toy.

My issues:

\- everything _has_ to be in a docker container (not all things can run in garden)

\- Just trying to find logs is a pain, and getting the UI to show a full log output is basically impossible

\- each install of concourse may need an entirely different version of fly

\- There was no way of retriggering a CI run on a single PR (without force pushing to the branch, which removes Github reviews)

Jenkins + Jenkinsfiles + Pipelines provides all of the good bits of Concourse, with none of the Pivitol enforced workflow + tools.

I get why CloudFoundry uses Concourse - the build process is so arcane that you basically need to run the full CI locally, but I really do not get the hype for other projects.

<comment><USER11> Jenkins has some serious problems when you want to work with CD.

Its currently impossible* to use Jenkins in a CD environment where you want to deploy by tagging your git project[1] and allow rollbacks because Jenkins doesn't treat tag's hashes (only the commits the tag points to).

Unfortunately, hack such a fix is not very pleasant either, because Java, and i don't think the Jenkins developers are interested in it either, at least not if just a few people ask for it.

It may look as a silly usecase, but i believe its one that mostly fits when you have a dynamic server farm (hosting on aws with autoscalling, for example) and don't want to auto-deploy on master commit.

[1][https://groups.google.com/d/msg/jenkinsci- users/mYxtDNMz1ZI/...](https://groups.google.com/d/msg/jenkinsci- users/mYxtDNMz1ZI/xbX9-xM9BQAJ)

edit-> * aktually its not impossible, but pretty hacky and ugly, it would be nice to be able to rely on jenkins own tools for this.

<comment><USER10> > Jenkins has some serious problems when you want to work with CD.

When you want to work with CD in a particular way.

A simple fix to the above issue would be to have a "production" branch. When you want to do a release, you merge to the production branch. A roll back would just be a revert of the commits since the last merge to production, and could even be tagged.

If you are doing CD by allowing devs to tag random branches, cool, but I would not call it a "failure" on the Jenkins side.

> Unfortunately, hack such a fix is not very pleasant either, because Java, > and i don't think the Jenkins developers are interested in it either, at > least not if just a few people ask for it.

Its a webhook - that can be in any language, or if you use Git(hub|lab) a config item.

<comment><USER12> > _When you want to work with CD in a particular way._

I think that I, as the user of the project, should get to decide how I want to deploy my software. So, yes, I do want to CD things in a particular way: one that matches my developer's expectations and my business requirements.

> _CD_

Jenkins isn't a deployment tool. It can act like one, but this (rollback) is a classic example of how it's a build, not release, tool.

To address OP's complaint: we rebuild steps from the previous pipeline to perform a rollback, and we have all our deployment logic in a separate system which works with commit IDs, if the previous pipeline has been pushed out of the list of recent pipeline runs.

<comment><USER10> > I think that I, as the user of the project, should get to decide how I want > to deploy my software. So, yes, I do want to CD things in a particular way: > one that matches my developer's expectations and my business requirements.

Sure - it definitely is. But we all have to do that within the limitations of the tools we use. Jenkins makes an assumption (in the default mode) of a continuously moving HEAD - which may or may not be what you want. It doesn't make Jenkins bad - it makes it not suitable for your use case.

FWIW - adapting to using a moving HEAD is not that hard - `git reset <old-tag> && git commit -a -m "Revert deploy <failed-tag> && git tag <failed-tag+1> && git push` - but I totally understand why people would not like to use that style. It is a personal / culture choice.

</comment></comment></comment></comment></comment><comment><USER13> Mostly off-topic but I've been looking for more than a CI for some quite time, more on the CD side. How are you guys handling some of these cases? Bonus points for hosted options.

\- Truly a pipeline based stages. No messing around with git branches/tags for each environment release.

\- Ability to combine multiple builds together.

\- Build dependencies. Ability to trigger project-B build when project-A build succeeds. (Docker Cloud Builds have something like this.)

\- Use the same artifact across multiple build stages.

\- An option to promote a build to the next step either manually or automatically.

Amazon's internal Apollo system was amazing. AWS Code Pipelines kind of does some of these things but it's every limited and hard to work with.

<comment><USER14> [https://buildkite.com](https://buildkite.com) has all those features, and very easy to work with.

It's a hybrid hosted model -you supply your own workers (they provide a CFN template to make that part easy), everything else (e.g. web UI/API) is hosted

<comment><USER15> Buildkite is hands down the best tool we have used.

</comment></comment></comment><comment><USER16> Avoid! Concourse looks good on the surface but it's really not that great.

The UI is clunky.

The abstraction layer is too low and leads to a lot of repeated YAML. Which leads to YAML programming.

There are simple scenarios like deployment rollbacks who are hard to do.

For some reason they decided to develop their own container engine which leads to all sorts of trouble and maintenance issue. It's generally slow and we had 100% CPU usage when the worker was doing almost nothing.

I have used for 4 months and it was only problems. Gitlab CI is much better. Or even Jenkins is better.

<comment><USER5> > Concourse looks good on the surface but it's really not that great.

I've used teamcity, jenkins, gocd, circleci, concourse, and travisci. For multi-project systems, concourse is king. (I like travisci for by-itself, non- system projects)

> The UI is clunky.

What? You just said the UI looks good... It's simple and clean; everything is async javascript (no page loads).

> The abstraction layer is too low and leads to a lot of repeated YAML. Which > leads to YAML programming.

Which is an intentional choice. If you don't like YAML, use one of the MANY yaml abstraction layers of your choice...

> There are simple scenarios like deployment rollbacks who are hard to do.

First of all, a CI system shouldn't be your answer to rollbacks. Your deployment system should handle that. Secondly, assuming your deployment system can do rollbacks, concourse has on-fail jobs that can trigger rollbacks just fine.

> For some reason they decided to develop their own container engine which > leads to all sorts of trouble and maintenance issue. It's generally slow and > we had 100% CPU usage when the worker was doing almost nothing.

garden is used because cloudfoundry builds it. It is not slow... it is a light layer on top of runc (as opposed to docker which is a rather heavy layer on top of runc). You should pretty much never have to care about it, and in 3 years of using concourse I haven't had to - and we have some pretty gnarly large pipelines.

Also I call rubbish on your 100% CPU. I have two workers t2.xlarge workers running 22 and 30 containers (like, right NOW) and neither is above 10% CPU (which, actually, I should make those a lot smaller). Don't run workers on a potato and you'll be fine.

> Gitlab CI is much better. Or even Jenkins is better.

Ohhh you're trolling.

<comment><USER16> > What? You just said the UI looks good... It's simple and clean; everything > is async javascript (no page loads).

I meant that when just looking at the website, Concourse looks good in terms of what it has to offer. For example it offers proper build pipelines, something that is lacking in most other CIs.

Maybe it's clean to you, I found that our developers where generally confused by how the releases were working.

> Which is an intentional choice. If you don't like YAML, use one of the MANY > yaml abstraction layers of your choice...

I don't mind YAML too much but adding a templating language on top is generally clunky. Gitlab CI also uses YAML, can also build pipelines and doesn't require to generate the YAML.

> Also I call rubbish on your 100% CPU.

It might have been due to a misconfiguration (we were using c4.4xlarge). I bet that you have deployed Concourse using BOSH, the only truly tested deployment method.

</comment></comment></comment><comment><USER17> We're using Concourse extensively at HelloFresh (>130 devs). It's not without its quirks, but I've little to complain about so far, except perhaps the polish of the UI.

<comment><USER18> Cool - did you folks know about the Concourse London User Group? The last two have been held just around the corner from you: [https://www.meetup.com/Concourse-London-User- Group/](https://www.meetup.com/Concourse-London-User-Group/)

<comment><USER17> All our developers are Berlin based, so the London office isn't a great metric. Thanks anyway, though!

</comment></comment></comment><comment><USER19> We use concourse.ci to successfully run cloud.gov at 18F.

</comment><comment><USER20> Disclaimer: I work for Pivotal, on the RabbitMQ team. We push Concourse to its limits every day. We work closely with Jenkins, GoCD, Travis & Concourse. They all have their limitations.

All things will break horribly if the conditions are right. It's unreasonable to assume that the things which work in [insert your current CI] will work in Concourse. It's still a new and relatively immature product, but it works well in most cases.

Half the secret to a good Concourse experience is not upgrading it in-place - stand up fresh deployments. The other half is gradually transitioning between Concourse deployments, because bad versions have been and will continue to be released - mistakes are only human. As long as you share the Concourse vision and are willing to keep up with the pace of change - not everyone can or wants to - then it's an amazing CI.

Concourse still makes me excited, even after many years of hard lessons, because it is a genuinely innovative approach to building better software. Most miss this, and I understand why, but give it time - the ideas behind it will mature and become the norm.

Even though Concourse can work really well, it's not always the best choice. Make it better if you can & want to, use something else if it's easier. There is no right or wrong, just preferences : )

</comment><comment><USER9> I'm late to the party.

Concourse is difficult to come to from other CI tools. A little more aloof. There have been real, serious implementation difficulties.

But I kinda love it, because it's a handful of simple ideas that unlock incredible power. It goes beyond "build and test each commit" to becoming a full project automation tool, a software manufacturing robot. When I talk to people about Concourse, I tell them: your pipeline and your tasks are _production code_. Keep the discipline, care and engineering practices that you bring to the apps and services you create.

The problem is that most of the deep tribal knowledge about how to get started and how to best apply it is locked up inside a handful of organisations, most notably Pivotal. I had been working on a video series which was meant to walk through both the concrete business of building pipelines, as well as the concepts of how best to do so.

Unfortunately visa conditions got in the way and I have abandoned that effort. Interested persons are welcome to email me for links to the first 4 episodes that I made, but be aware that it stops even more suddenly than Firefly.

Disclosure: I work for Pivotal.

<comment><USER21> Why not put it up on youtube?

<comment><USER9> It's an hour of work per minute of screen time. For now it ends at 4 episodes.

</comment></comment></comment><comment><USER22> Kind of odd to see the homepage show Vagrant as the install mechanism, even though it supports Docker as well. In 2017, I'd think more developers are likely to run Docker than Vagrant workloads on their machine.

<comment><USER23> Probably easier to run Docker inside Vagrant than Docker inside Docker.

<comment><USER18> Concourse doesn't run Docker, it uses something called Garden to run OCI containers from Docker images.

Those containers can run the Docker daemon though, if you like.

<comment><USER4> The version of Garden it uses basically just shells out to Docker. Run 'top' on a busy Concourse worker, it's fun.

<comment><USER8> Garden makes use of runc (an Open Container Initiative project with a lot of contributions from Docker), in the same way containerd (component included in Docker) makes use of runc to run images. You won't find the docker engine being used by Garden.

Edit: I _think_ what you're likely seeing (if you are seeing docker in the top output) is Concourse using the docker-image-resource to pull images for your tasks to a local docker registry.

</comment></comment></comment></comment></comment><comment><USER24> I've been eyeing Concourse and Go.CD over Jenkins for a while.

The main criticism I saw on Jenkins and Go.CD vs. Concourse was that Jenkins Pipelines aren't first class and that it's easier to export configuration(in that regard Concourse > Go.CD > Jenkins). On the other hand Jenkins and Go.CD supports extensions, which Concourse touts as a feature.

I also want the CI builds to create my base boxes with packer in multiple steps. And I somewhat want to be able to hand over the stuff to ops at some point to be able just stay alive for the next 5 years or more. Would anyone know if it makes sense to even consider concourse or go.cd or some other CI/CD solution and if so which?

Obviously the boxes need to be used as artifacts and everything has be on premise as well.

<comment><USER25> How are Pipelines not first class in Jenkins? Pipelines are provided in the default installation and they are the first thing we talk about in the docs [https://jenkins.io/doc/pipeline/tour/hello- world/](https://jenkins.io/doc/pipeline/tour/hello-world/)

Disclosure: I work on Jenkins.

<comment><USER26> I think that the features are very different.

Admittedly I ditched jenkins before pipelines became a full feature, but my understanding of them in jenkins was that they are mostly for scheduling of jobs. IE, run job A after job B

Concourse passes inputs, outputs and resources between jobs as the ONLY STATE, and jobs trigger based off of changes on resources or the availability of new inputs.

I think that when you sit down and look at the two products, jenkins is great at running scripts, and concourse is great at managing code versions.

In the end we actually run both concouse + a script runner; this allows concourse to manage the tagging, builds, releases, and testing; but still allows us to run ad hoc scripts that concourse doesn't do well.

I'd be interested in taking another look at jenkins now that pipelines and the groovy DSL is solidified, but I get the idea that they still fill slightly different needs

</comment></comment></comment><comment><USER27> My number one complaint with Concourse (which I suspect is due to Go) is that you need to have it hosted with a valid TLS/SSL cert in order to use the fly command. At least this was an issue in the 2.6.0 days, but I couldn't see anything to change this in the recent versions.

This is rather annoying if you want to run a copy on your local network say at home. Its very frustrating because the fly command solves the biggest issue with IWOMM (it works on my machine) by allowing you to run code and tests on another machine before committing anything.

I think from memory I tried using self signed certs and this also had issues for one reason or another.

That said it is still the best CI system I have used to date.

<comment><USER28> I issue valid TLS certs for my internal servers using letsencrypt DNS challenge (there is a nice cloudflare hook for dehydrated that I use). Runs on cron, haven't had to worry about it once I set it up. (Haven't tried with concourse, but don't think that would be a problem)

<comment><USER29> Do you worry about your internal names being exposed via Certificate Transparency?

<comment><USER28> Not much.

</comment></comment></comment></comment><comment><USER30> A little late to this discussion, but although I'm a big fan of many of the concepts in Concourse, I think it's lacking a lot of polish.

For example, I have actually implemented patterns within Jenkins to force all jobs to run inside of containers. A job is just a container + command with some linking using the jenkins pipeline plugin that reads some json configuration to determine how jobs are linked.

The primary issues I have surround the fact that my company uses kubernetes, thus we have no insight into the runc containers. Load balancing in concourse is non-existent. If a worker goes down due to load, if you bring it back up, it's going to go down immediately from all the jobs that have been triggered while the worker was offline. Not only that, the resource requirements seem pretty high. Recently a concourse worker stalled because the amount of volumes/images it was caching was over 100 gigs, and not knowing the internals, I wasn't sure what the best way was to clear this cache. Having to tell the infrastructure team that we just need to spend more money is a hard sell when we've upped the cpu, memory, storage, postgres disk, all more than once. I understand that different images have vastly different sizes, and jobs different amounts of work, but their need to be some clear suggestions for sizing. If there exist them now, I apologize but I haven't seen them.

So yeah I've had some fun developing in it, but more help making it reliable would be really nice. Also if kubernetes is absolutely the wrong way to run it, which it seems like, I'd have to be provided a better/easier alternative to really become an advocate.

Final note: has anyone actually setup metrics/monitoring for concourse that doesn't know BOSH? The docs describing it seem huge unless you already have the infrastructure pieces. Let's setup riemann (we already have statsd and no experience with riemann), emit to influxDB (we have prometheus, no experience with influxDB), then use Grafana (ok we already have that). I just wanted a better idea of disk, cpu, mem, # of containers, lifecycle without having to setup all these new pieces of infrastructure. Finally, just not that interested in BOSH, which all of the example metrics repo's are.

</comment><comment><USER31> I took part in evaluating Concourse CI for the needs of my company (30+ devs). While it has amazing CI pipeline capabilities, we ultimately didn't select Concourse because it felt much more like a CI toolbox, requiring some development to put those tools to use. And what we really wanted was more of a turnkey CI product.

Perhaps ironically, we ended up doing some development around the edges of the CI product we ultimately selected (GitLab).

<comment><USER32> Cool to hear you selected GitLab. We recently added multi project pipeline visualization. That was partially inspired by concourse CI. Is there anything else you would like to see?

<comment><USER31> Interesting you should mention that. Right around the time you implemented multi-project pipeline visualization, we built a bot that listens for GitLab build completion events, hits the (undocumented /unsupported) internal GitLab global code search ElasticSearch index, finds downstream dependencies of the completed build (as declared in setup.py/package.json), and triggers their builds as well.. Taking full advantage of your nifty visualization.

So I think the feature we'd want to make that easier is an actual global code search API!

<comment><USER33> Thanks for this proposal! It might be a cool feature indeed. I created an issue about it in our issue tracker, see [https://gitlab.com/gitlab- org/gitlab-ce/issues/35186](https://gitlab.com/gitlab-org/gitlab- ce/issues/35186).

We do not have immediate plans to ship that yet, but I pinged our product team member there. However we do have plans to ship multi-project pipeline with an "inversion of control".

You will be able to specify pipeline relation with an upstream project, and when someone pushes a commit to it, a downstream pipeline is going to be trigger automatically. See an issue about cross-project dependencies - [https://gitlab.com/gitlab-org/gitlab- ee/issues/1681](https://gitlab.com/gitlab-org/gitlab-ee/issues/1681)!

<comment><USER31> Awesome, thanks for filing!

</comment></comment></comment></comment></comment><comment><USER34> Excellently designed system. Very poorly implemented.

Our team's been using it since it's initial releases. It's been nothing short of disastrous for all but the smallest pipelines.

The design is great. Keeping configs in yaml instead of little white boxes in a Jenkins database is much better. Pipelines as a first class concept. It feels inspired by a functional programming language. You get great build reproducibility since there are no workers that get dirtier over time if you forget to clean up. The resource model is awesome. Very cool stuff. I'm hoping every CI system learns from what's here. Second to none in design.

However it performs like a dud. No scheduling to speak of, just runs everything as soon as it can. We've run into nodes dying under load (-not- underprovisioned, could run all these jobs manually at once on these monsters). We've run into problems with volume reaping, fork bombs, ui freezes, everything under the sun.

I really like Concourse and will hopefully one day be able to come back to it when its implementation is as solid as its paradigms are.

But I'd avoid use for now.

<comment><USER5> > However it performs like a dud. No scheduling to speak of, just runs > everything as soon as it can. We've run into nodes dying under load (-not- > underprovisioned, could run all these jobs manually at once on these > monsters). We've run into problems with volume reaping, fork bombs, ui > freezes, everything under the sun.

I've used concourse as a consumer for 3 years and I've very, very rarely seen any of the problems you're describing, even on the older versions and certainly not in the last year or so.

> no scheduling to speak of

Concourse has a massive scheduling system built into it.. [https://github.com/concourse/atc](https://github.com/concourse/atc)

Furthermore, you can configure jobs to run in serial (default is parallel).

> ui freezes

Put your `web` binary on a decently-sized VM and your problem should disappear. Also, don't have your workers on the same VM as your `web`.

<comment><USER34> UI freezes are completely client side and related to the elm implementation and your browser's execution of the code. The size of your VM doesn't matter at all.

ATC is a more of a dependency scheduler. The code [1] shows that it basically gets all pending jobs, and then runs them. There's no concept of queueing or maximum number of jobs, you just have to hope your limits are high enough (max containers, max tasks in systemd, max fds in the same) and that your machine doesn't fall over in the attempts.

The "massive" scheduling system also has no idea what nodes need work and which do not [2] so the idea is to heavily overprovision until it doesn't fall over (on top of already beefy requirements which others have alluded to in this thread).

You can not serialize multiple pipelines. Only within pipelines. If I have 10 pipelines, they will all run independently and there's nothing you can do about it other than attempt serialization with the pool resource (which we've recently had problems with - it also appears to be buggy and we're looking at submitting patches).

I've got a tremendous amount of experience with this system and I believe it's everything I made it out to be. The rebuttals you've provided to my issues are simply a lack of understanding of our context, not every user will have the same experience with any given product.

[1] [https://github.com/concourse/atc/blob/master/scheduler/sched...](https://github.com/concourse/atc/blob/master/scheduler/scheduler.go#L46)

[2] [https://github.com/concourse/concourse/issues/675](https://github.com/concourse/concourse/issues/675)

</comment></comment></comment><comment><USER35> We are using concourse at work right now and have 50-ish pipelines for various repositories. IMO, there's definitely some work you have to put into it because you'll sooner or later run into a problem with the existing resources and need a custom one. Writing custom resources is pretty easy however.

Concourse also isn't really made to work well with the Git Flow, there is no builtin way to run the CI on multiple branches (there's a git-multibranch community resource which requires redis at some point). we're basically thinking about changing our workflow to trunk-based, but it still feels weird to me that we might change our workflow to fit our CI better.

that being said, I personally still really like concourse and it's fun to work with.

</comment><comment><USER36> I like how it looks, but first impressions after trying to make it run on windows ended up in failures:

Refused to execute script from '[http://127.0.0.1:8080/public/index.js?id=932c553753eec4ef375...](http://127.0.0.1:8080/public/index.js?id=932c553753eec4ef375e1c17f4b28c10') because its MIME type ('text/plain') is not executable, and strict MIME type checking is enabled.

</comment><comment><USER37> I evaluated concourse CI and I was impressed by the concept.

Pros \- Setup was really easy.(I didn't use bosh). \- Both server and build pipeline configuration (Yaml file) are easy to backup and recreate.

Cons \- When I evaluated it , it had a few bugs which forced me to restart the server almost daily to keep it working. It should be more stable now.

-

<comment><USER5> Did your evaluation happen a long time ago? Or, another question... did you run the binaries and appropriately screen them? I had the same problem until I realized I sucked at making a binary live a long time, and my eyes were opened to screen :)

<comment><USER37> It was a known bug in the some go library. It was recently fixed. But at the time it caused concourse to stop working until restarted.

</comment></comment></comment><comment><USER38> We looked at Concourse deeply, while we didn't go with it, it's inspired a number of projects I know and I think has pointed out the obvious; representing how you think about a problem is how you should represent it visually, great insight that has been overlooked in CI.

</comment><comment><USER39> I hadn't heard of this, it looks nice! Would I be far off if I'd say that Concourse is an open source clone of Travis CI and CircleCI? Any strong pluses of the SaaS offerings over Concourse I should know about?

<comment><USER32> Concourse is developed independently, it is focused more on multi project workflows.

</comment></comment><comment><USER40> During my Jenkins build I am creating a file with the number of database queries per test, since I am generating the file it can be in any format.

How can I feed this to Jenkins and plot it over time?

</comment><comment><USER41> Curious to know why no one actually mentions Bamboo - definitely a great alternative to Jenkins, especially for those, who are into Atlassian infrastructure.

<comment><USER31> At least last I looked, Bamboo requires manual configuration of all builds. This doesn't scale to more than a handful of build plans. It also doesn't have much in the way of modern CI pipeline functionality.

</comment></comment><comment><USER42> Concourse is a really amazing piece of software, but the BOSH requirement I think will keep adoption low for small companies.

<comment><USER43> I use Concourse every day and I don't know yet what BOSH is and I hope I'll never have to. The installation was pretty standard. I followed a tutorial written by Justin Ellingwood, a great technical writer working for DigitalOcean - [https://www.digitalocean.com/community/tutorials/how-to- inst...](https://www.digitalocean.com/community/tutorials/how-to-install- concourse-ci-on-ubuntu-16-04).

</comment></comment><comment><USER44> Typo on front page.

Rather than "Rather than a myriad of checkboxes..." it should be "Rather than myriad checkboxes..."

<comment><USER45> [http://www.thefreedictionary.com/myriad](http://www.thefreedictionary.com/myriad)

> Usage Note: Throughout most of its history in English myriad was used as a > noun, as in a myriad of reasons. In the 1800s, it began to be used in poetry > as an adjective, as in myriad dreams. Both usages in English are acceptable, > as in Samuel Taylor Coleridge's "Myriad myriads of lives." This poetic, > adjectival use became so well entrenched generally that many people came to > consider it as the only correct use. In fact, however, both uses are > acceptable today.

<comment><USER44> There are many of reasons why I'm not going to calmly agree with the audacious claim that 'both uses are acceptable today'.

<comment><USER46> Of course both are acceptable, why is this an audacious claim?

<comment><USER44> Because it invites (not begs) the question 'to whom are they both acceptable?'.

People that contribute to thefreedictionary.com, evidently.

<comment><USER47> And people who contribute to the OED, in both the 1989 and 2003 editions, evidently.

[http://www.oed.com/view/Entry/124538](http://www.oed.com/view/Entry/124538)

[http://www.oed.com/oed2/00154839;jsessionid=AC4DB817EFBADA5F...](http://www.oed.com/oed2/00154839;jsessionid=AC4DB817EFBADA5F500874E63F5A5E38)

I have had similar experiences, though, where I was convinced that some word or phrase usage was just incorrect, and where it turned out that I had just not happened across it. Fortunately, a minute of research can today fix any such misconceptions!

</comment></comment></comment></comment></comment></comment><comment><USER48> We've been happy teamcity users for years.

</comment><comment><USER49> We're using Buddy - [https://buddy.works/](https://buddy.works/) \- and it seems to be easier, faster and more reliable than most of these other CI solutions so far.

<comment><USER50> Interesting. How does it compare to other CI software like Jenkins? I'm always a bit skeptic about tools that look nice but turn out to be limited when put to real work.

<comment><USER49> Works well for us, docker/container native so every step in a pipeline is a persistent volume with the environment changing based on what image you need. Also easily builds/pushes your own images and lots of other built-in task runners to make pipelines. Has .yaml or ui config and they have an enterprise version which runs on-prem.</comment></comment></comment>


Tell HN: Moniker is down - <AUTHOR>

It looks like all of monikers DNS services are down, that's a pretty big deal, they do DNS for millions of domains. <comment><AUTHOR> They're back up now, that was a 1 hour outage.</comment>


PayPal no longer serves users in Turkey - <AUTHOR> https://www.paypal.com/LV/webapps/mpp/ua/upcoming-policies-full?locale.x=en_LV <comment><USER2> What went wrong?

<comment><USER3> New banking law and PayPal failed to obtain the necessary license to operate further, presumably because it couldn't or didn't want to comply with requirements for having infrastructure in turkey. [https://techcrunch.com/2016/05/31/paypal-to-halt- operations-...](https://techcrunch.com/2016/05/31/paypal-to-halt-operations- in-turkey-after-losing-license-impacts-hundreds-of-thousands/)</comment></comment>


Quantitative Finance Reading List - <AUTHOR> http://www.quantstart.com/articles/Quantitative-Finance-Reading-List <comment><USER2> QuantNet maintains a _Master Reading List_ as well:

[https://www.quantnet.com/threads/master-reading-list-for- qua...](https://www.quantnet.com/threads/master-reading-list-for-quants-mfe- financial-engineering-students.535/)

A few words of advice for anyone interested in this field as a career:

1\. I would caution anyone on speculatively learning a new programming language in the hopes of getting a job. Your boss will tell you what to learn when you get hired. Don't ask strangers on the Internet for how to allocate your time.

2\. "Quantitative Finance" is a broad field. High-frequency trading and exotic options trading have no overlap. So putting together a list of what-to-learn- to-be-a-quant is really foolish. It's like creating a master knowledge base for all of programming without narrowing the field to web, embedded, gaming, etc.

3\. The best way to learn is to get a job in the field. I'm surprised at how often I have to repeat this.

<comment><USER3> >3\. The best way to learn is to get a job in the field. I'm surprised at how often I have to repeat this.

But to get one, you need to have worked through a lot of books already, right? Job descriptions of this field look to me like: PhD in math finance and senior programming skills.

<comment><USER4> I think the best way to get a tech job in finance is to live in a city where finance is a big industry (New York, Boston, Chicago, London). If you punch "C++" into Indeed for Chicago, IL, half the listings that come back are for positions at trading companies.

<comment><USER5> I apears that Algorithm trading Firms are slowly moving to SF/Silicon Valley. I see some postings for SF-jobs in this domain. May be they want to use the software talent in this area.

</comment></comment></comment></comment><comment><USER6> A couple of years old, but this is (IMO) a much better list: [http://quantivity.wordpress.com/2010/01/10/how-to-learn- algo...](http://quantivity.wordpress.com/2010/01/10/how-to-learn-algorithmic- trading/)

The 30+ generic C++ books in the posted list says a lot, and many of the more finance-oriented books on the list are very low on signal-to-noise.

</comment><comment><USER7> It seems like the topic of QF is appearing more frequently these days on HN. I guess it's a sign of the times after an absence after the implosion of 2008. There also seems to be general leveling off of posts on how awesome working for startups are compared to 2008/9 and the exodus of talent from wall street. Sign of the times I guess. Makes me think we're headed back to the 'good old days.'

This is all a bit depressing cause I feel like the conclusion a lot of society had reached in 2008 was that finance should be boring, simple, and shouldn't attract too much brain power from other fields.

<comment><USER8> I've noticed it too. I think many of us believe that the size of the financial sector is extremely disproportionate to the value it adds. It drains talent by offering smart people salaries that other industries can't afford. It gambles with the economy recklessly, but the public pays the debt when their gambles turn sour. And on, and on.

But let's face facts. I'd love to think everybody's really here to "change the world" and "create value." But the biggest stories in tech entrepreneurship focus on silly, time-wasting apps that are sold for millions. Does that really make any more sense than giving a guy a million dollar bonus for designing a high frequency trading algorithm?

If we don't expect more from our community, we can't really criticize theirs.

<comment><USER4> "It drains talent by offering smart people salaries that other industries can't afford."

Bullshit. The difference is that in finance (trading, at least), good technology is seen as a competitive advantage that makes a meaningful impact on P&L. When financial companies do well they tend to share the love in the form of _cash_ bonuses. This is a strategy other industries could adopt but choose not to.

Also one other nit, there are only a few companies (the megabanks) that are big enough to have a government backstop if they blow up. There is a whole world of smaller outfits doing business without any such guarantee.

<comment><USER8> The mean salary in New York City's finance industry rose from $80,000 in 1981 to $360,000 in 2011, while average New York City salaries rose from $40,000 to $70,000.

You may be right, but I just don't think there are a lot of tech companies that could offer a mean salary of $360,000 and remain profitable. Especially small to medium size companies. Compare that to hedge funds, which were basically printing money and could pay their small staffs extremely large salaries (and even larger bonuses).

<comment><USER9> Google and Microsoft are in the same ballpark of profitability per employee as say Goldman Sachs or Morgan Stanley. The difference is that financial firms take pride in how much they pay their employees, while tech firms do things like collude to depress employee wages.

<comment><USER10> This couldn't be more true. I got an offer at Intel after working there for the summer. A finance company asked what they were paying and decimated Intel's offer. Upon telling Intel the recruiter at Intel demanded to know where I was going. Luckily I wasn't going to Apple, Adobe (or one of the other companies they allegedly had illegal no competes with or I would have no job at all.

<comment><USER11> 'decimated', as in, paying you ten times less?

<comment><USER10> If you are going to pull out a dictionary, make sure to read both definitions. :)

decimated:

past participle, past tense of dec·i·mate (Verb) Verb

1) Kill, destroy, or remove a large percentage of.

2) Drastically reduce the strength or effectiveness of (something): "plant viruses that can decimate yields".

<comment><USER11> Even under definition 2, it still means 'reduce'. So if the salary was 'reduced', the second offer was less than the first one. It may have 'decimated the attractiveness' of the first salary, sure.

<comment><USER10> Please tell me you are trolling:

Drastically reduce the strength or effectiveness of (something): "plant viruses that can decimate yields".

decimated Intel's offer.

Drastically reduce the effectiveness of Intel's offer.

It fits exactly. Not only that, but you could also say:

Destroyed Intel's offer. (Def 1)

Decimate is the right word choice and it conveyed the intensity of how badly Finance Co. beat Intel's offer. Your definition of decimate may be used by heavy quants but it doesn't fit everyday usage.

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER12> I get a feeling that people somehow do not want to believe that quant finance is in very bad shape right now. It is very hard to get a job, especially without experience. Salaries are not growing, might be even decreasing. Starting base salaries are around 100k. You might get a bonus, but it all depends on firm performance. Before VP level the pay is not much greater than in tech.

You will likely need a PhD to get a job at all. Maybe a top MFE, but most MFE graduates are perceived to have very shallow knowledge, which has a grain of truth to it.

</comment><comment><USER13> Saying "I want to learn Quant Finance" is like saying "I want to learn programming," in that it really all just depends. There's an impossible number of fields and sub-fields, and many people know one field intimately but may not know another at all (high-frequency trading and derivative pricing, for instance, have little overlap). Also, some of these jobs (i.e. high-frequency trading) are about 90% programming, whereas other jobs tend to be significantly more mathematical. If you can't narrow your interests beyond just the exceptionally broad category of quant finance, then learn enough to figure out what subcategory you want to jump into, and then let me know and I'll give you a reading list, if I know anything about it.

<comment><USER13> To get it started, subcategories include:

1) High-frequency trading. Very programming oriented as trade latency is everything. You'll need to be really good at making C/C++ code really fast. Math is somewhat less important than other fields.

2) Derivatives pricing. Usually working for banks/hedge funds, and use various methods (Monte Carlo, PDE, transform) to price options. Will need to know C/C++ and/or probably Matlab, C# (Excel is king here, so MSFT languages is a big deal), and Python is on the up and up. Will need to have a serious math background. This is more or less what I do.

3) Algo trading. Uses mathematical techniques to find patterns and execute trades. Probably works for a hedge fund. Programming languages vary by fund and include Matlab, Python, C/C++, Java (one uses Ocaml). Less theoretical, lots of playing around with data.

4) Quant risk. Usually works for a bank or hedge fund in the risk department. Responsible for things like building VaR models. Usually requires light programming (VBA, although serious languages help), and some lightweight math.

<comment><USER14> I'd add that if you are truly interested in HFT the time when you could make C/C++ software fast enough is either gone or very short lived. Much more valuable would be FPGA programming or other similar skills.

<comment><USER5> For a layman, Can you please explain why FPGA skills are good to have? Some years, I could understand that processors were not fast enough so FPGA parallelism was useful...I think modern processors have caught up.

<comment><USER14> FPGA cards are the easiest/cheapest way to get programs into the chip level. You can bypass OS level operations, memory/cache operations etc.

That said, you probably could build bespoke machines out of modern processors that perform better (and someone might be doing this), but it would be very expensive. FPGA's offer a middle ground between that and software.

</comment></comment></comment></comment></comment><comment><USER15> Why learn C++? It's used because of A) history or B) it's speed is useful in HFT that isn't UHFT yet with microwave links and ASICs.

I work in python. I'm staggered by the range and quality of the libraries. Numpy/Scipy + Pandas + ipython = instant productivity.

<comment><USER16> Yup -- this is the most common stack for quant nerdotry

</comment></comment><comment><USER17> I like the topic, but I have no idea where to start on these books. Can anyone recommend one or two sources that would be good entry points for learning this kind of stuff?

<comment><USER18> A good corporate finance book is going to give you the foundation of all this. If you read a corporate finance book and then start going through the book list, it will make a lot more sense. A lot of the books in his book list assume a foundation in graduate level understanding of finance.

One I would recommend is:

[http://www.amazon.com/Corporate-Finance-3rd- Pearson/dp/01329...](http://www.amazon.com/Corporate-Finance-3rd- Pearson/dp/0132992477/)

This will go into some level of detail into all areas.

Once you are comfortable with this material, you can branch into more deeper study in each area.

A lot of trading systems attempt to exploit arbitrage opportunities (however time limited) in the market place. A book like the above will give you a deeper understanding of the stock market and the different financial instruments that are used in the market.

It will also provide real insight into how companies manage their capital budgeting (I have $100 million dollars, how do I decide what to spend this money on this year). Not every company will follow the same approach, but the fundamental concept of net present value is very common.

Once you have a good understanding of the basics then you can delve further into derivatives. It would be hard to get into derivatives without a solid background in what a MBA/graduate level corporate finance book gives you.

As an aside, anyone in IT that wants to expand their business know-how would gain quite a bit by reading a corporate finance book. If you want to know what the executive team spends their time on, the knowledge around capital budgeting and understanding how projects are assessed (ie. using WACC and NPV) is very valuable.

<comment><USER19> Hmm, I think that's backwards. If you're going to work in corporate finance, then you need to know about capital markets, derivatives, etc. But if you're primarily interested in quantitative trading then you don't need to know all the stuff in that book.

E.g. there's no need to know all about leasing or taxes or mergers in order to price derivatives. In fact I doubt that most people working in the area know much about corporate finance, whether they are math and physics PhDs working as quant researchers or regular programmers implementing the models.

<comment><USER18> I agree that one doesn't need to know all that stuff in that book for this area. You are right that leasing, taxes, mergers, or corporate governance don't apply. Those chapters in any corporate finance book should be considered the sandwich for the meat. If you don't like those chapters, be a picky eater and just go for the meat and leave the sandwich alone.

But if one has no clue about how stock markets work, how risk is quantified, the intricacies of call and put options, the various types of bonds, etc. than I would suggest a corporate finance book is a good place to start.

Why? It starts slow and builds the concepts on top of each other. These books take a long time to put together and they are usually pedagogically well put together. Just for this along I would recommend such a book. The effort that has been put together to make sure that students have the best opportunity to easily pick up the knowledge is really high.

I especially like the above book as it builds the concept of the law of one price and arbitrage very well.

I have made the assumption that kevincrane is at this starting point.

The reality of all this is that this stuff is _hard_.

It's quite amazing much the study of finance is so complex. A corporate finance book let's you gaze into the window without actually stepping inside.

As I suggested, its a starting point.

In my opinion, if one can't get their head around the knowledge in a corporate finance book the rest of the material in the more technical focused books on that book list will not be very accessible. There is a reason why people in the field have Masters and PhD's in strong heavy math focused specializations.

</comment></comment></comment></comment><comment><USER20> This was asked on HN more than a year ago: <https://news.ycombinator.com/item?id=3177815>

</comment><comment><USER21> Don't forget the Coursera courses on finance, econometrics and financial engineering. They might not cover everything, but they should be enough for getting you interested or bored. I'm not in a Finance field, but I like these courses a lot.

</comment><comment><USER22> Could you point me to quant analyst job salary listing on glassdoor. I couldn't find what the job title typically is.

<comment><USER23> I know the "pull of the markets" is strong and my words will likely fall on deaf ears, but here it is:

If you actually qualify for a quant job in today's crappy markets (solid technical PhD is a start; keep in mind your bonus potential is about half of what it was 5 years ago for HFT), please do yourself a favor and pick an industry where you actually get to produce something. You're a decade too late.

<comment><USER24> "an industry where you actually get to produce something."

an industry where you could actually earn something would be much better!

<comment><USER23> so true. if i had to go back, i'd (gulp!) actually consider an IT role over trading.

</comment></comment></comment></comment><comment><USER25> Here's a list I wrote up for a friend when he asked for a few "non mathematically hardcore" finance books to read.

<http://brotchie.github.io/favourite-finance-books.html>

It overlaps somewhat with the OP's list, but I have a tiny bit of commentary on each book.

</comment><comment><USER26> Don't suppose anybody knows of any service that provides historical market pricing data available to download en masse? The only services I can find just allow you to view data for a specific company, whereas I'd like to just dump the data for as many companies as possible.

</comment><comment><USER27> see also:

[http://quant.stackexchange.com/questions/2391/what-books- sho...](http://quant.stackexchange.com/questions/2391/what-books-should-any- quantitative-portfolio-manager-or-risk-manager-have-as-ref) [http://quant.stackexchange.com/questions/431/video- lectures-...](http://quant.stackexchange.com/questions/431/video-lectures-and- presentations-on-quantitative-finance) [http://quant.stackexchange.com/questions/1985/what-papers- ha...](http://quant.stackexchange.com/questions/1985/what-papers-have- progressed-the-field-of-quantitative-finance-in-recent-years-p)

</comment><comment><USER5> Many people say its too late to enter HFT..Is it true for developed world or is it true for ALL countries? (India/China/etc). For example, I am not sure if Indian stock markets have so much of HFT.

</comment><comment><USER28> I've been wondering if Quant trading really works? I mean that are really someone who earns money with this? It's just a "gold rush"?

<comment><USER29> Absolutely it does. Just about every hedge fund, trader, proprietary trading firm, etc..., uses it. But it doesn't guarantee success. Your algorithms, programs and analysis have to be better than the next guy, otherwise you'll just lose money very quickly.

This is simply the evolution of trading, another tool to help traders. And someone will always make a ton of money, while someone else will always lose. That's how trading works.</comment></comment>


Ask HN: How to automatically monitor keywords (eg. Java, DBA...) on job sites? - <AUTHOR>

If I look through IT/programming jobs posted on a service such as s1jobs.com, I'll start to notice patterns like lots of companies looking for J2EE programmers, not so many looking for Haskell programmers, and so on. I want to quantify these patterns. How could I implement some sort of script to crawl part of the site daily, count the occurrences of repeated words (obvious shortcut would be to discard any word not beginning with a capital letter, and then any other words that are non-helpful like simply 'The' or 'A' or 'Candidates'), so that I can see what skills are most in demand? <comment><USER2> I think most will give you an alert.</comment>


Google Has Given Up On Social - <AUTHOR> http://www.businessinsider.com/gene-munster-google-has-given-up-on-social-2011-1

<comment><USER2> Google has most likely given up on replicating the social graph, since Facebook dominates. No doubt.

But what about _social search_?

I'm positive that Google has something up their sleeve.

</comment><comment><AUTHOR> "Facebook is Google five years ago, and Google is Microsoft."

Ouch.</comment>


Learning Spring with Kotlin – Sprint 0 - <AUTHOR> http://thatsabug.com/blog/spring_learning_week0/ <comment><USER2> I've been using spring and Kotlin together for nearly 3 years. Since then, a lot of Kotlin specifics have crept into Spring. Also, there's an effort to move to a fully declarative Kotlin DSL for driving spring (kofu), which gets rid of basically all anotations, AOP processing, and other magic that makes spring both nice and hard to debug. At this point, if you are using Spring and not using Kotlin: you are doing it wrong. You are missing out on a ton of stuff the Spring people did to support Kotlin. Yes Java is still supported but it is increasingly in the same way that Google hasn't completely deprecated Java for Android yet despite that obviously very much being a second class citizen in that ecosystem.

In short, if you are a conservative institution like a bank and evolving your tech stack at a glacial pace; yes you can still use Java. Everybody else, you probably should be using Kotlin.

My advice to anyone looking to do this is:

1) Do it, it's easy and you end up with better code. You can start simple and mix Java and Kotlin code. I would advice starting with some tests or some new features.

2) Beware of the java idioms and cruft in Spring and do things the Kotlin way. So, DSLs instead of annotations + AOP, co-routines instead of spring flux, immutable data classes instead of java beans, etc.

3) Spring has a lot of stuff and not all of it is great or even good. IMHO the added advantage of e.g. spring-cloud or spring-data is very limited and sometimes it's just very limiting/backwards. I ripped out spring-data on several projects where people naively supposed it was the easiest/best way to interact with Elasticsearch. It turns out it is neither. Spring cloud sounds nice until you realize most of the stuff you need is supported in the amazon driver but yet not exposed in spring-cloud. Also all the code samples on Amazon are for their own driver and not for spring-cloud.

4) Re-assess if you actually want to go anywhere near hibernate. IMHO it made a lot of sense when java beans were hard to write. Kotlin data classes take away most of that pain and there are some nice alternatives. Also, you should be using non blocking IO for talking to your DB and hibernate is still stuck on blocking IO. That in itself is a good reason to not use it on new projects.

Once you get a bit further, you'll find yourself wondering if ktor would work. The answer is probably yes. It's actually pretty good. Spring does a lot of things but ultimately do you really need all of that and the associated complexity?

<comment><USER3> Do you have any thoughts about Groovy?

<comment><USER4> I still go with Groovy for scripting and tests with Spock.

Scripting is better because Kotlin doesn't have simple dependency management like Grape and compilation speed is way faster than Kotlin's. One of Kotlin's downsides is compilation speed and kts (kotlin script) is even slower. Also, Groovy still has ton of utility methods in GDK not matched in kotlin stdlib.

Other than that, I'm sad to say, Groovy is dead or in process of dying. I'm seeing companies rewriting Grails apps to Spring Boot and switching to Kotlin dsl in Gradle. It's a shame really, such a fun language to program in.

<comment><USER3> I feel like the sweet spot is using groovy where it makes sense, like parsing unstructured xml and json, reading in files, pojos, and of course spock. I was never the biggest fan of grails.

I'll have to check out Kotlin though, seems like the new wave for sure when it comes to simple readable jvm code.

</comment></comment></comment></comment><comment><USER5> Some guy writes an article where he says he is thinking about learning Spring with Kotlin and this is worthy of the front page of Hacker News??</comment>


The World's Most Spoken Languages and Where They Are Spoken - <AUTHOR> http://www.iflscience.com/environment/worlds-most-spoken-languages-and-where-they-are-spoken

<comment><USER2> Love the diagram.

One comment; I have a hard time not questioning why "Chinese" is presented as a single block here; many languages that are included under "Chinese" are mutually unintelligible. Other related-but-mutually-unintelligible languages are listed separately, e.g. Italian & Spanish.

I'm guessing it's a political/cultural choice.</comment>


Marijuana Can Stop Alzheimer's Disease in It's Tracks - <AUTHOR> https://www.greenrushdaily.com/2016/06/17/marijuana-can-stop-alzheimers-disease/ <comment><USER2> Sensationalist title is sensationalist. The actual study announcement:

[http://www.j-alz.com/content/marijuana-compound-may-offer- tr...](http://www.j-alz.com/content/marijuana-compound-may-offer-treatment- alzheimers-disease-university-south-florida)

First, this is a cellular model. Results have not been demonstrated in actual trials.

Second, the result is predicated on the assumption that amyloid build-up is the cause of Alzheimer's, which itself is only an hypothesis at this time:

[https://en.wikipedia.org/wiki/Alzheimer%27s_disease#Amyloid_...](https://en.wikipedia.org/wiki/Alzheimer%27s_disease#Amyloid_hypothesis)

Third, the announcement itself indicates:

"While we are still far from a consensus, this study indicates that THC and THC-related compounds may be of therapeutic value in Alzheimer's disease,"

Which is far from stating that THC can stop the disease "in it's tracks". Furthermore, they caution:

"Are we advocating that people use illicit drugs to prevent the disease? No. It's important to keep in mind that just because a drug may be effective doesn't mean it can be safely used by anyone. However, these findings may lead to the development of related compounds that are safe, legal, and useful in the treatment of Alzheimer's disease."

As an aside: I'm all for marijuana decriminalization, but this tendency to overhype weed and claim it's a cure-all for, well, basically everything, actively reduces the legitimacy of the advocacy movement.

</comment><comment><USER3> ... but can it cure the apostrophe?

</comment><comment><USER4> *Its</comment>


Ask HN: Fastest language for iteration and arrays manipulation? - <AUTHOR> I currently have a microservice written in js which basically query a redis db and iterate few hundred or thousands times while geting/pushing values from/to arrays in multiple for loops.
It's made to build line of sight of players for a mmo game.
The service run as a loop, ideally at 60FPS (1 iteration = 16ms). More players in the world = more time to iterate.
I have the belief that writing in C++ the same would be faster but:
1. That's a belief, I didn't do any benchmark 2. I don't have the belief that C++ will be the fastest to do it.
What HN does think about that ? <comment><USER2>
        query a redis db

General rule: if you're looking for "fastest", pulling data over the network is not part of the solution.

As for the choice of language, it's going to be heavily motivated by what languages _you_ (or your team) know how to benchmark and optimize. It doesn't matter if, say, Befunge is the "fastest" if you don't have anybody working on it that can optimize it.

<comment><AUTHOR> Basically the network query is done before the computing tasks, the network remark is very true. I don't know cpp, only java php and js but I know I need to learn a new one for this task, I'm just wondering which one.

</comment></comment><comment><USER3> It doesn't have to be fastest, it just has to be faster than what you're using now. C++ compiler has a suite of automatic compiler optimizations that'll make your code performant. Things like loop unrolling, strength reduction, vectorization, cache optimization can make your code many times faster than what you can muster with js.

There's a reason why many AAA titles use C++. No benchmark necessary.

<comment><AUTHOR> Why I wouldn't need the fastest and only take a faster solution? You are answering to the question of C++ usage but do you know other languages that could do it faster? I know that c++ has dozens of years of optimizations and it is considered as a very powerful and fast language. Is it the best choice I can make? What about c, rust, java, etc...? Will they be slower that cpp?

<comment><USER3> Because language choice is more than just about whether it's the fastest for your specific concern at the moment; it may be slow or non-ideal in other situations. C++ in general is pretty fast. I may be biased since I used to write C/C++ optimizations for a living.

I can't speak for other languages; languages with JIT optimizations could in theory be faster than fully optimized C++ code, since it could take advantage of run-time information, such as hot paths and hot fields.

If you care about the fastest performance possible, you're asking the wrong question. I'd benchmark different languages for the specific code I want.

<comment><AUTHOR> Thanks for the answer, I understand better your point. What I understand from your messages is that C++ should be a good solution.

</comment></comment></comment></comment><comment><USER4> You should probably identify and quantify the bottlenecks first. My guess is this in an XY problem and you're IO bound or your architecture is doing something bad like a linear complexity algo calling a linear complexity algo in your iteration, and a 1:1 translation to C++ probably won't have much benefit.

<comment><AUTHOR> Actually the IO happen outside of the loops I'm trying to optimize, I shouldn't have mentioned it as they are not part of the problem.

</comment></comment><comment><USER5> Maybe Fortran?

<comment><AUTHOR> I was thinking about perfored cards too</comment></comment>


New "Windows" logo unveiled in Windows 8 Consumer Preview  - <AUTHOR> http://www.fudzilla.com/home/item/25907-new-windows-logo-unveiled-in-windows-8-consumer-preview?tmpl=component

<comment><USER2> Fake. The hardware shot came from a recent article by Paul Thurrott on the Windows start button: [http://www.winsupersite.com/article/windows8/windows-8-secre...](http://www.winsupersite.com/article/windows8/windows-8-secrets- windows-8-dropping-start-button-142236)

</comment><comment><USER3> Do they really have to include TM and R in the logo itself?

<comment><USER4> Absolutely not. But that's what happens when you take a design decision and "run it past legal". The lawyers will do their best to "add value" and "manage risk" etc. You can imagine how that would have flown with Steve Jobs.</comment></comment>


Ask HN: Any compiled list of plain text User-Agent? - <AUTHOR> From what I've seen it's a common practice to use HTTP User-Agent to differentiate plain text clients like curl from html clients like browsers, in order to properly serve readable text if user is using a plain text client. Is there a good list of all relevant plain text User-Agents? On the top of my head I can only think of curl. <comment><USER2> Wouldn't it be better to read the Accept header and look for text/plain?</comment>


SPAC Man Begins - <AUTHOR> https://alexdanco.com/2020/07/24/spac-man-begins/ <comment><USER2> There is an older and less flattering term for this. It's called a reverse merger. Typically, you start with a company that sold aluminum siding in the seventies and has no assets but happens to still be on a public Exchange.

They call this the vehicle. The rest works just like the article says, you merge the shell company in with the company that actually has value and avoid the IPO process.

As the article honestly points out, the whole point of this is to avoid the scrutiny that you typically get from an investment Bank. It also skirts the spirit of a number of regulations put in place to keep people from losing their retirement nest egg on snake oil.

</comment><comment><USER3> Does anyone know how one should go about learning about all these different financing methods / vehicles if I wanted to read about it by myself? And say for some reason I wanted to do an IPO/finance through a SPAC or something myself, where would I start and what would I need (assuming I don't want to hire some investment bank for me).

I used to work as a trader at an investment bank and I've always been curious about the corporate finance side of the business. I have a friend who worked in corp finance at a boutique IB and struck out by himself. He eventually ended up running his own boutique bank in China and deals almost exclusively with SPAC's by helping his Chinese clients list on Nasdaq or whatever. I was always curious how he did it and what one needs to do if he wanted to follow a similar path

</comment><comment><USER4> > _" While the normal IPO process starts with a private business who wants to > go public, a SPAC is the opposite: it starts with a public business, that is > nothing.

A SPAC begins its life when a well-known promoter, like Chamath or Bill Ackman, raises money in an IPO with the following prospectus:

"I am raising money to take a company public. I don't know which one yet; the money is going to go in a bag until I find one. When I do, my publicly traded bag of money will merge with the private business, and in doing so, take them public. You, the investors, will have your SPAC shares convert to shares in the new business, at an attractively negotiated price, plus warrants to buy more so you can profit on the upside.

Because I'm so smart, I'm going to pick a business that's amazing and that you'll want to own. If you disagree, you'll have a chance to get your money back. But if I'm right, you'll make a ton on the upside.""_

An interesting sales proposition, to be sure...

</comment><comment><USER5> The IPO pop is definitely giving away value to a bunch of well connected investors who have added no value. Facebook solved it by pricing high and using the greenshoe. Most companies don't seem to have enough leverage to get that and even with Facebook there were plenty of influential people berating them about it. But Google's solution of a dutch auction IPO seems like a more natural solution. To manage the discontinuity of the moment the company starts trading set up the trading book early with the auction and then start trading at a well discovered price. Why hasn't that become standard? Just the gatekeeping of the banks again? Seems like the natural thing to offer next for one of the companies that does markets for still private startups.

</comment><comment><USER6> I'm missing what is good for a company when they go through a SPAC the 20% fee is close to what the average POP for an IPO which historically has been 13-15% and last year was 22%[1]and in exchange for dealing with a bunch of investment bankers you get to give up a board seat at best or being the CEO of your company to the promoter. Is it really just to dodge regulations?

[1][https://www.cnbc.com/2019/05/09/the-ipo-market-started- stron...](https://www.cnbc.com/2019/05/09/the-ipo-market-started-strong-and- could-soon-take-off.html)

<comment><USER7> The general tradeoff here is:

An IPO is a difficult, involved process. You jump through a lot of hoops, you have to write a bunch of documents, you do a road show, and at the end of it you don't even know how much money you're going to raise. As with WeWork, it's very possible the IPO may be derailed before it gets to the end, or the price will be cut in order to get it done (meaning you don't raise what you expected). Plus, the price may drop after the IPO (meaning many of your new investors are mad and you get bad press), or it pops a bunch after the IPO (meaning many of your old investors are mad and you get bad press). It's not a great process! On the other hand, you'll probably get something a bit _like_ a market price - as you note, IPO prices tend to be within 15-20% of the eventual market price.

A SPAC is different. You talk to one guy (or team, at any rate), you negotiate a deal for a fixed price, you sign, it's done. Sure, it's very, very expensive, because you're paying to get rid of that risk, but maybe you really hate risk?

(Note: the 20% fee is only one part of the cost of the SPAC. And there's still a chance of a pop, as with Nikola who surged enormously. Much of the money they raised was at $10 per share, but they closed at $33.97 on the first day, and $53.95 a month later. Good deal for some...but not for Nikola or their original investors.)

</comment></comment><comment><USER8> Obligatory Matt Levine take[1]

> I just want to spell it out a bit here because, again, the common story > these days is that a SPAC is a substitute for an IPO that is cheaper and > avoids mispricing. I have argued before that that is wrong, but in > particular here I want to point out that the SPAC is (sometimes) a marriage > and an IPO generally isn't.

> An IPO is, you sell stock to some people who want to buy it, and in general > you hope that they'll be good stable long-term investors, but you don't lock > them up forever. They get to vote for directors, usually, but the big IPO > investors don't generally get their own representatives on your board. And > in particular, the trend in tech unicorn IPOs in recent years has been to > minimize the power and input of public investors. Lots of tech unicorns go > public with dual-class stock so that investors can't really vote for > directors; the founders' goal in going public is to raise money (or at least > get liquidity for early investors) but keep control of their companies for > themselves. They are not looking for a marriage partner; they are looking > for a transaction, for money, while keeping their independence.

> Which is fine too! Some companies stay independent, some merge, some want > something in between, a powerful outside shareholder who can give them > advice and support. My point here is just that if you are a tech unicorn > founder, and your goals are (1) to go public without giving up any control > to outside public shareholders and (2) to go public without going through > the vagaries of the IPO process, you may have to make some tradeoffs.

[1] [https://www.bloomberg.com/opinion/articles/2020-07-24/goldma...](https://www.bloomberg.com/opinion/articles/2020-07-24/goldman- does-a-malaysia-deal)

</comment><comment><USER9> Spac is slang for stupid in England. Might want to reconsider the name.

<comment><USER10> It's worse than that, it's a nasty insult for disabled people from "spastic" (also used: "spaz" which may be familiar to US readers and is much more derogatory in the UK).

It's got _terrible_ connotations in the UK.

[https://www.wikipedia.org/wiki/Spastic](https://www.wikipedia.org/wiki/Spastic)

<comment><USER11> I mean, we have _git_ so I think we'll survive.

<comment><USER10> "Git" is not even remotely close to the nastiness of the words here.

</comment></comment></comment></comment><comment><USER12> So basically this article describes how not to give Wall Street crooks your money and instead keep more money for Silicon Valley crooks. Whats up with this "system" talk? Jeez I think SV types on the west coast need to grow up a bit. He even admits trying to SPAC a crypto company. That is more system than the system itself.

<comment><USER7> Not even that; the author didn't think through the math right. A SPAC is much _more_ expensive than an IPO:

1\. Sponsor takes 20% 2\. Warrants give investors the right to buy buy shares cheaply 3\. You still get the IPO pop (Nikola and DraftKings both had enormous pops after becoming public)

<comment><USER13> When you IPO you sell a (usually) significant percentage (generally from 20% to 80%) of your company for an opening price - you don't get to capitalize on the hoped-for increase except with the percentage you didn't sell, either personally or corporately, as there are usually many limitations to discourage loss of the established personnel to sudden wealth syndrome. In the SPAC (and similar) models, you "sell" 20% of your company for a fixed amount (in exchange for the SPAC's cash). The remainder is yours to nurture as needed. With a little planning you avoid a lot of work generally irrelevant to your core business by finding an available company to be acquired by and who is by experience a good partner for publicly traded relationships.</comment></comment></comment>


PG&E Sparked at Least 1,500 California Fires, Now Faces Collapse - <AUTHOR> https://www.wsj.com/articles/pg-e-sparked-at-least-1-500-california-fires-now-the-utility-faces-collapse-11547410768 <comment><USER3> In 2017, PG&E was fined for a fire in 2015. In 2017, they were implicated in 17 wildfires that killed dozens. All on her watch. The CEO does nothing to address the risk. Now she resigns, and:

 _PG &E is planning to award her a perhaps less-than-golden parachute that could range from $2.36 million to $4.46 million, depending on how her departure is categorized_

Step 1: devastate the company you're in charge of.

Step 2: collect millions.

[https://www.bloomberg.com/news/articles/2019-01-14/pg-e-s- fo...](https://www.bloomberg.com/news/articles/2019-01-14/pg-e-s-former-ceo- to-walk-away-with-millions-after-wildfires)

<comment><USER4> Knew someone who worked at PG&E. Her department's task was to support calculation of whether it was financially optimal to obey the law or risk paying fines, across PG&E's service footprint.

<comment><USER5> A new car built by my company leaves somewhere traveling at 60 mph. The rear differential locks up. The car crashes and burns with everyone trapped inside. Now, should we initiate a recall? Take the number of vehicles in the field, A, multiply by the probable rate of failure, B, multiply by the average out-of- court settlement, C. A times B times C equals X. If X is less than the cost of a recall, we don't do one.

<comment><USER6> Great scene - sad, but great

</comment></comment></comment></comment><comment><USER7> Privatization of public services cannot work unless the private companies have liability for their actions. This is the necessary consequence of their negligence.

<comment><USER8> > California law compels utilities to pay for damages from wildfires if their > equipment caused the blazes — even if the utilities weren't negligent > through, say, inadequate maintenance.

This would have happened whether or not PG&E was private. People want a scapegoat and PG&E is California's.

<comment><USER9> Nope, not a scapegoat. PG&E has a long history of negligence, poor planning, and straight up lying to consumers. I don't feel the same way about SoCal Edison, whom I only switched away from last month. They have their problems, sure, but it's nowhere near the level of malicious incompetence coming from PG&E for the past 20 years.

There are tons of news stories on PG&E if you look, many unrelated to the fires. They spent millions to lie to voters when Yolo County considered switching to SMUD. They routinely backdated maintenance reports to the government. They've had explosions in gas lines, and then there's the Diablo Canyon mess.

<comment><USER10> Non-Californian; from here, it looks like it would have burned sooner or later, thanks to changing climate & history of forest management practices. Whether lightning strike, hobo camp, stupid teenager, OHV, train, it was a tinderbox waiting to go. Is that an inaccurate perception?

<comment><USER11> I am living in California (but also consider myself non-Californian) near one of the areas that burned, and regardless of PG&E's incompetence, empirically the area was a tinderbox. The Carr fire was caused by a spark from a flat tire of an RV which caused the rim to create sparks against the road. Despite the forest service actively burning underbrush regularly, the fire was the largest in California history until the Mendocino fire unseated it a few weeks later. If sparks from a wheel rim can touch off a blaze, there isn't much margin for error.

<comment> j-c-hewitt In situations like this I think we would have to ask that considering that CA in general has high fire risk compared to other regions of the country, are the right people and entities bearing the risk and shouldering the costs of development there appropriately? I should think a lot of people have a vested interest in blaming PG&E, but if a completely ordinary event like a spark from a flat tire can cause a massive fire in the area that impacts a lot of human settlement directly, then someone is not assessing the risk of building and living there appropriately.

<comment><USER12> >then someone is not assessing the risk of building and living there appropriately.

This is the answer no one wants to hear.

For example almost 10 years ago Paradise almost burned. There was a grand jury review afterwards(that you can find online if you're willing to search some) that pointed out that the place was a tinderbox deathtrap and major changes needed to be implemented for safety.

The problem is populations in these places, along with most humans in general, like trees and shade and the idyllic beauty of nature. People would rather have a pine tree and bushes around their house, rather than a 100 foot cleared defensive parameter.

<comment><USER13> You are also leaving out environmentalists who fight against thinning and cutting down trees — source, the Angora fire in South Lake Tahoe, Canyon Creek complex in eastern Oregon, and many others. Playing politics and not acknowledging environmental lobby will not do anyone any good. If you know any actual foresters in the USFS, they will tell you as well because they are suid by environmentalists for stuff as simple as salvage logging.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER14> To what extent could PG&E mitigate these problems with improved safely devices? There are various devices that can try to minimize arcing when there is a fault, for example Petersen coils. [0]

California already has a distinct advantage here compared to the rest of the country. In the other 49 states (to the best of my knowledge), it's legal for the utilities to use a conductor that is grounded at multiple locations as a neutral. This type of arrangement induces nasty currents through the ground (which is, I think, why it's illegal in CA -- those currents can apparently be quite harmful to livestock that are being milked with metallic equipment, not to mention hazardous to people in swimming pools and such). But it also means that, if another phase shorts to ground, then there is necessarily a large amount of fault current available.

In CA, the distribution wires can't legally have any intentional current flowing between themselves and ground, which means circuits that detect residual current will work.

[0] [https://www.modernpowersystems.com/features/featurenew- roles...](https://www.modernpowersystems.com/features/featurenew-roles-for- the-recloser-in-the-modern-mv-network-4928052/)

<comment><USER15> The US uses overhead wires everywhere, because it's cheaper to install than underground wires. But then you're subject to damage from trees, wind, icing, etc. Everyone pays in worse uptime, and apparently also in fire risk.

<comment><USER16> No, not everywhere. Most of NYC uses underground wires. My neighborhood in Colorado near Boulder built in the 60s uses underground wires as well.

<comment><USER17> Underground wiring is far more efficient in big cities, as the major cost part is "miles of trench digged" and in a city the ratio of "customers per mile" is waaaay higer than in rural areas, thus reducing "investment per customer".

In addition this wiring is at the maximum 110 kV, you do NOT want a 380 kV feed in a major city. These things require a shitload of clearance, talk about 40m wide and 4m deep trenches!

</comment></comment></comment></comment><comment><USER18> I don't like PG&E, but if you have a gas leak in your house, and a guy walks down the street in front smoking a cigarette and everything blows up, does the smoker buy you a new house? The natural environment in parts of California has reached an inherently unstable state, with no logging, no prescribed burns, extensive suppression, global warming, draught, and limited brush clearing.

<comment><USER19> _I don 't like PG&E, but if you have a gas leak in your house, and a guy walks down the street in front smoking a cigarette and everything blows up, does the smoker buy you a new house?_

"Most ironic analogy ever"? You know PG&E is a gas company? Not only that, PG&E has killed people with gas line leaks?

[https://en.wikipedia.org/wiki/San_Bruno_pipeline_explosion](https://en.wikipedia.org/wiki/San_Bruno_pipeline_explosion)

<comment><USER20> this is a whataboutit response, and doesnt really engage with the users comment.

<comment><USER19> The users' comment was about multiple liability but his example strongly implied the gas leaker and not the smoker should take the blames. PG&E is only partly for all of the terrible things that have happened under their watch but liable enough that they deserve harsh penalties. I think it is telling when someone groping for a defense by analogy can't help but stumble onto an analogy that says the opposite of what's intended. It can be symptom that one's overall argument is quite thin.

<comment><USER18> As I said, I don't like PG&E. High rates, San Bruno, etc, etc. The problem is that a lot of the forests in California are just going to burn, whether it's tomorrow or in five years, because they are tinderboxes. Maybe any individual ones will be caused by poorly maintained electrical lines, or a careless campfire, or a cigarette or lightning, but given enough time, there will be a spark. In much of California, wildfire is part of the natural cycle. A lot of of California is dry, so downfall wood doesn't rot. The only way that energy gets released is for it to burn. Because we use forest management practices developed where downfall wood does rot, and because politicians don't like to tell people that their cozy cabin in the woods needs to burn down, we've backed ourselves into an ugly corner with way too much fuel on the forest floor. You can blame PG&E, but if it wasn't them with the ignition source it would have been someone else.

</comment></comment></comment></comment></comment><comment><USER21> Is there any part of California that runs power underground rather than overhead cables? I understand it's a significant cost and effort (sometimes infeasible) but at this point in time, just based on the past few years, I'm surprised not to have heard of any large scale pushes for underground power.

<comment><USER22> Nah, we just string the cables haphazardly wherever, through trees is just fine, because of course it's always sunny and nice in California. It seriously looks like third world infrastructure to someone from another part of the country, where weather events happen often enough that the utility has to give a shit.

<comment><USER23> PG&E can't afford to do that without raising capital. CPUC won't approve rate hikes to finance that capital. Legislators pass no new laws or regulations.

Then when it goes awry CPUC and legislators stand far back and decry PG&E for negligence.

<comment><USER24> They could afford to pay out a regular dividend, illegally skipped safety measures on pipelines, and "an independent audit from the State of California issued a report stating that PG&E had illegally diverted over $100 million from a fund used for safety operations, and instead used it for executive compensation and bonuses". Ratepayers and taxpayers should never have to pay for deliberate executive negligence.

[https://www.latimes.com/business/la-fi-pge-safety- investigat...](https://www.latimes.com/business/la-fi-pge-safety- investigation-20181214-story.html)

</comment></comment></comment></comment><comment><USER25> Same story from the Associated Press: Utility seeks bankruptcy protection over California fires - [https://apnews.com/5e70c4c99a044c4192d7ee52e38330f2](https://apnews.com/5e70c4c99a044c4192d7ee52e38330f2) .

</comment><comment><USER26> A preview of some of the economic toll of climate change

<comment><USER27> PG&E has been obviously negligent to any casual observer for at least the last 40 years. But yes, there is more to come.

<comment><USER28> \- Found negligent in the San Bruno gas pipeline explosion

\- Found negligent in the pollution in SoCal Hinkley groundwater pollution

</comment></comment></comment><comment><USER29> Ohhh, will Julia Roberts be in the movie adaptation of this _latest_ PG&E debacle?[0] She was so good in the last one.

[0][https://en.wikipedia.org/wiki/Erin_Brockovich_(film)](https://en.wikipedia.org/wiki/Erin_Brockovich_\(film\))

<comment><USER30> A tweet from the real Erin Brockovich was the top Google Tweet result for [pg&e] this morning:

[https://twitter.com/ErinBrockovich/status/108483037975907533...](https://twitter.com/ErinBrockovich/status/1084830379759075330)

</comment></comment><comment><USER31> [https://outline.com/5bxFLb](https://outline.com/5bxFLb)

Whole article

</comment><comment><USER32> Is municipal red tape at all a factor in PG&E's under-maintenance of its lines and equipment? I feel like CEQA lawsuits and municipal planning codes and committees are given as reasons why we don't have denser housing and more fiber-to-the-home and dedicated bus lanes in the bay area (the Van Ness CEQA report took something like six years to do). Do the engineers for AT&T and Comcast and PG&E find similar frustrations with the process here?

<comment><USER33> nice try -- cost-cutting by using sub-contractors to do legally mandated maintenance up and down northern California was well-documented. In addition, PG&E was found guilty of of obfuscating and avoiding legally required maintenance records for gas pipelines in the San Bruno case. Utterly unacceptable and sadly predictable.

<comment><USER34> "cost-cutting by using sub-contractors to do legally mandated maintenance up and down northern California was well-documented."

Contractors are licensed by the state to operate in their respective trade. Making the claim that contractors do sub par work is just as much an indictment of the state for licensing them in the first place.

</comment></comment></comment><comment><USER20> Consider the following:

 _" PG&E didn't anticipate how quickly the drought would overtake heavily wooded areas north of San Francisco and outside Sacramento, said Stephen Tankersley, who oversaw PG&E's vegetation-management program between 1999 and 2015. "It's hard to believe that anybody would have predicted that it would have been like this," said Mr. Tankersley, now a utilities consultant. "I've never seen anything like it."

Conditions on the ground worsened dramatically and quickly, said PG&E spokeswoman Lynsey Paulo. She said the utility has reacted with speed and urgency. "We are very aware of the risk and we are doing everything we can to keep our customers and the communities we serve safe," she said. "PG&E considers wildfire risk as a top-tier enterprise risk. It is evident in our actions."

The utility removed 451,000 more trees from 2016 through 2018 than it had originally forecast, she said, in an "amped up" effort to deal with massive tree mortality."_

...

 _" The task was complicated because some dangerous trees were on private land, forcing PG&E to negotiate with landowners, said Bob Fratini, a retired PG&E vegetation-management manager. Residents sometimes pressured crews, he said, to trim just enough to satisfy minimum requirements.

"Utilities should be given the right to remove any tree that could cause an outage or a fire," he said. California regulators recently gave utilities more latitude in this area, saying they could shut off power to homes or businesses that prevent tree crews from working. PG&E's Ms. Paulo said that isn't necessary very often.

Sometimes, PG&E's tree-clearing created new problems. After PG&E workers removed two trees in January 2015 southeast of Sacramento, a gray pine was exposed to wind and began leaning, according to a state investigation.

On a 102-degree day in September, the pine hit a 12,000-volt line and electricity ignited it, dropping embers onto dry grass and sparking the Butte Fire, which burned 70,868 acres and 921 buildings. Two people died."_

These two blocks of text to me make it sound more like what you'd traditionally call an 'act of god', and not malfeasance (unlike San Bruno). While its unfortunate people died, and that there were great losses, you when you have people living in land that historically (over tens of thousands of years) had regular burn cycles, then humans came along and kept putting the fires out, I have less sympathy for those who take a loss - they should have known it was a risk of living there.

Also, I work with power utilities, having to turn the power off in a windy situation is an unwinnable situation, you either get complaints to the PUC, or you get higher fire risk - both are expensive to an extent to deal with - <USER2> without a historical record of fires in that area, its not really rational to just turn the power off, to tens of thousands of customers, and its harder to justify to your customers on its face - just as they are being excoriated now a failure to exert enough caution, if they had exerted the caution and nothing had happened, they would have been excoriated for exerting too much caution, again - its an unwinnable situation.

<comment><USER35> >its an unwinnable situation

It becomes a bit of "boy who cried wolf" situation. In the mid-west/Tornado alley, a similar situation occurs. Weather forecast tech has become reliable enough to detect tornado conditions, and issue warning appropriately. However, these conditions do not always lead to fully developed tornadoes even though the warnings have already gone out. This leads to a lot of people ignoring the warnings and even getting annoyed at the interruptions to regularly scheduled programming.

</comment></comment><comment><USER36> Why is something as vital as ENERGY managed by a private company with profit motives?

Not trying to be NATIONALIZE EVERYTHING or whatever, but we decided its worthwhile for water... why not power?

<comment><USER37> Energy used to be fully regulated for a long time. All in all, they did a good job while getting a guaranteed return. Deregulation didn't really deregulate everything (the industry is still super regulated), but it brought in competition and more market based mechanisms. Stoft's power system economics textbook covers this subject pretty well. Deregulation helps incentivize innovation, but it isn't a slam dunk. We have a different system now that is definitely better in some ways and worse off in others. I doubt wildfire safety would have been radically different prior to deregulation.

</comment></comment><comment><USER38> I wonder if the cost of providing energy with so much liability will force California to solar e.g. individual homes and businesses providing their own power?

<comment><USER39> That's actually part of the problem. Profits have been going down for years because their most profitable consumers (rich people with big houses) have been going solar. They don't adequately separate the cost of infrastructure from the cost of energy, so they've had less for infra.

Also we just passed a law to make this even worse by requiring solar on new homes.

</comment></comment><comment><USER40> $10,000 bet not a single Exec will go to jail.

</comment><comment><USER41> Every year, they cause a fire or other natural disaster, then threaten bankruptcy, then get bailed out.

<comment><USER42> Capitalism for the poor. Socialism for the rich.

If they continue to get bailed out I would prefer that we just have state run power or something more heavily regulated and monitored.

<comment><USER43> Socialize the losses and privatize the gains, but only for those who are wealthy and in power. That is our current system in America, too big to fail.

</comment></comment></comment><comment><USER44> On a somewhat similar note, president Trump recently threatened to cut off funding for wildfires[0].

[0] [https://www.cnbc.com/2019/01/09/trump-threatens-to-cut- off-f...](https://www.cnbc.com/2019/01/09/trump-threatens-to-cut-off-federal- funding-for-california-wildfire-relief.html)

<comment><USER45> How many times does California need help until it, as its own entity, fixes the problem?

Hopefully this lights a fire under the regulators in Sacramento.

<comment><USER35> How many times would Florida, Louisiana, etc need help until they fix the hurricane problem? There's definitely situations that people create that makes them much more susceptible to natural disasters, and I would go along with the "fix the problem" approach if this is the point you are making.

<comment><USER46> Obviously they just need to establish a hurricane fighter force. I wonder why nobody has ever considered this before /s

</comment></comment></comment></comment><comment><USER47> How are Californians going to be better off? When they go out of business, some other entity will pick up the equipment and staff.

Probably a lot of institutional memory will carry over.

<comment><USER48> They don't murder the employees when the company goes bankrupt, you know.

<comment><USER47> Which is the whole point of my remark.

Why do people think it's going to help them to have essentially the same co running their power again?</comment></comment></comment>


A Python library to import contacts from Gmail, Yahoo and Live accounts - <AUTHOR> https://github.com/mengu/contact_importer <comment><USER2> This is awesome. Been procrastinating to add this functionality!

</comment><comment><USER3> We've been using Cloudsponge for quite a while now - I'll be looking into this.

Out of curiosity, do the providers change their login/api mechanisms regularly, or are they pretty static? I'm just wondering how quick something like this would get fixed in the event that 'something' changed, if that's a possibility.

</comment><comment><USER4> Forked already and experimenting currently. It is amazing how you get so much more in python community.</comment>