Why We Need Dynamic Secrets - <AUTHOR> https://www.hashicorp.com/blog/why-we-need-dynamic-secrets <comment><USER2> Secret management can be a really infuriating problem. No matter what you do there never seems to be a good solution between dealing with employees who know passwords leaving, automated processes needing credentials and other business processes. It's one of the toughest problems I have ever seen without a good solution. At least not one that the business people will accept without complaining that it's too expensive or burdensome.

<comment><USER3> Secret management tends to suffer from the logical holders of secrets (managers) not actually having the time or expertise in a lot of cases to deal with the implementation (or even understanding) of the process.

<comment><USER4> This. The lack of focus faced today from key custodians and the lack of undergoing continuous training to understand the different threats and keeping up-to-date with implementations is a gaping hole in this particular type of model. It's never a core competency, which is why the industry needs a different paradigm.

<comment><USER5> As in, "Job title: The Holder Of Keys"?

</comment></comment></comment></comment><comment><USER6> This is all cool, but... once you get to a setup like this (which has many moving parts and config rewriting), why isn't that solution compared to kerberos? You get the temporary credentials giving you similar possibilities.

<comment><USER7> Kerberos was a major inspiration for us!

The goal of Vault was to be a modern Kerberos, but invert the integration model. I think that is the Achilles heel of Kerberos, since it has a complex API and only works if the endpoint systems are tightly integrated.

Vault operates in much the same way, and could be viewed as a "KDC". However, instead of requiring the Authentication Service (AS) to be Vault aware, Vault uses authentication plugins to do the integration in the other direction. Similarly, instead of network services being Vault aware, Vault uses secret plugins to do the integration with endpoint systems.

This lets Vault easily be extended to support new authentication systems and endpoint systems, without needing those systems to be modified. Otherwise, its conceptually very similar to Kerberos!

Edit: I'm a co-founder of HashiCorp, and one of the early authors on Vault.

</comment></comment><comment><USER4> This is a great first step on building the right kind of infrastructure to stop applications from exposing secrets. Leases on secrets with dynamic policy enforcement is, IMHO, the future of application security. Happy to see companies like Hashicorp discussing more of the vision on how Vault can be used to secure applications that we depend on for infrastructure.

I would caveat, however, that encryption is just one part of the tool in the toolbox. A correct security posture is about having layers of defense. The next layer is absolutely the application and isolating secret management responsibility from the app layer to the infrastructure layer is the right strategy, IMO.

</comment><comment><USER8> I'm still stuck on "Yeah, but I need a credential to authenticate with Vault to get the other credential that I want".

Very chicken and egg.

<comment><USER9> That was my first thought, too.

"But... there's a root account... creating another account... which is typically a privileged action. What's protecting that? Is that root account being rotated, too?"

<comment><USER10> the master key is split into shards. See under "Why" here: [https://www.vaultproject.io/docs/concepts/seal.html](https://www.vaultproject.io/docs/concepts/seal.html)

<comment><USER9> Sure, the master key is split into shards...

I'm talking about the account creating the database user. Let's take MSSQL, for example. The equivalent to a root account there is `sa`. So, Vault will have control of the `sa` account in order to create leased database users.

If I'm a malicious actor inside the environment, what's stopping me from compromising the `sa` account and mimicking dynamic secrets? I'd need to be comparing Vault logs with Database logs constantly to ensure it was legit.

It's just madness, if you ask me.

<comment><USER11> In some cases, vault can change the password of the root account on the service, so that only vault is known to have the password. But you are correct, there is a problem here. There is nothing stopping you from attacking the DB or service directly.

You can do a lot to minimize that, many DB auth systems let you specify where connections for users are allowed to come from, only allow a single concurrent connection, etc. Plus you can encrypt and verify that connection with TLS, build up FW rules, etc.

Security is never "perfect" but to say dynamic secrets are wrong and crazy just because of this one problem isn't very well thought out. Your wordpress application will be hacked a LOT easier than your MSSQL sa account.

So dynamic secret people are saying, we all know our apps leak/get compromised all the time, minimize their leakage with a short lived, one-time use secret, that can only be used from X machine, etc, instead of a long-term, static secret that can be used anywhere, etc.

For sure, for high-value targets you best be auditing everything you care about, not just vault, but with a vault audit log, you can automate a lot of that comparison between MSSQL and vault, where before it was much harder to automate since the logs from every application using MSSQL will be different, etc. But if you are in a high-value org like this, you are almost certainly already auditing your MSSQL account creations, this doesn't change that.

</comment></comment></comment></comment></comment><comment><USER12> This is awesome. Somewhat reminds me of Short-Lived SSL Certificates (e.g., [https://medium.facilelogin.com/short-lived-certificates- netf...](https://medium.facilelogin.com/short-lived-certificates-netflix- fd5f3ae5bc9))

</comment><comment><USER13> Vault has always appeared to me to be a great technology for a larger tech org looking to implement more granular access control and increased auditability.

At a smaller scale, we've been satisfied with encrypting secrets using KMS and then placing the resultant ciphertext into environment variables that we commit with our terraform scripts. Our system does not allow granular access control, but it is relatively simple to implement using a couple of Bash scripts and allows committing all of the config, including secrets, all at once so that deploys are fully reproducible.

To allow a bit more granularity in isolating different environments, we use different encryption keys depending on whether we have a dev/uat/prod deployment. We grant the target application role access to particular secrets based on which KMS keys it can use.

The other trick is that we decrypt all of the secrets in our environment at the entry point of our application code, so that the environment is fully decrypted by the time the service runs. This means there is no trace of secret management in our application code.

There are intermediate steps of this scheme where we could revoke access to KMS keys for some developers, so that we could allow users to deploy services but not necessarily be able to use and access the encrypted secrets.

Regardless, I always appreciate knowing these kinds of technologies exist because they seem to me to solve very challenging problems at scale.

</comment><comment><USER14> So this is like authentication tokens that pretty much any developer who's had to use a web API probably knows already ? It definitely is welcome in a more "traditional" setup.

</comment><comment><USER15> IME, dynamic secrets don't scale well. First, you're going to have a tough challenge getting a large org to stop sharing secrets amongst apps/hosts. Tougher still is providing a justification for why host1 for Salesforce needs a unique credential from host2 for Salesforce that is doing the same thing. Second, ensuring privilege creep doesn't occur becomes nightmarish when you consider an environment today that has 500k secrets and having to audit the authorizations an ID was granted in multiple backend authentication systems after that ID has been deleted. Finally, generating dynamic ids for an environment like this is a staggering amount of compute workload. Taking that same 500k environment and breaking these out to unique credentials dynamically generated puts you into 7 figure numbers pretty quick. Even if we ignore the complexity of such an arrangement, just consider what the performance demands are going to be to generate an id, assign it privileges and a password, provide that to a client, then clean the same id up later. We'll ignore how this looks in an environment where you have to replicate the identity that was created across multiple authenticating backends. (think Active Directory domain replication)

In short, I think dynamic secrets will increase the threat surface dramatically, while simultaneously increasing complexity. Complex systems are expensive to maintain, hard to secure and notoriously fragile. There is a limited scope use case where dynamic secrets may be a good fit, but I for one wouldn't base a purchasing decision on whether or not my secret store can do dynamic secrets. Instead, I'd want a secret store that has some provision for secrets rotation that I can tailor to the needs of my specific use cases.

<comment><USER7> We work with many Fortune 2000 customers, and having 500K secrets is on the extreme side and most certainly puts you in an infrastructure where you have 50K-100K+ machines under management.

In terms of the "compute cost", for an infrastructure of that size this is a negligible amount of overhead. For dynamic secrets that live 30 days, rotating 500K secrets works out to 1 secret every 5 seconds.

The advantage would be avoiding an incredible number of static credentials sprawled across a very large estate, plus having a unique audit trail that lets you identify points of compromise. Treating those credentials as dynamic will also reduce the human overhead of managing so many credentials, instead focusing on roles and high level intents.

I question if there is an non-disclosed bias given the anonymous user, created just in advance of the comment.

<comment><USER15> How is a dynamic secret that has a TTL of 30 days avoiding static credentials sprawled across the estate? It's static (for 30 days) and the only difference I see between rotating the password of an ID every 30 days and a dynamic secret with a TTL of 30 days is that when I cycle the dynamic secret I have to reassign entitlements to the new user ID and correlate this new user ID with my monitoring systems and Vault audit data. I disagree that the compute cost to doing all of that is a negligible amount of overhead.

Like I said, I think there is a use case here for dynamic secrets, but I have questions about what it looks like when it comes to trying to do them at scale. If you have solutions to the worries I outlined, I'd love to hear them.

</comment></comment></comment><comment><USER16> This sounds an awful lot like .. Kerberos?

</comment><comment><USER17> Here's what I see happen with Vault in actual real world usage. Applications use the token auth method with extremely long lived tokens, and then the auth tokens are stored in source or in config files or whatever. It just moves the problem and makes it look better because they aren't called "passwords"

<comment><USER11> This is absolutely not how _WE_ use it, but I'm sure lots of people do stupid stuff like that.

We actually deploy vault tokens to every user(happens at login time, since login authentication happens against vault). So we can deploy secrets directly to end users using vault policy.

Applications that are not ran/started directly by the user find other means of getting their secret 0(VAULT_TOKEN), usually through nomad, but in the rare case it's a standalone application not run under nomad, we seed the startup with a token by hand, and then have a process handle renewals, etc.

</comment></comment><comment> t-moe Funny how hashicorp promotes security products on one hand and fails to implement proper security in their remaining products on the other hand.

For instance: Vagrant Images/boxes hosted by hashicorp (vagrantcloud) are neither signed nor is there any author information available (if not explicitly provided by the uploader).

<comment><USER18> Also, Terraform stores "all settings, including usernames, passwords, port numbers and literally everything else" in the tfstate file (1).

I believe that using Dynamic Secrets is HashiCorp's proposal for how to mitigate this; leave the secrets in the log files, but make sure they expire in a timely manner.

[1] [https://tosbourn.com/hiding-secrets- terraform/](https://tosbourn.com/hiding-secrets-terraform/)

<comment><USER19> Yea, I wish they had thought of sensitive stuff in the state files from the beginning.

</comment></comment></comment><comment><USER20> How does an application (say, a RESTful API server) deal with dynamic secrets? I suppose it could be simply restarted when a secret changes (assuming it is requesting the secret from Vault at startup). Is that how it is typically handled?

<comment><USER7> That is exactly right. At the bottom of the blog post we touch on this, but if you are using consul-template to provide secrets via a configuration file, it can either restart or reload (signal) the application to pickup the changes. Alternatively, an application could be Vault aware and use the SDK programmatically.

</comment></comment><comment><USER9> My concern is how you can tell me whether a database user was created by Vault and not by a malicious actor with knowledge of Vault's Dynamic Secrets method mimicking it.

How the heck can I know if it was a legit created database user or not?

<comment> FLUX-YOU I assume you can check Vault's audit logs and see if the user was created there or not.

<comment><USER15> I have 500k ids today. Now you want me to make those ids dynamic and correlate how much Splunk data with Vault audit data? And what is my pattern matching regex going to look like when the ids I'm trying to match on are randomly generated? And how do I pick out anomalous behavior from the noise I just intentionally created because Terraform can't stop leaking my secrets? And what about the performance? How does Vault scale to generate that many identities? And how do I audit my authorizations since on Vault I'd just see what groups the IDs were added to but not the groups those groups belonged to? What about replicating my authentication backends? Active Directory replication takes minutes to replicate a password in some environments, it's going to take longer to replicate a new identity and its group memberships. And while I can revoke an identity after some time, that doesn't mean existing authenticated sessions are terminated, it just means subsequent authentication with the same secret will fail.

<comment> FLUX-YOU Maybe this feature isn't a good fit for your organization?

But based on what you've just asked, I'd definitely never create anything like what you're working with because it sounds like ANY addition to your infrastructure is problematic.

</comment></comment></comment></comment><comment><USER21> How does this different from an SSO (single-sign-on server)?

<comment><USER22> SSO is for users, and credential lifetime is usually on the order of months. Vault is for service to service authentication keys/secrets, which are traditionally static because changing them is a complex process (you need to make sure you don't break services by forgetting them or updating credentials in the wrong order). Vault automates credential creation and rotation.

</comment></comment><comment><USER23> "Why you need to buy our product."

<comment><USER7> Vault is free and open source. We do have an enterprise product, but all the dynamic secrets capabilities exist in the open source!

<comment><USER24> There are also excellent free and open source frontend interfaces for vault that are out there, such as vault-ui and goldfish. We have used both at my org, since the HashiCorp enterprise pricing is unfortunately not within our budget.

<comment><USER4> I think they just released it: [https://www.hashicorp.com/resources/vault-oss- ui-introductio...](https://www.hashicorp.com/resources/vault-oss-ui- introduction)</comment></comment></comment></comment>


We need a GitHub of Science - <AUTHOR> http://marciovm.com/i-want-a-github-of-science <comment><USER2> > _\- GitHub's success is not just about openness, but also a prestige economy > that rewards valuable content producers with credit and attention_

I don't think I can agree with that. GitHub's success, IMO, seems to be based almost _entirely_ on its openness. It has turned contributing to open source software into a drop-dead easy task, which would never be found nor contributed to if they weren't open. And they _keep making it easier_. I've fixed a number of things with machines which don't have Git installed, simply because they have their on-site editor.

Imagine if GitHub were behind a paywall. Do you think it would still be the success it is today? And, I may be weird, but I very rarely look at the names associated with commit histories. The code should speak for itself.

The rest of it sounds about right, scientific publishing as a whole is massively backwards compared to GitHub, if you're looking at it from an "Open" perspective. But I think that a lot of that is that the _researchers_ tend to be insular compared to the _implementers_ (businesses guarding their IP aside - they're not really GitHub's target audience anyway). GitHub isn't used exclusively for comp-sci researchers to post their findings with code, it's more for people _doing_ things with ideas others have contributed to.

There are experiments on GitHub, absolutely. I have a few myself. But the main thing that GitHub has done is to make _final products_ easy to find, modify, and contribute to. I have significant doubts that it would fit a research workflow smoothly, without becoming something else entirely.

<comment><USER3> John Resig started a project to generate resumes from the contributions at github. Regularly I see posts at HN/reddit/etc sites of users saying: "I build this and that, check it out!".

Personally, a motivation (of a lot more motivations) is indeed prestige. Now I can show off my nice work.

Sure, openness started it all: Linus shared his VCS, which in turn sparked Github, which in turn initiated thousands of developers to share their code. But openness really isn't the sole driver for people to share their code any more; more incentives, of which prestige is an important one, drive the popularity of github.

<comment><USER2> > _"I build this and that, check it out!"_

But people do that with other open source sites as well. Does GitHub provide this feature better than SourceForge or others? You still need to go to the user's page, they don't advertise it anywhere else. Do people go to GitHub to see information about person X, or project Y? And for the posts on social sites, are they more often about _the creator_ or _what they created_?

<comment><USER3> > And for the posts on social sites, are they more often about the creator or > what they created?

You hit the nail right on the head. Github is about sharing code. The code is the creation. Github's unique selling point is the webinterface to show the code: It's the best designed interface in the industry to show code in a browser.

> Does GitHub provide this feature better than SourceForge or others?

Various software is available to show code in the browser, but none works as well and is polished as well as Github. The creation (code) therefore can be best shared on Github. As such, others willing to check out the creation are, from the moment they arive at github, mostly busy this exactly that: check out the creation (code). Every time I visit repo's visualizing the code in html on other places then github, I get agitated by the annoying interface. Example: some interfaces require you to click on a file, after which the postback returns a site containing a list of revisions, and new buttons to 'view' a revision. This causes me to wait for a postback, and click, twice per file I want to view. Github instead instantly shows the last revision.

All these small tweaks account for much better usability on Github compared to other sites.

I guess for scientists, there must be the same approach as Github approaches programmers. For programmers, it's about code, and for scientists, it's about data and the conclusions derived from it. Instead of showing code, show a paper with the possibility to drill down on data. The data being shared can then be treated the same as code being in a source repo, so the usual git stuff (branching, merging, pulling, pushing) can be applied on the paper+data.

</comment></comment></comment></comment><comment><USER4> I'm a Ph.D. student working in theoretical high-energy physics. In this field we don't rely on peer-reviewed journals. Instead, when a researchers wants to "publish" a paper she uploads it to the arXiv, and the paper appears on the site within a day or two. The arXiv is open in the sense that almost anyone can publish there [1]. Researchers in the field catch up on new research by scanning the arXiv daily for interesting papers. No one I know reads peer- reviewed journals. I know that many papers are also published in journals, but I believe this is a formality that has more to do with obtaining grants and such than with actual communication within the community. As far as I know there's no reason to publish in a journal before you become a professor.

The result is similar to the GitHub situation in many ways. Because there are no barriers to publishing, everyone makes up their own mind about which papers are interesting. If your work is relevant, others will build on it and cite you. They will discuss it in their group meeting, and so on. A scientist's reputation is then directly related to the quality of their work, as judged by the community, with no artificial barriers. This means that a self-respecting scientist would not publish a sub-par paper even though it's technicality possible to do so, because that would hurt her reputation.

So it seems to me that the situation in high-energy physics is close to ideal, with respect to ease of publishing and the social aspect of reputation. Having said that, there are certainly aspects of GitHub that I would love to see adopted.

For instance, when several researchers are writing a paper, generally no version control system is employed. Instead, at any point in time the draft is "locked" by one of the collaborators, and only that person can change it. Beyond the obvious inefficiency of this method, note that it is also difficult to track what changes were made in each lock cycle. I use diff for this purpose, but in my experience many scientists in the field aren't aware of such tools. So something that could really help is a simple way to collaborate on papers, just a basic source control system. Also, it must be possible to work on the paper in private within the collaboration, and only publish the end result.

[1] The few barriers that exist are in place to keep out the crackpots, who reduce the signal-to-noise ratio and in that sense resemble spammers.

<comment><USER5> "In this field we don't rely on peer-reviewed journals...papers are also published in journals, but I believe this is a formality that has more to do with obtaining grants and such than with actual communication within the community."

So peer reviewed journals are only important for grants, but your community doesn't rely on peer reviewed journals. Do you not rely on grants? Who funds your work? Do you all work for free, in your spare time?

<comment><USER4> I guess I wasn't clear enough. What I mean is that the communication within the community, and the reputation of a researcher among her peers, does not rely on publications in reviewed journals. These are the things that can be compared with open-source development.

If researchers also need to publish their work in journals, write grant proposals etc., how is it relevant to the idea of applying the GitHub model to science? Of course raising money is part of the job for a professor, but thanks to the arXiv it's decoupled from the actual research work. It's at a point where I, as a Ph.D. student, have no reason to consider publishing in reviewed journals. This is in contrast to my friends in optics or condensed matter, for whom a publication in Nature or Science practically guarantees a good postdoc position.

</comment></comment></comment><comment><USER6> I'm a former biophysics postdoc myself. Now I work for an open-source software company.

This post strikes me as charmingly naive. You have to love this guy. And yet any essay that discusses the incentive structure of science but doesn't use the word "grant" until the last sentence is beating around the bush. Follow the money, my friends.

The publications are a side issue. To the extent that your count of top-tier publications matters when trying to get an academic job, it's because it's correlated with your ability to bring in money. (Money comes from peer review too, and what your peers want to read about is also what they want to fund.) What the hiring committees really want is grants. Grant money pays for labs and salaries. It pays for grad students and postdocs. And grant money literally buys prestige: Big projects come from big grants, and big grants require strong track records and a bunch of preliminary data, which in turn comes from smaller grants, or from the shared equipment that your neighbor bought with _her_ grants.

The fact that there aren't that many top-tier peer-reviewed journals is a side effect of the limited number of top scientists, and the number of scientists is limited by available resources, not by lack of knowledge or connections or education. I could literally pick up the phone and reach a dozen Ivy-educated postdocs who would be full-time scientists if they could afford it.

Why can you find so much great software on Github? There are lots of reasons, but a fundamental one is: Moore's Law. Computer hardware has become so dirt cheap that you can be a programmer in your spare time. You can literally be a twelve-year-old kid with a $200 cast-off computer and yet do top-notch software work. If computers cost millions of dollars each, like they did in 1963, we wouldn't have Github. We'd have the drawer of a desk on the ninth floor of Tech Square. (After all, in the old days half the AI researchers in the world lived within a few miles of that drawer, and the others were just a phone call away.) That's how most advanced science works today: There's no need for more publishing infrastructure for scientific technique, because the available methods of getting the word out -- top journals, second-tier journals, email, the phone, bumping into people in the hallway at conferences -- scale well enough to meet the limited demand. Because just having the recipe for your very own scanning multiphoton microscope doesn't do you much good: You need a $150,000 laser, and a $200,000 microscope, and tens of thousands of dollars in lenses and filters and dyes, and a couple of trained optics experts to maintain the thing, and that's before you even have something to photograph.

I wish there were a magical way to turn everyone's suburban basement into a cancer research lab, the way Github has turned everyone's couch into a potential CS research lab, but there's no magic bullet. A few technologies, like DNA sequencing, are sufficiently generic, useful, and automatable to be amenable to Moore's-Law-based solutions, so we probably will soon be able to (e.g.) drop leaves into the hopper of a $1000 box and get a readout of the tree's genetics. But something like cancer research is never going to be cheap. To study cancer you must first have a creature that has cancer. Mice are as cheap as those get, and mice _are not cheap_ , especially if you know what the word _mycoplasma_ means.

<comment><USER7> Do you need exclusive access to your very own scanning multiphoton microscope? Seems like laboratory virtualization would go hand in hand with open source science.

<comment><USER6> You can share them. Many labs do. Indeed, most of these things are probably shared in one way or another.

And you're correct: The trick to encouraging open source science is not to focus on the social networking tech -- that will be ready when you need it -- but to first attack the problem of doing quality lab work on the cheap. That's where the bottleneck is.

The biggest problem with shared facilities is the tragedy of the commons. In engineering -- or machining or woodworking or cooking, for that matter -- you quickly learn the importance of having your own tools. It only takes seconds to ruin a good tool. It only takes seconds to contaminate your cell culture, or your neighbor's cell culture, or an entire room full of your department's laboratory mice.

And mailing your samples off to a distant "virtual" lab is fine if you're studying disposable samples, or inorganic samples, or samples that have been permanently fixed and preserved on a glass slide. But living cells ship poorly even when you're allowed to ship them at all, and animals ship even _more_ poorly than that. So often you've got to live _next door_ to the equipment you're trying to share, and that's still expensive.

<comment><USER7> The cost of maintaining your own tools is quite high, as is the expertise required to maintain them in working order. It seems to be that the inexperience or lack of skill which leads to shared equipment getting ruined is due to the fact that extremely sensitive equipment is being handled by students and junior scientists with little engineering background. The same argument could be made for the value of having your own servers. The scientific equivalent of a datacenter, would necessarily entail staffing of highly qualified, experience personnel. Such expertise would undoubtedly be costly, but it would be amortized over a large amount of equipment.

<comment><USER8> Staffed by who? There are actually some user facilities for materials synthesis--the problem is that if someone is making someone else's material instead of working on something that they're interested in, then I don't think that you're going to get the same level of productivity out. Also, materials synthesis takes time and running through lots of dead ends. When I was a grad. student, a postdoc from a collaboration would drive in to our lab with powders she had made--try to grow a crystal for a week (sleeping for maybe an hour or two a night on a floor in our office) and then go back to her home institution and come back in a few weeks. That just doesn't work--you don't have the responsiveness to be able to figure out what dead ends you're wandering down...Imagine if you were writing code and could only run it once every few weeks. Now, imagine that it was thousands of lines of nontrivial code and you're trying to debug it that way...

<comment><USER7> Staffed by scientists and engineers, I imagine. You can get some pretty sophisticated parts built by foundries. At one point, making a microchip was prohibitively expensive. Nowadays, when you create chips basically by coding them in high level synthesis languages. Spinning a chip does take weeks. Obviously when a process takes longer to carry out, with high iteration cost, careful methodology is called for. Simulation and error checking software becomes valuable. Heck, once upon a time computers themselves were massively expensive, among the most expensive machines built. Computing wasn't born cheap. Mass production and decades of technological advances made them so. Machines for combinatorial science may someday be more effective than graduate students.

</comment></comment></comment></comment></comment></comment><comment><USER9> My goal for <http://bibdex.com> is to be this. I based the software on a wiki (original name was Bibwiki). The idea was to build lit reviews on topics that you could reuse and share with colleagues privately or publicly with the world.

I realized after starting that scientific communication is more complex, or at least it tries to be for various reasons. I could use help learning what people want from such a system.

I am keen on feedback or insights to drive my development. Please, if you are interested, you can reach me at sunir at bibdex com.

</comment><comment><USER10> Perhaps not at the academic level, at least not initially, but drawing more people into science by making it easier to ask questions and get answers couldn't hurt. <http://area51.stackexchange.com/categories/7/science>

Someday there might be 1,000,000 well-defined science/math questions, along with great answers.

<comment><USER11> There are science sections in quora as well, but people tend to post too many general/popsci/naive questions.

</comment></comment><comment><USER12> I agree ([http://www.quora.com/What-online-tools-do-scientists-wish- ex...](http://www.quora.com/What-online-tools-do-scientists-wish-existed-to- facilitate-their-work/answer/Marius-Kempe)). Why don't we just start using GitHub itself to do this and go from there? The pain points will suggest ways that a real science-focused github could improve on GitHub itself.

<comment><USER3> The problem is currently GUI. There are no good GUI's to work with git. Windows and Mac OS X have some GUI tools scratching the surface of what's possible with git, but none come close in opening up the full possibilities of git. Linux has a few very alpha, specific (like showing branches) GUI tools.

If we want non-programmers to use git, we need GUI's to instantly visualize the state, commands and other possibilities. No non-programmer is going to learn git using a CLI.

<comment><USER13> Perhaps if it caught on enough. There are many people that had to learn LaTeX who were not programmers, which is IMHO not a trivial feat.

Not that I'm disagreeing with you--But making git point-and-clickable doesn't strike me as being very simple.

</comment></comment></comment><comment><USER14> <http://science.io> was featured on HN recently. Not github, but at least a place to discuss and sift research.

</comment><comment><USER15> I tried to start a science network a few years ago, knowble.net, and I know this problem well. The main roadblock we faced was the "publish or perish" mentality. Luckily this mindset seems to be shifting & the idea of a 'GitHub of Science' is very powerful. Much more than a Science LinkedIn, which is what Knowble was.

The main unanswered questions for this idea are 1) Funding & 2) Maintenance. Knowble was a for-profit venture, but should have been a non-profit organization. So where can you/someone get the funding to build & maintain the site?

If you need a python hacker to help out - my email is emile.petrone (at) gmail.com

<comment><USER16> Thought of starting a science network myself... mind if I email you? Want to know more about what your roadblocks

</comment></comment><comment><USER17> One issue I see is what branch of science are we talking about? Physics largely seems to have this figured out via arXiv.org, but funding for molecular / biology / medical research is heavily dependent upon publication record. I'm not sure about Computer Science. But my point is when one says "Science" needs X or Y for "Science", no one is speaking the same language.

These comments are enough evidence of this. Some have already mentioned arXiv.org, and others Science.io which seems to be specifically targeted at CS. When you add medical research, the needs for these branches is _vastly_ different.

</comment><comment><USER18> Good ideas, but I disagree that you need a Bill Gates to make it happen.

The way this will happen is a grad student hacker who is avoiding working on his thesis will start coding it, and then create a kickstarter asking for support to spend the summer working on it. If she's a credible engineer, she'll get the support easily, and every subsequent kickstarter grant will also be fulfilled and it'll get built.

If you build it (right) they will come.

<comment><USER19> You may not _want_ to believe it, but this is not an engineering challenge.

</comment></comment><comment><USER20> The programs on Github were written by amateurs. Professionals can do better - compare Python, PHP and Gnuplot to Mozilla, Scheme, Haskell, Plan 9 and Mathematica. But evidently people can keep their day jobs and still write good programs.

Science is different. The amateurs are called cranks, and a small community of professionals does the good stuff. (There are exceptions, but few.) The basic issue is who will pay their living expenses, and buy the million dollar machines that they work on.

These days, almost all research money is spent by governments. They spend most of it rewarding people for publishing in prestigious journals. Scientists will keep packaging their research that way until someone starts buying it in a different package.

<comment><USER21> Most of the interesting projects on Github were written by professionals and are far better quality than the average "professional" day-job application because it is a developer's passion and not constrained by business development.

I do not believe open source solves all problems, but you dismiss the incredible value and quality of so much that I find it difficult to take the rest of the comment seriously.

It sounds more like "People with credentials (whether scientists or professional programmers) are the only people who can produce quality work. I have credentials. I'm part of the elite who can do quality work."

</comment></comment><comment><USER22> We also need a GitHub of government / legislation.

</comment><comment><USER23> The most pressing problem of our modern information society is the abundance of crap.

The service peer-review provides is the filtering of crap, so that not everyone has to do that by himself. This makes science possible, as not everyone can be a master of all trades.

Publication without review is called "journalism".

As a side note, I believe that Elsevier has acquired an extreme market dominance in the scientific publishing sector and is abusing it in alarming ways.

</comment><comment><USER24> Yes we do. Not just a replica of it with different content though, but a separate product tailored to the needs (and wishes) of science, sharing only some of the core ideas of GitHub. Sometimes I wonder if I should welcome the surfacing of ideas that have a large overlap with my own, or be anxious knowing that my lead has possibly been somewhat reduced.

</comment><comment><USER25> I just met the brains behind Opani (<http://opani.com>) last night and they are a huge step in this direction.

<comment><USER26> Opani is actually a huge step in the right direction. I think Marcio's post is fantastic. I would in fact add an additional idea to it.

In addition to the open prestige inherent in GitHub, there is also the fact that one's work is vetted by a community. It becomes very difficult if not impossible to publish crap and claim that it is quality. In science this is not the case. The peer review system is supposed to protect us against that. However, my understanding is that a surprising percentage of research in top ten journals can't be reproduced either because key details about implementation are missing or because it is actually not reproducible.

A GitHub for science could also meaningfully move the ball forward in making science reproducible as it should be easy to wrap ones scripts in a specification of an "environment" that can be readily setup, deployed, and run. A lot of work would be required to develop corollaries for non- computational scientific domains, but it would be a hugely valuable effort as discussed in the general reproducible research community (<http://reproducibleresearch.net/>).

</comment></comment><comment><USER27> arXiv.org?

</comment><comment><USER28> I think we need a github of any kind of information.

</comment><comment><USER11> Do you think something like <http://pubcentral.net> could be useful in that direction?

</comment><comment><USER29> The biggest problem with CS academicians have been in their misinterpretation of computers. Its a very different field from traditional sciences like Physics, Chemistry etc.. In traditional sciences, we study the world, understand it and express those ideas formally. With computers, its upto one's imagination what they can do with it. We just get so lost in the depths of formalism, that we forget that hacking and exploration are what can break boundaries and enable people to make computers do what they could not.

Funnily, academia harbours the most brilliant minds of CS, and barely produces usable software. Its people who identify problems, and provide software/ideas who actually get things moving. Github/Blogosphere etc allow such solutions to emerge more efficiently by allowing a lot of people to look at such solutions. In academia, a publication is taken as a end point for problem solving. There are no incentives to build real software or real systems.

If computer science wants to make a difference, it must move away from its publish or perish culture.

<comment><USER30> Computer Science is not about building real software or real systems, as opposed to Software Engineering. Your comment strikes me as missing the whole point of CS. Maybe it is so because of misleading English name -- as someone said, Computer Science is not about computers, just like Astronomy is not about telescopes -- but what kind of person would call Astronomy a Telescope Science?</comment></comment>


C++98 Support Costs Extra (or why we should switch to C++11 now) - <AUTHOR> http://marcmutz.wordpress.com/2011/09/20/c98-support-costs-extra/

<comment><USER2> I used C++ for years, and then switched to C#, and recently switched back to C++ in my new company (choice of language reflects only the nature of the software we develop, not language superiority). Just yesterday I was evaluating the new features of C++11, and I was pleasantly surprised by (1) the new language features, and (2) the extent of support already by existing compilers (gcc 4.4 and VC++ 2010).

I can see that we'll immediately benefit from a few improvements, such as auto variable, lambda support, standard support for unique/shared/weak_ptr. Even old school C++ users who have no interest in fancy new idioms would benefit from these "syntactic sugar" features found in other modern languages. There are a lot more that I'm still exploring, but so far I like what I saw.

Also these new features just work on the compilers. Given how long the standard process has been, a lot of the features were already baked in. Modern g++ and VC++2010 both have very good support. I listened to a BUILD presentation on C++ and the Microsoft folks are very committed to full C++11 compliance in VS11 (next version of Visual Studio), but AFAIK, VS2010 is already quite capable.

So, I agree, if you have a choice, switch NOW.

</comment><comment><USER3> And in the (rather depressing) real world, I have gotten flack in the past year for making changes to my source code that required C++98 compatibility, which made said code no longer compile on Microsoft Visual Studio 6.0.

<comment><USER4> Do they still run Windows NT4 on their dev systems, too? This is quite surprising.

<comment><USER3> No idea, and I'm kind of scared to ask.

</comment></comment></comment><comment><USER5> "How many Mac applications still support OS X 10.2? 10.3? 10.4? Half of the web doesn't work anymore on KDE 3.5′s Konqueror."

I'm not familiar with the author's product and can't say whether C++11 is the right choice for him, but I wouldn't consider the constant API churn of OS X or the constant web standard and web standard extension churn to be a good argument in favor of anything.

<comment><USER6> For existing large applications (in particular, Adobe's and Microsoft's) OS X's relatively quick change has been somewhat frustrating, and for any application there's no need to replacing working code with newer different code just for the sake of it... but for new projects, OS X's API "churn" has been great news, letting smaller teams tackle harder projects and create richer user interactions in less time.

I think it's pretty reasonable to argue in favor of using new language features and APIs for new projects, assuming enough of the target audience is using platforms which support them.

</comment></comment><comment><USER7> GCC, Clang, Visual C++ 11 provide charts where they show how far they have completed c++0x support (renamed to c++11):

GCC: <http://gcc.gnu.org/projects/cxx0x.html>

Clang: <http://clang.llvm.org/cxx_status.html>

Visual C++: [http://blogs.msdn.com/b/vcblog/archive/2011/09/12/10209291.a...](http://blogs.msdn.com/b/vcblog/archive/2011/09/12/10209291.aspx)

<comment><USER8> Apache also maintains a list: <http://wiki.apache.org/stdcxx/C++0xCompilerSupport>

And IBM: [https://www.ibm.com/developerworks/mydeveloperworks/blogs/58...](https://www.ibm.com/developerworks/mydeveloperworks/blogs/5894415f-be62-4bc0-81c5-3956e82276f3/entry/xlc_compiler_s_c_11_support50?lang=en)

And Digital Mars: [http://www.digitalmars.com/ctg/CPP0x-Language- Implementation...](http://www.digitalmars.com/ctg/CPP0x-Language- Implementation.html)

<comment><USER7> That Apache one looks like it is a comparison matrix, which is extremely handy.

</comment></comment></comment><comment><USER9> This is analogous to the "let's drop IE6!" movements. If your client doesn't need backward compatibility, great for you. If they do, then good luck charging them for a new version that's _worse_ (no longer works/compiles) in their environment.

</comment><comment><USER10> I want to completely agree with this guy, I really do. But Visual Studio 2010 supported more of C++11 then most other major compilers (except I think for Comeau and GCC) when it was released.

Apple didn't choose Clang over GCC for political reasons, a lot of Clang devs are actually on the Apple payroll.. and Clang is just a far superior compiler to GCC architecture-wise -- it just needs a few more years to catch up to full C++11 support.

The fact of the matter is that unless you're starting a new project you're probably tied to your old version of the compiler, to upgrade to something newer would require weeks of updating the toolchain.

<comment><USER11> > Apple didn't choose Clang over GCC for political reasons, a lot of Clang > devs are actually on the Apple payroll.. and Clang is just a far superior > compiler to GCC architecture-wise -- it just needs a few more years to catch > up to full C++11 support.

Do you have any insider knowledge of this? Because the prevailing wisdom around the net is that Apple didn't like to be dependent on a GPL3'd product (which gcc became at some point), so they propped up the only viable alternative at the time, which was LLVM.

<comment><USER10> That may be so (and I'm not an Apple insider, although I do know a few LLVM devs). I guess you could say a driving motivator to find an alternative solution to GCC was to not be reliant on a GPLv3 license. Traditionally I don't think the license has stopped Apple from contributing a lot back to the OSS community (just look at WebKit).

Besides, a compiler is just a compiler. Maybe someday down the road they will start producing their own ISA, but until then even if they're making their own ARM chips it's really hard to get it to do special Mac/iPad-only magic.

What I do know is that Clang was funded by Apple almost from the start, it was engineered right from the start to be more than just a compiler but something you could plug into the IDE as well (as the latest XCode demonstrates, it has completely unbeatable C insight).

LLVM itself is also far more extensible than GCC, just look at the slew of new-ish (last 3-4 years) of projects that have launched using LLVM. Have you heard of any such projects being launched with GCC? I can't think of any (maybe if they did, they weren't proud of it).

<comment><USER11> > Traditionally I don't think the license has stopped Apple from contributing > a lot back to the OSS community (just look at WebKit).

GPL3 makes a world of difference (KHTML/WebKit is GPL2 if I'm not mistaken); e.g., it may interfere with Apple's executable signing policy

> Have you heard of any such projects being launched with GCC?

Not as many as LLVM, but ... GCCXML, GCC Go frontend, GCC D frontend. Things still happen in GCC world.

</comment></comment></comment></comment><comment><USER12> C++11 would be nice, but I think it will be a while before I can use it. I don't think any of the mobile platforms support it...

I am now using GCC 4.4 and haven't used MSVC in a while.

</comment><comment><USER13> Our company's been C++0x-heavy since the first bits rolled out in GCC. We're lucky enough not to have any legacy, so we can afford this.

The other day I was stuck on a system with only a C++ (03) compiler, and it was a truly crippling experience without lambdas and the new STL.

Anecdotes aren't worth much, sure, but I find it difficult to disagree with this article.

</comment><comment><USER14> Some entertaining bits from the article:

"How many Mac applications still support OS X 10.2? 10.3? 10.4?"

"Then why should we be so conservative when it comes to C++, the very core of what we work with?"

"C++11 is a much more productive language than C++98. We as an industry can, however, only reap that productivity gain if we stop throwing C++ productivity out of the window by the bucket-load in the hopeless pursuit of compatibility with non-standard implementations."

I'm as much for iconoclasm as the next person, but this whole C++ madness is starting to make me worry. While the rest of us have been off in the trenches of maintenance and production, some cabal of academics and chuckleheads has been cooking up this monstrosity of a language, and now it is finally coming to a head.

Folks, conservatism in software design is appreciated--in language design, essential. Python 3 is still not support by many useful libraries. Perl 5, over ten years later, is still in production and use. ECMA/Javascript is only now really taking off, after 15 years.

And on the darker, even more depressing side of things, you've still F77 codes in "production" and _active maintenance_ \--a friend of mine who is a graduate student is dealing with this right now. These are souls being fed to the gaping maw of legacy code.

I'm sure C++ has some interesting features. I understand, for example, why the "auto" keyword is being given new meaning (because the syntax of the STL, classes, and templates is so mindnumbingly awful!). I understand why explicit defaulting and deleting of function might lead to cleaner code (regardless of, say, an explicit that already exists). I understand why I might want 3 different declensions of 'u' in front of a string to support various forms of Unicode (where is your liberalism about forcing everyone to using UTF-8 and being done with it?).

I also understand that D probably has a better execution on these ideas than C++.

And that Javascript and C# have a better implementation of lambdas and closures.

And that for raw power and readability, C with embedded assembly (still!) can probably eat the lunch of any C++. (for an example of rational language evolution, see C1X: <http://www.open-std.org/jtc1/sc22/wg14/> )

And that C++ still doesn't have anything other than a crackheaded set of half- idioms for exporting classes for dynamic linking in a crossplatform way, or even for binding to another language! Go ask the poor bastard who wrote SWIG's comments on the same--he quit programming for several years to stop dealing with C++'s increasing garbage.

I feel reasonable in saying this: C++ is not going away soon. One day (even sooner if you are in academic circles and dealing with libraries) we maintenance programmers are going to have to start dealing with code written to take advantage of the new features in C++11.

And on that day, all of us will be wondering why the hell we are still using that katamari of language features, instead of something less baroque--perhaps Perl.

<comment><USER15> I think your missing a key point: C++11 is a better, cleaner language. It's not C++98 with a bunch more not-fully-baked and overly complex stuff thrown in. Much of the new stuff that's added is there to make the old complexity go away.

Most of the old mile-long redundant type definitions can go away now, being replaced with just the 'auto' keyword.

The old std:: argument binders were like poor man's lambdas. Now we have real lambdas and don't need them.

The old practice of defining tons of template overloads is no longer required because we have proper variadic templates now. To unfairly pick one example at random of what should no longer be necessary: [http://svn.boost.org/svn/boost/trunk/boost/type_traits/detai...](http://svn.boost.org/svn/boost/trunk/boost/type_traits/detail/is_function_ptr_helper.hpp) Of course this is special purpose-library code that isn't the kind of things most applications would need to write, but it could easily end up in a compiler warning or error message.

Sure, of the people who really didn't like C++ before then C++11 might not change all their minds. But for those of us who used and liked C++ already, these new features represent great _simplifying_ improvements that are only going to help with ongoing maintenance.

<comment><USER14> The issue is that there are already many better, cleaner languages than C++98/03.

C++ is a good language, with some clunk to it. The best parts of it are arguably the bits inherited from C (which C++11 looks like it might break with). The OOPy parts (classes, polymorphism, etc.) have been done better by languages like Java, Javascript, and Smalltalk. Templates exist to get around grossness of the rest of the language, and were done better by Java's generics anyways. Multiple inheritance was done better by Java in the form of interfaces.

The rhetoric is that these changes somehow simplify the language--that's cool and all, and may even be true for the most trivial of examples or the most basic of beginners. The problem is that these new features are going to be released in the wild alongside many libraries who don't use them, being old, and who won't be updated, being cumbersome or trusted. And so, the burden of making the ends meet is going to fall on the rest of us who now have to support, in effect, two languages.

If the improvements are incremental, then this wasn't worth doing, If the improvements are such a new, cleaner, simpler language, why the hell call it C++? Again, see D.

Lastly, most of the fixes are syntactic sugar which do nothing to help fix some very real issues with the deployment of the language.

How do I bind these things to other languages cleanly? In C this is trivial-- why isn't this fixed in C++11?

How do I load libraries of classes dynamically at runtime? From C wrappers?

When interfacing with old libraries (which actually do exist), how often am I going to have to write some kind of NULL-nullptr glue code?

Why do we not have variable length arrays (they're in C99)?

<comment><USER15> I'm nothing like an official C++ spokesperson, but I'll take a shot at these questions.

 _If the improvements are incremental, then this wasn't worth doing_

Well maybe C++ is not your cup of tea, but I find the improvements worthwhile.

 _If the improvements are such a new, cleaner, simpler language, why the hell call it C++? Again, see D._

Yeah, I think it's a better C++ not an entirely new language.

 _Lastly, most of the fixes are syntactic sugar which do nothing to help fix some very real issues with the deployment of the language._

Systems languages _are_ syntactic sugar for the underlying runtime model. I consider C++ to be a systems language (the ISO spec explicitly defines a minimal profile suitable for embedded systems for example). But C is probably a slightly more "systems" of a language (in no small part because it's the language which defines most system's APIs).

 _How do I bind these things to other languages cleanly? In C this is trivial --why isn't this fixed in C++11?_

If you think this is trivial in C I suspect it's because you have so much familiarity with it and/or these other languages are implemented in C and define their external APIs directly in C. C has an unfair advantage in this regard, but it's just not that hard to integrate C++ and C.

Sometimes C++ has advantages too, some people swear by Boost.Python for example. Looks much simpler than C to me.

 _How do I load libraries of classes dynamically at runtime? From C wrappers?_

That's one way. The main reason dynamic loading works in C is because the types are all so primitive (and it doesn't have "classes"). You can use those primitive types for your APIs and in C too.

Another way is to define the classes interfaces in terms of something binary stable over the scope you wish to separately compile these libraries. This requires some understanding of the guarantees of the platform and complier/linker. MS-COM for example gets a lot of mileage out of some relatively minimal abstract class layout conventions.

 _When interfacing with old libraries (which actually do exist), how often am I going to have to write some kind of NULL-nullptr glue code?_

I've been writing some C++11 with C++4.6.0 and a bunch of old libraries and haven't run into that. I don't use pointers in my code very much, so I don't think I've even had to use nullptr yet.

So I would think "not very often", at least if your coding style is similar to mine.

 _Why do we not have variable length arrays (they're in C99)?_

I don't know why the decision was made in the language committee, but std::vector with the new initializer syntax works well for me for that purpose. Perhaps the C99 construct could be a little more efficient, but I don't see a reason that an alert compiler couldn't also allocate the storage on the stack.

<comment><USER14> Thank you for your helpful response!

 _C has an unfair advantage in this regard, but it's just not that hard to integrate C++ and C._

This is very true... as you pointed out C is very primitive in a lot of respects, and this doubtless simplifies working with it from other languages. The question is, how do I easily call the new C++11 code from something like Lua, Java, Python, Ruby, etc.? How weird will my wrapping code end up looking?

I'm sorry to harp on this, but I'm in the middle of a medium-size engine codebase written in C++, and even with our restraint so far on the use of some of the more advanced language features, things like smart pointers and whatnot are so useful that they become a concern (multithreaded code without smart pointers is a recipe for madness).

 _I've been writing some C++11 with C++4.6.0 and a bunch of old libraries and haven't run into that. I don't use pointers in my code very much, so I don't think I've even had to use nullptr yet._

Out of curiosity, how much have you done with OpenGL/OpenAL/OpenCL, etc.? Have you had any issues there yet? I'd like to know if that stuff fits okay with C++11.

<comment><USER15> _The question is, how do I easily call the new C++11 code from something like Lua, Java, Python, Ruby, etc.? How weird will my wrapping code end up looking?_

Well, no weirder at least than the older C++. Which is certainly a matter of perspective! :-)

In the ideal case, someone will write a C++ binding to your language that's even easier than C. Like I said, some people love Boost.Python but I haven't tried it.

I did play with Lua and C++ recently. The Lua interpreter itself can be compiled as C++! It will then throw C++ exceptions instead of using setjmp/longjmp, which is much saner IMHO. (I am really impressed with Lua.)

Just a feeling, but I bet that sooner or later somebody is going to come along and do something really cool with C++11 lambdas and that multi-language bindings problem.

 _I'm sorry to harp on this, but I'm in the middle of a medium-size engine codebase written in C++, and even with our restraint so far on the use of some of the more advanced language features, things like smart pointers and whatnot are so useful that they become a concern (multithreaded code without smart pointers is a recipe for madness)._

Yeah. How does one code without smart pointers in their toolkit? I couldn't go back..(except when I have to thunk back to C for something).

Also, for multithreaded stuff, I'm getting addicted to the new std::atomics in C++11!

Not new with C++11, but have you tried Boost.ASIO for multithreaded IO? It is awesome.

 _Out of curiosity, how much have you done with OpenGL/OpenAL/OpenCL, etc.? Have you had any issues there yet? I'd like to know if that stuff fits okay with C++11._

I have, in fact, been doing some experiments with C++ and OpenGL in my spare time. Made some wrapper classes for compiled shaders and buffer objects and the like.

I had much of that code running before I switched to G++4.6.0 and starting allowing myself to depend on C++11 features, but I know I've extended it some since. I really don't recall noticing any difference in the switch.

One of the biggest advantages of C++ is something that many people miss. Much of that "ugly" complexity is in support of some design principles that are critical for large-scale software development: backwards compatibility (often back to C), and you don't pay for new features you don't use.

</comment></comment></comment></comment></comment></comment><comment><USER16> If your supported platform list looks like this:

      Windows

Switch to C++11 NOW. You will kick yourself for not doing it earlier.

<comment><USER17> If you want your application to run on Windows 2000, you can't. VS 2008 doesn't have "auto" etc but it can produce binaries which run on Windows 2000. The libraries for VS 2010 simply removed the code that allowed them to run on Windows 2000. I can imagine you'll ask "who uses Windows 2000," well, if home users don't, the companies still do.

<comment><USER16> I know which companies do. I chose not to support them for my software. There are enough other companies to make money without it.

<comment><USER18> You could do that, or based on what the author of the article was saying, you could just charge more for anyone that wanted Windows 2000 support.</comment></comment></comment></comment>


Ask HN: How do I profit from my idea? (new tagging system with many benefits) - <AUTHOR>

I have a idea but do I have to patent it or is there any other way to profit from it? I live in sweden and dont have the money or time to patent it. What should I do?<p>Thanks! <comment><USER2> i think the only really tried and true method is to bring it to startup in a company and demonstrate that it is really a new and beneficial addition to the market.

take your tags and make a business out of it. until you do it has no value and can't really be safeguarded as just "an idea".</comment>


The Evolution of a Software Engineer - <AUTHOR> https://medium.com/@webseanhickey/the-evolution-of-a-software-engineer-db854689243

<comment><USER2> Posts like these are somewhere in the middle of the Evolution.

</comment><comment><USER3> Oh dear. This again.

<comment><USER4> Apparently 9 previous threads (with almost no comments) just isn't enough. [https://hn.algolia.com/?query=The%20Evolution%20of%20a%20Sof...](https://hn.algolia.com/?query=The%20Evolution%20of%20a%20Software%20Engineer&sort=byPopularity&prefix&page=0&dateRange=all&type=story)

For those not in the know, it's a variation of an old theme. [http://www.willamette.edu/~fruehr/haskell/evolution.html](http://www.willamette.edu/~fruehr/haskell/evolution.html) , which is a Haskell version, says it goes back to at least 1995.

The traditional version (that is, the one I remember) is more akin to [http://www.ariel.com.au/jokes/The_Evolution_of_a_Programmer....](http://www.ariel.com.au/jokes/The_Evolution_of_a_Programmer.html) .</comment></comment>


Assassin's Creed IV: Black Flag Lets You Be a Pirate - <AUTHOR> http://thegadgetflow.com/blog/assassins-creed-iv-black-flag/

<comment><USER2> Just imagine the irony when this game gets pirated.</comment>


Atlassian Confluence $5 for 5 users - <AUTHOR> http://www.atlassian.com/starter/

<comment><USER3> The company I work for has a site-wide license. Before Confluence, we used Lotus Notes and other proprietary databases (we joked they were data roach traps - data entered in, very/very hard to get out). After a small pilot, Confluence spread quickly through our company and is now indispensable. The only thing that could replace it is probably Sharepoint. Confluence is basically the concept/power of the Wiki without the Wiki-awkwardness. A normal-user friendly Wiki.

Confluence has traditionally free for open source projects. I upvoted this because I think this limited-time offer ($5 for 1 year) would fit many small, distributed startup's needs for a centralized knowledgebase (if they are not already using Drupal or Backpack). Jira is also included in this $5 offer too [separately - not 2 for 1] (we use that too - our testers/business analysts love it)

<comment><USER4> > Jira is also included in this $5 offer too (we use that too - <USER2> > testers/business analysts love it)

Really? They do? That's surprising, because the last 3 places I've worked with Jira, everyone hated it. I always felt like it was a super-configurable, really lame to use issue tracker.

<comment><USER5> Odd. The last several places I've worked have switched to Jira (from Mantis, Bugzilla, some home grown tool, Trac, Excel) and everyone was happier (devs, testers, PMs). Perhaps it wasn't setup well for your uses? I'm not sure.

Personally I love Jira, it's easy to configure to work the way you work, it has tons of plugins, great integrations (Eclipse, Hudson, SVN). It's by the best task/bug/project management tool I've every used.

</comment></comment></comment><comment><USER5> Confluence is a really amazing product! I've used it for document and project management at companies large (Fortune 100) and small (my 3 person startup), and it's worked great.

It has a ton of excellent features and polished interface out of the box, but the real strength is the ecosystem of plugins available for Confluence that let you easily extend it to be tailored to your needs.

With a deal like this, I'd recommend checking it out.

</comment><comment><USER6> Confluence is a great product and way too good to pass up at $5, done deal!

</comment><comment><USER7> This is an awesome deal! I use the free JIRA personal license for my own projects but I will get this license so that I can officially use it for my small business.

</comment><comment><AUTHOR> I submitted it because with this price tag it could be a serious competer to some start ups out there.

</comment><comment><USER8> "5 users for 5 dollars for 5 days" is a misleading headline (it's for a year, but the _offer_ is only available for 5 days).

But Atlassian are a very marketing-smart company, so maybe this is intentional, to generate discussion, and so make the message more sticky.

<comment><USER9> I didn't even notice the "5 days" part until my boyfriend pointed it out. And even then it sounded odd, since it says they give you a year of full support.

A quick reading of the FAQ (which you should always do to avoid surprises :)) clears up all doubt.

BTW, thanks to the submitter. We use this at work and it's fantastic. I really wanted to be able to use it for my personal projects, too!

</comment></comment><comment><USER10> Just as a note, it's $5 per year of support/upgrades for up to 3 years. Then after that, I guess you have to pay whatever the base value is at the time for support/upgrades.

</comment><comment><USER11> Thanks a lot for the info, I can't believe I just bought two applications I always wanted and never could afford for 10 bucks!

</comment><comment><USER12> Don't forget that JIRA is available for $5 as well! I love its dashboards for project summarization.</comment>


Mapping the Shadows of New York City - <AUTHOR> http://www.nytimes.com/interactive/2016/12/21/upshot/Mapping-the-Shadows-of-New-York-City.html?hp&action=click&pgtype=Homepage&clickSource=story-heading&module=second-column-region&region=top-news&WT.nav=top-news <comment><USER2> Huh, just yesterday I tried something like this, but lacking elevation data for NY I settled for most of Europe: [https://i.imgur.com/b7h3ulR.jpg](https://i.imgur.com/b7h3ulR.jpg) (I downscaled the resulting 198'000x126'000 image to 4'950x3'150.)

And some more high resolution ones, e.g. Salt Lake City [https://i.imgur.com/R352z2l.jpg](https://i.imgur.com/R352z2l.jpg)

Using an extremely fast tool built by coworkers.

<comment><USER3> It's really cool to see the natural borders of countries in much higher contrast than a regular elevation map. Especially the Alps walling off Northern-Italy from the rest of the continent and the way the Carpathian basin is surrounded by terrain that made it hard to invade. Makes sense that it was such a heavily contested area around the 9th century.

</comment></comment><comment><USER4> Pretty awesome visualization. Does anyone know if the authors published a paper on the "shadow computation that is 10 to 15 times faster than older methods"? I couldn't find it on their websites [1-3].

[1] [https://vgc.poly.edu/~fmiranda/](https://vgc.poly.edu/~fmiranda/)

[2] [https://vgc.poly.edu/~csilva/](https://vgc.poly.edu/~csilva/)

[3] [http://www.harishd.com/home/](http://www.harishd.com/home/)

<comment><USER5> It sounds like a way to take larger simulation steps by linearising the movement of projected points on shadow outlines. I'm guessing a bit, but the idea of figuring out how to take larger steps is an established trick for speeding up simulations (e.g. the Siggraph paper "Large Steps in Cloth Simulation")

"Instead of explicitly tracking the movement of the sun, which is how shadow accumulation is usually measured, the researchers are tracking the movement of the shadows. Using the key insight that shadows move linearly within a short time range, they designed an efficient algorithm that performs an order of magnitude faster than those used in existing approaches. "This also allows users to interactively compute and explore shadows over different time ranges and allows them to quickly test multiple scenarios," said Doraiswamy."

</comment></comment><comment><USER6> It'd be interesting to see a composite of reflected sunlight included around the builings with highly reflective exteriors.

<comment><USER7> This strikes me as a definite perk of the glass-and-steel skyscraper design. Older brick and stone designs seem to waste a lot more light, especially as they get dirty and lose albedo.

<comment><USER8> Brick and stone buildings have mostly been kept clean the last 15 years or so, and they don't darken as quickly as they did in the 20th C when both transportation and industry produced far more soot.

That said, I've seen buildings that were nearly black with 100 years of accumulated soot at the end of the 1980s undergo deep cleaning (sandblasting) in the 90's and come out off-white. It's pretty remarkable.

25 years later, those same buildings are now almost exactly the same color they were immediately after the cleaning, without any further maintenance.

<comment><USER7> It is a pretty stunning difference. Honestly, I think I'm mostly thinking of a handful of black stone buildings (famously, American Radiator) and buildings which haven't been cleaned since the '70s or '80s.

</comment></comment></comment></comment><comment><USER9> Is this the first time somebody's constructed an interactive map of every single building in the greater NYC metropolitan area? It's really the part of the article that I find most impressive.

<comment><USER10> Seems like a good opportunity to sell data to those in real estate. For example, why would someone pay the same for two different apartments on the same floor when one gets direct sunlight door 30% of the day, and the other gets none!

<comment><USER11> For that matter, you can market _both_ of those as advantages.

Apt A: "Save hundreds on air-conditioning in the summer - this apartment never feels the full brunt of the scorching sun!"

Apt B: "Wake up every morning to a New York sunrise - the best way to beat those winter blues."

Just gotta know what the tenant is looking for.

<comment><USER12> Almost always you want Apt B. Bright sunlight can be controlled with electronic transition glass, curtains and other methods. You can't reproduce sunlight in your apartment.

<comment><USER13> I find it surprising you mentioned electronic transition glass - I had thought it's not used pretty much anywhere as of yet, and prohibitively expensive.

<comment><USER11> The only place (I think?) I've seen it was on the Boeing Dreamliner.

<comment><USER14> Didn't some Maybachs have it? Or is that a different kind of glass?

<comment><USER11> I have no idea :/

</comment></comment></comment></comment></comment></comment></comment></comment><comment><AUTHOR> New York NIMBYs often refer to shadows when trying block the construction of a new building. But as covered in shadows as New York is, it doesn't have a shadow problem. It has an affordability problem. Lots of people are willing to spend crazy amounts of money there (many of whom could live anywhere in the world) because it's still an amazing place to live. More people would love to live there, but there is, of course, a shortage of affordable housing...

</comment><comment><USER15> "It is said that the sun rises in the east and sets in the west. But at New York's latitude, that's not quite the case. On the summer solstice, the sun rises in the northeast and sets in the northwest, while on the winter solstice, the angle changes and the sun rises in the southeast and sets in the southwest. "

I thought the sun's path was always south of due East-West for locations north of the Tropic of Cancer at all times of the year.

<comment><USER16> Above the tropic of cancer, the sun is south of you at noon even in the summer, but rises and sets a little in the north because the earth is more spherical than cylindrical.

I think [http://www.suncalc.org](http://www.suncalc.org) is a useful site for seeing where the sun will be at different times of the year. Zoom out the map, then move the dynamic icon north and south in June and December.

</comment></comment><comment><USER17> Don't think I could live there, I like sunshine. Some people love living in the city because of all the amazing stuff you can do there, but it's too big (and dark in many places) for me. I wonder what NYC will look like in 200 years. Can you build higher?

<comment><USER18> As much as I like the interactive map the truth is it makes things look way more shadowed than they actually are because its showing the averages of shadows from all time of the year and all times of the day. The reality is that most of the streets have a shady side and sunny side at all times of the year. It's shady on one side of the street in the morning and shady on the other side in the afternoon.

This is actually great because you can choose the side of the street to walk on. If you are too hot in the summer you can go walk on the shaded side, or if you are too chilled in the spring/fall/winter you can walk on the sunny side. During the summer it is actually very nice to have a long stretch of tall buildings where you can cool off by walking on the shady side of the street.

It's kind of like enjoying walking down a street that is shaded by a lot of trees, although sadly the buildings don't absorb as much heat as the trees do, and in some cases glass buildings actually seem to reflect the heat harshly.

<comment><USER19> Good point. They could have projected only the shadows before noon, or separate morning/afternoon shadows with a toggle.

</comment></comment></comment><comment><USER20> Very cool. The explaination of "shadow accumulation" would have been a bit closer to reality if they had used hours in their example. shadows don't move anywhere near that far in 1 minute.

</comment><comment><USER19> Some cool accompanying photos. I also enjoyed seeing the buildings' shadows projected onto the Hudson River

</comment><comment><USER21> Awesome work!</comment>


Real-time Curiosity Simulator with Landing Preview - <AUTHOR> http://eyes.nasa.gov

<comment><USER2> This is a great link, thanks. I'm watching it and I have a question about reentry (should apply to all capsules, like Apollo, SpaceX not only MSL): basically, how is this thing stable?

I see the animation and how the capsule twists and turns, and I'm nervous, why doesn't the capsule destabilize and start to tumble? It's not spinning (like a bullet) so how come it stays heat shield down?

Perhaps some kind HN-er with aerodynamic knowledge can write a brief answer for the layman (I'm sure others here might have the same question).

Thanks

</comment><comment><USER3> This doesn't work in Linux. Not in Firefox, not in Chrome. I'm using Java 7 + Icedtea 7 plugin. The plugin starts Java, but it sits there chewing 200% CPU time and doesn't do anything for 10 minutes.

This is a Java development box, and I tried everything I could think of to fix it. Obviously, other Java apps (even LWJGL using ones) work fine.

Edit: I tried getting it to work under XP in VBox using VBox's guest drivers' OpenGL passthrough, it still didn't work.

<comment><USER4> Working in Safari on osX. <http://i.imgur.com/J05jP.png></comment></comment>


Some People Are Smoking Scorpions to Get High - <AUTHOR> http://health.howstuffworks.com/wellness/drugs-alcohol/smoking-scorpions-get-high.htm <comment><USER2> Some people use genetically modified yeast to get high. Very soon, some people may use genetically modified viral agents to get the bodys own opiod glands to "step it up".

The war on drug is a war on human nature.

</comment><comment><USER3> Time to ban scorpions, and scorpion venom products. Anyone who gets within sight of one must be thrown in prison for decades.

Or we can just let adults choose what they want to do with their own bodies, and stop freaking out because somebody might have "got high."

<comment><USER4> Clearly these people that chance a substantial risk of 6 hours of pain followed by paralysis or death have made a wise decision and must not be interfered with.

Truly a travesty that the state invests its resources into peoples lives and then cares when they throw it away in selfish stupidity.

<comment><USER3> We should be forced to seek state approval for every decision we make, so that our behavior is consistent with the needs and interests of the state. That will also make sure that nothing "unwise" is ever accidentally decided. Bureaucracy is the answer; it will save us. We should cede our individual sovereignty to the more enlightened among us, for the good of the collective. It's simply too dangerous to allow humans free agency. The person is dumb but the people are smart.</comment></comment></comment>


Ask HN: Best practices for open sourcing your project? - <AUTHOR> Hi all,
I've been working on https://algodaily.com for the better part of a year now, and would like to open source both the curriculum and platform.
Aside from taking the current spaghetti code and beautifying it, are there any suggested practices for opening up a formerly private repository? I'm wondering about things like secrets management, ads (I have ads on the hosted site, should I remove them from the public repo?), how much documentation to write before shipping, etc.
Appreciate any advice! <comment><USER2> In line with the "release early, release often" mantra, you can just go ahead and do it.

Secrets are usually split into separate "configuration" repositories that remain private, and that's the only gotcha.

Any other time you invest in beautifying your code, writing documentation, stripping ads and making them configurable, might be in vein: most open source projects do not attract any contributors, yet having these "easy pickings" is also stimulating for someone to contribute.

Your users are better served having an ability to tinker with the code than waiting for the code to become nicer. At least, the free software persona in me says so :)

</comment><comment><USER3> * focus on documentation. Good documentation is the thing that differentiates good open source from bad open source.

* separate out all things external. This includes services, dependencies, and especially ads. Manage them like managing a foreign business partner.

* provide a demo tool if applicable.

* focus on code organization. My applications are well organized large files. This scares the shit out of people who don't really read code but is preferred by people who do read code, so know your audience. I would rather a drastically smaller audience of higher quality interested people but that makes your application appear less popular.

* automate everything. If automation is not a primary consideration your code is just a toy.

* everything is broken. Encourage challenging critical feedback

</comment><comment><AUTHOR> I should say, I'm also open to any resources like articles/podcasts/talks on this topic.</comment>


Review our new project:team brainstorm app - <AUTHOR> http://www.groupzap.com

<comment><USER2> exceptions.KeyError not callable

<http://gyazo.com/564cc7065e1cf337b3a5034c68b87972.png>

:(

<comment><USER3> Yes, sounds promising but it crashes when trying to create a new whiteboard...</comment></comment>


Show HN: If Snapchat and Foursquare had a baby - <AUTHOR> http://www.partyhere.me/ PartyHere is my first attempt at developing a sizeable, yet simple mobile web app. The idea is that users can easily create temporary "parties" at their current location, and then their friends can check in to said parties. The parties only last a few hours, so the user doesn't have to worry about the party after creating it.
The idea came out of 3 Day Startup (http://cornell.3daystartup.org/) in November. Since then I've been working nonstop on coding up the project, and have been having way too much fun with it! It has provided me with an incredible opportunity to play with open source technologies like socket.io, backbone.js, and Phonegap/Cordova. Currently, the iPhone and iPad apps are under review by the app store.
My request to you is for any kind of feedback. Please be honest and let me know what you feel needs to be fixed/changed (UI/UX, marketing, features...).
So, what do you think? <comment><USER2> Cool. I just signed in, but couldn't find any of my FB friends (obviously none are connected) but can't I invite them?

Is this just using the FB/Google maps APIs?

Good luck with it!

</comment><comment><USER3> Why did you copy bootstrap exactly?

<comment><AUTHOR> The app is very much in it's alpha stage. I don't have a designer, and have focused mostly on the product. Don't worry, the first thing to get an overhaul is the design of the landing page as soon as I get the chance.</comment></comment>


Does Rising CO2 Benefit Plants? - <AUTHOR> https://www.scientificamerican.com/article/ask-the-experts-does-rising-co2-benefit-plants1/ <comment><USER2> I wonder if the benefits of doubling CO2 would be greater for nitrogen fixing plants, as they would be less likely to become nitrogen-limited.</comment>


What are some easy-to-implement tools for newly remote teams? - <AUTHOR> In light of the coronavirus, my team of mechanical engineers is now working from home. Can anyone recommend any tools that are easy to get started using that might be helpful for us? <comment><USER2> Teams or Slack for communication. If you're using Office 365, Teams is a absolute no-brainer. Teams has the added benefit of being a viable video conferencing tool too.</comment>