Bad science persists because poor methods are rewarded - <AUTHOR> http://www.economist.com/news/science-and-technology/21707513-poor-scientific-methods-may-be-hereditary-why-bad-science-persists <comment><USER5> The article reminded of me of Robert Axelrod's excellent "The Evolution of Cooperation." Axelrod uses similar computer modeling to create a tournament to help tease out how cooperative strategies emerge from primarily self- interested behavior. It is one of the best things I have ever read in my life, and I'd recommend that book to anyone.

The real issue with scientific publishing is that there is simply no penalty for publishing shoddy research. I know several academics who made quite a big name for themselves on research that was later partially or fully retracted. No one cared about that; there was no real reputational damage done. To tackle poor science, such "poor" scientific inquiry should be "punished" in some way. Similarly, it is terrible for the advancement of science that only novel or significant results get published -- there should be a way for researchers to benefit from publishing well-designed research which simply did not yield interesting results.

How to do that? I think some of Axelrod's tournament provides an answer. Like in his examples, the individual incentives align to yield a pretty poor outcome to the members of his population (he runs an iterated prisoner's dilemma game). However, correctly setting up the iteration parameter's, slowly a cooperative strategy becomes the evolutionarily stable strategy.

I can see how this could also be the case for academics. There is no law from up above that dictates that "number of papers published" is the ultimate metric of success. There is a culture, and processes, and institutions which have led that to be a leading indicator of academic success. If there were real motivation and impetus to change this, there is no reason to imagine that other metrics (and processes) could emerge that would much more highly value scientific integrity and thoroughness.

<comment><USER6> > I know several academics who made quite a big name for themselves on > research that was later partially or fully retracted. No one cared about > that; there was no real reputational damage done.

cf. the "power pose"

<comment><USER7> It's hard to find information on what the problem was with that one, so here's a link to an article from Slate:

[http://www.slate.com/articles/health_and_science/science/201...](http://www.slate.com/articles/health_and_science/science/2016/01/amy_cuddy_s_power_pose_research_is_the_latest_example_of_scientific_overreach.html)

<comment><USER8> Ironically, "failure to replicate" can also be spurious.

<comment><USER6> IIRC there were on the order of ten attempted replications, all of which failed.

</comment></comment></comment></comment></comment><comment><USER9> "universities and funding agencies [should].. stop rewarding researchers who publish copiously"

You have to look at the media as well. When was the last time you read an article about a failed experiment, ie: an hypothesis was DISPROVED? There's rare/no coverage of this yet it is an important aspect of science[1], and ties into what the OP talks about.

If we're going to improve how science is done, this is an equally important area to focus on because it's the current bias for positive results that plays a part in driving labs to produce lots of papers.

1 - [https://www.elsevier.com/connect/scientists-we-want-your- neg...](https://www.elsevier.com/connect/scientists-we-want-your-negative- results-too)

<comment><USER10> We'll they do when there's common myths that get disproved. Like that guy who cracked his knuckles on only one hand for nearly all of his life. But yeah I get your point.

[http://www.foxnews.com/health/2014/03/13/health-myth-is- crac...](http://www.foxnews.com/health/2014/03/13/health-myth-is-cracking- your-knuckles-really-bad-for.html)

<comment><USER11> Thats the only time when things have a general interest. Negative results are very important to people directly involved in studying something but what general interest (or value) is there in a story "chemical xyz123 does not work as an effective catalyst for the reaction between abc123 and def678" for anyone not directly researching those things?

<comment><USER10> There is no interest to the general public. But that doesn't mean scientists can't carve out their own online community like Wikipedia and start to categorize all of this information. It would for them probably be what stackoverflow is to programmers.

</comment></comment></comment></comment><comment><USER12> I would say it is less that poor methods are rewarded, and more that the investment in proper methods is expensive, and thus penalized.

That is, nobody is particular looking for people doing poor work and handing out rewards. However, the "proper" methodology that we want necessitates time. Something that is expensive and there are already plenty of things eating at the budgets of work out there.

I do think this can be made better. But I see no reason to think it is just us chiding people for not doing better.

<comment><USER13> > That is, nobody is particular looking for people doing poor work and handing > out rewards.

I'd challenge that. If you look for people with high impact factors you are asking for people doing poor science. If you only publish "significant" results and refuse to publish negative results you are asking for bad science. There are structural problems that lead to bad science, and often it's not about costs.

<comment><USER14> Given that plan to decimate the economy of America's 4th largest and most racially diverse city (Houston, TX), I am quite keen to know if these issues also affect climate science.

Don't get me wrong, I'm still pretty convinced that anthropogenic global warming is happening. But if there is a change that it isn't and we can avoid all of the murders and suicides that come with throwing large numbers of people in one area out of work, we should investigate the possibility.

<comment><USER15> These issues affect all science. That's not an argument for ignoring science when it's politically convenient. (At least not any more than you already were based on the results. Whatever your all-in number for how often science is right or wrong is, that number already incorporates all these issues).

If anything I would expect standards to be much higher in climate science than less controversial areas. There are a lot of people who would stand to gain (or, equivalently, avoid losing) huge amounts of money if the climate science was wrong, they will be going over papers with a fine-tooth comb in a way that doesn't happen to most papers.

<comment><USER14> > ignoring science when it's politically convenient

And I'm not making an argument for it. I'm making the argument that "getting this wrong is personally costly for a lot of people, therefore we should be examining these results with a fine-toothed comb before basing massive social change on them."

But you're right, there would already be monetary incentive to do that.

</comment></comment></comment></comment></comment><comment><USER16> _"... his finding also suggested some of the papers were actually reporting false positives, in other words noise that looked like data. He urged researchers to boost the power of their studies by increasing the number of subjects in their experiments."_

Conclusions based on low sample rates, should be seen as poor technique, an indicator of potential bias and suggest a _lot more_ validation is required before acceptance.

</comment><comment><USER17> ...20 researchers try to study something. But only one of them gets statistically significant results (p < 0.05), so they publish and the rest don't.

[http://xkcd.com/882/](http://xkcd.com/882/)

If we don't address this, then the whole research publication process is on fire.

<comment><USER18> The easiest way to address this is to simply move the p value necessary for significant results - instead of p < 0.05 use p < 0.001 or even p < 0.0000003 (3 x 10^-7), as is needed in particle physics. Though the 5 sigma threshold is a bit ridiculous. Lowering the p-value threshold significantly increases the cost of hunting for positives due to random chance, at the expense of needing more sensitive experiments.

<comment><USER19> Nope. It just means that you have to come up with a shittier null hypothesis. Of course, it still has to get past peer review, but this can often be accomplished by complication and obfuscation, which have the side benefit of giving you plausible deniability in case you are discovered. Sufficiently advanced cluelessness is indistinguishable from malice, and science, in its present form, rewards them both.

</comment></comment></comment><comment><USER20> Which is a mild way that our education is poor since rewarding status over competence.

Would parents sending their kids to Ivy leagues accept a kid from downtown going to public school being given a better recognition as theirs?

We let parents influence education, and let schools bend the «natural» competition in order to satisfy influential parents.

This cannot happen without regulations to tie diploma to a job. Without academy's and authorities. The first winner in the bending of the knowledge competitions all other the world are the kids of teachers.

There is a worldwide corruption without bribe of teachers achieved by a social pressure.

What could be done? Burst the educational bubble?

</comment><comment><USER21> It is my personal experience that bad science persists not only because it is rewarded but also because a large portion of scientists just don't do good work.

</comment><comment><USER22> This is the one part of the article that is counterintuitive:

> Worryingly, poor methods still won—albeit more slowly. This was true in even > the most punitive version of the model, in which labs received a penalty 100 > times the value of the original "pay-off" for a result that failed to > replicate, and replication rates were high (half of all results were subject > to replication efforts).

How can bad results still confer a net reward on their producers with a penalty like that?

<comment><USER23> I went back and read the original description of the model, this is what I think is going on:

The average performance of a bad lab is worse then a good lab. However, a bad lab might get lucky and not have any of their false-positives subjected to replication. As a consequence, the top-performing labs tends to be bad labs that got lucky. The selection method heavily favors being the top performer, and thus the poor but lucky lab tends to win out, which is why it takes over the population.

This casts doubt for me on their model, since fitness proportionate selection would probably have quite different results.

</comment></comment><comment><USER24> There is another dimension to this problem; universities in developing countries are pushed to match what their colleagues in developed nations are publishing. In doing this professors where I am currently studying (Brazil) foist off research on undergrads and masters students.

The system gets even more ludicrous when it comes to translation and publication, but that is a tangential issue.

<comment><USER25> Same here, quantitiy over quality is way more preferred. Trying to improve some research that has no value just to increase publication count.

</comment></comment><comment><USER7> >> Ultimately, therefore, the way to end the proliferation of bad science is not to nag people to behave better, or even to encourage replication, but for universities and funding agencies to stop rewarding researchers who publish copiously over those who publish fewer, but perhaps higher-quality papers.

I don't see why being prolific should be punished. No, a better solution is to reward research that is replicated more than research that is not yet so.

This would be much easier to do than punishing anyone. It's hard to find a good reason to punish someone because they published something; after all, even very thorough research has a chance to be wrong, and let's not get into what is "correct" anyway. And it's a bad idea to discourage people from going out and tyring things that have a very slim chance of working.

Rewarding replication might even result in some labs specialising in replication studies and that would be a net benefit for everyone, especially the researchers who would love to do replication but really don't have the resources to do it.

<comment><USER26> Yeah, but it's hard to get funding to do a rigorous replication of something that doesn't have an immediate commercial justification (for example legal liability or FDA requirements in drug trials). It's a big investment where the best case scenario is that you can be a little more confident in prior results. And if you can't replicate the results, it doesn't necessarily mean the original finding was wrong, it could just mean the original scientists were better than the replication scientists.

Also, among scientists who have invested the time to earn a Ph.D., there is a culture of wanting to break new ground and push humanity forward. There would have to be a strong incentive to motivate them to invest the time in replicating research for which they won't get the glory.

You would have to change the way science is funded to make research replication a required step in the modern scientific method.

</comment></comment><comment><USER27> The current model is just not efficient for science and I'm honestly continually surprised that it manages to accomplish anything at all.

<comment><USER28> Lying by omission seems to be widely accepted now as necessary and expected in order achieve tenure, secure funding, etc. I find it abhorrent. If you go back and read experimental papers from the 1800's, before science became an industry, they are full of detailed methods (including where to purchase items), pitfalls encountered along the way, and potential detractors from the main result. Not so today. Today, it seems to be all about selling a result that may or may not be significant.

<comment><USER27> It's a society-wide problem that is caused by competition and status seeking. Pretty much everything is optimizing for "winning", whatever that means, instead of doing things correctly, conscientiously, and carefully. The latter will often make you "lose", and there's no system rewarding you for it besides an internal one. Then there's nothing left for people to do but try to "win", because resources are scarce. The more you optimize for "winning", the more nasty your methods get, because there's always something you can sacrifice, until nothing is left.

Another unseen side effect of this is that a lot of people see this, combined with the low returns from science, and decide to go do something else, because if they are going to put up with the garbage they can as well earn more money doing it. And some among them then realize that they can have a greater effect by generating billions of dollars and then using them on whatever they want, instead of wasting time in a lab and asking for scraps from the government.

Untangling this requires a complete mentality shift.

<comment><USER29> > Untangling this requires a complete mentality shift.

I agree with this.

> It's a society-wide problem that is caused by competition and status > seeking.

I don't agree with this.

I agree that competition and status seeking can lead to bad outcomes.

However this is wholly about how competition and status seeking are channeled. They are instincts, they come naturally.

Clearly in the present order they are not channeled in ways that lead to superior or interesting outcomes since mediocrity is everywhere.

<comment><USER27> > However this is wholly about how competition and status seeking are > channeled.

You can't truly channel competition anywhere because it's always routed to itself. Competition implies a provider of gifts and those who chase those gifts. The provider will always use faulty proxies to determine who to provide to, thus becoming disproportionately important to the process. Acquiring the gifts is always disproportionately more important than anything else. Status is one such gift, with no redeeming value.

Competition is always about the agent that is winning, and the provider that arbitrarily bestows prizes. That is where the focus lies by definition - <USER2> objective of a competitor is to eliminate all others, not to produce anything of value. Science should be about the results, and should bow to no provider.

> They are instincts, they come naturally.

A lot of bad things are caused by instincts that come naturally. That's actually specifically why I pointed those two out: they're natural, and they're extremely dangerous and we should always be aware of them creeping up and keep them in check.

Instead, we created whole institutions to worship them.

> Clearly in the present order they are not channeled in ways that lead to > superior or interesting outcomes since mediocrity is everywhere.

The "mediocrity is everywhere" thinking is part of the problem, as it drives everyone to worry about how "mediocre" they look and rush after the best thing ever all the time, instead of calming down and doing the right things. Publishing a failing study is mediocre. Helping people in need is mediocre. Doing a little bit of fitness to keep yourself healthy is mediocre. Yet, that is exactly what we need more of.

Mediocrity will always be everywhere because it's a mathematical fact - <USER4> people are in the middle. So, yes, it's everywhere. That's tautological.

I think you were trying to say something else.

<comment><USER29> > Competition implies a provider of gifts and those who chase those gifts. The > provider will always use faulty proxies to determine who to provide to, thus > becoming disproportionately important to the process. Acquiring the gifts is > always disproportionately more important than anything else.

I think we are on the same page with respect to the dangers.

I am reminded of the experiments in which rats are getting hits of dopamine. What is it called? Behavioral conditioning? Skinner conditioning? Very simple, but very powerful.

And this I think is the specter that haunts Science/Education. Sheer force of collective habit maintains the status quo. There is lots of evidence for 'work' (even workaholicism) but paradoxically diminishing results. I am reminded of those students who write down everything the lecturer says in class but don't have the time to understand what the lecturer is trying to get across.

> You can't truly channel competition anywhere because it's always routed to > itself.

Here we sort of disagree. I agree that it is potentially an uncontrollable positive feedback loop but I think competition (and also our sense of fairness, demotism, voting, equality) can be channeled.

To use an analogy, these instincts we have are like Water or Electricity. Dangerous but controllable to great effect. The goal of good governance is to 'traffic shape' these forces into productive causes.

Competition in the wild is simply Hobbesian violence, one man against the other. It requires the halter of a market with its price system and exchanges to be funneled productively. There are huge problems of course but we at least know a halter actually exists.

What I am calling 'fairness' in its natural state is a powerful raw instinct to enforce conformity and uniformity among the tribe members. There is an obvious biological connection to these instincts, to our genetics, our genes in their voting blocks trying to spread their influence. This is known in biology as R selection, where the system is optimizing for volume (the overall hypothesis is called R-k selection).

In my opinion there exists no 'halter' (as of yet, but I hold out hope) for fairness. I do not believe a proper mechanism has yet been invented by humankind. Yes humans build institutions to control it, like standardized education, democracy, republics, but I am unconvinced as to their strength against a sudden surge in the tide of popular feeling. Sooner or later, like the rise of the Communists, brute strength wins out over elegant attempts to moderate, basically mob rule followed shortly by blood rule with tyrants. Year Zero. A Reset. It has happened countless times in history.

Democracy leads to entropy. Competition has the potential to take us to Moloch.

[http://slatestarcodex.com/2014/07/30/meditations-on- moloch/](http://slatestarcodex.com/2014/07/30/meditations-on-moloch/)

Basically you either need to choose your poison, or we need to come up with a better system. I currently err on the side of Moloch, because it is a slower death than the alternative.

> A lot of bad things are caused by instincts that come naturally. That's > actually specifically why I pointed those two out: they're natural, and > they're extremely dangerous and we should always be aware of them creeping > up and keep them in check.

> Instead, we created whole institutions to worship them.

That's about the size of it, yes.

> The "mediocrity is everywhere" thinking is part of the problem, as it drives > everyone to worry about how "mediocre" they look and rush after the best > thing ever all the time, instead of calming down and doing the right things. > Publishing a failing study is mediocre. Helping people in need is mediocre. > Doing a little bit of fitness to keep yourself healthy is mediocre. Yet, > that is exactly what we need more of.

> I think you were trying to say something else.

This is interesting, thank you. You're quite right that over-optimizing is a threat (Moloch) but I shall reformulate my complaint here.

Mediocrity _is_ a threat (Entropy), _but_ , and I feel this is important, if we have enough fields or subfields of exploration, avenues of research then Mediocrity basically loses its meaning. The division of labour allows everybody to potentially be a winner.

It does not eliminate average results but it disperses them over a large area, which should mean net progress across the board is higher. Does that make sense?

> Mediocrity will always be everywhere because it's a mathematical fact - <USER4> > people are in the middle. So, yes, it's everywhere. That's tautological.

We all know smart people achieving mediocre results (via Molochian competitions).

We all know not-as-smart people achieving, if not success in the sense of being on the first page of a top ten Google search results, achieving success in their local maximum.

In my opinion the second is in some sense smarter than the first because they are better adapted to their niche, they have a better foundation on which to ultimately progress.

I suppose I could summarize my feeling on the subject by saying I care more about increasing the whole population's IQ (or other metric) by 1 point, than about collecting the 'cream of the crop' and curating talent, because although the second initially appears most impressive, the truth is that the whole network benefits much more from the first.

<comment><USER27> I started responding to your post but after a while I found that we're not on the same page on so many things that you just seem to be stating as fact that I'm not sure if this is bridgeable. All of these things are worthy of a major discussion of their own.

> Sooner or later, like the rise of the Communists, brute strength wins out > over elegant attempts to moderate, basically mob rule followed shortly by > blood rule with tyrants. Year Zero. A Reset. It has happened countless times > in history.

> Democracy leads to entropy. Competition has the potential to take us to > Moloch.

> Mediocrity is a threat (Entropy).

> We all know smart people achieving mediocre results (via Molochian > competitions). We all know not-as-smart people achieving, if not success in > the sense of being on the first page of a top ten Google search results, > achieving success in their local maximum.

These are major statements that you can't really just assume to be true or that the other person finds them true. And I don't agree with any of them. Not to mention the lack of context on highly vague terms like entropy.

A lot of this smells like neoreactionarism.

<comment><USER29> > fter a while I found that we're not on the same page on so many things that > you just seem to be stating as fact

I try to make clear what my priors, assumptions are. And which portions I think are personal opinion.

I don't mind my assumptions being attacked, but I do mind that people "show their work".

How many internet discussions have you had with people who did not clearly delineate what their premise was?

That leads nowhere because the premise has a tendency to evolve (ever more detailed refinement) when disagreement.

If you know what I think you're not shadow boxing with a set of stereotypes.

> These are major statements that you can't really just assume to be true or > that the other person finds them true.

Of course not, but I'm not making an argument for them here. I'm writing an internet comment, not a book.

> And I don't agree with any of them.

That is fine, although I don't really yet know your interpretation of what we see.

> Not to mention the lack of context on highly vague terms like entropy.

Here I must take askance. Entropy is widely used across topic areas but it is not a vague concept. Entropy is another word for randomness and disorder.

In the context of a society, you could say a Dark Age was less ordered than the previous age when the Roman empire ran the show. Typical hallmarks of anti-entropy effort include roads, aqueducts and cities. Ruined cities, disintegrating roads and falling bridges are results of entropy. The social and physical meanings of entropy are intertwined.

If that is not evident to you then you are correct to say our discussion is not bridgeable. If there is one thing I find disagreeable it is a relativism that posits all states are equally good. At that rate people will be flinging their brains into bins.

> A lot of this smells like neoreactionarism.

And what if it is? You shall find everything interesting happens at the peripheral. Conservatism and liberalism are after all, if nothing else, definitely repeated routines of thought. Hard to take good observations there. Of course if there is nothing to fix then there is no need to!

<comment><USER27> > How many internet discussions have you had with people who did not clearly > delineate what their premise was?

Too many. And I often have the same problem since my premises are often far from what is expected. I try to bridge it as I can, but limitations of the human language and text space and all that.

> Of course not, but I'm not making an argument for them here. I'm writing an > internet comment, not a book.

But it seems like you're making an argument /from/ them. Your definition of mediocrity is different from mine (mine is defined as: a person who will by others be considered mediocre in some context, which often resolves to a person who has low-to-medium relative ability in a skill). I don't have a definition of "entropy" to begin with, because it's not really something I care about. Randomness and disorder is the official definition, yes. I can't splice "randomness and disorder" onto reality. Reality seems to actually tend in the opposite direction, at least on a planetary level (i.e., dust -> planets -> creatures on planets). Mediocrity, given these definitions, seems utterly orthogonal to entropy. A football player called "mediocre" by others because he fumbled during a random game, or a person who is moderately OK at tech support being considered "mediocre" because they're not great, to me do not at all contribute to entropy. So I can only imagine your use of the word "mediocre" is something else.

> In the context of a society, you could say a Dark Age was less ordered than > the previous age when the Roman empire ran the show. Typical hallmarks of > anti-entropy effort include roads, aqueducts and cities. Ruined cities, > disintegrating roads and falling bridges are results of entropy. The social > and physical meanings of entropy are intertwined.

My problem is that the definitions are too vague. As well as culturally biased. Rome is more ordered than mud huts. But is Rome more or less ordered than modern USA? Mongol clans? Soviet Russia? Czarist Russia? Maybe there are different flavors of orderliness? I'm not really sure. Not to mention, if you consider it on a global level, you have to sum up the results in some way, and then determine whether the entire Earth is more or less ordered than some other timeframe.

I think there's a difference between "everything is relative and all states are equally good" and "everything is not relative but I'm not convinced on what's more ordered and I don't trust your methods, either".

Nonetheless, I still fail to see the relevance of the football player being called mediocre or the tech supporter not doing too great of a job to the fall of Western civilization.

> We all know smart people achieving mediocre results (via Molochian > competitions). We all know not-as-smart people achieving, if not success in > the sense of being on the first page of a top ten Google search results, > achieving success in their local maximum. In my opinion the second is in > some sense smarter than the first because they are better adapted to their > niche, they have a better foundation on which to ultimately progress.

From the earlier post. This to me sounds like an amalgamation of just-world hypothesis and "nature is good". I don't correlate any specific outcomes to one's "smartness" since I don't believe in either. I don't respect "niches" too much since I don't believe in the latter. I disagree with both, and THAT disagreement is fairly fundamental, it's very hard to find a bridge between the "nature is bad and the world is not fair" and "nature is good and the world is fair" groups, or even the other combinations.

> And what if it is? You shall find everything interesting happens at the > peripheral. Conservatism and liberalism are after all, if nothing else, > definitely repeated routines of thought. Hard to take good observations > there. Of course if there is nothing to fix then there is no need to!

Sure, let's say all the interesting ideas are there, but there is still a problem in assuming a specific vocabulary or axioms when talking to someone who is not already familiar. I have my own framework that's not conservatism, liberalism, or neoreactionism, but I try to keep it more or less under wraps or contextualize parts of it, because otherwise discussion gets very confusing. The whole line about communists and democracy and entropy and year 0 didn't make any sense to me until I considered neoreactionist thought. For other people I think a large portion of the post is likely just gibberish.

<comment><USER29> > Your definition of mediocrity is different from mine (mine is defined as: a > person who will by others be considered mediocre in some context, which > often resolves to a person who has low-to-medium relative ability in a > skill).

That's a definition I'd accept. Perhaps we're not alien species after all ;-)

I definitely see that mediocrity is context dependent. Nobody is a super-man excelling in all. People like Sam Altman and Peter Thiel have rotten days as do us all.

However it seems like there are _pools_ of mediocrity in society. It is not a stand-alone individual phenomena. You don't find many Nobel Prize winners in council estates in Britain. Or any kind of winner apart from the Lottery.

While nobody can excel at everything, we should be a bit suspicious when we see a place where there is mediocrity in everything apart from alcoholism and gambling addictions. That would suggest either extreme assortative mating (highly unreasonable, in the time scale of centuries) or that a group of people is being subdued systemically on the memetic level. What does one say when we see an entire area code suffering from what seems like depression or some existential crisis of purpose? The contrast with former glories is particularly sharp in the North of England, but if you've ever been to a similar region you get it. It is like a giant question mark in the landscape, unexplained. We _should_ see a roughly random distribution of talents, there would be elites, yes, but there would also be a long tail. I don't see much of a long tail in present society in America or Europe, whereas I am convinced there _was_.

These thoughts lead me to the belief that mediocrity is primarily caused by societal structure. This suggests that talent is allowed to be exposed sometimes and not others, which depending on how you look at it, is either a very dystopian view of society and/or a potentially very hopeful one.

And of course we are seeing these same 'pools' arising in Science and Education. Whatever the 'rot' is, it is surely spreading.

> But it seems like you're making an argument /from/ them.

I have a proselytizing streak in me. When I believe something is _so_ , then others must _know_. However like yourself I am not a purist, I cherrypick my way through various lines of reasoning and often am of two or more contradictory minds on one topic. Currently I believe NRx's positions on Western society and its politics (The Cathedral) are useful insights with predictive power. I think also that every model has limits. The utility of Conservatism and Liberalism has run its course in the West and now it is time for something new or old again. In the Asian countries such as China this may not be true.

> Randomness and disorder is the official definition, yes. I can't splice > "randomness and disorder" onto reality.

> Reality seems to actually tend in the opposite direction, at least on a > planetary level (i.e., dust -> planets -> creatures on planets).

Let us stick to centuries or thousands of years at most!

I think depending on the scale of resolution you look at physical reality, you'll see different things, but this is another topic.

> My problem is that the definitions are too vague. As well as culturally > biased. Rome is more ordered than mud huts. But is Rome more or less ordered > than modern USA? Mongol clans? Soviet Russia? Czarist Russia?

I can easily imagine comparing them! There are absolutes, like average life expectancy, and then confounding factors such as efficiency gains making less things to measure. As a brute estimate, you could multiply the number of years of life by the number of people the empire or system can support. That by itself should give us a fair guess at complexity. Or we could look at density, since average city size should correlate with complexity.

> Not to mention, if you consider it on a global level, you have to sum up the > results in some way, and then determine whether the entire Earth is more or > less ordered than some other timeframe

We already do this really. GDP. I realize all these metrics will have flaws, but if you are consistent about using them you can obtain a fair idea of trends and won't miss unusual events e.g. The Black Death. If GDP is negative from 2017 - 2117, projektir will be far from shocked if armed with that information he/she steps from the Time Machine and observes humankind.

> I think there's a difference between "everything is relative and all states > are equally good" and "everything is not relative but I'm not convinced on > what's more ordered and I don't trust your methods, either".

That is fine, my methods I just came up with a minute ago notwithstanding there may be good objective metrics for orderliness.

> I still fail to see the relevance of the football player being called > mediocre or the tech supporter not doing too great of a job to the fall of > Western civilization.

Many functions in society are options or choices.

Items like the ability to travel over distances, to not be killed, to feed and water oneself, to provide for a family, these are not really options. If enough people can't accomplish those kinds of tasks then society does actually collapse.

Fundamentally, are you saying that if the symptoms of a degeneration appear in a society, that the people of that society just don't or can't be trusted to recognize them?

I mean we have records, letters from Rome and Egypt, and they seem to indicate a very acute sense of catastrophic decline.

> From the earlier post. This to me sounds like an amalgamation of just-world > hypothesis and "nature is good". I don't correlate any specific outcomes to > one's "smartness" since I don't believe in either. I don't respect "niches" > too much since I don't believe in the latter. I disagree with both, and THAT > disagreement is fairly fundamental, it's very hard to find a bridge between > the "nature is bad and the world is not fair" and "nature is good and the > world is fair" groups, or even the other combinations.

You don't believe some people's native intelligence is higher or lower than others?

You don't believe societal niches exist?

Perhaps this is a straight disagreement but maybe I just don't understand what you're saying.

> but there is still a problem in assuming a specific vocabulary or axioms > when talking to someone who is not already familiar.

Usually, true, but not in these circles I find, after all you understood what I was saying.

> The whole line about communists and democracy and entropy and year 0 didn't > make any sense to me

Entropy I explained, Year Zero is a reference to the Khmer Rouge.

[https://en.wikipedia.org/wiki/Year_Zero_%28political_notion](https://en.wikipedia.org/wiki/Year_Zero_%28political_notion)

The only thing I can think to add here is that in NRx thought Democracy is taken as being Communism Lite, like how there exists a Diet Coke soda.

> For other people I think a large portion of the post is likely just > gibberish.

That is partly by design. Journalists have short attention spans and suffer from buffer overflows. They then fall back on 'Fascists!', which not even they wholeheartedly believe.

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER30> While I'd like to slag science for this, we also have a bit of counterexample in other fields.

In art, we have studies that show that those who produce _more_ also produce _better_. And it compounds. There is no reason to believe that at least some of this isn't operant in science.

The real problem is the lack of positive incentives for negative results. And I don't know how you fix that.

<comment><USER31>
      we have studies that show that those who produce more also produce better.

This is true of people producing more of _the same thing_. An artist who makes a lot of pottery doesn't increase their ability in painting, or piano-playing.

In other words, do _the same thing_ lots of times, and the results become better and better. This sounds a lot like replication in science.

</comment></comment><comment><USER32> "Science" has now many definitions. My personal understanding of "science" is this: [https://www.youtube.com/watch?v=EYPapE-3FRw](https://www.youtube.com/watch?v=EYPapE-3FRw)

</comment><comment><USER33> Would it help to move away from print journals, to online journals with comments sections so the community can discuss any potential flaws?

Publishing the data seems like it would help, although might not be possible where there are privacy concerns.

<comment><USER34> The active, open, free discussion on findings and theories has been regarded as one of the big ideas of modern science. That's what journals were for.

However, for some reason I'm doubtful if the "online comments section" is the most conductive for the behaviour we want to encourage, after witnessing how that particular concept did manifest in the newspapers when they went online (compared to the traditional opinion / to the editor page).

</comment></comment><comment><USER35> >"Not only are dodgy methods that seem to produce results perpetuated because those who publish prodigiously prosper—something that might easily have been predicted. But worryingly, the process of replication, by which published results are tested anew, is incapable of correcting the situation no matter how rigorously it is pursued."

This isn't referring to science, and these studies aren't being done by scientists. It is research, and these are researchers. I wish people would stop dragging science through the mud over these problems.

</comment><comment><USER8> Is it just me, or is there a lot less "bad science" being done in chemistry and (testable) physics?

Maybe one way to decide if a discipline is "scientific" in nature is how well the scientific process actually works within that discipline?

And if we discover evidence that the scientific process doesn't work well in a particular discipline (for whatever reason—doesn't matter why, actually), maybe we should stop calling that discipline a "science" and come up with some other descriptor?

<comment><USER26> What evidence/data have you used to form your opinion in regards to hard sciences? I don't necessarily disagree, but considering the subject matter, I feel compelled to press you for a data-supported justification.

<comment><USER36> I've seen copious bad papers in EE and MechE journals. I'm not qualified to evaluate chem or physics research, but you'd think if any field would be hard to fudge, it would be engineering. Turns out this is not the case. The bad engineering papers I've seen tend to draw overly-broad conclusions while failing to document the methods adequately.

</comment></comment></comment><comment><AUTHOR> BMC Psychology is trying out "results free" peer review - reviewers judge only the method then decide whether to publish [http://www.the- scientist.com/?articles.view/articleNo/47081/...](http://www.the- scientist.com/?articles.view/articleNo/47081/title/Reviewing-Results-Free- Manuscripts/)

<comment><USER37> That would be great if it were a form of pre-registration, but since nobody puts p-hacking in the method section, it's a little pointless.

</comment></comment><comment><USER38> We can generalize: for all X, bad X persists because some incentive structure makes it locally optimal for people to produce bad X.

<comment><USER39> Well, it's fundamentally easier to fail than it is to succeed. Especially when you try to do something difficult.

</comment></comment><comment><USER40> 1 measure all the things

2 regress variables until p < 0.05

3 publication!

</comment><comment><USER41> Poor methods which is a result of inadequate data/descriptions being provided by researchers to the publication (also an issue with publications not requiring it).

In many of the hard sciences there is not a requirement to list the products used such as chemical reagents or plasticware in a given experiment.

<comment><USER42> The publications don't seem to _want_ adequate data and descriptions. They want papers that have the traditional format and fit in 6-8 pages.

Every paragraph you spend on practical details is one less paragraph you can spend on making a good pitch for the relevance of your work, and citing lots of people in "Related Work" so that the reviewer's favorite people are in there.

They don't want your code, because that would interfere with the supposedly blind review process.

Occasionally, I've seen a submission process that includes a way to submit supplemental data, but it's always extremely half-assed. There's no guidance about what this supplemental data should be beyond "maybe an Excel file or a zip file or something", nobody will ever see it, and if you include too much data it probably gives you an error and maybe even crashes and makes you start over.

</comment></comment><comment><USER43> "A new scientific truth does not triumph by convincing its opponents and making them see the light, but rather because its opponents eventually die, and a new generation grows up that is familiar with it" -Max Planck

</comment><comment><USER44> similar to:

[https://www.theguardian.com/science/2016/sep/21/cut- throat-a...](https://www.theguardian.com/science/2016/sep/21/cut-throat- academia-leads-to-natural-selection-of-bad-science-claims- study?CMP=share_btn_fb)

</comment><comment><USER45> Why, being a member of a popular rich sect is more comfortable and rewarding than being a seeker after truth.</comment>


Git for Windows: Change non-inclusive naming - drevil-v2 https://github.com/git-for-windows/git/issues/2674 <comment> drevil-v2 "Don't you see that the whole aim of Newspeak is to narrow the range of thought? In the end we shall make thoughtcrime literally impossible, because there will be no words in which to express it."

\- George Orwell, 1984</comment>


That China Parenting Article - <AUTHOR> http://paul.kedrosky.com/archives/2011/01/that_china_pare.html

<comment><USER2> I saw this a couple days ago and was turned off of her opinion just by the arrogance of her tone.

She is very extreme. It's a case where something "works" for her kids though I'm not sure the end goal, so she jumps to all kinds of conclusions. Besides the obvious questions of whether her kids are actually going to be happier or more successful in the longrun, she thinks that every part of what she's doing is what makes it work. I think not owning a TV is good for kids, but I can't see where most of this other stuff would help the child.

From my experiences watching high schoolers in China I am very happy I had lax western parents. Many will study 14 hours a day and are not allowed to date or hang out with friends.</comment>


How to fix knockout/playoff tournament format - <AUTHOR> http://letuscode.blogspot.com/2009/09/why-every-other-playoff-system-sucks.html A nice hack for fixing the knockout tournament format. The discussion on soccer (http://news.ycombinator.com/item?id=850202) reminded me of this piece I wanted to write about for a long time. <comment><AUTHOR> A hack for fixing knockout tournaments. The discussion on soccer yesterday (<http://news.ycombinator.com/item?id=850202>) reminded me of this piece I wanted to write.</comment>


What Brands Are Actually Behind Trader Joe's Snacks? - <AUTHOR> https://www.eater.com/2017/8/9/16099028/trader-joes-products <comment><USER2> In the SF Bay Area, their European-style organic yogurt is supposedly supplied by the wonderful Straus Family Creamery in Petaluma, Marin County. After I moved to Rhode Island, the same great product was available in my local TJ's store. How do they do it?</comment>


Google Antitrust Complaint Threatens Plan for Auto Dashboard Dominance - <AUTHOR> http://fortune.com/2019/05/17/google-android-auto-enel-antitrust/ <comment><USER2> What happens to the car when the computerized dashboard crashes or fails an update leaving it in a crashed state?</comment>


First Randomized Trial on Parachutes Finds They Are Not Safer Than No Parachute - <AUTHOR> https://motherboard.vice.com/en_us/article/d3m5d7/are-parachutes-safe-skydiving-deaths-randomized-controlled-trial <comment><USER2> This is like an onion article except it's not very funny

</comment><comment><USER3> The humor is kind of lost by not linking to the BMJ.

[https://www.bmj.com/content/363/bmj.k5094](https://www.bmj.com/content/363/bmj.k5094)

</comment><comment><USER4> This is absurd. A waste of 5 minutes.</comment>


Herbert R. J. Grosch, 1918-2010 - <AUTHOR> http://bit-player.org/2010/herbert-r-j-grosch-1918-2010 <comment><USER2> _Grosch had proposed a ternary architecture for the Whirlwind, Jay Forrester's enormous early electronic computer_

I have the notion that, in one of his novels, Heinlein "let slip" that the story was not only in the future, but also in an alternate timeline, by casually mentioning ternary computers.

Which reminds me - in early Heinlein stories, analog computers were "de rigueur", and the other day, in a Fortune mag archive someone posted, I bumped into this 1949 summary: <http://www.fulltable.com/vts/f/fortune/gschwind/08uu.jpg>

(Part of an amazing set of illustrations by Max Gschwind, indexed at <http://www.fulltable.com/vts/f/fortune/gschwind/a.htm> \- the CAD/CAM workstation of 1956 and the lunar lander of 1962 are neat too.)</comment>


Ask HN: Is it worth learning a new language for your startup?  - <AUTHOR>

<comment><USER2> I assume you mean programming language.

If you're a programmer, it is good to periodically learn the basics of new languages. You won't be expert in them, but it is less work than you probably fear, you'll know how to find your way around, and you'll come back to your main languages with greater perspective.

Whether it makes sense to learn one for your startup is a different question. Beware of the fact that while working on a startup there will always be plenty of opportunities to engage in yak shaving. (See [http://projects.csail.mit.edu/gsb/old-archive/gsb- archive/gs...](http://projects.csail.mit.edu/gsb/old-archive/gsb- archive/gsb2000-02-11.html) if you don't know what yak shaving is.) Letting yourself get consistently sidetracked is a quick way to fail to produce anything.

 _However_ it is also true that there are many technologies that your startup might benefit from which require you to know specific languages. For example you might want to use Puppet, but that requires you to know some Ruby, which you might not know. (Ditto Chef and Python in the same problem space - so you get a choice of new languages to learn!)

With all of that said, here is my answer. Ask yourself why you want to learn the language, what the benefit will be, and how long you'd give yourself to do so or pull the plug. Decide that up front. If you decide that it is a worthwhile investment of your time, then plunge in, but DON'T TRY TO MASTER IT! Instead recognize that for whatever you need to do you likely only need to understand some _subset_ of the language. Try to identify that subset, learn that, and then get done what you need to get done. (A reasonable way to learn that subset is to focus on learning what seems necessary to start working on your problem, then learn everything that you encounter as a blocker while trying to solve said problem.)

But put boxes and limits around this task. Because your goal should be to get something out there and iterate. Anything that distracts you from that goal needs to be removed for that goal.

</comment><comment><USER3> If you are not a programmer, then get one for your startup but if you are, then stick to the language you already know because thats the best way you can express yourself (provided a startup is concerned). You can learn a new language later but not during startup. Starting a startup is a journey you can only go with familiar tools

</comment><comment><USER4> for a real startup no. a startup must start up fast. for that the best tools are what you already known.

but for side projects a new language is absolutely a must!

<comment><AUTHOR> Yes, that's what I thought but wasn't sure!

<comment><USER5> I don't agree with the 'real startup' comment. For howareyou.com I learn't Ruby, and chicly.co.uk I learnt Objective-C and I would say, although maybe bias, both are 'real startups' :) Choose the right language for the job and be prepared for that language to change as you encounter different challenges.

</comment></comment></comment><comment><USER6> In most cases, no. If you are very early, you can achieve product market fit for almost anything, with any language. Then, you can change the language later or develop tools to make the language better fit your needs.

</comment><comment><USER7> I would say it's worth learning a new language if that language will achieve whatever goal(s) you have for your startup.

</comment><comment><USER8> It depends!

If you can afford to hire a programmer to code for you it'd be fine, so you can focus on other things.

</comment><comment><USER9> Is your goal to build something or to learn how to build something?

<comment><AUTHOR> I have an idea and want to implement it myself. I'm confused between web and mobile prototypes of my future product! to make it on mobile I have to learn a new programming language such us Objective-C.

<comment><USER9> Learning a new language _for a new platform_ is a little bit different.

Learning a new language when you have an existing skillset that can do the same job [eg switching from ruby to python for fun] is a costly investment that probably won't pay off.

</comment></comment></comment><comment><USER10> Do _you_ want to learn a new language? Then go for it!

</comment><comment><USER11> Depends on what you are building.</comment>


What Is the Oldest Code Written Still Running? - <AUTHOR> http://developers.slashdot.org/article.pl?sid=08/05/11/1759213&from=rss <comment><USER2> If you take a rigorous view of the question (and don't, for example, take some early COBOL test code and run it on an emulator) the actual answer is probably some old control code operating an industrial servo motor. Or similar.

</comment><comment><USER3> Windows 3.1 Font Dialog Still in Vista.

Win 3.1 was released in 1992, yikes!

[http://bink.nu/photos/news_article_images/images/13552/origi...](http://bink.nu/photos/news_article_images/images/13552/original.aspx)

</comment><comment><USER4> The simulation that we are all living in.

</comment><comment><USER5> I heard from a buddy who worked on IE7 while he was a summer intern that the IE team still had code from IE 1.0 sticking around.

</comment><comment><USER6> The space shuttle and Nasa command center

</comment><comment><USER7> Hammurabi's</comment>


Lessons I learned while creating my SaaS with Rails - <AUTHOR> http://samurails.com/projects/7-lessons-i-learn-while-creating-my-new-saas-yoursetup-io/ <comment><USER2> Extra (maybe obvious) tip for #7: Get yourself a free Slack/Hipchat account and start curling those errors and exceptions to it. Super easy, way less hassle than email, and free push notifications to your phone if you've got the app installed.

<comment><USER3> Also do that with new relic. I finally found peace at night again.

</comment></comment><comment><USER4> Good writeup for your experience with Rails and EBS. I've built my project, Zejoop dot com, in Java/Groovy/Grails and deployed on AWS Elastic BeanStalk - and I'm very happy with my setup so far.

As for your comment about the multitude of services available on AWS, I agree. They've recently added ACM certificate manager which simplified my maintenance considerably.

Their ACM easy implementation probably saved me at least 20 hours of cert renewal agony, plus it was free!

</comment><comment><USER5> You're going to cry the day when you try to clone your Elastic Beanstalk environment to do a rolling upgrade, and the termination of the first wipes away your RDS db with it.

It's undocumented, but you'd be better off creating the db separately, not within the beanstalk wizard, to avoid this

<comment><AUTHOR> Wouah, thanks for the heads up. I had no idea about this, I'm gonna look into it and re-create the RDS db myself.

<comment><USER5> Take one of your RDS daily snapshots (which you're taking of course), and recreate a new instance from it. Point your app to it, and you should be good.

</comment></comment></comment><comment><USER6> Good post. But "[http://www.yoursetup.io/"](http://www.yoursetup.io/") is returning 502s most of the time... Maybe something's wrong with your setup :/

</comment><comment><USER7> Thanks for the insight into Elastic Beanstalk; I'm looking into it now.

Regarding Coffeescript and classes, I think that's a really poor reason to use it. IMO Coffeescript has had its day and if you want class sugar, just use ES6. You're already using a compilation step in your process so why not use it to work with something standard?

<comment><AUTHOR> You're probably right indeed! I was just looking into refactoring some code, found that and thought it was pretty cool. Plus Rails comes with CoffeeScript by default.

ES6 is pretty sweet (pun intended) and will probably make CS kinda useless in the future. I'll look into it however I think that currently CoffeeScript is a bit of a standard in the Rails world.

<comment><USER7> That is a fair point, if I'm using Rails I actually just use plain old JS as I don't personally like CS.

I just found a comment by a Rails contributor [1] which says ES6 will be supported out of the box in Rails 5, which would be ideal. I can't find anything newer to back that up though. I think it might be via ES6 support baked into sprockets.

[1] [http://kryptonlabs.com/blog/2015/10/25/whats-new-in- rails-5/...](http://kryptonlabs.com/blog/2015/10/25/whats-new-in- rails-5/#comment-2326469442)

</comment></comment></comment><comment><USER8> Off topic in regards to this specific blog post but if you haven't already check out mr money mustache's blog. I think his philosophy mirrors your own based on reading through the rest of your posts on creating financial freedom.

</comment><comment><USER9> The actual site (not this article) is currently serving up a 502; I think the author needs to play with his autoscaling options in EB!

<comment><USER10> Or just, you know, cache it correctly.

@OP there's a more DHH way of caching, but these will serve you well in the meantime: [https://github.com/rails/actionpack- page_caching](https://github.com/rails/actionpack-page_caching) and [https://github.com/rails/actionpack- action_caching](https://github.com/rails/actionpack-action_caching)

</comment></comment><comment><USER11> Great blog post! Can't you use Angular or React with RoR when rendering in the backend? I am pretty sure you can.

<comment><AUTHOR> Yes, you can just use Rails as an API and create a frontend SPA. But in my opinion, it's faster to write in the 'old way' for a MVP instead of creating 2 different applications (one frontend, one backend). For the future, I'm actually thinking about rewriting the frontend using SPA technologies.

<comment><USER12> Way faster. We are using a rails API and a React+Flux app as a front end. The whole setup time alone is a considerable pain. I feel that the JS tooling is still too fragmented and sometimes messy, but I'm probably biased by years of rails and iOS development. Writing the app takes more time since you need to write everything from the ground up (e.g.: no Devise views to just style and forget), but I really hope that in the future the investment will pay. I wrote about it a while ago (although now we are using the more terse Alt implementation): [http://fancypixel.github.io/blog/2015/01/28/react-plus- flux-...](http://fancypixel.github.io/blog/2015/01/28/react-plus-flux-backed- by-rails-api/)

<comment><USER13> Completely agree - just setting up a workable environment for webpack is a disastrous exercise. Even things like cache-busting signatures, etc are a implement-it-yourself deal.

</comment></comment></comment></comment><comment><USER14> Elastic Beanstalk drove me nuts. I went with Opsworks and was much happier.</comment>


How We'd All Die Instantly If The Sun Suddenly Went Supernova - <AUTHOR> https://www.forbes.com/sites/startswithabang/2019/12/27/this-is-how-wed-all-die-instantly-if-the-sun-suddenly-went-supernova/ <comment><USER2> For anyone interested in a fictional take on a similar event, see Cixin Liu's Supernova Era.</comment>


PyPy for low-latency systems - <AUTHOR> https://morepypy.blogspot.com/2019/01/pypy-for-low-latency-systems.html <comment><USER2> Reminds me of a time at Quora in 2011 where we saw Python GC impact 99th percentile server-side site speed. So drawing from HFT inspiration where some companies would disable JVM GC during trading hours and perform them offline, I thought about how to take some backends periodically offline in order to have GC not happen on user requests. A simpler operational solution emerged though where I just had to disable GC on user requests and make it happen only on a special "/_gc" endpoint. I then dual purposed the frequent nginx/haproxy backend health-check functionality to use that endpoint, thereby ensuring all backends had frequent GC and the time spent there only impacting the health check requests, and not that of end users.

edit: added more details I remembered later

<comment><USER3> This is a very interesting approach, what happened with memory footprint when you did this?

<comment><USER2> Thanks, don't think I saw much impact at all in aggregate - our memory consumption on these web servers were dominated by objects we intentionally stored per request or globally, and not temporary/unreferenced python objects.

<comment><USER4> Even while GC is delayed Python (CPython at least) will free some stuff through reference counting. Only circularly referenced stuff should stick around until the next GC run. So that can avoid lots of stack temporaries and stuff.

<comment><USER5> Theoretically with your code structured right you can disable the cyclical garbage collector outright: It only deals with reference cycles which you can explicitly avoid by using the weakref module.

Not entirely sure how you'd go about writing code like that, but it's possible.

</comment></comment></comment></comment></comment><comment><USER6> Nice examples and graphs. What really confuses me is the definition of "low- latency" nowadays. The meaning suffers a slippery slope in recent years. It used to refer to a scale of microseconds at HFT shops, then it came to web request latency at a scale of milliseconds. Now every GC-based language claims they are "low-latency" because 90%/95%/99% GC stop is within 16.67ms/50ms/whatever. Therefore today some HFT developers invented words like "ultra-low latency"[0] to name their work.

[0]: [https://en.wikipedia.org/wiki/Ultra- low_latency_direct_marke...](https://en.wikipedia.org/wiki/Ultra- low_latency_direct_market_access)

<comment><USER7> On the flip side, I've also heard "low-latency" (or even "instant") to mean "within 30 seconds", as an upgrade from "the mainframe runs a batch job every hour (or day)".

I don't think there's any way to consider a phrase like "low latency" without considering what you're talking about.

<comment><USER8> Good point, the word "low" itself is relevant.

</comment></comment></comment><comment><USER9> This could possibly be combined with multiprocessing for great effect? I'm imaging something like having a pool of workers executing tasks (/reacting to events/serving requests/etc), and only running gc after the task has been done, but before indicating readyness to the supervising process (/loadbalancer/etc).

<comment><USER10> Even a very simple mode with two processes that never have GC enabled at the same time would greatly improve things.

</comment></comment><comment><USER11> Seems quite useful, allowing the developer to guide the garbage collector into the right direction by carefully placing statements to tell the garbage collector when it is (not) ok to run. But make sure to add some comments describing the purpose of those statements, and how to profile your code to check whether it's still working correctly. Don't want to accidentally stop garbage collection altogether, either.

</comment><comment><USER12> This is a nice bit a progress and addresses a major concern about using PyPy in real-time systems.

</comment><comment><USER13> I'd be curious to see if any of the work done here could be applied back to the main CPython project. I doubt it could happen immediately -- at least, not with the way GC is currently implemented -- but PyPy has been a source of innovation for CPython in the past (see: new dict implementation).

<comment><USER14> CPython already has a lower latency GC than PyPy, gc.disable() already works, and allowed manual memory management when needed.

Reference counting allows you (if needed) to keep references to memory in your python code, and free them in the right spots.

This is PyPy becoming useful for a _lot_ more production use cases. From web APIs that have a latency SLA, to audio, games. In many cases peak performance is not important, it's the minimum performance.

<comment><USER15> Refcounting comes with its own in-thread gc pauses whenever you exit a block or context and the local variables are collected.

<comment><USER14> Yeah. However you have the option to not pause if it is important. You can control where the memory management happens. You can either keep references to the memory, and call gc.disable(). When you are ready you can let go the references and enable the gc.

PyPy now lets you control where memory management happens. Making it possible to control worst case performance. For many production apps this is a big deal.

<comment><USER15> You can never prevent the GC cycle in CPython at the end of a block (context). You can only prevent the GC that tries to break reference cycles. If your class does crazy things at destruction, like "time.sleep(10)", and you create an instance of the class inside a function, when that function returns you will pause CPython even if you call gc.disable()

You also cannot disable the minor collections in PyPy, only the major collections, but once the JIT kicks in PyPy can prevent some of the object churn by optimizing instances away.

<comment><USER14> Yeah. Avoiding slow things like classes, threads and adding time.sleep(10) is the trick.

</comment></comment></comment></comment></comment></comment><comment><USER16> But a new event could be submitted at anytime.

This is certainly an improvement, but not a complete solution.

<comment><USER17> Ideally you have enough copies of the server process to handle the events that come when another process is running GC.

You already have to have enough of them to handle events while other workers are busy.

</comment></comment><comment><USER18> Relevant: "Blade: A Data Center Garbage Collector" (2015, [https://arxiv.org/pdf/1504.02578.pdf](https://arxiv.org/pdf/1504.02578.pdf))

Terrible title, but basically the same idea.

</comment><comment><USER19> _Combining these two functions, it is possible to take control of the GC to make sure it runs only when it is acceptable to do so._

I'm conflicted on this. My gut tells me if I'm going to manually take control over the garbage collector I should reconsider my design decisions.

Disclaimer, I don't know the first thing about PyPy or Gambit Research, so probably this is the right approach for them?

<comment><USER20> It's explained in the article that this is to solve one specific issue that Gambit Research had: in some parts of their code they need to take action with very low latencies (<10ms) and hence they can't wait for the GC. This way they manually execute the GC in other sections of the code were timing requirements are relaxed.

</comment></comment><comment> alex-wallish This is awesome!</comment>


Photosynth 3D - <AUTHOR> http://photosynth.net/preview

<comment><AUTHOR> Microsoft was showing the new version at Web3D 2014 a couple of hours ago:

[https://twitter.com/mattatheque/status/498537142386049025](https://twitter.com/mattatheque/status/498537142386049025)</comment>


Backlash over 'single-shaming' banking ad - <AUTHOR> https://www.bbc.com/news/business-47117649 <comment><USER2> Society has become intolerant and totalitarian. Everything causes outrage and lynching on social media.</comment>


Plume is turning home Wi-Fi into a subscription service - <AUTHOR> https://www.theverge.com/circuitbreaker/2018/6/12/17416982/plume-superpod-adaptive-wifi-subscription-service-announced <comment><USER2> Apparently it is the week I really live up to my current HN username.

I'm sorry I _do not want_ a subscription for everything. There are a few things that make a lot of sense on a subscription. Phone service, Netflix, cloud services etc. Car leasing can make sense.

There are an awful lot of things that are being forced into a subscription model. Editors, routers, throwaway printers with their delightfully expensive DRM'd inks. Purely as a means to squeeze a little more cash out of the punter.

 _" It seems inevitable that subscription services will come to more routers. We rarely buy new ones, and subscriptions give these companies a way to keep making money"_

It does? Why? After 30 years of free, mostly junk, modems and routers from the ISP that rarely got replaced? Routers get replaced when the magic smoke escapes. Unless you're an outlier, like me, who reads HN the only interest you had in the router was connecting the phone and laptop when you unboxed it.

No, can't make sense of this.

No, I don't want an extended warranty with my £29 toaster either.

</comment><comment><USER3> It could make sense if you were essentially _leasing_ Wi-Fi gear, so that you would always get the latest and greatest as the Wi-Fi standard evolves - receive new gear and send back the old gear every year or so.

But this does not make any sense. You are already _buying_ a router, and they're tacking on a subscription on top, for what appears to be snake oil.

<comment><USER4> Wifi doesn't work like that. Replacing all your installed wifi gear every week with the latest and greatest doens't improve much if you aren't also upgrading your individual devices. There is some room to play, but most all developments in wifi need upgrades to both routers and devices.

Furthermore, as wifi is a shared medium, your experience is heavily dependent on the behavior of your neighbors. Routers can hop channels all they want, that doesn't make more room in the spectrum. If you want to stream a billion gigs per second over your fancy AC++ router, something that requires exactly half the available spectrum, you better live deep in the woods.

<comment><USER5> It may not, but it'd make more sense than their current offering.

</comment></comment></comment><comment><USER6> The ars article about it has a lot more info about why they're doing it and how it works etc.

[https://arstechnica.com/features/2018/06/exclusive-plumes- ne...](https://arstechnica.com/features/2018/06/exclusive-plumes-new-superpod- hardware-is-here-and-its-fast/)

<comment><USER7> This helped frame the subscription for me:

> ...If you let an annual membership lapse, your pods do not turn into > paperweights; they just revert to the same status they had when you first > turned them on. That means they'd have a very simple "we found each other, > good enough" topology rather than a more optimal one determined by Plume's > cloud NOC; you'd also lose any other features dependent on the NOC, like > client-specific rulesets. > The other perk of a membership in good standing > is a rolling lifetime warranty for all pods... even after the original > design is no longer manufactured. If you have a v1 pod that fails in 2022, > you'll get a replacement with whatever 802.11ax-compliant replacement design > is current then, for example. > I'd recommend that anybody who's super angry > about the "membership" concept to look into an apples-to-apples comparison > that encompasses this lifetime cost. For example, in this article I compared > a four-Superpod startup pack at $259. Add a $200 membership and it's $459 > compared to the cost of an Eero + 2 Beacons (currently $389) or an Orbi > RBK-53 (currently $530). If you're on the fence, you can make the initial > buy with an annual membership and upgrade the annual to lifetime with a > prorated refund of the remaining time on the annual. There's also a 60-day > full-refund policy on both the hardware and the membership fee if you decide > the system is not for you.

This seems a bit better than my initial impression, but I hope their messaging around the subscription frames it similarly to this article.

<comment><USER8> > Update June 12th, 12:10PM ET: After publication, a Plume representative > reached out to say that the company had changes its policies between my > phone call with Diner and today. The company will, in fact, require a > subscription for Plume Pods to stay functional.

<comment><USER6> Sheesh. They just got taken back off my list. That's just so dumb.

</comment></comment></comment></comment><comment><USER9> I don't quite understand the negativity. Everyone complains about how IoT devices are a security nightmare because manufacturers don't update the software. Now you are paying the ongoing cost to get that engineering done.

You could pay upfront, and many people do (that's what "enterprise" is and why it's $2000 for a $200 router)... but now the price of ongoing support is in reach of the average customer. (But they're mad that ongoing support is actually expensive.)

Now of course, if they do the same half-ass job that other router manufacturers do with ongoing support, the subscription fee won't be worth it. But if the product does continually improve relative to other offerings, then maybe it's the right business model.

Anyway, it is pure honesty to split out the hardware cost and the software engineering cost. The question is whether or not it's actually worthwhile to pay the software engineering cost. It's a pretty new idea, and there are plenty of reasons to not trust it... but in theory, I like it.

<comment><USER10> The negativity is easy to understand. Somehow, Apple managed to sell routers for 20 years, and keep them updated for over a decade after purchase, without charging exorbitant fees for routine software updates. How did they manage that? They didn't do it in a "halfass" way, either.

<comment><USER9> How sustainable a business was that, though? Obviously it was not making them huge amounts of money, because they stopped doing it.

</comment></comment></comment><comment><USER11> Great, so now in addition to paying a monthly fee for internet service and a monthly fee for the modem rental, you also have to pay a monthly fee for your router?

<comment><USER12> What service doesn't let you outright buy the modem?

<comment><USER13> Comcast let me use my own router, then decided it was theirs and charged me rent anyway.

A few years later, I was stuck with them again in another state and bought my own from their approved list...then a few months later that model was no longer approved and any issue was blamed on using an unapproved/outdated model.

<comment><USER14> Same. I used to have a monthly calendar reminder to call Comcast and complain about the modem rental charge for my personally owned modem.

Every month they would agree to waive the fee and promise to fix it, and every month it would be back on my bill.

This went on for about a year until I cancelled my service, and then they hounded me weekly to "return" the modem.

</comment></comment></comment></comment><comment><USER15> Is there some sort of consumer psychology behind this whole 'as a service' push? It seems like monthly bills kind of go in the back of people's heads and they rarely consider the full cost in the way they do normal purchasing. Maybe we should have a law surrounding subscription services since it seems like they're making increased profit off exploiting a tick in the human psyche as opposed to actually making a better product.

<comment><USER16> It's the same "thinking" that makes many people care more about low monthly car payments by taking out 6- or even 7-year car loans. They totally ignore how much that is actually costing them to get a new, shiny car that they really can't afford or the fact that their new car will only be worth a fraction of what they paid for it at the end of the loan. There's certainly no law preventing car dealers from taking advantage of that. Why is this different?

<comment><USER15> Interesting, I'm actually testing out my own thought here. I guess I would say there is a difference between these two scenarios:

(1) I can't afford to outright purchase a product that would normally cost $10,000 so I choose to finance it and pay $12,000 over 5 years (or whatever.)

(2) I could afford to outright purchase a product that would normally cost $100 but instead I have to pay over time, say $500 over 5 years.

It seems to me that a lot of the more recent SaaS stuff is associated with a very large price increase probably because there's no on-prem price to compare to, no stated rate of interest or any of that information you get from a car loan. So I suppose my problem is that the premium on these products is much higher due to obfuscation and these companies (like Plume) don't offer a comparable option to buy outright.

</comment></comment></comment><comment><USER17> This seems identical to Meraki's ([https://meraki.cisco.com/](https://meraki.cisco.com/)) business model, but the enterprise space seems like a lot smarter way to push a subscription model.

</comment><comment><USER18> All these comments dismissing this model are just reacting to the flippant tone in the article. ~70% of service calls to ISPs are for issues that are within the home, be it interference, or a misconfigured or even unplugged router. Managed WiFi, where a rented router is remotely administered, is a very good way to deal with this stuff. It greatly reduces the number of truck rolls that an ISP has to do. Sure, maybe ATTs service sucks, but everything they do sucks. I am a little puzzled by the choice to sell this straight to consumer, since most managed WiFi manufacturers sell their routers to the ISP to be leased to the consumer (or supplied free).

<comment><USER19> They never claimed to provide support. Only something vague called active management. What could you possibly imagine that would actually be that would actually help anyone in any actual real-life case?

<comment><USER18> Typically managed wifi involves both remote administration of the router(s), and phone support. Having the remote access means that the user can just pick up the phone and describe the problem, and the technician can solve it remotely, or with a little assistance from the user "try moving the router over near the door". I assume that they are doing this as well.

<comment><USER19> wiling to bet that they are not. Callcenters and people do not scale.

</comment></comment></comment></comment><comment><USER20> If we held the companies responsible for the security of their IoT devices, this wouldn't be necessary. Instead none are liable when our smart refrigerators get hacked.

Let's change that instead of pushing the cost on consumers.

<comment><USER21> Reminds me of paying for identity protection services.

</comment></comment><comment><USER22> It never ceases to amaze me how arrogant some tech startups can be with their pricing. Plume is one of many that seem to target consumers that simultaneously have a lot of disposable income and are stupid with their money.

A subscription fee for a router that I still need to pay for, on top of the internet that I'm ALREADY paying for? Get outta here. How does it make any sense to charge subscription pricing for hardware that you have to pay upfront for?

</comment><comment><USER23> This seems like just a money grab. No thanks.

</comment><comment><USER24> How about this?

$10/month for three units with free hardware upgrades every 3 years (and no upfront cost).

That comes to $120/year, and I don't have to worry about replacing the hardware to get the latest wi-fi standards / features / hardware fixes.

Google wi-fi is $299 for 3 units. Expected life time is 3 years before obsolescence sets in.

<comment><USER25> How about no. I just bought a single unit and it covers my entire house and gets security updates for a few years and it costs way less than $360 every 3 years. I literally can buy multiple routers and come in less than that for better than what they offer. Not to mention that this "model" requires a government mandate to work otherwise cheap Chinese knock offs with good enough security will flood the market. As a liberal, I've seen enough of the government and their idea of "the internet".

</comment></comment><comment><USER26> > _Plume's subscription service will cost $60 per year, or $200 for a lifetime > membership._

I really, really hope that buying the lifetime membership unlocks your pods (or whatever, or however it works) - and that Plume going out of business doesn't fuck you over and lock you into the "free" tier.

<comment><USER27> It sounds like non-free-tier stuff happens on their servers, so Plume going out of business would lock you into free tier anyhow.

</comment></comment><comment><USER28> If you can setup your own router and troubleshoot problems I don't think it'll be worth it.

However, if this can solve my parents' Wifi problems so they don't have to wait till next Thanksgiving for me to come fix it, I'll totally tell them to sign up for it.

</comment><comment><USER29> Anyone else hear echoes of Cisco Meraki? I can see that model working in commercial environments where the network itself is often managed by a third- party. But for home users, it's paying a premium for nothing of value.

<comment><USER30> In fairness, plume discounts the hardware with a subscription, so the pricing is fairly competitive. However, I cannot see bothering with a subscription when the competition is at $85/access point and works without an account.

</comment></comment><comment><USER30> Has anyone tried out Ubiquiti's mesh offering? A three access point solution is priced similarly to this,l plus a lifetime membership (but presumably keeps working without the cloud)

</comment><comment><USER31> Click bait alert. They are turning wifi router hardware into a subscription, not Wi-Fi itself. In case the author or any readers were unaware, home Wi-Fi is already a subscription from Comcast, AT&T etc.

<comment><USER32> If you bothered to read the article, you would see that you still purchase the equipment, but pay a subscription just to keep it running.

At least your Comcast/AT&T "Wi-Fi Subscription" actually comes with Internet access and rental hardware. Plume is having you buy the hardware and then pay $5/month just for the privilege of keeping the firmware running.

This business model is already the norm for enterprise firewalls and network security appliances, but that's not what Plume is. Plume is just a Mesh network, no different than just meshing a few OpenWRT devices together...

<comment><USER33> > This business model is already the norm for enterprise firewalls and network > security appliances, but that's not what Plume is.

One reason this makes sense for enterprise hardware is that you get a real person on the phone if something goes wrong. I doubt I'd be able to get a Plume representative to help me no matter how many years I subscribe for.</comment></comment></comment>


Ask HN: How do you provision new developer environments? - <AUTHOR> We're hiring 5-6 new developers, and I know that setting up a development environments can be a nightmare. Trial and error, things going wrong, what solutions have you learnt over the years that makes this as pain free and easily replicable as possible?<p>We will be providing all hardware/equipment for the new onsite developers, but looking at having a workflow in place that helps people write new code on day one, rather than spending two days installing everything and then having to repeat it again in a weeks time for the next recruit. <comment><USER2> Manually and inefficiently. It helps that turnover is low.

At least now the process is documented. Two years ago it was trial and error.

All the dependencies take a while to compile, but we've done it enough times now and have the operations written down precisely enough that aside from some data indexing tasks that have to run overnight (and which aren't strictly necessary to get started developing) we can set up a new environment in about 3 hours.

Next step is probably a bash script to automate most of the process on each of our 2-3 most commonly used platforms.

</comment><comment><USER3> docker and Makefiles as task runners. I have a git repo that is the skeleton for this idea: [https://github.com/undernewmanagement/flight- deck](https://github.com/undernewmanagement/flight-deck)

On mac we have homebrew to pull in all the desktop apps and cli tools. Then a simple bash script to automate all of it.

I've managed to "retrofit" this into a team of 20+ devs and product owners for a variety of services.

We are even piloting this with sales and support so that they can stay abreast of bleeding-edge development and provide valuable feedback.

<comment><USER4> There has to have been a solid solution before docker or containerization in general, what do giant companies use?

<comment><USER5> At the giant company I worked at. A central server which kept a canonical mirror of all the tools (except visual studio which was installed using whatever tool large companies use to install and update applications on windows machines) and all the dependencies for everything and then just a handful of .bat files that basically just used robocopy to make sure your build environment was up to date and correct.

Not pretty, but it worked. The big advantage was that since everybody was getting all their tools and dependencies from the same place everybody was using, not only the same version, but also exactly the same binary of everything.

</comment></comment></comment><comment><USER6> Binary packages. Once you package all your software and all your software's dependencies into packages, a development environment (the one that needs deps in their compiled form and your software in its source form) is just one `apt- get build-dep' away, which is a bonus to the main point of easy production deployment.

<comment><USER5> The important thing is that, no matter what solution you go for, all build dependencies should always be pulled directly from a central server that you control. If your build process involves grabbing arbitrary stuff directly from a github account not directly under your control you will, sooner or later, end up in trouble.</comment></comment>


How I Got Started: Software Developer at 60 (almost) - <AUTHOR> http://blogs.jobdig.com/wwds/2008/07/17/where-to-start/

<comment><USER2> The title is misleading. The author clearly states that he got started as a programmer at 31, not 60.</comment>


Boléro: Sergiu Celibidache (1971) [video] - <AUTHOR> https://www.youtube.com/watch?v=gy5Ve3338-E <comment><USER2> An interesting and a rather unsual pick on HN. Celibidache, the Romanian conductor seen in the video, was nortorious for his perfectionism, similar to his contemporary Carlos Klieber. He would take 3 times as much more time for rehearsals, much to the chagrin of concert organizers. Despite or because of his limited repotroire, he was known to be very precise in his directions, often reciting every phrase for every instrument while adjusting the orchestra during reheasals.

His strongest renditions are that of Bruckner. Unlike other conductors, such as the classical "rockstar" Karajan, he would approach the symphonies in a glacial pace, often ending up 30 minutes longer than a typical rendition. However, this is where his craftmanship shines, with precise and transcendent harmonics that no other rendition can achieve. His 8th symphony is an experience one can not miss in his lifetime.

It's actually my first time hearing his rendition of Bolero, but I've already noticed the mark of the conductor: taking time to make sure each phrase is accurately pronounced. I'll have to re-listen for more observations.

In addition, Celibidache was not a perfect person. He was later known for his sexism, refusing to hire a qualified trombonist as first trombone after she passed a screen-blinded audition. [1]

[1] [https://www.timescolonist.com/entertainment/music/female- tro...](https://www.timescolonist.com/entertainment/music/female-trombonist- fought-battle-against-orchestra-sexism-1.628198)

<comment><USER3> Can you recommend a recording for Bruckner's 8th?

<comment><USER4> If you want Celibidache, he recorded the 8th with the Munich Philharmonic. My personal favorite (the opening to the fourth movement was my ringtone for some time) is Furtwängler with the Vienna Philharmonic.

</comment></comment></comment><comment><USER5> I don't know why this is in HN, but thanks for posting this. I have always loved Bolero for its hypnotic transcendental feeling, that can be found in music of Terry Riley, Arvo Pärt and electronic music. I started to reading about Sergiu Celibidache and he seems a very interesting composer. A teaser from Wikipedia:

"Celibidache frequently refused to release his performances on commercial recordings during his lifetime, claiming that a listener could not obtain a "transcendental experience" outside the concert hall. Many of the recordings of his performances were released posthumously. He has nonetheless earned international acclaim for celebrated interpretations of the classical music repertoire and was known for a spirited performance style informed by his study and experiences in Zen Buddhism. He is regarded as one of the greatest conductors of the 20th century."

<comment><USER6> Thank you for describing this "hypnotic transcendental feeling", which I now realize is the common quality that I so enjoy in both Bolero and Terry Riley's "In C" (especially the Ictus & Blindman quartet version [https://play.google.com/music/m/Be26kwoxcafkgcyxad6c25y6m3y](https://play.google.com/music/m/Be26kwoxcafkgcyxad6c25y6m3y)).

Anyway, now that I've discovered this, I'd love if you'd share some pieces (or a playlist!) that you've found that have this quality?

</comment></comment><comment><USER7> Background story about this piece, presented by RadioLab: [https://www.wnycstudios.org/story/unraveling- bolero](https://www.wnycstudios.org/story/unraveling-bolero)

<comment><USER8> Gives me goosebumps just to see the URL

</comment></comment><comment><USER9> Richard Adeney, long time flautist with the London Symphony Orchestra has some amusing anecdotes about Celibadache in his autobiography ('Flute')."He was a jolly young man and his concerts were full of surprises. In the intervals of his chaotic concerts, he'd drink a bottle of sea water (bought from Boots the chemist), vomit and the after the concert he always went to the nearest fish and chip shop and wolf down three portions of fish and chips". Adeney's opinion (not being in the audience) was that he was 'a truly awful conductor' but maybe he was from the point of view of an instrumentalist under his baton.

</comment><comment><USER10> I must add Maya Plisetskaya performing Bolero: [https://youtu.be/sc_DuM8h6wU](https://youtu.be/sc_DuM8h6wU)

<comment><USER11> And I must add Beck's bolero :)

[https://www.youtube.com/watch?v=F1hijzkOrnk](https://www.youtube.com/watch?v=F1hijzkOrnk)

</comment></comment><comment><USER12> Wow! This guy is the Artur Schnabel of composers. I love everything about his philosophical and technical approach to music. Thank you for sharing. This is exactly what I've been looking for.

Can anyone help me find his recording of Dvorak's 7th? Supposedly the recording exists but I'm not able to find it at retail anywhere I look.

Too bad there's no recording of Sibelius 4 or 7...

</comment><comment><USER9> Richard Adeney, long time flautist with the LPO has some amusing anecdotes about Celibadache in his (RAs) autobiography ('Flute'). "In the intervals of his chaotic concerts, he'd drink a bottle of sea water (from Boots, the chemist) and then vomit; after the concert he'd always visit the nearest fish and chip shop and wolf down three portions of fish and chips".

</comment><comment><USER13> What makes this piece so captivating? Despite the length, I can never start it without finishing it.

Anything with similar magic but a different tone?

<comment><USER6> Since I'm also looking for this, let me share my favourite (and only other I know of):

In C by Ictus (Terry Riley): [https://play.google.com/music/m/Be26kwoxcafkgcyxad6c25y6m3y?...](https://play.google.com/music/m/Be26kwoxcafkgcyxad6c25y6m3y?t=In_C_- _Ictus)</comment></comment>


Show HN: This startup acquisition announcement does not exist - <AUTHOR> https://thisstartupacquisitionannouncementdoesnotexist.ajnisbet.com/ <comment><USER2> I love it. Your title caught my eye as I was submitting a GPT2 project as well! I had the same thought that GPT3 might not be funny because it could be too accurate.</comment>


What happens on the Internet in 60 seconds - <AUTHOR> http://whathappensontheinternetin60seconds.com/ What happens on the Internet in 60 seconds. <comment><USER2> cool!</comment>


Robotic monopoly player competitive against good human players - <AUTHOR> https://github.com/giannitedesco/kapital I threw this together in a handful of evenings. It plays monopoly using the monopd protocol/servers. Enjoy! <comment><USER2> You mean "Monopoly" of course ;-)

<comment><AUTHOR> lol thanks, fixed!

<comment><USER2> nps :)

</comment></comment></comment><comment><USER3> Outstanding!</comment>


Ftp Must Die - <AUTHOR> http://wooledge.org/mywiki/FtpMustDie

<comment><USER2> FTP can return sane directory listings that can be analyzed by a program - something that's still missing in HTTP.

<comment><USER3> http can do that too although I agree that mod_autoindex doesn't.

</comment></comment><comment><USER4> I haven't used FTP in years. SCP all the way!

<comment><USER2> And that's overkill, as most of the time we don't need data to be encrypted. In fact secure authentication and insecure data transfer would be enough, but SSH lacks this possibility.

<comment><USER4> I never want any of my business data going across the Internet unencrypted.

<comment><USER4> Why was I downmodded for this? The only time I really use SCP is to transfer backups, and I want those encrypted. I don't see why that is an unreasonable thing.

<comment><USER2> Well, in theory anything that you copy over the network can be intercepted and modified to do harm to you or your business. However, the cost of a man-in- the-middle attack is usually higher than the content you are transferring _in most cases_.

For example, if I'm downloading sources of an open-source program from a public web site, what's the point of attacking me and changing the contents?

(And I have no idea why you were downmodded.)

<comment><USER4> I'm not generally concerned about a man-in-the-middle attack. What I am concerned about is someone just capturing my data and then having it or using it. For example, in my backups will usually be a copy of various databases.

<comment><USER2> Ok, I understand that, but back to my question: why do you need encryption for something that's openly available anyway, for example an open-source program? And it's not everyday that we copy private business data over the Net, so my point is, it would be nice to have an option for faster unencrypted transfers.

<comment><USER5> Look into rsync [1] if you want transfers to be _fast_. You have the option (but not the requirement) to encrypt by using rsync in concert with ssh.

Incidentally I do transfer private business data over the network, encrypted, every single day, usually many times per day.

[1] <http://samba.anu.edu.au/rsync/>

<comment><USER3> I strongly agree with the article and with mdakin's point; scp is good but rsync is better. ftp is awkward, flawed, and obsolete. I am amazed that it is still used as much as it is. Perhaps the problem is largely due to Windows admins. ssh still doesn't ship with Windows and too many Windows admins aren't familiar with it.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER6> What really perplexes me are the sites that still use FTP for serving large files, as if FTP is better than HTTP for that purpose.

<comment><USER7> It could be about their particular software (eg apache vs proftpd) rather than the underlying protocol.</comment></comment>