Why Many People Perceive the Study of Human Behavior as Unscientific (2011) [pdf] - <AUTHOR> http://web.missouri.edu/~segerti/capstone/LilienfieldPublicSckepticism.pdf <comment><USER3> 18 pages of pure bullshit. I devoted 10 minutes to reading parts of it; I couldn't find anything of value.

It goes as far as to compare the "highly probabilistic predictions" of psychology to those of quantum physics, and it even cites Richard Feynman. It's just laughable.

In one instance it uses the word "stochastological".

<comment><USER4> Thanks for saving me the trouble of reading it.

I'd also suggest that one of the reasons people perceive the study of human behaviour as unscientific is that it so frequently _is_.

Studies of human behaviour are inextricably bound up in politics. Many studies are designed, whether consciously or unconsciously on the part of the authors, to advance the authors' own political or social agenda.

Even if you avoid that problem, many studies are just plain statistically lousy, and employ deeply non-random samples.

And finally you've got the annoying problem that people know they're being studied and may modify their behaviour accordingly.

<comment><USER5> What's with the resentment towards psychology on YC?[Insert Freudian joke here]

</comment></comment></comment><comment><USER6> Here is an interesting excerpt.

> "Psychology Does Not Yield Repeatable Results" How replicable (repeatable) > are the results of psychology compared with those of the hard sciences? > Larry Hedges (1987) decided to find out. He compared the replicability of > findings in particle physics, ostensibly one of the most rigorous domains of > physics, with those of several areas in psychology, including the effect of > teacher expectations on students' IQ scores, gender differences in verbal > and spatial ability, the effects of desegregation on educational achieve- > ment, and the validity of student course evaluations. Using various > statistical metrics of consistency, Hedges found that the results of > particle physics studies aimed at estimating the mass or lifetime of stable > subatomic particles (e.g., the muon) were in general no more consistent that > those of psychology. Hedges's findings suggest that the claim that > psychology's results are far less dependable than those of physics are not > supported by data.

> Still, we should not overstate the implications of Hedges's (1987) findings. > As Hedges acknowledged, he did not sample randomly within either physics or > psychol- ogy, so his results may be unrepresentative of the domains within > these broad fields. Nevertheless, Hedges observed that the results of > studies in several other domains of physics, including the estimation of > chemical and thermo- dynamic constants, appear to be about equally > consistent (or inconsistent, depending on whether one chooses to view the > glass as half full or half empty) as those within many domains of > psychology.

<comment><USER7> These quotes fail to notice that, notwithstanding all the non-repeatable results in physics that come and go, physicists eventually settle on a few extremely reliable theoretical precepts, precepts that lead to new technology. A classic example is quantum physics, the most successful and reliable scientific theory we have, a theory that makes numerical predictions about nature that prove accurate to better than ten decimal places:

[http://scienceblogs.com/principles/2011/05/05/the-most- preci...](http://scienceblogs.com/principles/2011/05/05/the-most-precisely- tested-theo/)

Psychologists have many non-repeatable results but don't shape reliable theories about their subject. Physicists have many non-repeatable results but _do_ shape reliable theories about their subject. The computer I'm sitting at is proof of that.

</comment></comment><comment><USER8> "In a larger sense, I see the AS chapter as part of a long-term program to pathologize normal behavior, one that hopefully will backfire and destroy unscientific psychology." \- Paul Lutus

From: The Trouble with Psychology [http://www.arachnoid.com/trouble_with_psychology/index.html](http://www.arachnoid.com/trouble_with_psychology/index.html)

Mr. Lutus has many articles on psychology. Asperger's By Proxy ([http://www.arachnoid.com/psychology/aspergers.php](http://www.arachnoid.com/psychology/aspergers.php)) is extremely relevant to the software industry.

<comment><USER7> Thank you! As it happens, I just published a new article today entitled "Science of Mind":

HTML: [http://arachnoid.com/science_of_mind/index.html](http://arachnoid.com/science_of_mind/index.html)

PDF: [http://arachnoid.com/science_of_mind/resources/science_of_mi...](http://arachnoid.com/science_of_mind/resources/science_of_mind.pdf)

<comment><USER8> I am several months from adequately responding to this article due to delayed neural development(1). Until I can properly respond, please(2) consider the following:

You seem to be overlooking inflammation. If only for fun, run your web backwards - starting from dementia (e.g. Alzheimer's), then sleep disorders, then anxiety.

Appropriate diet, moderate exercise, and a good night's sleep are vital as a foundation for good health.

Acceptance that normal behavior is pathologized leads to the understanding that "mental illness" is legislated.

Thank you for making your research publicly available, and open to review and criticism.

\---

1) For over five decades I had been 'locked in'. Comedy at it's finest - <USER2> cannot contribute, as being misdiagnosed (word needed for profit motivated diagnosis - prodiagnosed?) prevented the ability to communicate that I could not communicate.

2) Tens of millions suffer needlessly. A few months away from improper 'medicine' and glowing rectangles would provide much needed data points.

</comment></comment></comment><comment><USER9> [https://www.youtube.com/watch?v=gfZWyUXn3So#t=37](https://www.youtube.com/watch?v=gfZWyUXn3So#t=37)

Tom Lehrer's recent(-ish) song "Sociology" seems very relevant here.

</comment><comment><USER10> How about: human behavior is a complex system exhibiting chaotic behavior and is mathematically proven to be unpredictable? And don't even get me started on the absurd over-reach of the conclusions of almost all social science papers. They'll study Westerners and proclaim to have discovered something about humans - ignoring the monumental variable of culture. They also project their own values on research constantly. How many times have you seen research reported as "health benefits of sex"? And how many times have you seen the same research reported as "health dangers of abstinence"? They're the exact same thing, but a matter of cultural perspective. If you view sexuality as a normative part of human life, then observing that those who have fewer orgasms develop prostate cancer is a health danger of abstinence. But if you view sexuality as an aberration, something that's not a normal part of everyday life, then you report it as a benefit of increased sexual activity - establishing that you think the current level is 'normal' rather than 'so low as to present actual biological danger'(which is the truth).

<comment><USER11> That it is seemingly chaotic doesn't make it chaotic, and I'm interested in the mathematical proof that behavior can't be predicted. Don't confuse limited ability to predict with impossibility. We can make many confident statements about the likelihood of certain behaviors/reactions/response, in the aggregate, under given real-world circumstances.

You are correct in that much research, especially research older than 10 years ago, tended to ignore culture. I know my field of study (I/O Psychology) references cultural frequently, either through overt study when possible (desirable given the publication potential) or at least remarking on the limitations of any findings. I don't read much outside I/O research though, so maybe it hasn't progressed much on the whole and the global nature of I/O work for businesses rewards more recognition of culture. It's hard though - accessing globally-relevant datasets for studies of human behavior is often impossible. That doesn't make all studies meaningless, or unscientific.

And yes, you're correct about value-laden framing of results. That doesn't invalidate the science of the results, it just makes it more necessary to think critically when interpreting them rather than relying on abstracts and headlines alone.

<comment><USER7> > That it is seemingly chaotic doesn't make it chaotic,

Very true. A chaotic system has specific properties that we would be very pressed to identify in human behavior.

> ... and I'm interested in the mathematical proof that behavior can't be > predicted.

Think abut what you're saying. Proof that behavior _cannot_ be predicted would require proof of a negative. Proof of a negative, or (the same thing) an evidentiary burden requiring evidence of absence, is a logical error. So this is a non-starter.

Let's say I believe in Bigfoot, and I think others are obliged to dissuade me from by belief. The problem is that this also requires proof of a negative, and Bigfoot could be hiding under a rock somewhere on a distant planet. _Bigfoot cannot be proven not to exist_ , so I'm secure in my evidence-free belief.

This is why scientists adopt the _null hypothesis_ precept, the idea that an idea is assumed to be false until evidence supports it -- it's a simple way to avoid logical pitfalls like believing in Bigfoot.

> We can make many confident statements about the likelihood of certain > behaviors/reactions/response, in the aggregate, under given real-world > circumstances.

Yes, true, but until those observations lead to a falsifiable theory of human behavior, it's not science. Until then, we can only describe, we can't explain. Testable explanations are the threshold of science.

> And yes, you're correct about value-laden framing of results. That doesn't > invalidate the science of the results ...

No, of course not -- thee reason is that such results aren't science. An observation without an explanation isn't science. Here's an example -- if I sit under a clear nighttime sky, then report that I saw many tiny points of light, I've made an observation of very little scientific usefulness, and one that can't meaningfully be falsified by repeated observation. But if I claim those points of light are actually distant thermonuclear furnaces like our sun, my assertion can be tested and possibly falsified. By moving beyond observation, by offering a tentative explanation, I've crossed the threshold of science.

<comment><USER11> > Think abut what you're saying. Proof that behavior cannot be predicted would > require proof of a negative. Proof of a negative, or (the same thing) an > evidentiary burden requiring evidence of absence, is a logical error. So > this is a non-starter.

Yes, I know. That was the point of the comment in response to the parent's statement: "[human behavior] is mathematically proven to be unpredictable." But thank you for the lecture.

> Yes, true, but until those observations lead to a falsifiable theory of > human behavior, it's not science.

There are plenty of theories. They are typically considered theories on motivation and behavior rather than behavior itself. It's a notable difference, but a sensible distinction. Many of these theories are falsifiable, and in fact have been falsified (e.g., expectations-based models, for example). But they are typically piecemeal, as the complexity of human motivation and behavior currently requires. If the key requirement to be considered "science" is for The One Great Unified Theory, then I agree that it will never be considered a science in my lifetime.

You seem to think psychology is all observation with no attempt at explanation, much less construction of logical theories supporting those explanations. That is not the case in my little fiefdom (though I have no doubt it is true in some disciplines, or acceptable in some journals). Much work has gone into proposing, testing, and refining many theories on motivation and behavior (e.g., reinforcement theory, goal-setting theory, social-cognitive theory or workplace behavior, job design theory, motivated action theory, etc.).

It's not perfect, and it's got a long way to go, but it's also come a long way. Yet it's hardly just observing the stars and making claims, and it's certainly shown utility in helping design systems to influence behavior.

But there, I've said my piece. I've usually got the discipline to stay out this pet topic for the HN crowd.

<comment><USER7> > There are plenty of theories. They are typically considered theories on > motivation and behavior rather than behavior itself.

Those aren't theories as science defines the term. Among other things, they aren't falsifiable and they don't make predictions about unobserved phenomena, both hallmarks of scientific theories.

> You seem to think psychology is all observation with no attempt at > explanation ...

Wait -- that's not my idea. That's the reason the NIMH gave for abandoning the DSM -- it's all symptoms (descriptions) and no search for causes (explanations/theories):

[http://www.nimh.nih.gov/about/director/2013/transforming- dia...](http://www.nimh.nih.gov/about/director/2013/transforming- diagnosis.shtml)

Quote: "While DSM has been described as a "Bible" for the field, it is, at best, a dictionary, creating a set of labels and defining each. The strength of each of the editions of DSM has been "reliability" – each edition has ensured that clinicians use the same terms in the same ways. The weakness is its lack of validity."

"Unlike our definitions of ischemic heart disease, lymphoma, or AIDS, the DSM diagnoses are based on a consensus about clusters of clinical symptoms, not any objective laboratory measure. In the rest of medicine, this would be equivalent to creating diagnostic systems based on the nature of chest pain or the quality of fever. Indeed, symptom-based diagnosis, once common in other areas of medicine, has been largely replaced in the past half century as we have understood that symptoms alone rarely indicate the best choice of treatment. Patients with mental disorders deserve better."

> Much work has gone into proposing, testing, and refining many theories on > motivation and behavior ...

You aren't using the scientific meaning of theory -- a formalization of observations using empirical evidence, that generalizes specific observations, that predicts things not yet observed, and that is falsifiable.

> Yet it's hardly just observing the stars and making claims, and it's > certainly shown utility in helping design systems to influence behavior.

There are many ideas that meet this description, but they aren't scientific theories as that term is defined.

Prove me wrong. Point to a theory in the class you describe that has been used to conclusively falsify someone else's theory in the same field, a case in which the defender of the falsified theory acknowledges that his theory can been proven false and has no further utility. I say it this way because it happens all the time in science, and it never happens in psychology.

Consider the example of Asperger Syndrome. It was thrown out over a year ago by the editors of DSM-5, described as an unscientific and meaningless term, but I am still getting emails from parents whose children are being assigned the diagnosis.

Consider recovered memory therapy. Many people have been accused of imaginary crimes based on it, and many studies prove the unreliability of this therapy, but it's still offered to anyone foolish enough to ask for it.

Falsifiability is a big issue with respect to psychological theories, for the reason that it doesn't exist.

> But there, I've said my piece. I've usually got the discipline to stay out > this pet topic for the HN crowd.

This isn't a contentless, rambling argument in which people say their piece -- at least, I hope not. It should be a discussion of ground truth, with the possibility of a conclusive outcome, like in science, in which someone says, "My position has been falsified". The fact that this never happens within psychology doesn't mean it can't happen here -- in principle.

I hope you see the degree of self-reference that's present here.

<comment><USER11> My hesitance to continue this debate is that I find places like HN and reddit to be pretty hostile audiences, and engaging in the conversation occupies more of my thoughts than I find it should. It's a form a self-regulation to know when I should stop, particularly given the audience.

And I want to make one thing clear about psychology which just never seems to be clear: psychology is way, way bigger than clinical psychology, yet clinical psychology (and stories from it) seem to be the only thing that is ever brought up in these conversations. I can't speak to clinical psychology. There are 50 divisions of the APA, some significantly better than others. My division is #14.

> Among other things, they aren't falsifiable and they don't make predictions > about unobserved phenomena, both hallmarks of scientific theories.

Almost all of those theories are falsifiable, elements of them have been falsified, and they make predictions for unobserved circumstances.

> That's the reason the NIMH gave for abandoning the DSM

The DSM is a diagnostic tool from a branch of psychology I know little about. It is not the singular representative of the social sciences, or even halfway related to much work being done in behavioral prediction. It's not theory- based, as I understand it it is not meant to be, and I've no interest or ability to defend it.

> Point to a theory in the class you describe that has been used to > conclusively falsify someone else's theory in the same field, a case in > which the defender of the falsified theory acknowledges that his theory can > been proven false and has no further utility.

The last part of your request is unreasonable - proving that there is no utility whatsoever in old theories is unnecessary. Elements of older theories can have utility. And as for the original theory's proponent abandoning it? I don't see that often, nor see it as a requirement. Journals I read don't publish mea culpas from those who were ultimately proven wrong, they just publish the evidence proving them wrong.

I pointed out one specifically already in my previous comment (edit: my mistake, I took that out just referenced expectancy theories more generally). Victor Vroom introduced the VIE expectancy model for explaining and predicting behavior in 1964. Over 20 years the research indicated that the theory as proposed did not jive with the observed evidence, and the model transition from one based on "psychological force" to "outcome attractiveness" as chronicled by Pinder in 1984. Van Erde and Thiery, in 1996, that the preponderance of evidence on the model, revised or otherwise, demonstrated that even the revised theory was inadequate. The individual elements had predictive power, but as a model it just didn't work. Did Victor Vroom abandon it? Who knows? The field certainly did. But did the field note that nothing in theory had any utility? No, certainly not. Locke and Latham had integrated some of the elements into explaining goal commitment, a critical component of goal-setting theory.

There was also the feedback theory of workplace behavior popularized by Komaki in the 80s and early 90s, and was effectively dismantled as a stand-alone theory both logically and empirically Locke and Latham in the mid-to-late 90s. I have no idea whether Komaki acknowledged they were wrong publicly, but the field moved on. Even something so basic as goal-setting theory has wholesale change in just the past 5 years with the measurement and observation of subconscious goals, and how they don't work with traditional goal-setting theory.

Deci and colleagues outlined and empirically supported Cognitive Evaluation Theory in the 90s and early 00s. Among others, Rynes dismantled the bulk of it in the mid-to-late 00s, and as far as I know it's essentially dead.

Another one is job characteristics theory, which I don't recall the whole story of off the top of my head. It came, it didn't prove to be reliable, it was abandoned. Again elements have found their way to better homes (i.e., links of autonomy to workplace behavior patterns) but on the whole it didn't cut it.

If that is too old, a more modern one is the debate between Social Cognitive Theory in the workplace (from Wood and Bandura, at some point). It's a theory, lots of testable hypothesis, and great empirical support. A pillar of explaining aspects of workplace behavior! A professor named Vancouver out of Michigan State has demonstrated empirically multiple times that the model's predicted outcomes are often at odds with reality. There's been some back-and- forth on this one, and I'm not up to date on it. But it became clear the theory as proposed was not adequate.

> Consider the example of Asperger Syndrome... recovered memory therapy.

They are not my domain of psychology. Psychology, and social science in general, are very broad terms. There are bad examples. I cannot and will not defend those things, but you should not accept those as archetypal representations of all psychological and social science research.

EDIT: One more thing to add. These things tend to have longer timescales than in other harder sciences because of the nature of the subjects. One or two empirical studies may be enough to definitively disprove ideas elsewhere, but because getting large numbers of people across adequate contexts is so onerous, particularly with non-student, non-US samples, the lag-time for debate can be large.

<comment><USER7> > ... psychology is way, way bigger than clinical psychology, yet clinical > psychology (and stories from it) seem to be the only thing that is ever > brought up in these conversations.

Yes, and there's a reason. The ground truth about the medical field is what happens in clinics. If clinicians are shaking dried gourds over patients, that tells us about medicine itself, from top to bottom -- the only way something unscientific could happen in a clinical setting is if there's no reliable science anywhere in medicine.

In the same way, if someone builds a bridge by guess and by golly, without any consideration of loading and stresses, this calls into question the scientific basis of civil engineering.

So when I hear a psychologist complain that clinical psychology isn't representative of the entire field, I have to say that kind of compartmentalizing doesn't work anywhere else -- it suggests that psychology isn't unified by a respect for science, or by reliable, falsifiable theory.

>> Consider the example of Asperger Syndrome... recovered memory therapy.

> They are not my domain of psychology.

This sounds increasingly like one of those joke conversations about which kind of Southern Baptist you are. Religion works that way, science doesn't. In science, evidence and tested theories forge a consensus that unifies all legitimate fields, at least until someone produces a better theory, then the process repeats.

Do you suppose that a doctor could say, "I don't care what biologists are doing, it doesn't affect me of my practice"? Do you suppose that a geologist could say, "I don't care what the physicists are doing in Earth Science, that has nothing to do with my work"?

This is not realistic, because the example fields are united by a common respect for the primary role of evidence and the scientific method. I say this to remind you how you sound then you try to describe psychology as though it's a bunch of disconnected pieces. Not that that isn't so, but it provides us with a measure of the significance of science in psychology.

> ... but you should not accept those as archetypal representations of all > psychological and social science research.

If that's true, then psychologists lose the right to describe themselves as scientists, or to receive funds earmarked for scientific activities. Taxpayers are willing to contribute to physics because (a) it's a science with an enviable track record, and (b) physicists are on the same page -- they don't say things like, "Don't listen to the particle physicists, they're not the real thing." Only psychologists say things like that.

> The last part of your request is unreasonable - proving that there is no > utility whatsoever in old theories is unnecessary.

Then you do not understand science. When the existence of the ether was disproven, to save limited resources the falsification was accepted by everyone -- opinion played no part. When the Phlogiston theory was disproven, it was disproven for everyone, everywhere. This kind of certainty and consensus is what makes science science.

<comment><USER11> > Yes, and there's a reason. The ground truth about the medical field is what > happens in clinics.

True, but psychology is not the medical field. You've picked the flashiest "branch" of it to make your archetype, but it's probably the least representative of the whole (though admittedly perhaps the best funded). Clinical psychology is more focused not on what is normal, but what is abnormal, how to diagnose it, and how to respond. Almost all of the rest of psychology is focused on what is normal, typically within a given context (childhood, workplace, developmentally, etc.)

> psychology isn't unified by a respect for science, or by reliable, > falsifiable theory.

Psychology is not unified by a single reliable theory. I have no problem admitting as much. Maybe someday, but not while I'm alive. Again, if that's your baseline for what it takes to be considered scientific, I'm fine saying "then by that definition it's not scientific" and feeling no shame.

> This sounds increasingly like one of those joke conversations about which > kind of Southern Baptist you are.

Then you're not thinking critically. Your doctor-biologist example doesn't fly because one's work (biology) directly and materially affects the work of the other (medicine). What clinical psychologists do, more than almost any other branch, has no impact on me _presently_ , nor I on them. Our populations are different, and the goals of our research with respect to our populations are different. Ridicule the division if you wish, but what do you want me to do? Do you expect my field to reconfigure itself entirely to become experts on abnormal behavior and diagnoses? Should I, in my effort to increase workplace productivity and satisfaction, educate myself on the ins-and-outs of autism or psychological disorders? Human behavior is so large and complex to support these various divisions.

Many, many branches are connected in some way. It's a web. I have no ability to speak beyond the basics of clinical psychology. I can speak to psychometrics, social personality, experimental, educational and consumer psychology, and behavioral analysis (all of which are branches of the APA). Outside of a general interest in human behavior with no regard for context, my goals are as different from a clinical psychologists as a biologists are from neurologists. It's hard enough to do good work in specific contexts, unfortunately. Psychology right now is a bottoms-up field because we're just too ignorant as a species (despite efforts) to do it top-down right now.

> I say this to remind you how you sound then you try to describe psychology > as though it's a bunch of disconnected pieces. Not that that isn't so, but > it provides us with a measure of the significance of science in psychology.

I don't have to _try_ to describe it that way, because it is. It's a pragmatic response to the complications of understanding people on the whole. The whole is too disparate, so look at the pieces. That bothers you, but most don't seem as perturbed. I'm sure you see this as concrete evidence for your point, but you won't be surprised that I think that's poor criteria. "Psychology" as a term is almost certainly overly broad, which is why so many divisions exist in the first place.

> If that's true, then psychologists lose the right to describe themselves as > scientists

It is true, and I disagree. Bad science is found everywhere. Look at what bad science has done for vaccines. Do I hold it as an archetype of medical science? Your argument seems to be "we're not perfect at explaining literally all cognition and behavior in a Single Unified Theory now, mistakes have been made in the past, so sod the whole thing."

Physicists are on the same page because their work is infinitely more provable. Ultimately human cognition and behavior is a much more practically difficult, nebulous topic at the present time that's going to take longer to get right on the whole. It's all probably just math itself in some respect, and maybe we'll know the equation someday à la Hari Seldon in The Foundation. We're not there yet.

>> The last part of your request is unreasonable - proving that there is no utility whatsoever in old theories is unnecessary.

> Then you do not understand science.

Then you're not thinking critically. It's this kind of thing that turns me off these conversations completely. For better-or-worse, many theories in psychology are not all-or-nothing. While the theory on the whole may prove false, certain elements of the theory or the direction may prove useful in new theories. Old, falsified theories are invaluable for paving the way towards new theories, and not all elements of all falsified theories need be disposed of whole-hog. You can completely reject and recognize the positive contributions it has made, and the utility elements of it still possess.

I think we know where each other stand on this now. At the very least, you got your examples.

EDIT: Passovers for grammar.

<comment><USER7> >> Yes, and there's a reason. The ground truth about the medical field is what happens in clinics.

> True, but psychology is not the medical field.

This is becoming circular. I say psychology isn't scientific, you say I have to say which part of psychology I'm referring to, I say the very question indicts the scientific standing of the field, because scientific fields are united by consensus on the meaning of evidence. I use medicine as an obvious example.

Then you reject my comparison with medicine by saying that psychology isn't medicine. One, psychiatrists would vigorously disagree (their livelihood depends on an association with medicine), and two, it's a red herring. The issue is whether a field can call itself scientific if it's not guided by evidence, and if there's no consensus about the meaning of the theories that define the field. In the case of psychology, there are no such field-defining theories, consequently the field is splintered into warring factions like sects of a religious order.

> For better-or-worse, many theories in psychology are not all-or-nothing.

You're missing the point that there are no scientific psychological theories. There are no such theories that are open to falsification by empirical evidence. And there has never been a scientific falsification of any psychological theory, ever, anywhere. Your use of the word "theory" is misguided, because it resembles how scientists talk.

>> If that's true, then psychologists lose the right to describe themselves as scientists

> It is true, and I disagree.

This is not a literature debate and it is not a matter of opinion -- it must be resolved by evidence. No scientist, when given a chance to present evidence, instead says, "I disagree".

> Bad science is found everywhere.

Psychology is not bad science, it is non-science. In bad science, theories are weak and short-lived, but they are falsified, as with astrology, phrenology, the Phlogiston theory, the N-ray theory, and so forth. In non-science, theories aren't open to a comparison with empirical evidence, consequently they cannot be falsified.

No falsifiability, no science.

<comment><USER11> > This is becoming circular. I say psychology isn't scientific, you say I have > to say which part of psychology I'm referring to, I say the very question > indicts the scientific standing of the field, because scientific fields are > united by consensus on the meaning of evidence. I use medicine as an obvious > example.

No, that is _not_ what I said. I said I personally cannot speak to certain branches of psychology, nor defend them, nor think they should be considered archetypes. I won't speak to something I know little about. I see it doesn't seem to stop you.

I didn't reject your comparison because psychology isn't medicine, I reject it because it is a bad comparison. I'll use a more apt one. Different branches of psychology don't interact much because they don't share populations or, often, goals. You want to use medicine as a comparator.

I know for a fact that most pediatricians (my wife among them) don't stay up to date, or much care, what's happening with geriatricians. I know that radiologists aren't terribly interested in the work of physical therapists. Ophthalmologists and cardiologists? Probably not reading the same stuff. And yet they are linked by care and interest in the general human physiology. But does one's work impact the other? Not really, not much. But biologists can certainly impact both. For psychology, the cognitive psychologists probably is the most "upstream." I care what they do like dermatologists care what biologists do.

> You're missing the point that there are no scientific psychological > theories. There are no such theories that are open to falsification by > empirical evidence. And there has never been a scientific falsification of > any psychological theory, ever, anywhere. Your use of the word "theory" is > misguided, because it resembles how scientists talk.

I'm only "missing the point" because your insisting that's the critical issue, and I'm saying you're wrong not in premise but in reviewing the state of things. If your use of the word "theory" is for a Single Unified Theory of all Human Behavior and Cognition, then I agree. I don't think that's the baseline for a scientific psychological theory, and there have been plenty of theories within psychology that have been falsifiable, have been falsified, and no longer exist. You even asked for some examples, which I provided. But what you do then is twist the criteria for what counts to meet only the standard you set. You'll say "it isn't my criteria" I'm sure, but I'm not seeing anything that says otherwise.

>>>>> Consider the example of Asperger Syndrome... recovered memory therapy.

>>>> ... but you should not accept those as archetypal representations of all psychological and social science research.

>>> If that's true, then psychologists lose the right to describe themselves as scientists

>> It is true, and I disagree.

> This is not a literature debate and it is not a matter of opinion -- it must > be resolved by evidence.

I'm not entirely sure of your point here. What scientific evidence are you asking for that some examples of bad practice / science / outcomes should not stand as the primary representation of all psychological research?

> Psychology is not bad science, it is non-science... No falsifiability, no > science.

See, this is the issue. You start with it as a non-science and work your way back. I've already listed several falsified theories in my own line of work. Outside of that: Skinner's theories? Falsified. Piaget's core theories of cognitive development? Falsified. McGregor's Theory X and Y? Falsified. Unlike other branches they don't vanish entirely, but elements remain through new theories and contributions.

But yet, the DSM exists, so it's all hogwash. We are indeed talking in circles.

<comment><USER7> >> I say psychology isn't scientific, you say I have to say which part of psychology I'm referring to, I say the very question indicts the scientific standing of the field, because scientific fields are united by consensus on the meaning of evidence. I use medicine as an obvious example.

> No, that is not what I said. I said I personally cannot speak to certain > branches of psychology, nor defend them, nor think they should be considered > archetypes.

Thanks for that clarification, which changed nothing about the exchange or its meaning.

> See, this is the issue. You start with it as a non-science and work your way > back.

No, I start with how science is defined, note that psychology cannot meet the definition, and draw the sole possible conclusion. As did the chairman of the NIMH as he ruled that the DSM may no longer be used as the basis for scientific research proposals, on the ground that it has no scientific content:

[http://www.newyorker.com/tech/elements/the-rats-of-n-i- m-h](http://www.newyorker.com/tech/elements/the-rats-of-n-i-m-h)

> there have been plenty of theories within psychology that have been > falsifiable, have been falsified, and no longer exist.

This has never happened in a scientific sense. Ideas that become unpopular or that fall out of clinical use aren't falsified by those circumstances.

> I've already listed several falsified theories in my own line of work.

Nonsense. None of the were falsified as that term is defined in science, by way of empirical evidence, the kind of evidence not accessible to those who study the mind.

> Skinner's theories? Falsified. Piaget's core theories of cognitive > development? Falsified.

Your claim is absolutely false. These ideas weren't stated clearly enough to be open to scientific falsification, and they were certainly not falsified -- they both have active practitioners who would be surprised to hear your claim.

[http://en.wikipedia.org/wiki/B._F._Skinner](http://en.wikipedia.org/wiki/B._F._Skinner)

No mention of the falsification you claim exists.

[http://en.wikipedia.org/wiki/Jean_Piaget](http://en.wikipedia.org/wiki/Jean_Piaget)

No mention of the falsification you claim exists.

[http://en.wikipedia.org/wiki/Phlogiston_theory](http://en.wikipedia.org/wiki/Phlogiston_theory)

Falsified, with a clearly explained reason, based on empirical evidence.

> Different branches of psychology don't interact much because they don't > share populations or, often, goals.

Like science? The same might be said about different branches of physics, except that physics is a science, consequently particle physicists, who study nature at the smallest scales, and cosmologists, who study science at the largest scales, productively read each other's literature and attend each other's conferences. They have science in common.

<comment><USER11> Alright, this is the end of the road for me.

> Thanks for that clarification, which changed nothing about the exchange or > its meaning.

It boggles the mind that you'd even say that. The difference between me saying "you must choose a branch of psychology to call unscientific" and "I do not personally possess the knowledge or capability to stand up for branches of psychology I do not know enough about it" seems pretty substantive.

> No, I start with how science is defined, note that psychology cannot meet > the definition

If you're relying on wikipedia, allow me to do so as well: science is the systematic enterprise that builds and organizes knowledge in the form of testable explanations and predictions about the universe. You take issue with the latter, but many psychological theories do them, no matter how much cotton you stick in your ears.

I do not argue that the DSM lacks scientific credibility.

> None of the were falsified as that term is defined in science, by way of > empirical evidence, the kind of evidence not accessible to those who study > the mind.

No matter how loud you say it, it doesn't make it more true. You demand that it is true, but it is not. You sit at the armchair, reading no research or studies whatsoever, and judge me for 1) being unwilling to comment on issues which I'm ignorant on and 2) not being an expert at all things psychology.

Re: Skinner - you are correct, though he had few "theories" to be wrong about. I meant E.L. Thorndike, Skinner's predecessor (and the other one with a famous "box"). His theory of learning was his greatest achievement, but he eventually repudiated it.

Re: Piaget - that is the person wiki, this is the relevant one: [http://en.wikipedia.org/wiki/Piaget%27s_theory_of_cognitive_...](http://en.wikipedia.org/wiki/Piaget%27s_theory_of_cognitive_development#Challenges_to_Piagetian_stage_theory) Nobody is a classic Piaget psychologist anymore. Did they throw it all out and start over? No, they did not, the iterated and keep testing. If that's the red flag to you, then there you go.

> Like science?

Sure, like science. Whatever you say. I don't disagree that physics is a more mature science, where connections between different branches are more clear.

Have a nice weekend.

<comment><USER7> >> No, I start with how science is defined, note that psychology cannot meet the definition

> If you're relying on wikipedia ...

No, Wikipedia is not my source. I rely on a much better source, the thousands of sources that define science in the same way, unambiguously, including many court rulings, for example those that keep Creationism out of public school classrooms by defining science as something that requires empirical evidence and objective falsifiability.

> Alright, this is the end of the road for me.

Of course it is. Before we had this conversation you had no idea what constituted science and falsification, and now, you still have no idea.

I want to thank you for taking part in this conversation -- I often think it's not possible for someone not to understand science basics -- it's really rather simple -- but then I have a conversation like this, and I get it all over again.

> I don't disagree that physics is a more mature science, where connections > between different branches are more clear.

What you think, your opinions, _do not matter_. Only evidence matters. And it occurs to me that you will never get this.

Scientist A says, "There is a largest prime. I don't know which prime it is, but there is a largest one. Infinity is not a number, it's a concept, therefore there cannot be an infinity of primes. Therefore there is a largest prime. Call it p."

Scientist B says, "I will falsify your claim. I agree that we shall call the largest prime p. Now let q = p factorial, i.e. q = p * (p-1) * (p-2), etc. At the end of the process, q is either a prime larger than p, which falsifies the claim, or q is composite, composed of primes, at least one of which is larger than p, which falsifies the claim. There are no other possibilities. I have falsified the claim."

Scientist A says, "Yes, you have falsified the claim, my opinion doesn't matter, how I feel about it doesn't matter, the falsification relies entirely on incontrovertible, objective evidence, _evidence on which all educated observers must agree_."

That is science. That is manifestly not psychology.

Another voice on this topic:

[https://www.cfa.harvard.edu/~ejchaisson/cosmic_evolution/doc...](https://www.cfa.harvard.edu/~ejchaisson/cosmic_evolution/docs/fr_1/fr_1_site_summary.html)

Quote: "Today, all natural scientists throughout the world employ the scientific method. Normally it works like this: First, gather some data by observing an object or event, then propose an idea to explain the data, and finally test the idea by experimenting with Nature. Those ideas that pass the tests are selected, accumulated, and conveyed, while those that don't are discarded — a little like the evolutionary events described on this Web site. In that way, by means of a selective editing or pruning of ideas, scientists discriminate between sense and nonsense. We gain an ever-better approximation of reality. Not that science claims to reveal the truth—whatever that is — just to gain an increasingly accurate model of Nature."

"Despite an emphasis on objectivity, some subjectivity does affect the modern scientific enterprise, for this is work done by human beings having strong emotions and personal values. Yet, with the test of time and repeated observations, objectivity eventually emerges and then dominates, enabling us to reach conclusions free of the biased viewpoint of any one scientist, institution, or culture. As a rational investigative approach used to formulate descriptions of natural phenomena, the scientific method is designed to _yield a reasonably objective consensus on the nature, contents, and workings of the Universe_." [emphasis added]

Couldn't have said it better myself.

Have a nice day.

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER12> The study of human behavior is based primarily on subjective measurements and double blind studies are generally not feasible. The level of rigor often expected in hard sciences is generally not possible in the study of human behavior.

<comment><USER13> This doesn't mean it's not scientific by definition.

If science _required_ double-blind trials, then you'd be throwing out a lot of physics, too (for example, the study of planetary motion). Science is 'robust reporting and observation', not 'double-blind trials'. The more robust your methods, the stronger your case - double-blind trials add a lot of robustness. Predictive value is also helpful, but again not required to be considered science (eg dark matter, which we still have no idea about).

<comment><USER7> > This doesn't mean it's not scientific by definition.

On the contrary, that is exactly what it means. Science requires empirically testable, falsifiable theories backed up by empirical observations. If we only have observations, it's not science, because observations are neither particularly useful nor falsifiable.

> Predictive value is also helpful, but again not required to be considered > science ...

Many commentators on this issue argue that an empirical, falsifiable theory that is able to make predictions about things not yet observed is a requirement for science.

[http://en.wikipedia.org/wiki/Scientific_theory](http://en.wikipedia.org/wiki/Scientific_theory)

Quote: "A scientific theory is a well-substantiated explanation of some aspect of the natural world that is acquired through the scientific method, and repeatedly confirmed through observation and experimentation.[1][2] As with most (if not all) forms of scientific knowledge, _scientific theories are inductive in nature and aim for predictive power and explanatory force_."

"The strength of a scientific theory is related to the diversity of phenomena it can explain."

Also, very important, observations (descriptions) can only be a preliminary to shaping an explanation, a testable theory (an explanation). Observations without a theoretical dimension are indistinguishable from stamp collecting and are a step toward science, but are not science themselves.

</comment></comment></comment><comment><USER14> APA should ditch the Goldwater rule.

<comment><USER15> It's the first time I heard of this rule, thanks.

As far as I understand, it only limits professional diagnostics, shouldn't it be OK if it's not done as a practicing psychatrist (i.e. call it an analysis or interpretation and not a diagnostic)?

<comment><USER14> The end result is that the public doesn't know about the rule, so they see psychologists acting a certain way and frequently think it's actually epistemically bad, or impossible, to reason about the mental states of other humans based on their actions, writings, and other artifacts.

Imagine if computer security experts couldn't talk, in general, about the security of a line of products offered by a company, or about the probable cause of a specific issue that had been observed in a particular product.

The parallels ARE actually there; it's not as strange an analogy as it looks at first glance.

I understand some of the reasons the rule exists. I think this is probably not the best solution, in that it creates a public who have very strange ideas about psychiatry, and that is actually a great harm.

I'm neutral on most of this article, but like, specifically, if public perception is an issue, finding a better way to resolve the ethical dilemma that spawned the Goldwater Rule sounds like a pretty good idea to me.</comment></comment></comment>


CoreOS Overview, Part One - <AUTHOR> https://deis.com/blog/2016/coreos-overview-p1 <comment><USER2> Pardon my ignorance... Is CoreOS an alternative to Docker? And if yes, what are the tradeoffs between the two?

<comment><USER3> CoreOS is a Linux distro based on Gentoo. It is noteworthy for being very small, built for "all-or-nothing" automated updates, and geared for clustering.

Basically if installed on disk (you can also boot it over the network or from a USB stick etc.) it updates by having two system partitions - one "known good" because you've successfully booted from it, and one used to download new updates to, and then will (by default) automatically reboot and switch to update.

The system you end up in is basically systemd + Etcd + Fleet (with Kubernetes on the way) + Docker and rkt, with very little else included.

If you want to spin up a large cluster where everything will run in Docker or rkt containers, that's something CoreOS is well suited for.

<comment><USER4> But it seems that CoreOS is more suitable as being the underlying OS on which containers are run, rather than being the basis of a container image?

<comment><USER5> Container images do not _need_ operating systems, and I will forever be angry with Docker for tainting the concept in popular usage. Research a static binary as a container to see what I mean.

Nobody who built containers before Docker came along shipped an operating system in the image. Now we have gigabytes of Ubuntu alongside nginx thanks to Docker. Cool.

<comment><USER6> Disclaimer: I work at Docker.

> _Nobody who built containers before Docker came along shipped an operating > system in the image. Now we have gigabytes of Ubuntu alongside nginx thanks > to Docker. Cool._

First: I believe shipping the entire distro in each container was in fact the most common use of containers before Docker, since openvz/vserver/lxc were focused on "mini-servers" (to this day you can still rent openvz-based vps). So I think your statement is incorrect.

I also think your (eternal?) anger might be misplaced. Yes, today most people who use Docker build containers with unnecessary distro bits in them. But before Docker, 99% of them didn't build containers at all, and struggled with non-repeatable deployments and dependency hell. Even better, now all these people use a standardized Dockerfile which can be changed and improved over time - for example to move away from a heavyweight distro like fedora or ubuntu, to something like alpine or even more minimalistic.

Incidentay, we have hired Natanael Copa, the awesome creator of Alpine Linux and are in the process of switching the Docker official image library from ubuntu to Alpine. You can help us with pull requests to [https://github.com/docker-library](https://github.com/docker-library) if you want :)

Even one such pull request could go a long way to promoting smaller containers, since those images have been downloaded over half a billion times.

<comment><USER7> > Yes, today most people who use Docker build containers with unnecessary > distro bits in them.

>... to move away from a heavyweight distro like fedora or ubuntu, to something like alpine or even more minimalistic.

The distro package management is one of the biggest reasons why people still stick with heavyweight distros like Ubuntu.

This is why DockerSlim [1] was created (at one of the Docker hack days, by the way :-)) It lets you use heavyweight distros as-is eliminating everything you don't need. You get the best of both worlds where you don't have to change your process and you still get small images (7, 10, 20 and even 30 times smaller than the original).

[1] - [http://dockersl.im](http://dockersl.im)

P.S.

It is safe for use in production :)

<comment><USER8> There's a packer manager in alpine. Maybe not as complete as ubuntu but pretty decent imho.

<comment><USER7> Yes, it does have a package manager, but... , like you said, it's not as complete and it is different, which means you need to put in extra efforts to convert and make things work (also don't forget about a potential lib C compatibility fun). Why go through all this trouble though? Why not use the distro you are already using as-is and still get Alpine-like image sizes? One nice thing about Alpine is its hardened kernel, but you won't be using it in your Docker container. Alpine is nice, but it's not worth it when there are other alternatives that don't require extra time and effort I'd rather use to build a better product.

And if you are using 3rd party Docker images Alpine won't help :)</comment></comment></comment></comment></comment></comment></comment></comment>


Python at Scale: Strict Modules - <AUTHOR> https://instagram-engineering.com/python-at-scale-strict-modules-c0bb9245c834 <comment><USER2> More and more I want someone to create a new language that amounts to a strict subset of Python, with mypy built-in, and is compilable into machine code. Python has by far my favorite syntax, community, and in my experience leads to the greatest productivity. There just happens to be a lot of overly dynamic features, that aren't even used by most, but used just enough to hold back optimization and structural improvement.

<comment><USER3> I like Python, but I often wonder how many developers use Python because they actually use dynamic language features versus just liking the languages' clean syntax and library ecosystem. I'm surprised languages that offer both REPL (for development) and AOT native compilation (for production), like OCaml, are not more popular. Evidence that syntax matters, I guess. :)

mypy and mypyc are interesting but their compile-time checks and optimizations are still hampered by Python's dynamic language semantics.

<comment><USER4> Don't underestimate inertia. I've worked with Python and Django for seven years. I know the libraries in the ecosystem. I know the framework. It's far easier for me to start a project with Django than to learn another framework or language.

</comment></comment></comment><comment><USER5> > This means that just by importing this module, we're mutating global state > somewhere else.

Yes, this !

That's why I hate Django and some flask app the most for, the fact that by importing a module, you're implicitly creating a database connection, and a lot of other magic stuff, which mean that now I can't import a constant defined in said module outside of `python manage.py`

Also as said below in the article, suddenly it's much harder to handle smoothly the "the database is momentary unavailable" (because someone has put the line starting the database connection in the global space of a module somewhere)

I much prefer frameworks/modules for which code is executed only once you invoke their "setup" function

<comment><USER6> _I much prefer frameworks /modules for which code is executed only once you invoke their "setup" function_

Django _does_ have a "setup" function. You can't import and use Django database connections outside of a running application without it.

Flask also has a "run" method and does no i/o without it.

<comment><USER7> Every time I hear a comment alike parent's, it makes me think how many times a day I actually read a comment in the same fashion, but about something I actually know nothing about.

<comment><USER6> With credit to the original poster, they might be complaining about the fact that Django is a monolithic framework and you can't really use Django code _without_ spinning up the i/o portion. Which is legitimate criticism, but frankly if that's what you need then you shouldn't be using Django.

<comment><USER7> If by I/O you mean web i/o then Django provides a perfectly functional management utility interface accessible through cli.

If, however by i/o you mean the database portion of it then Django works without database configuration, too.

</comment></comment></comment></comment></comment><comment><USER8> I love the idea, but it feels like just an idea at this point. I'd rather read about them releasing their 'compile-time' analyzer and revealing their measurements for how much startup time it saves.

In our codebase, we have pretty strict developer-enforced rules about not doing I/O at the module level, usually through the use of simple "Lazy" wrappers for module-level objects. I'd be curious to know what other approaches people have taken with Python here.

<comment><USER9> It is an interesting approach, though I feel like this could introduce some nasty unintended consequences given how dynamic and introspective Python can be (admittedly I haven't studied this particular implementation).

I always treated this a bit like single underscore private functions/methods, i.e., follow a convention that produces code that's easy to reason about, even if it's not strictly enforced by the language/compiler. So in practice this equates to separating out modules that mutate global state, and placing the majority of logic in "strict" modules that only declare a bunch of "pure" classes/routines. So the "non strict" code is really just a thin layer of wiring gluing everything together. For instance my Celery task files tend to be very thin.

<comment><USER8> well, we also heavily use static typing, so you end up with something like

my_db_conn: Lazy[DbConn] = Lazy(lambda: make_db_conn(...))

and MyPy will tell you if you're doing something silly when you try to use it.

EDIT: After typing up this response and submitting I realize you were talking about their strict approach rather than ours. whoops :)

</comment></comment></comment><comment><USER10> > How do we know that the log_to_network or route functions are not safe to > call at module level? We assume that anything imported from a non-strict > module is unsafe, except for certain standard library functions that are > known safe.

It's hard to know anything about the stdlib as it can be monkey patched, e.g. [1]

That said, you could solve this with diagnostics; calculate signatures of stdlib functions and classes to find any known safe ones that were patched. Run that check in your test suite to find problematic imports.

> If the utils module is strict, then we'd rely on the analysis of that module > to tell us in turn whether log_to_network is safe.

I like this. It seems far more usable than proposals like adding const decorators.[2]

[1]: [https://github.com/gevent/gevent/blob/master/src/gevent/monk...](https://github.com/gevent/gevent/blob/master/src/gevent/monkey.py)

[2]: [https://github.com/python/typing/issues/242](https://github.com/python/typing/issues/242)

</comment><comment><USER11> It's interesting to me that they are going down this path instead of the microservices path. This seems like something ripe for slowly breaking down into microservices.

Someone made a change that took down production because of non-deterministic outcomes? How about break out whatever they were changing into it's own service? With proper fallbacks, breaking that part shouldn't take down all of production again.

To be clear, I'm not saying microservices will solve all their problems or be less work. I'm just saying that with an equal level of effort, they would probably get more overall reliability by having multiple services, they'd be able to use multiple languages, whatever is suited to the task at hand, be able to deploy even more often with less risk, and be able to isolate these types of "change on import" behavior to a much smaller surface on any given deployment.

<comment><USER12> > _Someone made a change that took down production because of non- > deterministic outcomes? How about break out whatever they were changing into > it 's own service? With proper fallbacks, breaking that part shouldn't take > down all of production again._

Yeah, now you'll have 10 interconnected services, 10x the complexity, and everything will have the ability to take down all of large parts of production, plus all the extra pain points of a distributed system...

<comment><USER11> You won't have 10 times the complexity if you are taking a monolith and making each section services. You'll have to same dependency graph, it will just use the network to make calls between them instead of being local.

You'll have added complexity with the network calls, which is why I said it wouldn't be any less work, just different work.

<comment><USER12> > _You won 't have 10 times the complexity if you are taking a monolith and > making each section services. You'll have to same dependency graph, it will > just use the network to make calls between them instead of being local._

Merely "use the network to make calls between them instead of being local" will add 10 times the complexity -- you suddenly have a distributed system, latency, delays, parts that can be on or off, de-centralized configuration (which can also get out of sync), and so on.

</comment></comment></comment></comment><comment><USER13> This is yet another example of the divide between wizarding and engineering[1]. When you're a small startup, what matters is the expressiveness of your language, and the ability do do a lot of things very very quickly. Type safety, performance, readability, those things don't matter. You're just a bunch of engineers who know the whole codebase inside out, you're pretty certain of what you're doing. In short, you're wizarding. If you grow big enough, this approach slows you down greatly, and you need to switch to engineering. You sacrifice some speed for making the codebase more understandable to a larger group of people, you can no longer assume everyone knows all the code, you write unit tests, need types and dislike metaprogramming because of the confusion it creates. This is why languages like Python, Ruby, Lisp or Smalltalk are amazing for small startups, but Java is what enterprises use. They're different ends of the wizarding/engineering spectrum. I wish there was a language that let you move gradually from one end to the other, exactly when you need to.

[1] [https://www.tedinski.com/2018/03/20/wizarding-vs- engineering...](https://www.tedinski.com/2018/03/20/wizarding-vs- engineering.html)

<comment><USER14> Who is starting large-scale new projects in Java in 2019?

<comment><USER15> It's a good question. Why would you pick Java over Go or C++?

<comment><USER16> Why would you pick Go or C++ over Java?

C++ is a hydra of complexity, sure it has it's place, but it's not nearly as productive as Java for your typical web application.

Go is almost the opposite, so simple it lacks features like generics. The last time I used Go it had fundamental usability issues around dependency management(although I think recent versions have improved on vendoring a little).

<comment><USER17> > C++ is a hydra of complexity, sure it has it's place, but it's not nearly as > productive as Java for your typical web application.

Modern C++ well is as productive ( probably even more productive ) as Java. The main issue with C++ is recruitment, C++ engineers are rare because C++ is barely teached.

<comment><USER18> C++ is barely taught in ProgrammerGenerationFactories because "modern" C++ still allows "old" C++ and makes it difficult to stop developers from doing that.

Just like MISRA tries to constrain C programmers from doing dumb things in the embedded world, "modern" C++ tries to the same in the business world. But there isn't an easy way to enforce it, especially when you're outsourcing to some code sweatshop.

<comment><USER17> > But there isn't an easy way to enforce it

Every mature enough language has a subset that you need to avoid. Including Java. This is precisely due to this kind of things that every company need to have coding guidelines and proper static analysis tools.

> especially when you're outsourcing to some code sweatshop.

If you outsource your dev to cheap, other side of the world, low quality engineers. Then you deserve your problem, in any language.

I worked in the past for a company (embedded programming) that had an entire team of expensive engineers in Luxembourg just to fix the stupidities of an other team of outsourced engineers in India.

</comment></comment></comment></comment></comment></comment></comment><comment><USER19> Another thing that I would like to see in some kind of strict mode is the ability to mark explicit exports like in JavaScript modules. I often want to import multiple things globally at the top of a module because they are shared by multiple class or function definitions that I am writing. However, such imports end up being exposed to and usable by the consumers of my module, even though the consumers should really have imported those things at their source instead of via my module.

There are currently maybe two ways to tackle this "problem", without a strict mode:

1\. Don't import at the global module scope; but that's a bit tedious.

2\. Import with rename, like `import os as _os`, and then leave it to the principle of "we're all consenting adults". I.e. if anybody imports and used things that start with an underscore, it's clearly their fault, not mine.

<comment><USER20> 3\. Import as normal, and leave it to the principle of "we're all consenting adults"; unless something is explicitly called out as being part of the public API I consider Law of Demeter[1] "violation" the same as accessing _var.

[1] [https://en.wikipedia.org/wiki/Law_of_Demeter](https://en.wikipedia.org/wiki/Law_of_Demeter)

</comment></comment><comment><USER21> I think this is an interesting idea, which appears to embed a stricter subset of Python within Python itself. Have the Instagram engineers tried floating this with the wider community via established channels like Python-Ideas or discuss.python.org?

</comment><comment><USER22> I like the idea, but it feels a bit heavy handed outside of a very large team.

I think the first step here is to get away from the assumption that importing a module will have "interesting" side effects. This is not only a problem with Python...

I tend to create mini "dependency injection" frameworks that create a pattern for loading module code at some point well after import. This patterns tends to reduce to wrapping whatever code you have in the module in a function/closure instead of just running whenever.

Again, I like the idea of enforcing constraints with code, but I don't think it's a substitute for educating developers to avoid certain patterns and giving them infrastructure that makes the alternative easy.

</comment><comment><USER23> [https://docs.python.org/3/library/importlib.html#importlib.u...](https://docs.python.org/3/library/importlib.html#importlib.util.LazyLoader)

</comment><comment><USER24> Wow. Talk about solving the wrong problem!

Millions of lines of code in a monolith. 20s start up time. Meta monkey patching. One unit test per process... Yikes!

Software architecture, anyone?

Maybe Instagram should get a copy of Michael Feathers' book...

</comment><comment><USER25> I like that idea, it's just not that easy. How to do define module versions and inheritance, when you are not allowed to do global assignments in the module. declarations only, and no IO or global side effect is fine, but declaring versions and inheritance need to be allowed in global scope.

I added these ideas here: [https://github.com/perl11/cperl/issues/406](https://github.com/perl11/cperl/issues/406)

</comment><comment><USER26> > ... many of whom are new to Python.

well, if you ask me to write language X, I would definitely make mistakes for the first couple of weeks/months/years, that is why you need code review, mentoring and education plans for your hires.

> Here's another thing we often find developers doing at import time: fetching > configuration from a network configuration source.

      MY_CONFIG = get_config_from_network_service()

I am pretty sure this an anti-pattern, if this code passed the code review, you should make your review process more strict.

      def myview(request):
        SomeClass.id = request.GET.get("id")

> Likely you've already spotted the problem

Well, yes, why would you do this? why would this pass code review? why do we we have linters and other checks for dynamic languages

> It works great for smaller teams on smaller codebases that can maintain good > discipline around how to use it, and we should switch to a less dynamic > language.

It seems we are here blaming python for shortcomings of a monolith also, instead of chunking out specific businesses modules to separate services/micro-services.

TO be honest the strict mode seems interesting, but I believe the problems they seem to be facing can be solved by a couple of changes to their pocess and code:

\- everyone gets a mentor if they are not experienced in python or django

\- code review atleast by two experienced python developers(does not count if you have coded for Java for 20 years)

\- teams should try to move their logic outside the monolith(it sounds like they have a monolith)

\- write CI tests to measure how much time it takes to import a file, if it takes more than T(line count * LINE_PROCESSING_THRESHOLD) you have to fix your code.

\- prepare config and load it before running the actual server, no network call for getting config

All in all, python is suitable for big companies also, the thing is if don't care about the best practices, you would also have problems when you are a small startup, but in a big co it would make it impossible to move forward, trick is to independent of the company size follow best practices and have code review.

<comment><USER27> That's a long post to say "do more code review instead of investing into technical solutions to technical problems".

Clearly, Instagram's solution saves them time. That means faster code reviews which incidentally makes them more accurate. Your post doesn't really make sense.

</comment></comment><comment><USER28> It's very important to think about objects lifecycle management.

It's also important to... use pytest fixtures instead of arbitrarily patching around in tests.

</comment><comment><USER29> I have a question about a detail in the article:

> But if we moved the log_to_network call out into the outer log_calls > function, [...] this would no longer compile as a strict module.

My current understanding is that the log_calls method would NOT get executed during module load time!?!

Why would having a side effect in this function violate the intention of __strict__ ?

<comment><USER27> > _My current understanding is that the log_calls method would NOT get > executed during module load time!?!_

That's incorrect. log_calls gets executed on import because it's a decorator, so equivalent to `hello_world = log_calls(hello_world)` at the top-level (which does also get executed).

log_to_network in the _wrapped() definition doesn't get executed until hello_world gets called; but outside of the definition of _wrapped does get executed.

<comment><USER29> Right! I missed the fact that log_calls is used as a decorator further down.

</comment></comment></comment><comment><USER30> Avoiding module side effects and making classes and modules immutable seem like two separate concerns

<comment><USER31> Not really. Mutation in general, and in modules in particular, inhibit a lot of reasoning about the code, and thus stops a whole lot of optimizations from being possible. Guile (a scheme dialect) recently got declarative modules for that reason, where a top level binding cannot change (i.e. you cannot set! a binding, but you can wrap in it a mutable container and change the contents of that container). This makes procedure calls and variable lookup a lot faster. Andy Wingo wrote about it here: [https://wingolog.org/archives/2019/06/26/fibs-lies-and- bench...](https://wingolog.org/archives/2019/06/26/fibs-lies-and-benchmarks) .

Those optimizations won't mean much for cpython, since Cpython doesn't try to run things fast, but for something like pypy this could be a big deal.

<comment><USER31> To quote the article (from.memory): "adding static modules is probably the single most important optimization guile can do in the near future".

The quote is probably wrong, but it is right in spirit.

</comment></comment></comment><comment><USER32> Try Zope.Interface and Pyramid for a framework. You'll be really happy.

</comment><comment><USER33> who would have thought Instagram is a python monolith. ?

</comment><comment><USER34> > Instagram Server is a several-million-line Python monolith

That's bananas.

Nothing Instagram does requires that much code.

Also, that much Python code means you're doing it wrong.

<comment><USER34> No, I'm seriously you guys.

Python is too expressive to require mega-LoC for that site.

You could implement an OS, relational DB, spreadsheet, and optimizing compiler all in less than that.

<comment><USER35> You have no idea about their codebase, the implementation details of their features nor how they counted the lines (comments included?). So stating that it's dumb is beyond ridiculous.

You are right in that it's certainly a high LoC count for Python, but still...

<comment><USER34> I didn't say "dumb" I said "bananas".

And yes, knowing nothing else about their code base than A) It's in Python, and B) it's several million lines of code, I feel very confident that there is at least an order of magnitude too much of it. Instagram is just not doing anything that complicated.

(I should mention I specialize in maintaining and refactoring legacy Python code. I know what I'm talking about here.)

<comment><USER36> Features that are "not complicated" can actually very easily be "very complicated" at scale. Which Instagram does have. 500 million users, every single day.

<comment><USER37> Fewer LOC would actually benefit them at scale.

If you need _several millions_ of lines of Python to do what Instagram server does, the code is bloated.

My bet is that they let too many Java devs loose on the code base, without experienced Python devs reviewing the commits and managing the deluge of unnecessary classes. I've seen it happen before.

<comment><USER38> >If you need several millions of lines of Python to do what Instagram server does

I have this feeling that you're probably not all that aware of 95% of what their code actually does, and thus probably not in a position to make judgements as to whether their code base is truly bloated relative to what it does.

</comment></comment></comment></comment></comment></comment></comment><comment><USER39> I like this a lot.

</comment><comment><USER40> This article is among the best argument for using a typed language I've yet seen.

<comment><USER41> This has nothing to do with types. It's more about static guarantees the language gives about module import behavior.

<comment><USER42> In OP's defence:

 _> So that's a third pain point for us. Mutable global state is not merely available in Python, it's underfoot everywhere you look: every module, every class, every list or dictionary or set attached to a module or class, every singleton object created at module level. It requires discipline and some Python expertise to avoid accidentally polluting global state at runtime of your program._

 _> One reasonable take might be that we're stretching Python beyond what it was intended for. It works great for smaller teams on smaller codebases that can maintain good discipline around how to use it, and we should switch to a less dynamic language._

 _> But we're past the point of codebase size where a rewrite is even feasible. And more importantly, despite these pain points, there's a lot more that we like about Python, and overall our developers enjoy working in Python. So it's up to us to figure out how we can make Python work at this scale, and continue to work as we grow._

Those are literal quotes from the article. That is quite damning. How did they get to this point? By starting when Python was appropriate, and taking it day by day.

<comment><USER41> How is "our developers really like Python even on a million-line codebase, despite its global mutable state requiring discipline" "quite damning"?

</comment></comment></comment></comment><comment><USER43> It still blows my mind that people don't use strongly typed languages in the first place and spare themselves from all this future pain.

My guess (based on my experiences) is that companies wind up in this position from having inexperienced people building early versions of products instead of hiring experienced engineers (who are usually more expensive).

<comment><USER44> Python is strongly typed, just not static.

<comment><USER43> Python uses duck typing: [https://en.wikipedia.org/wiki/Duck_typing](https://en.wikipedia.org/wiki/Duck_typing)

I would categorize it as a subset of dynamic typing, and that's what Wikipedia says too.

<comment><USER9> Dynamic typing vs. static typing is on a different axis than strong vs. weak typing. Python is a strong dynamically typed language, with some "static lite" features introduced in Python 3.

Dynamic typing means that types can be changed arbitrarily at runtime, compared to statically typed languages which define all types at compile time.

Strong/weak means that type coercions rarely/never happen automatically. For instance JS has some interesting behavior enabled by weak typing `[] + [] -> ""`. Whereas Python rarely coerces things for you. The division operator in Python 2 was strongly typed, while they changed it to weak typing in Python 3 (inline with the practicality vs. purity convention).</comment></comment></comment></comment>


Your Post Has Been Removed: Tech Giants and Freedom of Speech (Open Access Book) - <AUTHOR> https://link.springer.com/book/10.1007/978-3-030-25968-6 <comment><USER2> 'Tech Giants'? Did they mean facebook, twitter, google? I guess.. We should stop calling them 'Tech'. If they were once, not any more. Those are giant advertisement companies, generating most if not all of their revenue from advertisement. No one would call big investment banks 'Tech Giants' event though they produce one of the most sophisticated software. Google departed from being a Tech company when their former CEO reached out to the election campaign manager of a certain political party and offered their 'influential power' to them. Facebook never was a tech company. Those managed to accumulate huge amount of money and use it to attract very talented people. However, these talents put to work to extract data from 'the users' (young zuckerberg named them very well once) and sell it to a highest bidder, literally. It is very much the same as those mentioned above investment banks attract most of the talents from physics and math to work on investment algorithms. Clear definition would help to understand the motivation of advertisement companies to remove a post, block a video or a user, to sensor one opinion or the other, to offer their platforms to political campaign.</comment>


Ask HN: What are the current standards for online password storage? - <AUTHOR>

I know there are legal standards for credit card companies, ecommerce sites etc., but what should any other website do? I'm building a joke aggregation website right now(I know, I know) and I'm planning on sha1 with a 16bit salt. Too much? Too little? <comment><USER2> Wrong algorithm. Here's what you need to know:

* The whole reason leaking user's passwords is dangerous is because users tend to re-use passwords.

* The purpose of hashing a password is to make it hard for an attacker to know the plaintext

* Many users are probably using fairly predictable passwords (like 'abc123' for example) so if you run them through sha1 these predictable passwords will always come out the same. This means I can precompute them and just look them up. (the table of hashes is called a rainbow table) This is why you use a salt.

* The size of the salt doesn't matter too much (once you're over a certain size) and it doesn't really matter if your attacker knows it. The point is just to add entropy to the user's password.

* The sha family of hashing algorithms are designed to be fast. This is a problem for protecting your data. It means once the attacker has your table, they can rip through the calculations very quickly. You want a slow algorithm that can be tuned to get slower as computers get faster.

* This means for you it might take 1 second to hash a person's password but that means it also takes an attacker 1 second per attempt at brute-forcing. That's not worth it.

* There are several of these but the most popular are scrypt, bcrypt, PBKDF2:

<http://www.tarsnap.com/scrypt.html>

<http://en.wikipedia.org/wiki/Bcrypt>

<http://en.wikipedia.org/wiki/PBKDF2>

* Any one of the above three would be a good choice. Popular web frameworks like rails and django used to just sha256 passwords but they have realized that the speed of that algorithm makes it non-ideal for password hashing.

* If you don't know this stuff, then do spend the time to get familiar with it. Many people will tell you not to implement hashing or encryption algorithms yourself. That's probably a good idea but it's no excuse for not educating yourself.

<comment><AUTHOR> Hey! Thanks for taking the time to write out such a detailed reply. I did read up. A salt needs to be large enough to avoid clashes with other salts in my database and the algorithm needs to be slow enough to calculate that it's impossible for hackers to crack but fast enough that it doesn't bog my site down. Right?

I picked sha _because it's fast to compute_. The idea behind the level of security I'll get is that it is trivial to provide and in the occurrence of a data leak, it will require a non-trivial effort to get any plaintext passwords out. Any less than this and the hacker just needs to dump the data on the internet and my users will immediately be SOL.

I think I over-thought the performance bit though. Sign-ups and sign-ins are not a regular enough event to be too much of a performance issue. Thanks again!

<comment><USER2> _the algorithm needs to be slow enough to calculate that it's impossible for hackers to crack but fast enough that it doesn't bog my site down. Right?_

That's exactly right. It's kind of an insidious problem because it goes against the intuition of most web developers, which is to be as fast as possible always.

</comment></comment></comment><comment><USER3> Jeff Atwood has a great layman's introduction to rainbow tables and why this stuff deserves thought here: [http://www.codinghorror.com/blog/2007/09/rainbow-hash- cracki...](http://www.codinghorror.com/blog/2007/09/rainbow-hash- cracking.html)

The follow up by Thomas Ptacek, a security expert, goes into a lot more detail about what to use and how: [http://chargen.matasano.com/chargen/2007/9/7/enough-with- the...](http://chargen.matasano.com/chargen/2007/9/7/enough-with-the-rainbow- tables-what-you-need-to-know-about-s.html)

Both are very readable articles, which I find helps immensely.

</comment><comment><USER4> I'm all in favor of making the passwords as secure as possible for your users, but I hate it when I'm dictated how my password has to look. For less than critical accounts I always use the same password, which is easy to remember to me. If any account is hacked, it's not a disaster. I just hate it when I sign up for something the website throws back 'your password should be at least 8 charachters long, contain one of the following characters: @!#$ etc. etc.'. From a usability point of view this is a nightmare, and it will cost you users.

In the end, the user is responsible for choosing a secure password. You can only direct.

On a side note: the 'unbreakable' Enigma cypher was cracked (partly) because the German re-used part of the encrypting key, for example, one operator often used the name of his girlfriend as a key. These easy to distuinguish words were called cillies (see [http://en.wikipedia.org/wiki/Cryptanalysis_of_the_Enigma#Ope...](http://en.wikipedia.org/wiki/Cryptanalysis_of_the_Enigma#Operating_shortcomings) ).

My point is that you should do everything possible to encrypt the passwords of your users without sacrificing usability.

</comment><comment><USER5> Bcrypt should be the ideal starting point.</comment>


Google's Social Playground Grows to 235 Million Active Users - <AUTHOR> http://www.maximumpc.com/article/news/googles_social_playground_grows_235_million_active_users

<comment><USER2> Does anyone really use Google plus?</comment>


SF JavaScript Meetup #2 -- first one was awesome - <AUTHOR> http://javascript.meetup.com/4/calendar/7922755/

<comment><USER2> booyah. lets do this.

<comment><USER3> didnt know you did javascript</comment></comment>


Has anyone else noticed a change in Heroku's pricing structure? - <AUTHOR>

I logged into one of my apps today (one that I created only a week ago) and noticed that I was now being offered a choice between "Legacy" and "Basic" pricing models. Unsurprisingly, there's no information to be found about this anywhere else on their site, including in the documents that they link to titled "Learn more about Heroku's pricing structure."<p>http://i.imgur.com/tlnpHC4.png <comment><USER2> How did you get to this? I am not seeing it, just the usual dyno sliders.

<comment><AUTHOR> It just showed up when I went to manage my app. Weird right?</comment></comment>


Ask HN: What do you build, what tools and editors do you use? - <AUTHOR> Just want to explore what is out there that I am not aware of. <comment><USER2> Although I started as a developer, my programming these days is mostly personal/hobby, and some automation coding at work. So my needs, pace and focus are probably not the same as yours.

From memory, and running:

      $ history |sort -r |less

      Anaconda Python
      pip
      pydoc
      ipython
      jupyter (ipython-notebook)
      ipdb
      pudb
      unittest
      setuptools
      (Starting to use sphinx)
      git
      github
      tig
      pypi.python.org
      pandas
      Vim
      restructuredtext
      rst2html
      Linux
      bash scripts
      urxvt
      tmux
      ssh (If I'm making something "up there.")
      cygwin at work
      conemu with clink at work
      virtualbox
      vagrant

<comment><AUTHOR> Thanks for sharing. Comemu is awesome. I'll have to find what clink is.

<comment><USER2> [https://conemu.github.io/](https://conemu.github.io/)

[http://mridgers.github.io/clink/](http://mridgers.github.io/clink/)</comment></comment></comment>