Facebook's unethical experiment manipulated users' emotions - <AUTHOR> http://www.slate.com/articles/health_and_science/science/2014/06/facebook_unethical_experiment_it_made_news_feeds_happier_or_sadder_to_manipulate.html <comment><USER3> 155,000 users for each treatment of the experiment, says the paper. Let's presume random selection, and that the occurrence rate of mental disorders is the same for Facebook users as the general public (probably not too far off). Then Facebook intentionally downgraded the emotional state of:

10,000 sufferers of Major Depressive Disorder

5,000 people with PTSD

4,000 bipolar disorder sufferers

1,700 schizophrenics

and a plethora of others with various mental disorders.

11/100,000 people commit suicide each year in America. How many were part of that treatment of the experiment, without consent or knowledge?

As a scientist, I'm fascinated by the research. As a human being, I'm horrified it was ever done.

[http://www.nimh.nih.gov/health/publications/the-numbers- coun...](http://www.nimh.nih.gov/health/publications/the-numbers-count-mental- disorders-in-america/index.shtml)

<comment><USER4> > How many were part of that treatment of the experiment, without consent or > knowledge?

How would this be determined in offline experiments where people volunteered?

<comment><USER5> All subjects would have the potential consequences explained to them so they could make an informed decision about whether to take part. Informed consent is a very important ethical principle in human-subject experiments.

</comment></comment></comment><comment><USER6> I'm puzzled about the outrage.

FB _already_ filters out updates based on some blackbox algorithm. So they tweaked the parameters of that algorithm to filter out the "happier" updates, and observed what happens. How is this unethical? The updates were posted by the users' friends! FB didn't manufacture the news items; they were always there.

I detest FB as much as the next guy, but this is ridiculous.

<comment><USER7> They altered people's feeds for a psychological experiment with the specific intent of manipulating their mood. That is _highly_ unethical.

<comment><USER8> But why if it's done for science it's unethical, but when done by news stations, politicians, advertising agencies, motivational speakers, salesmen, etc. it's suddenly ok?

<comment><USER9> Users of Facebook see it as a neutral platform for communicating with people they know. Consumers of the things you listed know it's top-down messaging coming from people they don't know or necessarily trust.

So, there is a difference. It's still a complex question, though -- is filtering or prioritizing based on emotional sentiment really different from what they are already doing with inserting ads and such?

<comment><USER8> I see it this way: they did a study, so it's fair.

Were they to filter posts by emotional sentiment as a part of their normal operations, I'd find it unethical, or at least something I might not want. But I'm totally fine with them subjecting users (including myself) to random research studies, as those are temporary situations, and with Facebook's data sets, they can have great benefits for humanity.

Perhaps Facebook should provide an opt-in option to for user to be a subject of various sociological experiments at unspecified times. I'd happily select it.

</comment></comment></comment></comment></comment><comment><USER8> I strongly hope that they won't care about any of this "outrage" and continue to do more and more experiments. Maybe even open it up as a platform for scientists to conduct studies.

Facebook is in the unique position of possessing data that can be orders of magnitude more useful for social studies than surveys of randomly picked college students that happened to pass through your hallway. There's lot of good to be made from it.

But the bigger issue I see here is why it's unethical to "manipulate user emotions" for research, when every salesman, every ad agency, every news portal and every politician does this to the much bigger extent and it's considered fair? It doesn't make much sense for me (OTOH I have this attitude, constantly backed by experience, that everything a salesmen says is a malicious lie until proven otherwise).

<comment><USER10> It's an interesting question. I have the same averse reaction to this story that a lot of people here have, but I admit I also thought, "If Facebook hadn't published this as research, but just had it as a business decision to drive more usage or positive associations with the website, no one would care."

My own way to reconcile this -- and I admit it's not a mainstream view -- is that advertisement and salesmanship should be considered just as unethical. I don't know how to quantify what "over the line" is, but it all feels like brain-hacking. Things like "The Century of the Self" suggest that in the past century or so we've become extremely good at finding the little tricks and failings of human cognition and taking advantage of vulnerabilities of our reasoning to inject the equivalent of malicious code. The problem is that when I say "we" I don't mean the average person, and there's an every-growing asymmetry. Like malware developers adapting faster than anti-malware developers, most people have the same level of defense that they always have had, while the "attackers" have gotten better and better at breaking through defenses.

Sometimes I'll see discussions about "what will people centuries from now think was crazy about our era?" and there's a part of me that keeps coming back to the idea that the act of asymmetrically exploiting the faults of human thinking is considered normal and "just the way things are."

<comment><USER8> > _My own way to reconcile this -- and I admit it 's not a mainstream view -- > is that advertisement and salesmanship should be considered just as > unethical._

I agree with that, or probably think even more strongly - <USER2> advertisements/sales are more unethical than research. It's difficult to put limits though, because even if many salesmen clearly act maliciously, pretty much everything you do or say influences people this way or another; it's how we communicate.

What I'd love to see is Facebook creating an opt-in option for an user to be a part of further sociological research. I'd gladly turn it on and be happy that I'm helping humanity, while Facebook could limit their studies to people who explicitly consented (there's an issue with selection bias though). They have too good data to be not used for the betterment of mankind.

<comment><USER10> Good point. I guess my concern -- recognizing this makes me sound like a Luddite or someone going on about "humans were never meant to know about this" \-- is that the results of research like this aren't going to be used for the betterment of mankind. Rather, it'll be all about how to use a new mental vulnerability to get more eyeballs on someone's content or to increase the dopamine hits from browsing the site.

What I would love -- and what I would eagerly opt-in to -- would be a system where Facebook could educate users on irrational behaviors. "We noticed that 60% of users like you spent an average of 30 seconds more looking at this kind of content... this is because your brain etc etc etc". Creepy, perhaps, but if there were a way to help people be more aware of and defend against advertisement that would be neat.

<comment><USER8> > _Rather, it 'll be all about how to use a new mental vulnerability to get > more eyeballs on someone's content or to increase the dopamine hits from > browsing the site._

Sadly, you've made a great point here. It's very likely that the end results of research will be used exactly for that - as it already happens with most of psychology.

I hope though that some of that research will be used to create better policies and help the society.

> _What I would love -- and what I would eagerly opt-in to -- would be a > system where Facebook could educate users on irrational behaviors._

I'd happily opt-in to that as well (and opt-in all my relatives too ;)). I don't expect Facebook to ever do that, as it'd exactly opposite to their goal to be able to a/ influence their users, and b/ cater for advertisers, but there already are websites doing exactly that (e.g. LessWrong). They're niche places though; I'd love to see something popular enough to reach general audience.

</comment></comment></comment></comment></comment><comment><USER11> _Facebook's Unethical Experiment: It intentionally manipulated users' emotions without their knowledge._

I'm not defending Facebook or the experiment, but if you're going to call them out for "manipulating users' emotions without their knowledge", then you need to call out every advertising, marketing, and PR firm on the planet, along with every political talkshow, campaign, sales letter, and half-time speech...

<comment><USER12> With that in mind, I'd find it important if our social media analytics publicly accounted for such emotional manipulations.

It could be wise to begin to share such emotional-information influences to equally let users and admins be so sensitive (and acknowledge its a real part of our systems).

<comment><USER11> Users aren't fed every post by every one of their friends -- users would be overwhelmed with posts flying by so fast they couldn't keep up -- so it's no secret FB tweaks its feed algorithm to keep users engaged. And experimenting with network effects is what social networks do.

Every time you see a picture of someone you like it's like a little shot of dopamine goes off in your head. Facebook wants to optimize those dopamine shots to continually bump engagement and create an experience where everyone is habitually checking their feed every 10 mins.

This type of behavior design/economics research is done by Dan Ariely at MIT ("Predictably Irrational" [http://www.ted.com/talks/dan_ariely_asks_are_we_in_control_o...](http://www.ted.com/talks/dan_ariely_asks_are_we_in_control_of_our_own_decisions.html)) and the Stanford Persuasive Tech Lab ([http://captology.stanford.edu/](http://captology.stanford.edu/)).

<comment><USER12> Exactly why I'm trying to articulate importance of giving public facing language to such persuasive technologies, by actually having formal public markers for emotions being manipulated.

(e.g. Apple ad with astericks linking to emotional signaling research and analytics, to help users determine deeper interests [this ad attracts people who feel X, Y, Z based on...]).

</comment></comment></comment></comment><comment><USER13> To say this response is unnecessary and unfounded is disingenuous. Marc Andreessen (whom I respect) tweeted _" Run a web site, measure anything, make any changes based on measurements? Congratulations, you're running a psychology experiment!"_ I could not disagree more.

This isn't simply a matter of looking at metrics and making changes to increase conversion rates. The problem is that the whole of users have come to expect Facebook to be a place where they can see any and all of their friends' updates. When I look at an ad, I know I am being manipulated. I know I'm being sold something. There is no such expectation of manipulative intent of Facebook, or that they're curating your social feed beyond "most recent" and "most popular", which seemingly have little to do with post content and are filters they let you toggle.

What FB has done is misrepresent people and the lives they've chosen to portray, having a hand in shaping their online image. I want to see the good _and_ the bad that my friends post. I want to know that whatever my mom or brother or friend posts, I'll be able to see. Someone's having a bad day? I want to see it and support that person. That's what's so great about social media, that whatever I post can reach everyone in my circle, the way I posted it, unedited, unfiltered.

To me this is a disagreement between what people perceive FB to be and how FB views itself. What if Twitter started filtering out tweets that were negative or critical of others?

<comment><USER14> There are better ways to question the ethics of the experiment. Here's a simple approach:

Would anyone want their emotions manipulated to be unhappy or unhealthy?

The corollary in a medical experiment would be, would a healthy person want to undergo an experiment that could make them sick?

Some people mentioned advertising as a counterpoint, that what Facebook does is not at all different from advertising's psychological manipulation. Well maybe some forms of advertising ought to be regulated too. Would a child voluntarily want their emotions manipulated by a Doritos ad to make the sicker or fatter?

Even if it's not known what the outcome is, the two points are:

(1) Facebook's various policies specify you will randomly participate in their studies, but

(2) It matters if an experimental outcome can harm you.

So even though you agreed to participate in experiments, you weren't told the experiments could hurt you. That is a classic medical ethics violation, and it ought to be a universal scientific ethics violation.

<comment><USER15> > The corollary in a medical experiment would be, would a healthy person want > to undergo an experiment that could make them sick?

Yes, many people would willingly volunteer to take experimental drugs that

i) might not work

ii) might uave severe side effects

because those people are dying and want some months more life.

<comment><USER16> Note the parent comment said "healthy". Last I checked, healthy != dying.

</comment></comment></comment></comment><comment><USER17> I just wrote a blog post about the ethical/professional obligations of the researchers associated with this study: [http://www.maxmasnick.com/2014/06/28/facebook/](http://www.maxmasnick.com/2014/06/28/facebook/)

When you publish a paper, you are supposed to write in the body of the manuscript if it's been approved by an IRB and what their ruling was. I'm surprised it was published without this, even though it apparently was?

It's also appropriate to address ethical issues head-on in a paper about a study that may be controversial from an ethical perspective.

If it really was approved by an IRB, then the researchers are ethically in the clear but totally botched the PR.

If not, then I think the study was not ethical.

<comment><USER18> It was approved by an IRB.

</comment></comment><comment><USER19> The difference between this experiment and advertising or A/B testing is _intent_. With A/B testing and advertising, the publisher is attempting to sway user behaviour toward purchasing or some other goal which is usually obvious to the user.

With this experiment, Facebook are modifying the news feeds of their users specifically to affect their emotions, and then measure impact of that emotional change. The intention is to modify the feelings of users on the system, some negatively, some positively.

Intentional messing with human moods like this purely for experimentation is the reason why ethics committees exist at research organisations, and why informed consent is required from participants in experiments.

Informed consent in this case could have involved popping up a dialog to all users who were to be involved in the experiment, informing them that the presentation of information in Facebook would be changed in a way that might affect their emotions or mood. That is what you would expect of doctors and researchers when dealing with substances or activities that could adversely affects people's moods. We should expect no less from pervasive social networks like Facebook.

</comment><comment><USER20> Oh, please.

Every single time Facebook changes anything on their site it "manipulates users' emotions". Show more content from their friends? Show less? Show more from some friends? Show one type of content more, another less? Change the font? Enlarge/shrink thumbnail images? All these things affect users on all levels, including emotionally, and Facebook does such changes every day.

Talking about "informed consent" in the context of a "psychological experiment" here is bizarre. The "subjects" of the "experiment" here are users of Facebook. They decided to use Facebook, and Facebook tweaks the content it shows them every single day. They expect that. That is how Facebook and every other site on the web (that is sophisticated enough to do studies on user behavior) works.

If this is "immoral", then an website outage - which frustrates users hugely - should be outright evil. And shutting down a service would be an atrocity. Of course all of these are ludicrous.

The only reason we are talking about this is because it was published, so all of a sudden it's "psychological research", which is a context rife with ethical limitations. But make no mistake - Facebook and all other sophisticated websites do such "psychological research" ALL THE TIME. It's how they optimize their content to get people to spend more time on their sites, or spend more money, or whatever they want.

If anyone objects to this, they object to basically the entire modern web.

<comment><USER8> Exactly. I find this situation to be an example of ridiculous pattern matching. Is it published? Then it's a psychological experiment, and needs to be evaluated by an ethics board. Is it just A/B testing? Then it's not "science", so no need for ethics board.

<comment><USER21> So as long as you aren't publishing the results of your experimentation, none of the ethics that apply to experimenting on humans apply?

That's an...interesting...theory.

<comment><USER20> Any A/B test is "experimentation on humans". Facebook and all other web giants constantly do such behavioral studies. The only difference is that this one was published.

In other words, if someone argues that this was unethical experimentation on humans, then there are 1,000 other studies we never heard of that are far, far worse. But we know they exist.

It doesn't make sense to argue that. Websites have to experiment with different ways of doing things, and seeing how that affects their users. This isn't just a web thing either, of course - businesses need to try different things in order to optimize themselves. And to measure which is the best to get people to spend more.

</comment></comment></comment></comment><comment><USER22> I'm torn about this. In some ways, I can see how mental health issues can be detected which can hopefully help us avoid these horrifying events (mass shootings off the top of my head). But then again, I can see how the Army or the government in general can control any type of popular uprisings. FB, Twitter, etc have given us tools to connect and join in efforts to fix what is wrong (I'm thinking the Middle East though that can be said about the Tea Party or even Occupy movement). If the price is right, FB can hand over that power (i.e. NSA) or through these secret courts, the Army/government can have direct control of FB. It's crazy to think that this only occurs in countries like Russia and China but wake up America! This is happening here as well!

</comment><comment><USER23> You know why I think they are doing this? Because there have been studies showing that people are miserable on facebook (see below) and I think people are starting to pick up on it. So FB feels some pressure to lighten the mood a bit. But as usual they do it with the subtlety of a drunken fool.

Also, the comparison to an A/B test is a false one. This is specifically to alter the moods of the user and test the results in a study, not to improve the users experience or determine which app version works better.

Regarding the study mentioned above: [http://www.newyorker.com/online/blogs/elements/2013/09/the-r...](http://www.newyorker.com/online/blogs/elements/2013/09/the- real-reason-facebook-makes-us-unhappy.html)

</comment><comment><USER24> > Facebook intentionally made thousands upon thousands of people sad.

Hang on. Wasn't the experiment to see _whether_ users would post gloomier or happier messages respectively? This very different from _intentionally_ making people sad.

<comment><USER25> "I only silently removed happiness from your life because I was curious what your reaction would be!"

<comment><USER26> Silently removed happiness from _my web application_ that you visit on a voluntary basis. If you think my web app is too gloomy, feel free to stop coming.

<comment><USER25> What if Gmail started silently removing happy emails from your Inbox by auto- archiving them?

<comment><USER8> You expect GMail to show you every non-spam message sent to you in your inbox. On Facebook, on the other hand, you expect a curated list of recent posts - otherwise you wouldn't be able to keep up with what your 500+ friends and 1500+ liked pages post every day. So comparing GMail to Facebook makes no sense at all.

<comment><USER25> Gmail users have an expectation that Google won't start silently diverting their legitimate email as an experiment on them. That's the comparison if you didn't quite grasp that.

You're claiming that users would "expect" Facebook to do something like filter out all the happy posts from their friends and family without telling them? I don't think many would agree with you.

<comment><USER26> I think he's merely claiming that you expect Facebook to curate the news feed. _How_ they do so (and for what purpose) is ever-changing and has never been fully transparent, thus your expectations for those particular factors is irrelevant.

<comment><USER25> Yes. He's saying that their lack of transparency justifies their abuse. I'm trying to explain that I disagree.

<comment><USER8> I don't see any abuse here, and I believe that their lack of transparency wrt. filtering algorithms is justified.

First off all, the shortest description of what they do wouldn't probably be far from publishing the algorithm iteslf. An algorithm that's ever changing and probably different depending on where you live or to what group you were randomly assigned. 99% of people wouldn't care anyway, and being transparent about the algorithm would likely make them less happy - right now they accept Facebook as is and don't think twice about it; give them the description of how things work and suddenly everyone will start saying that Facebook filtering sucks because random-reason-511.

Moreover, the only people that stand to benefit from knowing Facebook's algorithm are advertisers, who will game the hell out of the system for their own short-term benefit, just like they do with Google. It's something neither Facebook users, nor Facebook itself want.

<comment><USER25> Sorry, you failed to comprehend what I said and I even made it really short. Maybe try reading it a few more times.

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER27> This study really makes me feel vindicated for unfollowing all of my friends along with every brand on facebook. I could've been part of the study but I'd never know, since the only way I see my friends' posts is to visit their pages directly where I can see them all unfiltered. I've been doing this for the past six months and it has dramatically improved the way I interact with the site. I can still get party invites and keep in touch with people, but I'm immune to the groupthink.

</comment><comment><USER27> I have a feeling a lot of college courses on research methods are going to use this as an example of a grave ethics breach for years to come. With an experiment group as large as they used, statistically it's almost inevitable that someone in that group will commit suicide in the near future. If that person is in the group that was targeted for negative messages, even a rookie lawyer could make a sound case before a jury that Facebook's researchers have blood on their hands.

<comment><USER28> surely people have committed sucide after using facebook even without this study. is facebook guilty of that, too?

you may argue that facebook was "trying to make people depressed" but that simply isn't true. what if showing more of my friends negative status updates actually _helps_ them? depressed people are shunned in our society; facebook gave a voice to the voiceless. that's wonderful!

<comment><USER27> > you may argue that facebook was "trying to make people depressed" but that > simply isn't true.

Legal culpability issues aside: did facebook manipulate people's emotions intentionally? Did they inform them that they were going to do this, and of the risks involved? Did they get their consent? If the answers to the last two questions aren't unequivocally yes, then facebook is in deep trouble.

Edit: this also misses the problem that the subjects were never screened for their basical ability to give informed consent. Merely clicking through the ToS does not mean that you're not suffering from a mental illness that nullifies their agreement to the ToS.

Lastly, this experiment clearly involved deception, since the test subjects weren't informed up-front that they were being manipulated. This is problematic[1] if the subjects weren't debriefed after the study:

>It is stated in the Ethical Principles of Psychologists and Code of Conduct set by the American Psychological Association, that psychologists may not conduct research that includes a deceptive compartment unless the act is justified by the value and the importance of the results of such study, provided that this could not be obtained in an alternative way. Moreover, the research should bear no potential harm to the subject as an outcome of deception, be it physical pain or emotional distress. _Finally, a debriefing session is required in which the experimenter discloses to the subject the use of deception in the research he /she was part of and provides the subject with the option of withdrawing his/her data._

[1] [http://en.wikipedia.org/wiki/Informed_consent#Deception](http://en.wikipedia.org/wiki/Informed_consent#Deception)

</comment></comment></comment><comment><USER29> FWIW, the HN discussion on the study published on PNAS here:

[https://news.ycombinator.com/item?id=7956470](https://news.ycombinator.com/item?id=7956470)

<comment><USER30> Yes, and today's wave of media controversy about it hasn't added significant new information, so I think this post counts as a dupe.

<comment><USER25> Heavens forbid the fluff piece on a Google executive gets pushed down the page. I'm dumbfounded that you would kill this story. Hacker News is changing.

<comment><USER30> Well, it was a borderline call, so I've restored the thread.

Perhaps I should explain our thought process. There were at least half a dozen major web publications today putting out variants of this indignant post about "Facebook's unethical experiment". Did all these authors suddenly develop a passion for science ethics? Of course not. It is simply the internet controversy du jour. Those have never made for good HN stories, and the policy has always been to penalize them, because otherwise they would dominate the site.

In cases of pile-on controversy like this one, when the original story has already been discussed on HN—which is pretty common, because HN users tend not to miss a day in posting these things—we usually mark the follow-up posts as dupes unless they add important new information, or at least something of substance. Does this article add anything of substance? It didn't strike me that way, but arguably it does.

As for the PR fluff piece you think is on the front page, why haven't you flagged it? It's impossible for us to catch (or even see) all such things. We rely on users to point them out.

<comment><USER25> The idea that this story is "controversy du jour" is wrong in my view. I think it's an incredibly important story and the underlying issue may be the biggest in technology. At the very least it is not spam, gossip, or other obvious junk.

The explicit HN policy used to be to allow controversies like this to wash over the site. We all remember seeing the home page covered in many submissions on the same topic. The fear that this would cause a topic to "dominate the site" has been proven false numerous times. I'm not sure why that would be a consideration.

I wasn't objecting to the puff piece on the home page. I don't think lightweight stuff like that can dominate the site either.

<comment><USER31> Complaints about stories taking over the entire front page of the site are as old as the site itself. This comment might be the first one I've ever read suggesting that the phenomenon was a good thing that we should preserve.

<comment><USER25> Who is going to decide how many stories on a topic we get to have? Should there have been one Mt.Gox related submission? One Snowden related submission? Up to one submission per day per topic? I'm not suggesting it's "good" I'm suggesting it's better than the alternative.

Killing dupes when there is more than one active discussion is one thing. This submission was the only active discussion on this topic. Removing it is just editorial curation that is of no benefit to anyone at all.

<comment><USER30> Those are good questions and I'd be happy to discuss them, but you seem to be under the impression that HN didn't use to be intensively moderated. That model is wrong. HN was always intensively moderated, curated, or whatever one calls it. That's the only reason why you are able to write something like this:

 _The fear that this would cause a topic to "dominate the site" has been proven false numerous times._

That didn't prove itself false, nor did the community make it false; it was PG who made it false. He poured countless hours into managing the site and countless more into writing code to help manage it.

That model hasn't changed. It's more transparent now, because users asked for it to be. Transparency has the side-effect of making it seem to some people like we've fundamentally altered HN when it doesn't work like they assumed it did.

<comment><USER25> Few people know better what PG, yourself, and others have done for this site or appreciate it more than me. I've seen lots of threads get penalized or killed and reversed. I know it hasn't been perfect in the past either.

I regret saying anything and I won't comment in the future. Thanks.

<comment><USER30> Please don't regret saying anything and for heaven's sake please don't stop commenting! This stuff is messy, unobvious, and unsatisfying. I'm painfully aware that there's no way to make HN consistent, to satisfy everybody any of the time, or anybody all of the time. The least bad job is all we can strive for, and we can't do that without feedback.

Also, sorry for the snippiness in my tone above. I don't always succeed in responding the way I want to.

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER32> So does this mean that people can increase their happiness by using plugins that hide negative posts from their social media?

<comment><USER33> Possibly, but only in the short run, as skewed perception of reality tends to have long-term negative consequences. Which is precisely one of the reasons why this kind of stuff is evil.

</comment></comment><comment><USER34> Author falsely assumes that people changes their sharing behavior due to changes in their mood. More likely they just feel like "everyone's posting cats on Facebook, so that's a place for sharing cats, let me do too", or otherwise.

</comment><comment><USER35> Before and/or after the fact, research participants are made aware that they were part of a psychology experiment.

I wonder if Facebook plans on alerting subjects of this experiment to their participation?

</comment><comment><USER36> Isn't Slate in the business of exactly that: manipulating their readers emotions?

<comment><USER37> Yes, but they're better at it than Facebook. They've got a bunch of gullible illogical peasants about to ban A/B testing... or at least drive it underground. For the children.

</comment></comment><comment><USER38> I just use Facebook to bookmark youporn at this point

</comment><comment><USER39> Been kind of surprised there hasn't been more of a reaction to this. I guess the Internet has reached peak Facebook outrage.

</comment><comment><USER40> Another nail in the FB coffin.

Edit: for me at least.

<comment><USER41> And only 90,000 more nails to go before your average non-tech user who has Facebook as their homepage drops them.

Until a replacement comes about and a large number of contacts move, it has become such a large part of these peoples lives it isn't going anywhere. Arguments and reasons don't sway them. Sadly.

I've never even been on facebook. But my girlfriend and extended family use it religiously. My dad and a couple of other members finally dropped it as the result of my rants but the rest (the vast majority) just think I'm suspicious and nutty and go right on posting their entire lives.

So, facebook can pretty much do as they please. And apparently they do.

</comment></comment><comment><USER42> people still use facebook?

</comment><comment><USER43> > "If you are exposing people to something that causes changes in > psychological status, that's experimentation"

Or art, or journalism, or advertising, or football etc.

</comment><comment><USER44> Every business that makes sense will try to make its customers happier.

Showing people bad news to get more engagement has roughly the same moral standing as the evening news.

I guess I don't get it.

[It must be wrong because they learned something from it, I guess?]

<comment><USER4> What's your position on creating fake news to get more engagement? Some lines are defended because the slippery slope on the other side is infinite.

<comment><USER45> The thing about the slippery slope is that it is far more often a logical fallacy than it is a real danger.

<comment><USER4> [http://pando.com/2014/06/28/facebooks-science-experiment- on-...](http://pando.com/2014/06/28/facebooks-science-experiment-on-users- shows-the-company-is-more-even-powerful-and-unethical-than-we-thought/)

"Facebook itself could target certain users, whether they be corporate rivals or current/former employees. Having such strong psychological control over your workforce would certainly have its benefits. And if Facebook ever gets caught? Why, the company could claim it's all part of a social experiment, one that users tacitly agreed to when they signed up.

With over one-tenth of the world's population signing into Facebook every day, and now with evidence to back the emotional power of the company's algorithmic manipulation, the possibilities for widespread social engineering are staggering and unlike anything the world has seen. Granted, Facebook's motives probably are simply to convince people to buy more stuff in order to please advertisers, but the potential uses of that power to impact elections or global trade could be enticing to all sorts of powerful interest groups."</comment></comment></comment></comment>


Applying to YC – GetStream.io - <AUTHOR> https://getstream.io/

<comment><USER2> I'm not particularly in the market for this product, but two quick bits of feedback:

1\. What is a "FanOut"? I see it on your pricing table, but I'm not sure what it is exactly. I could guess, but I probably shouldn't have to if it's critical to your pricing.

2\. (Minor) In a couple of spots on the pricing table you have the "$" after the number.

Screenshot with comment: [http://screencast.com/t/rRqJCbjb](http://screencast.com/t/rRqJCbjb)

EDIT: I should mention that you have FanOut defined in the FAQs, but I think maybe a tooltip or something on the pricing table would help (especially on the homepage).

<comment><AUTHOR> thanks!

</comment></comment><comment><AUTHOR> Guys critical feedback is much appreciated!</comment>


Are you guilty of making these mistakes when handling images on your website? - <AUTHOR> http://cloudinary.com/blog/are_you_guilty_of_making_these_mistakes_when_handling_images_on_your_website <comment><AUTHOR> Images account for almost 63% of the bytes of the average web page. This article talks about 3 recent image handling mistakes of developers and how to solve them, as well as a review of the top 10 common mistakes that web developers make in handling images of websites.</comment>


Can I use a computer without RAM? - <AUTHOR> https://www.quora.com/Can-I-use-a-computer-without-a-RAM?share=1 <comment><USER2> I think the narrow answer is, that if the computer has on-chip L1 cache, then if you can arrange for the L1 cache to contain all the information you need to run a program, there is no reason in principle why you couldn't run entirely in L1 cache off the die, and never touch the off-chip memory access channels.

Arranging for state to lie in L1 cache without RAM is a challenge, I wonder if some combination of things from a JTAG header and chip behaviour could get you there?

the Boot ROM, in some sense, is the information you minimally are looking at. So, a boot rom, with state to prep L1 cache and then a run command, which operates totally in L1 cache?

(I imagine some operations on L1 attempt write-back and would be bad, but perhaps that can be mitigated by some clever tricks to act like the write-back worked?)</comment>


The Greatest Regex Trick Ever (2014) - <AUTHOR> http://www.rexegg.com/regex-best-trick.html <comment><USER2> Regular expressions match regular languages (hence the name). If your language involves pairs of things (e.g HTML), it's not regular. Perl hacked support for this in via backreferences and other extensions, but these are slow and illegible. Use a proper context-free grammar parser if you need to parse a context free grammar, you know?

More broadly, people fear and misunderstand regexes because they have no idea how they work. It becomes much easier if you understand how they map to deterministic finite state machines. Recommended reading: [https://swtch.com/~rsc/regexp/regexp1.html](https://swtch.com/~rsc/regexp/regexp1.html)

Once you understand how they work, you can basically read a regex left to right and intuitively know all the strings they'd match. There is no such thing as an unmaintainable/illegible basic regex - they're just words with some placeholders in them - it's when you cram in extended functionality (which is basically a programming language where all the keywords are single characters) that shit hits the fan.

<comment><USER3> I use regexes all the time for parsing data on a variety of 3rd-party websites. Just because they aren't _perfect_ at matching every potential theoretical situation doesn't mean they shouldn't be used. In practice, regexes can be a simple and reliable way to grab data out of HTML. Don't dismiss them out of hand!

Another point, generally overlooked by the theoretical purists, is that HTML in the wild is rarely correct, and your perfect HTML parser will barf when trying to process it. Regexes on the other hand don't have to care about exact syntax and can cope with horribly mangled data.

<comment><USER4> An ordinary regex obviously can't parse html, because html is not regular (given nested elements and the pumping lemma). But what you can easily do with a regex is to tokenize html - extract anything that looks like a start tag, for example. The simply approach will obviously get some things wrong - a tag inside a comment is meaningless, for example - but for a lot of uses, this difference simply isn't important.

If the job is to extract all links from a webpage, regexes will do just fine, and will probably be easier to write and understand than alternate approaches. (This is absolutely not a fair comparison of course; you could compare writing a regex engine to writing a html parser. But I digress.)

If the job is to determine whether a given webpage is a member of the set that includes all valid html documents. then a regex is not sufficient.

If the job is to extract a list of syntax tokens from a webpage, a regex is likely fine.

If the job is to assign semantic meaning to every token in that list, a regex just won't work.

Either way, the point is to know what you're doing. Much "parsing" of webpages is not parsing in the formal language sense, and who cares that it isn't because it doesn't need to be.

<comment><USER5> No, tasks like "extracting all links from a webpage" are absolutely trivial using an HTML parser. Run the following XPath query on an HTML object:

      //a[@class='specified_string']/@href

Yes, you have to understand the syntax XPath to write such expressions, just like you have to know the language of regexes. Or at least be able to google them.

The answer to "who cares" is "you", because you're the one who's going to catch hell when your regex failed to capture some hyperlink that utilized some feature of XML that exceeded your test-cases. The one-liner above is guaranteed to Just Work on all valid XML documents, so why even create such a monstrosity?

Everyone knows that Regexes Cannot Parse HTML, and yet _people still try it_ because they think they're smarter than Noam Chompsky. The real truth is that everything looks like a nail to these people, because all they have is a hammer.

<comment><USER6> Because valid documents are rarer than documents where regex parsing is good enough.

<comment><USER5> Then you're not talking about parsing HTML/XML, are you? How could you possibly know which links or syntax tokens are actually going to be displayed on a page if you feed the browser's parser an invalid document?

There are fault-tolerant HTML parsers like TagSoup that are specifically designed to handle dirty HTML and spit out a valid document object. If you have sources that are malformed badly enough that it's still not working, you can define custom SAX properties to handle them. But a task like that is certainly a best-case effort and the interpretation of such a library is no more or less valid than the interpretation of the browser's parser. It's not a valid document to start with and nothing can make it so.

If you are only parsing values out of a single specific data template, you know it's not going to parse as HTML or XML, you know that it's never going to contain weird values, and you know it's never going to change - then go hog wild. But it's fundamentally a brittle approach that only holds as long as those assumptions do. I've made the mistake of believing some of those about my data and it's bitten me before. And I really question the implicit assertion that "most html parsing" would fall into that exceedingly narrow category. Especially after a couple years of feature creep.

Just keep your logic general and normalize your data. Offer a failover to a fault-tolerant parser in your data layer with a logged warning. This is much more durable and doesn't silently generate invalid tokens or silently fail to capture valid tokens. Regexes simply cannot offer the capability to fail loudly. So once you are no longer actively babysitting your custom regex parser it could have started failing at any time - how would you even know?

</comment></comment></comment></comment></comment></comment><comment><USER7> Here is the __TL;DR __. This regex matches Tarzan but not "Tarzan":

        "Tarzan"|(Tarzan)

You can also include more than one case of what you don't want to match. This one also finds only the cases of Tarzan that don't match the first three patterns:

        Tarzania|--Tarzan--|"Tarzan"|(Tarzan)

You can even use more complex regexes. This matches all words not in an image tag:

        <img[^>]+>|(\w+)

And likewise this matches anything not surrounded by <b> tags:

        <b>[^<]*</b>|([\w\s]+)

<comment><USER8> That's not exactly it, the regexp does match both "Tarzan" and Tarzan, but the capture group 1 will only be set for strings that contain Tarzan without quotes. So by examining that group after matching you know in which case you are. (that also means you can only use this "trick" when you can examine capture groups, i.e. not generally in editors).

</comment></comment><comment><USER9> I once used the wonderful perl module [Regexp::Assemble]([https://metacpan.org/pod/Regexp::Assemble](https://metacpan.org/pod/Regexp::Assemble)) to produce a regexp to match [every single suburb/town name in Australia]([https://gist.githubusercontent.com/singingfish/d43c884fbac00...](https://gist.githubusercontent.com/singingfish/d43c884fbac0089d8523/raw/63eeaf88a2e8d896c07da3b2440080233dd48395/regex%2520for%2520every%2520suburb%2520name%2520in%2520australia.txt)) from a csv file download from the post office website. It was blazingly fast ... considering (better than the recdescent parser I'd been previously experimenting with).

Here's the code that generated the regex:

        use Regexp::Assemble;
        my $ra = Regexp::Assemble->new;
        while (<$FH>) {
            $csv->parse($_);
            next if $. == 1;
            my @fields = $csv->fields;
            $ra->add($fields[1]);
        }

        my $suburbs =  $ra->as_string;

<comment><USER10> Can you still get that csv file? I thought AusPost had started charging for everything.

<comment><USER11> Woah, do you mean this? If so, wow, never thought my tiny little data collection would come in handy. [http://badcunt.club/cant/stop/information/babe/every- austral...](http://badcunt.club/cant/stop/information/babe/every-australian- postcode.csv)

<comment><USER10> Might be good for hacks, but it'll go out of date eventually and can't be used for commercial purposes without incurring the wrath of AusPost. Thanks for hosting it, though.

</comment></comment></comment></comment><comment><USER12> This technique is unreliable in practice, and the author's discussion is confused.

First, their explanation doesn't make sense. They're supposing that there's some determinacy in the order in which a matcher can be expected to examine the different possible matches. But that's provably not the case: if it were, then deterministic and non-determinsitic finite automata would be inequivalent.

But the technique in question does seem to require some determinacy as to which of several alternatives will match against a string. Where does that determinacy come from? The semantics of the alternation operator (the '|') as usually formulated don't specify any preference among alternations. For that reason, POSIX _additionally_ requires that a matcher return the longest possible match (and if there are several such, the leftmost is what must be returned). Where you do find an explicit guarantee concerning which of several different possible ways of matching will be preferred, it's almost certainly because the engine is aiming at POSIX compliance.

Such compliance has a significant cost, though, as it requires the matcher to consider _all_ possible matches (in order to find the largest). For that reason, most regex engines forego strict POSIX compliance and only guarantee that some match will be returned if one exists, not that that match will be the leftmost longest. Some engines offer the option of requesting strict POSIX behavior, but the default will always be to eagerly return the first match encountered (and recall the point above that there provably can't be a guarantee about the order in which matches are encountered, in general).

You should never do this in production code unless you're sure that your matcher is POSIX-compliant.

</comment><comment><USER13> I almost wrote this off as it seemed to be about how to write unmaintainable regex soup but the author pulled something quite elegant out of the hat at the end.

</comment><comment><USER14> this one is great, too. matches all printable ascii characters:

[ -~]

[http://www.catonmat.net/blog/my-favorite- regex/](http://www.catonmat.net/blog/my-favorite-regex/)

<comment><USER15> English only.

<comment><USER3> Hence the ASCII in the description.

<comment><USER15> Since accented characters are used in the English language it's not even 'English only'.

<comment><USER14> "printable ascii" describes everything that it is and everything that it isn't; no further clarification is needed.

[https://en.m.wikipedia.org/wiki/ASCII](https://en.m.wikipedia.org/wiki/ASCII)

<comment><USER15> I will spare us an answer because it wouldn't be printable English.

<comment><USER16> You are the only one who said "printable English". Everyone else has said "printable ASCII". Do you perhaps not understand what ASCII is?

</comment></comment></comment></comment></comment></comment></comment><comment><USER17> > Match Tarzan but not "Tarzan"

Unfortunately it doesn't work.

Let's say I wanted to match a string following Tarzan but not "Tarzan", I will try his technique:

        ("Tarzan"|(Tarzan))\s+and JillOfTheJungle

Unfortunately this matches both:

        "Tarzan" and JillOfTheJungle

and

        Tarzan and JillOfTheJungle

Or maybe he meant:

> Capture Tarzan but not "Tarzan"

<comment><USER18>
        a=/(?:"Tarzan"|(Tarzan))\s+and JillOfTheJungle/;
        matched = !!a.exec(x)[1]

Works fine.

People often forget to solve the problem they need to solve (match x), and instead work on other things (find a regex to match x).

<comment><USER19> "People often forget to solve the problem they need to solve (match x), and instead work on other things (find a regex to match x)."

Completely agree. One common mistake is building a complex regex to match the elements you want to _find_ from a string, when an easier approach is to split the string on a simple regex that matches the things you want to throw away.

</comment></comment></comment><comment><USER20> This "trick" is simply exploiting a bug in regex implementations.

The regex

        "Tarzan"|(Tarzan)

should match the string

        "Tarzan"

in _two_ ways: first, matching the entire string; and second, matching the substring "Tarzan" in the whole string "\"Tarzan\"". But most regex implementations drop extra overlapping matches. I argue this is incorrect behavior, because it complicates understanding what a regex _means_ \- you have to understand the /order/ in which your regular expression matcher interprets your regular expression, which is an implementation detail. I conjecture that a DFA-based regex engine would not be able to exhibit this order-biased behavior, at least not with the standard approach.

However, it's interesting that this "bug" turns out to be a "feature" for the case of excluding other behavior. I'm not sure what conclusion to draw from this.

<comment><USER21> Given a string "aaabbb"

what should the results be, of the regular expression "(aa|aaa)(abbb|bbb)"?

$1 = ?

$2 = ?

<comment><USER20> I believe it matches in three ways:

\- the whole string, grouped as "(aa)(abbb)",

\- the whole string, grouped as "(aaa)(bbb)";

\- the substring "aabbb", grouped as "(aa)(bbb)".

<comment><USER21> Sorry, I made an assumption there that you were talking about the practical applications of regex, not the theoretical applications, and I was asking you to explain how you would practically return multiple matches in... any environment.

This was a small regex designed to create multiple answers to see how you resolved the issue, obviously we can engineer regexes that return far more results. So something's got to give. I don't agree with you that regex's innately imply all matches are valid.

<comment><USER20> The original article _already_ relies on finding multiple matches, in order to ignore the matches that don't contain the group that we're interested in.

Python's regex library, for example, can return multiple matches. It has three functions:

\- `re.match`, which checks whether the whole string matches.

\- `re.search`, which checks for the first location in the string that matches.

\- `re.findall`, which finds "all" non-overlapping matches.

I was simply suggesting that the "non-overlapping" constraint in findall is a "bug", in some sense, because it exposes implementation details of the regex engine.

But, again, given that it is apparently a _useful_ bug, maybe I am wrong. But that leaves open the question what the right spec for regex matching is, anyway.

</comment></comment></comment></comment></comment><comment><USER22> This article desperately needs a tl;dr, but major props for the two regex tricks at the very top of the page.

</comment><comment><USER23> That reminds me of something else with regex which I thought was extremely clever: implementing an A* search: [http://realgl.blogspot.com/2013/08/battlecode.html](http://realgl.blogspot.com/2013/08/battlecode.html)

<comment><USER24> that is a very nice hack. this kind of 'do as much work per standard library call as you can' approach is pretty much the way to go in interpreted languages like python or octave, which is in part why numpy is so popular — not only does it allow you to program at a higher level of abstraction, it also makes your program run more efficiently.

however, i think that technically, that's not an A* search he implemented, just a breadth-first search. i'm not an expert (i've never even implemented A* search), so i could be mistaken. i'm interested to hear whether other people agree.

</comment></comment><comment><USER25> Meh.

I think an even greater Regexp trick is the regular expression that determines primality:

[http://stackoverflow.com/questions/3296050/how-does-this- reg...](http://stackoverflow.com/questions/3296050/how-does-this-regex-find- primes)

<comment><USER26> The article actually talks about the prime validator regex, and goes on to say that while it's an awesome trick, it is not the "best ever" because it has limited scope.

I for one think the trick described in the article is pretty useful, and might even use it someday (and possibly have already without realizing it).

</comment></comment><comment><USER27> Very nice trick, while using of foo|(bar) is very simple, somehow I don't see such approach being used very often, and it looks like it could simplify a number of things.

</comment><comment><USER28> Maybe I'm too old. I tend to think of a regex as either matching or not matching.

Finding a bit of code that uses a capture to determine whether a match was found seems like it would easily be confusing/inobvious.

Some pretty clear commenting and it would be ok... maybe.

Also... I wonder how well it would work as part of a larger regex, one that already uses captures (or non-capturing groups)? The examples are all nice, short and sweet... but how often do regex based solutions stay short and sweet? A few maintenance cycles/years and suddenly you've got this funky regex/capture thing that only Bob understands and he's way to busy to talk to you for 5 minutes... and once you change things then Bob suddenly finds time to review your code to complain how you broke it for such a simple change. There goes your bonus you told the wife you were sure to get so you could take her and the kids on vacation. The day after your divorce finalized Bob sends you a fix request to use that improved scheme of yours because the old regex one isn't flexible enough anymore.

</comment><comment><USER29> I thought the answer was going to be "tricking the world into thinking regex was a good idea". I've always considered regex to have the rare and elusive "write only" flag. Write only. As opposed to read only. Because once you write a regex that's it. You will never know what it does ever again.

<comment><USER30> Just use tests and comments. For instance, in many language it's trivial to declare regexp in multiple lines like this:

        regex = "(^[0-9])" //catch the initial digit (group 1)
              + "/"        //skip the following slash
              + "([A-Z]+)  //capture the identifier (group 2)
              ...

Regexps are too powerful a tool to ignore. I'd much rather write a simple regex than a whole screenfull of code, especially when the former does the work faster (because regex engines are pretty efficient).

<comment><USER31> Another way of doing it is to use variables and helper functions, particularly if you're going to be creating lots of regexps from similar components. This is what I usually do in Python. For example, while making no promises about the value of this code specifically:

        def any(x): return x+"*"
        def many(x): return x+"+"
        def capture(x): return "("+x+")"
        def lit(x): return re.escape(x) # shorthand
        bol="^"; eol="$"
        ident="[A-Za-z_][A-Za-z0-9_]*";

        find_function=(bol+any(" ")+lit("def")+many(" ")+
                       capture(ident)+any(" ")+lit(":")+
                       any(" ")+eol)

You can handle more or less stuff this way according to how much you and/or your readers like regexp syntaxp. The above is probably further than I'd take it in practice; I'm familiar with the regular expression syntax, but I'd still probably at least use something like the `ident' variable just to keep clutter out of the regexp.

(A nice demonstration of this sort of thing is emacs's rx module (see, e.g., [http://emacswiki.org/emacs/rx](http://emacswiki.org/emacs/rx)). I couldn't find any good non-emacs documentation about this, nor much that would make sense to people unfamiliar with lisp, but when you're in emacs you can get help on it using C-h f rx RET.)

<comment><USER30> I agree. In terms of variables, I think a good strategy is assigning names to meanings, e.g.

        PRODUCT_ID = "[0-9]{2}[A-Z]{1,5}"
        ...
        CUSTOMER_ID = "[0-9]{2}[A-Z]{1,5}"
        ...
        ...
        regexp = "(" + PRODUCT_ID + ")" // capture product ID in group 1
               + "something something"  // something something
               + "(" + CUSTOMER_ID + ")"; // capture customer ID in group 2

In this example, even though the two constants contain the same regular expresison, they refer to two different concepts. Part of the problem with understanding regexp-based code is connecting parts of the expression with what they _mean_. Above strategy addresses this.

I also like assigning names to capture groups I depend on. So instead of, later in code, asking for e.g. matcher.group(2), I ask for matcher.group(GROUP_CUSTOMER_ID). Makes for a much more readable code.

Speaking of Emacs's rx, it's absolutely amazing and I'm sad that I only discovered it just few days ago :(. Another similar concept, from the other side of "code = data" equality is Common Lisp's ( _of course_ it's Lisp again) CL-PPCRE and its internal representation of regular expressions:

        * (parse-string "(ab)*")
        (:GREEDY-REPETITION 0 NIL (:REGISTER "ab"))

        * (parse-string "(a(b))")
        (:REGISTER (:SEQUENCE #\a (:REGISTER #\b)))

        * (parse-string "(?:abc){3,5}")
        (:GREEDY-REPETITION 3 5 (:GROUP "abc"))

You can encode any regexp you want as an S-expression, trading off conciseness for legibility. See [http://weitz.de/cl-ppcre/#create- scanner2](http://weitz.de/cl-ppcre/#create-scanner2) for more.

</comment></comment></comment></comment><comment><USER32> Very elegant, I like it.

Except I'm now having a major case of semantic satiation for the word "Tarzan"...

</comment><comment><USER33> I love regexes but they make my head hurt. I really rather badly need to spend some time properly learning it inside and out.

<comment><USER34> I first learned them properly by reading through [[http://www.regular- expressions.info/tutorial.html](http://www.regular- expressions.info/tutorial.html)]. I'd also recommend the website as a good reference for when you forget things.

</comment></comment><comment> bro-stick Unimpressive. The author of this article obviously didn't have a compiler class where one learns how regexes are basically glorified NFAs that are deterministicly convertible into a much more efficient DFA state machines (read: PCRE JIT), instead of assuming regexes are processed by O(N^2) algorithms.

<comment><USER24> 'convertible into a much more efficient DFA state machines (read: PCRE JIT)'

pcre supports reduction to dfa and also jit, but not only are they not the same thing, they are mutually exclusive. also, ever regexp engine i've seen that supports capturing and backreferences uses not worst-case quadratic-time but actually worst-case exponential-time algorithms, although i'm pretty sure this isn't actually unavoidable.

may i suggest that the next time you think about posting a comment that begins with 'Unimpressive. The author of this article obviously didn't', that you include less than one major technical error per sentence in it.

</comment></comment><comment><USER35> Generally I found it strange how difficult it is to do a search for everything except something.

The script looks elegant, but like the author mentions, doesn't work in a text-editor, so I would consider it the greatest.

<comment><USER15> Many searches that you can easily explain to your grandmother are difficult with regex.

</comment></comment><comment><USER36> How does this handle '"Tarzan" Tarzan'?

</comment><comment><USER37> Convincing the world that they don't exist?

</comment><comment><USER38> More like the author tricked you by changing the problem halfway through the very long article. Try this instead:

      (?:(?<!")|(?!Tarzan"))Tarzan

</comment><comment><USER39> ... and now you have two problems.

</comment><comment><USER40> Reminds me of this quote from Jamie Zawinski: "Some people, when confronted with a problem, think "I know, I'll use regular expressions." Now they have two problems."

<comment><USER19> Why does this remind you of that quote? Because it's about regular expressions? Because it seems to me that if you have the problem described in this post, and you solve it the way the post describes, using regular expressions, you have _solved your problem_ , regardless of what jzw says.

<comment><USER4> When I'm faced with a problem, I think about using recursion. Then I have two problems. Then I think about using recursion. Then I have three problems. Then I ...

</comment></comment></comment><comment><USER41> Parsing HTML with a regex? You should read this answer on Stack Overflow: [http://stackoverflow.com/a/1732454/84250](http://stackoverflow.com/a/1732454/84250)</comment>


About the security content of macOS High Sierra 10.13.2 - <AUTHOR> https://support.apple.com/en-us/HT208331 <comment><USER2> I find it interesting how many of those are attributed to project zero members

<comment><USER3> Google has long been Apple's security division. Often I wonder if Apple has any security people at all. The last Safari update had 11 CVEs from Google. Most of Apple's updates credit one or more issues to Google, and often Apple credits OSS-Fuzz, which is also a Google project.

<comment><USER4> >Often I wonder if Apple has any security people at all.

It just feels like they don't since they don't let their security people have social media presences. For example, their recent hire Jonathan Zdziarski

<comment><USER5> It looks like you were cut off there…

<comment><USER6> No, reread it as "For example, [consider] their recent hire Jonathan Zdziarski[, whom you'll see is a leading iOS security researcher from a cursory Google search]"

The GP just omitted a bunch of implied statement, which isn't immediately obvious especially if you don't natively speak English.

<comment><USER7> He forgot a period at the end, so it does look like he got cut off potentially.

</comment></comment></comment></comment></comment></comment><comment><USER8> Just let my Mac take in this update, now sitting in front of it watching it say

"About 3 minutes remaining"

And then jump to

"About 29 minutes remaining" :-( The price I pay for being dumb to let it update during the work day. OSX is starting to feel more like the old Windows....

<comment><USER9> And what time did it actually take in the end?

<comment><USER10> For me, about a hour and 2 (or 3?) reboots. And this is minor version update that consists only in bugfixes. I don't understand why overwritting few megabytes of files takes so long time and requires multiple reboots.

</comment></comment></comment><comment><USER11> From a cursory glimpse, it seems Apple only pathes CVEs in OSS components when the OS itself gets an upgrade.

The next time there is a problem in Apache, the chances seem pretty high it will remain unpatched on macOS for weeks, if not months.

<comment><USER12> Why does macOS ship with Apache ?

<comment><USER13> Before Mountain Lion, a personal web server was available under System Preferences > Sharing > Web Sharing.

They removed the UI to enable it in Mountain Lion, but the functionality is still built in and can be enabled if you install Apple's MacOS Server app from the app store. Or you can just enable it from the command line.

<comment><USER14> heh, remember when you could actually host your own website from your home connection on port 80? Dynamic DNS services, etc... ISPs put a quick end to that, though :(

<comment><USER15> I... still do?

This is more about ISPs where you live than anything else. Most people don't want the hassle.

<comment><USER14> Yeah, guess it varies, but a lot of ISPs block incoming port 80 connections. Common enough that noip.com has a "port redirection" feature, interestingly enough: [http://www.noip.com/support/knowledgebase/my-isp-blocks- port...](http://www.noip.com/support/knowledgebase/my-isp-blocks-port-80-what- can-i-do/)

</comment></comment></comment></comment></comment></comment><comment><USER16> I was hoping this would fix my "Month 13 is out of bounds" error. It doesn't I still have apps I cannot run now because of this. Looks like it is time to back everything up and wipe my disk back to 10.13 with no other updates.

<comment><USER17> Wow, thanks for mentioning this. My Mac has been freezing when opening tons of apps lately, making it basically useless, and I couldn't figure out what was wrong until I checked this. I never would have guessed it was a core OS issue. What a ridiculous bug to not patch immediately.

Apparently you can at least mitigate it partly by disabling ReportCrash.

<comment><USER16> Can you share how to do this? Anything I can try to be able to launch some of my critical apps might help.

Edit: for those who are curious: [https://www.gregoryvarghese.com/reportcrash- high-cpu-disable...](https://www.gregoryvarghese.com/reportcrash-high-cpu- disable-reportcrash/)

</comment></comment></comment><comment><USER18> How to update when App Store is not working?

> The operation couldn't be completed. (NSURLErrorDomain error -1012.)

Same error is shown on terminal too.

<comment><USER19> Do you have any antivirus or (shady) anti-malware software installed? Not necessarily the problem, but it wouldn't be the first time..

<comment><USER18> No. Last successful update was just before this root bug.

</comment></comment></comment><comment><USER20> Maybe Apple should hire a few more of those mythical C developers that never make mistakes.

3 x out of bounds errors

6 x memory corruption issues

</comment><comment><USER21> People are reporting problems on Reddit [https://www.reddit.com/r/apple/comments/7hzy3a/macos_10132_u...](https://www.reddit.com/r/apple/comments/7hzy3a/macos_10132_update_combo_10132_released/) with the update. Anybody here tried it yet?

<comment><USER22> It took several minutes on a couple of Macs with fusion drives. It seemed stuck at "Calculating time remaining..." but eventually finished, rebooted, and continued installing, this time displaying a reasonable time remaining value.

<comment><USER23> I had this problem with the last Sierra update. Have not pulled the trigger on High Sierra yet.

</comment></comment></comment><comment><USER24> Direct download link from Apple Support: [https://support.apple.com/kb/DL1946](https://support.apple.com/kb/DL1946)

</comment><comment><USER25> I find it interesting that the most notable names from P0 team aren't native US citizens.

Even with dual citizenship they won't get clearance easily to work for NSA.

<comment><USER26> How on earth can you tell if someone is a native citizen from their name?

And what difference does it make if they're native or naturalized? One of the bedrock principles of American democracy is (or at least is supposed to be) that a citizen is a citizen. There's a reason that the phrase "second-class citizen" is supposed to have universally pejorative connotations.

<comment><USER27> He's not wrong about it being more difficult for people with dual citizenship to get security clearance, though. At least in that sense you can be a "second class citizen."

<comment><USER26> I'm a naturalized U.S. citizen with a dual citizenship, and I had no trouble (well, no more than the usual trouble) getting a security clearance.

But what does any of this have to do with anything anyway? The linked-to page doesn't mention the NSA, P0 team, or security clearances.

</comment></comment></comment></comment><comment><USER28> This is their way of saying: upgrade from Sierra to the seemingly still supremely buggy High Sierra or you'll get owned?

Gee, thanks.

<comment><USER29> Long time mac user, versed in Linux but have been using Mac for its "convenience" for years: Upgraded to high sierra, and my power modes started working totally irrationally with seemingly no explanation. When I closed the lid it suddenly started going crazy and nearly burnt a hole in my desk. I think it burnt out the logic board in this way, the GPU and kernel started panicking after 2 minutes running. When turned off it would turn itself on and go into this crazy hyper swap mode, the box when I was shipping it to applecare seemed like it would catch on fire. Had to keep using SMC shutdown to get it to turn off. I dont know if the issue was High Sierra, macbook pro 2016 (which are total crap in my opinion why in the world would you hardwire the hard drive into the logic board??), or both, but it suffices to say I'm buying a Thinkpad, and Im only using Ubuntu on it.

<comment><USER30> Make sure it is a new Intel CPU too so you can't get power management to work there either. #skylakeWasFun</comment></comment></comment>


Ask HN: How do you stand at a standing desk? - <AUTHOR>

I know it sounds weird, but I have been using a dresser as a test for a standing desk before making the investment.<p>I seem to really be moving as a stand at my computer.  I lean on one leg or stand really wide.  What does everyone else do? Am I weird? <comment><USER2> Careful if the dresser isn't at an ideal height where your forearms can be comfortably parallel with the floor. Otherwise, you can end up with neck and shoulder pain. The fidgeting might be due to the dresser's height forcing you into an odd posture. Just a guess...</comment>


Deaf and Unemployed: 1k Applications but Still No Full-Time Job (2019) - <AUTHOR> https://www.npr.org/2019/01/12/662925592/deaf-and-unemployed-1-000-applications-but-still-no-full-time-job <comment><USER3> I have a good friend of mine who I worked with years ago. He's a usability engineer and he's completely blind (gradual; could see until he was a teenager). He keeps reading up on passive income and he wants to do something else, but he's with a good company and there's honestly not much else he thinks he can do.

Even for disabled people who get work, it may be difficult for them to get other work or be as mobile as those without serious disabilities.

Part of the problem is simply America as well. I met a women in a hostel from Norway who was on disability payments. It was enough to make her house payments, food and even some travel. People I know on disability in America can barely survive. We have no trains or good mass transit, and ride- shares/cabs are expensive.

</comment><comment><USER4> Not sure if there are any jobs left that don't require voice interaction over phone/skype/whatever. Heck, any kind of realtime multiplayer gaming neither!

I can telephone but it's very unreliable and tiring. Am self-employed and it helps, but still any communication is constant source of anxiety - <USER2> contact has became universal habit.</comment>


FBI Fugitive Bobby Khan - <AUTHOR> https://youtu.be/0tDtWqa5xfE <comment><AUTHOR> TLDR: OP turned problem wants passage for family and trial. OP parent does not want Bobby speaking and is using family as leverage.</comment>


Bruce Brown, 80, Dies; His 'Endless Summer' Documented Surfing - <AUTHOR> https://www.nytimes.com/2017/12/12/obituaries/bruce-brown-documentarian-of-surfing-is-dead-at-80.html <comment><USER2> "Endless Summer" is a classic. It's up there with "A Sunday in Hell" (paris- roubaix cycling race) and "On Any Sunday" (motorcycling) for quality mid- century documentary sport films.

<comment><USER3> Total classic. I would recommend even for non-surfers. "Dogtown and Z-Boys" too..

<comment><USER4> Also, "Step Into Liquid", "Riding Giants" and just about any Jack McCoy film.

</comment></comment></comment><comment><USER5> More than document surfing, he documented a spirit and passion.

Do not watch the movie if you are feeling restless and have wanderlust. You will end up quiting your job and traveling the world.

<comment><USER6> Just hearing the name of the film, that I last watched 10+ years ago in a completely different point of my life and mindstate makes me start to wonder how I got to this. Today is going to be a slow day.

<comment><USER7> My thoughts exactly. Well put.

</comment></comment></comment><comment><USER8> Experimental guitarist Fennesz classic track Endless Summer: [https://www.youtube.com/watch?v=lGcPwGqPHO0](https://www.youtube.com/watch?v=lGcPwGqPHO0)

</comment><comment><USER9> > "With enough money from his filmmaking and investments, Mr. Brown did not > work after making "The Endless Summer 2." He pursued hobbies like target > shooting, collecting old cars and racing rally cars."

<comment><USER10> And:

> In addition to his son, who wrote and directed a sequel to "On Any Sunday" > in 2014, Mr. Brown is survived by his daughter, Nancie Brown; another son, > Wade; eight grandchildren; and two great-grandchildren

Surfing and a nice family. He won at life.

</comment></comment><comment><USER11> Every time I watch that movie I want to retire and chase waves. It's an extraordinarily quaint piece of mid-century nostalgia.

</comment><comment><USER12> I grew up near a beach town. I remember seeing Endless Summer when I was about 13/14\. From that point forward, I loved the idea of finding a "new break" somewhere around my small stretch of A1A. This guy really poured some passion into it. Thanks Bruce.

<comment><USER13> Similar. In my mind, I always pair this with "Morning of the Earth" another classic surfing movie.

[https://en.wikipedia.org/wiki/Morning_of_the_Earth](https://en.wikipedia.org/wiki/Morning_of_the_Earth)

</comment></comment><comment><USER14> The article mentioned how Brown rented out the Kip's Bay cinema in NY to try to get distribution. That is where I in fact saw it as a small boy with my sister and parents. I never surfed myself but relatives in California did and I always remembered the line about "the dirty Wedge" somewhere near LA. Fond memories... I'll have to see it again.

</comment><comment><USER15> Just watched this with my wife a couple of weeks ago. Made me nostalgic for the state of the world 50 years ago (before I was born...)

</comment><comment><USER16> [https://www.imdb.com/name/nm0113164](https://www.imdb.com/name/nm0113164)</comment>


Show HN: Pomodroido - a pomodoro timer with a side of game mechanics - <AUTHOR> http://mennoluiten.com/pomodroido-my-pomodoro-timer-with-a-side-of-g

<comment><USER2> At this point, I don't think "leveling" should be considered a game mechanic -- there are so many apps and things that I can "level" in that leveling has become... ubiquitous, I guess, and therefor uninteresting. Achievements are kind of getting the same way.

Game mechanics only work if they're interesting and motivating. When something becomes uninteresting, I don't think it should be considered a "game mechanic."</comment>


Code Scribbling with Fizz Buzz - <AUTHOR> http://hundredminutehack.blogspot.com/2016/01/code-scribbling-with-fizz-buzz.html <comment><USER2> Art for the sake of art.

I like doing things in an esoteric way. I wrote fizz Buzz weeks ago without out any literal digits nor any arithmetic operators, also, I outputted Roman numerals.

[https://news.ycombinator.com/item?id=10599801](https://news.ycombinator.com/item?id=10599801)

</comment><comment><USER3>
        for(i=1;i<101;i++)console.log((!(i%3)?'Fizz':'')+(!(i%5)?'Buzz':'')||i)</comment>


Vfmd: Markdown with a spec - <AUTHOR> http://www.vfmd.org/ <comment><USER2> Hm... Isn't that what CommonMark is for?</comment>


R Notebooks - <AUTHOR> https://blog.rstudio.org/2016/10/05/r-notebooks/ <comment><USER2> I was really hoping that this was better support for an R kernel for Jupyter.

Instead, R is reinventing that particular device.

Reinventing can be a good thing: but Jupyter is universal, not just Python. This is just Markdown + R, which, though cool, has been a problem heavily handled by someone else.

Jupyter in R could have solved this problem, and offered R up as more of a competing platform.

<comment><USER3> RStudio is a platform, and a company, focused exclusively on R programming – it dates back to 2009. That's several years before the launch of Project Jupyter, though IPython has been in development since 2001.

I'd love to see RStudio branch out and provide feature-for-feature Python functionality (at least), but I don't think that's likely to happen.</comment></comment>


List of Open Source Real-Time Operating Systems - <AUTHOR> https://www.osrtos.com/ <comment><USER2> For others who didn't know what a RTOS was:

[https://en.wikipedia.org/wiki/Real- time_operating_system](https://en.wikipedia.org/wiki/Real- time_operating_system)

</comment><comment><USER3> Love this page. After wanting to try something other than FreeRTOS, and having been frustrated with very public projects RIOT and Zephyr, I found TNEO from this page and have used it successfully in multiple gadgets. Still very happy with it.

<comment><USER4> Interesting, I am looking at Zephyr for BLE devices, so you say TNEO is a good option?

<comment><USER5> Comparing Zephyr and TNEO is like comparing apples and oranges. Zephyr is full fledged operating system with networking etc while TNEO is more like basic building blocks of os for those who want to implement everything themselves.

I can warmly recommend Zephyr for BLE usage. It is up to date with recent bt specs and implementation is really solid.

<comment><USER4> Thanks, I had no idea how many options there are

</comment></comment></comment></comment><comment><USER6> One of the listed ones, ChibiOS, is in the QMK firmware I use on my custom mechanical keyboards: [https://docs.qmk.fm/#/](https://docs.qmk.fm/#/)

</comment><comment><USER7> Can Linux itself be compiled/built as a realtime OS? I mean without adding additional software or hacking.

<comment><USER8> The short answer is NO.

The long answer is that there's patchsets like linux-rt which give it better "soft realtime" behaviour, but no hard realtime (deadline guarantees), thus not actual realtime.

Linux-rt allows "coexistence" with "hard realtime", but it's basically a cop- out: Just reserve a processor to run something else than Linux for hard realtime tasks.

In reality, while linux-rt patchset (not mainline) suffices for most pro audio work, Linux has over a million LoC of trusted code base running in supervisor mode; It should not go anywhere near where anything requiring any kind of assurance is running. This happens to be the case with tasks requiring hard (i.e. actual) realtime, as hard realtime is all about guarantees.

My advice for hard realtime would be to look at seL4 and eChronos, both open source as per OSI definition, and pick the one best suited to your design.

<comment><USER9> For some use cases, a Beaglebone black can allow for "real" real time while running Linux. It has two microcontrollers (PRU) that share memory with the main ARM cores. So if the real time part can be satisfied by the PRU, it's a neat solution.

<comment><USER8> Same idea as linux-rt "hard realtime": Reserve a processor (doesn't matter if it's the same chip or architecture or not).

The issue with this is precisely the shared memory. If Linux can interfere with the PRU, then there's no assurance. This breaks hard realtime: No assurance that the system will react timely in a certain way to a certain event.

<comment><USER9> It does work well enough to, for example, drive large LED display boards, emulate old video cards, processing audio, etc. Things that don't work well with traditional soft rt-linux. I'm sure, as you say, that it isn't good for all use cases.

<comment><USER8> Yes, it has its uses but ultimately it is not hard realtime.

Hard realtime is all about guaranteeing deadlines. If Linux (which is not and will never be high assurance due to its TCB size) can interfere with guaranteeing deadlines, then it's only soft realtime.

</comment></comment></comment></comment></comment></comment><comment><USER10> The first entry FreeRTOS shows MIT as license, but says it's distributed as GPL in the description. I don't know where they got the GPL part from: it's not mentioned anywhere at [https://www.freertos.org/a00114.html](https://www.freertos.org/a00114.html)

<comment><USER11> FreeRTOS used to be dual-licensed GPL (with an LGPL-esque linking exception) and commercial. Amazon acquired the copyright in 2017 and relicensed FreeRTOS under the MIT license.

</comment></comment><comment><USER12> Nice, I didn't know about a lot of these. We've used ChibiOS in a few of our products, it's really great.

</comment><comment><USER13> So many of the best are missing. Eg from the L4 family with widest adoption and best HW support only the smallest SeL4 is present, but not OKL4, L4Re, Fiasco, L4Ka::Pistachio, L4/MIPS, WrmOS, Genode.

</comment><comment><USER14> Isn't TempleOS RTOS?

<comment><USER15> It's too bad you're being downvoted. It's a legitimate question.

Looking at the source, a case could be made either way IMO. On one hand the scheduler doesn't have priorities, but on the other, you have to explicitly pin tasks to cores so you could actually guarantee real time behavior pretty easily on a fixed platform...

</comment></comment><comment><USER16> Is TI-RTOS really not in development? It's the OS TI gives you when you download their tools.

TI is weird, they have their own C compiler.

<comment><USER15> What was the licence? Did you have to run it on TI parts or something?

<comment><USER16> BSD license according to this: [http://www.ti.com/tool/TI-RTOS- MCU#descriptionArea](http://www.ti.com/tool/TI-RTOS-MCU#descriptionArea)

It does compile with gcc, but I don't trust it with gcc. At least with TIVA TM4C processors, it would not boot. I suspect they don't test with gcc as well as with their own compiler.

</comment></comment></comment><comment><USER17> I want to study OS dev. Linux seems to be pretty complicated. Which one from the list do you folks recommend to study?

</comment><comment><USER8> What's the criteria the list is sorted by? It seems completely arbitrary or unsorted from a glance.

</comment><comment><USER16> One way to deal with the many available operating systems is to use OSAL. It's an abstraction layer for embedded OS (and POSIX). Admittedly I have mixed feelings about it since it creates work when debugging in a new OS: you need to trace which OS call OSAL is using.

<comment><USER18> But the RTOS is already an abstraction layer around the hardware, how would adding another abstraction possibly help?

It's not like you often find yourself in the situation where you want to run the same application on multiple RTOSes, it's usually way too specific to the device.

<comment><USER19> > It's not like you often find yourself in the situation where you want to run > the same application on multiple RTOSes, it's usually way too specific to > the device

Perhaps that's the point: make your program more portable across different hardware that may only support specific OSes.

</comment></comment></comment><comment> gjsman-1000 Where is ThreadX? (ThreadX runs on the RPi GPU cores)

<comment><USER20> ThreadX isn't open source.

</comment></comment><comment><USER21> I'd be curious how many of these actually provide real time guarantees the way QNX or VxWorks does. I think quite a few of them may simply be "embedded OS" rather than RTOS.</comment>