Standing desk experiment and consequent knee injury - <AUTHOR> http://nazargargol.com/standing-experiment-and-consequent-knee-injury/ <comment><USER2> Summary: a guy with an undiscovered previous injury aggravated it with a lifestyle change, and now it's on the front page of HN scaring people away from living a healthier (less-sitting) lifestyle.

You have one unfortunate story, but I have the opposite positive story. As usual, it means that people are not all the same - and what works for one doesn't necessarily work for another.

What's your bodyweight? Are you carrying a bit more weight than is ideal? Did you have any previous injuries with your legs/knees? (Yes, we know that now.)

I suspect that you weren't snowboarding for a fraction as many hours as you were standing, so it's not really surprising that one activity hurt and one didn't.

I spent four years standing 10-16 hours a day - or rather, standing/shuffling/dancing. Not only did I not develop problems, but my lower back pains (from sitting) went away. And for the last few years I've been standing at work and doing pretty heavy weight lifting. I still feel great, and I'm in my 40s. I hope my story does as much to encourage people as yours does to discourage them.

<comment><USER3> I think if we take the mid-point between your story and his, we're getting there.

Everything in moderation.

That means sitting is fine and standing is fine, just not too much excess of either.

My wife, an ex-carer, knows a number of athletic people who have _very serious_ problems in their 50s and 60s as they've worn their hips and spine out. Conversely she knows a number of people with diabetes and heart problems in the same age range.

The interesting thing I did find out is that those who do not focus on any particular fitness goal, eat what they need and don't objectively go out and exercise are perfectly healthy and run rings around the other groups.

A word of caution: all of these people were quite happy with their physical state into their 40s. The same with drug use, particularly hash etc which really kills you when you get to your late 30s and early 40s.

A standing desk has little to do with any of the above. Moderation of all things does.

<comment><USER4> The problem is that modern life is not geared toward moderation. The way our economy is structured, most people have to spend their entire day working in order to make ends meet. And for a lot of people "work" means being tied to a computer for 8 or more hours. The human body is really not well suited to doing any one thing for longer than 8 hours at a time, but unfortunately our lifestyle almost requires it.

<comment><USER3> It most certainly doesn't require it. I think most people are just resigned to it.

</comment></comment></comment></comment><comment><USER5> This discussion topic, which never ends here on HN, of "the correct working position", is pretty silly when properly analogized to "the correct sex position".

Everyone knows there's only one position that feels the best and it should be done the same way for your entire life regardless of temporary or permanent health level or other conditions and the world would be a better place if all diversity were forcibly stomped out and the one true position was legally required for all. Because there can only be one correct answer. You know, just like programming editors. See? Now everyone is happy.

<comment><USER6> Not really. One is something that you (mostly?) do for fun, and for relatively short periods of time. The other is something that you do by necessity for the majority of the day nearly every day, which makes it easy to injure yourself in ways that won't be immediately apparent.

</comment></comment><comment><USER7> In my experience, typing while standing is really good, but watching Youtube videos (most of what programmer does) felt stupid/tiring.

</comment><comment><USER8> Humans are not designed to stand for 8 hours every day. It's about switching poses frequently.

<comment><USER2> Humans are not designed to sit in chairs.

<comment><USER9> Our fatty asses beg to differ.

<comment><USER2> Which came first, the chair or the fatty ass?

<comment><USER10> does it matter?

</comment></comment></comment></comment></comment><comment><USER11> There's a right way and a wrong way to go about something like this. You can't design a good setup before you even know how standing most of the day feels for you. Get a good adjustable stand, don't skimp on "accessories" like gel mats, ease into it, and above all _listen to your body_. If you're uncomfortable, sit for a while. If you're sitting too much, try standing differently next time - e.g. monitor/keyboard higher or lower, feet closer or further apart, shifting weight. Your ideal standing position/habit for work might not be the one you've been using in other contexts. Maybe standing for long periods, in a body shaped by long years of not standing longer than absolutely necessary, just isn't for you. I hope more people realize the benefits of standing more and sitting less, but even more than that I hope nobody hurts themselves trying.

</comment><comment><USER12> Standing in one place all day is the whole reason I never tried standing desks. It's painful as hell to stand all day. A treadmill desk might not be as bad, but my knees are generally unhappy whenever they absorb any kind of impact force. And sitting up/laying back isn't great because it basically atrophies your muscles from not being used at all for 8 hours a day.

It doesn't make much sense for us to work a set number of hours, anyway, since most tech jobs aren't repetitive labor, they're projects which can take anywhere between 15 minutes and two months. I'd rather just work less hours, have a more dynamic schedule, or just take an hour or two in the middle of the day to go climb a tree.

</comment><comment><USER13> My father-in-law works (and stands) on street market for past 40 years. He is in similar shape as other man in his age. So I think there is no miracle here, just some physical work and exercise.

<comment><USER14> I'll bet your father-in-law moves about a fair bit on his stall assuming the market has barrows or stalls. I think the issue might be simply _not moving_ much in whatever stance.

I'm a teacher and in term time in classes (6h a day usually) I'm not sitting much at all, but then I'm not standing in one place much either. No issues as yet...

</comment></comment><comment><USER15> Try standing on a wobble board[1] while using a standing desk. I don't know if it's any healthier, but it is more fun, feels more comfortable, and keeps you moving.

[1] [https://en.wikipedia.org/wiki/Balance_board#Wobble_boards](https://en.wikipedia.org/wiki/Balance_board#Wobble_boards)

</comment><comment><USER16> Sharing my experience here. I have had a standing desk for over 6 months. I built it myself and cannot adjust the height, but I am actually really happy about that. I have a high stool I can sit on whenever I want. I don't know how much I stand and how much I sit, I just sit whenever I feel like it but stand for most of the time. I think that's the key. Not having fixed rule but listen to how you feel and always have the possibility to sit. The fact that I don't have to adjust my desk makes me more flexible, I can sit for just 10 seconds sometimes. My legs felt heavy during the first few weeks, but now I feel great and it gave me a lot of energy. The only drawback is that I have such thick skin on my heels now that I lost sensitivity there. But heel sensitivity was not a feature I was really holding on to...

</comment><comment><USER17> It seems that I am not the only one gotten hurt by standing all day long.

In my case, the knees are fine, the joint between my right thigh and body got hurt baddly.

So, do not stand a lot, do not sit all day long either.

Do drink water regularly, (kidney stones for sitting programmers) too much water will cause kidney problems too.

<comment><USER18> Maybe lying on a couch is the safest choice then. :-)

<comment><USER19> That's why I've been waiting forever for a certain one-handed chorded keyboard called Twiddler. :) (I'm not affiliated with them).

</comment></comment></comment><comment><USER20> > At this point I decided it was a bad idea to stand all the time and switched > to 2 week standing/sitting cycles.

Well there's your problem. You should alternate every couple hours, not every couple _weeks_.

Buy a tall stool or drafting chair so you don't need to move your desk up and down.

</comment><comment><USER21> I've tried standing desks a few times and what I've found is that if your going to work more than regular hours it's a bad idea. That is you should not stand more than 8 hours a day for 5 days a week. And even then that should be punctuated by breaks such as lunch, meetings, etc.

The main difference I found say compared toa cashier is that you barely move. You don't twist side to side and so on. This might seem small but it's important.

The biggest issue though is that if you're doing a startup with long hours you need to be very careful. Like everything else overdoing anything is bad.

<comment><USER22> The recommendation I've seen from others before building my standing desk, and certainly the pattern I follow, is to alternative sitting and standing. My desk is set up for a standing situation, but I can undock my laptop and sit at any time. Some days I stand 65% of the time, some days I stand 10% of the time.

Mix it up, feel better.

<comment><USER23> [http://www.nextdesks.com/](http://www.nextdesks.com/)

</comment></comment></comment><comment><USER24> I've been downvoted for writing this before, but here goes: exercise for exercise's sake may not be all it is cracked up to be. For example, standing for long periods is associated with an increased risk of varicose veins.

[http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1740939/](http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1740939/)

I know many people believe all their physical ailments are traced to a sedentary lifestyle, but i don't think prolonged periods of abusing your body is the answer.

<comment><USER25> Standing is hardly exercise.

<comment><USER26> It's not as easy as sitting down though. Being on your feet for hours on end is definitely a skill that needs to be learned (in the military, for example).

<comment><USER25> OK, on that I agree. I bet this is not a skill the OP acquired.

</comment></comment></comment></comment><comment><USER27> One suggestion which I got from here on HN is to have some sort of small stool or object on which to rest one foot at a time while standing. It's much more comfortable than keeping equal weight on both feet and the reason why there's a brass rail on the floor in bars.

I use my desk in sitting mode most of the time. Of course, I've been breaking up my working day with a 45 minute run in the middle, but it's still great to have the option sometimes to stand.

</comment><comment><USER28> I hope that your knee heals.

I'm gently curious about the people trying standing desks: this article mentions research on the dangers of sitting, but it doesn't mention any research on standing. Since many people are expected to stand all day at work I'd have thought there would be some research about standing.

Have people looked at that? How about investigating some of the products meant to counteract problems caused by standing all day (such as anti-fatigue matting).

<comment><AUTHOR> Standing whole day is also not a good thing, that is why I was switching between the two all the time. Standing position is giving to much stress on the keens and develops things like atherosclerosis, hip arthritis, varicose veins...

</comment></comment><comment><USER29> I'm pretty sure chairs were invented for a reason.

<comment><USER30> So were cigarettes (which were even touted as a healthy thing for a period of time, and later found to be harmful). The point being: lots of thing are invented for a reason - but is that reason valid? Does it lead to secondary effects that outweigh the benefits?

<comment><USER29> Are you really comparing chairs to cigarettes? I'm not really sure that supports your point.

Snark aside, it seems to me that sitting down a lot is perhaps a contributing part of a lifestyle that probably isn't very healthy (not getting much exercise, not seeing much sunlight, etc). But it seems far from conclusive that replacing sitting down with standing up is really much of a solution for many people.

</comment></comment></comment><comment><USER31> I bought my standing desk this spring. I started out using it almost all the time, but I was having similar problems after a long time standing.

Now, I use it in sitting mode most of the time, but use it in standing mode for conference calls. That works out to a nice balance during the day. It's one button press and 15 seconds or so to adjust from sitting to standing and back, so it's easy to switch.

</comment><comment><USER32> I enjoy my sitting time and make up for it with a regular exercise regimen and lots of after work activities. At one big corp. I have been at, it was like a competition to see who could stand the longest and who appeared the most health conscious.

</comment><comment><USER33> Has anyone looked into some sort of walking desk? Over a treadmill? It seems like that would alleviate the problems with standing.

<comment><USER34> Like the one that Linus Torvalds use? [https://www.youtube.com/watch?v=HSgUPqygAww](https://www.youtube.com/watch?v=HSgUPqygAww)</comment></comment>


Facebook is Great for Dead People - <AUTHOR> http://www.craigormiston.com/post/5606512383/facebook-is-great-for-dead-people

<comment><USER2> I work for a Dutch social network. We introduced In Memoriam status for member who passed away. The family members can ask for it. We will then show their date of death on their profile and won't send out birthday notifications and things like that anymore.

Here is an article about it: [http://thenextweb.com/eu/2011/03/23/dutch- social-network-hyv...](http://thenextweb.com/eu/2011/03/23/dutch-social- network-hyves-adds-in-memoriam-status-for-deceased-users/)

<comment><AUTHOR> I think this is a great idea! Facebook could use this functionality.

You can declare certain people on Facebook "family members." Perhaps you could assign those people certain permissions to control your profile in specific use cases, like death.

</comment></comment><comment><USER3> This seems like it would be a bad thing for 1000memories. They need to show why they are a better platform for memories than Facebook. I can definitely see advantages to them- seems a bit classier to me at least. But why they are better than Facebook for this purpose should probably be the focal point of their marketing.

<comment><USER4> I'm a co-founder of 1000memories. One of the motivations for starting the site was a bad personal experience on Facebook after losing a friend. My friend's page got memorialized after a week, and all of his wall content was hidden. In response, three group pages were then created, and the experience became fragmented and disjointed.

I don't think of using FB as a memorial as necessarily bad or good; the truth it's somewhere in between and varies by individual. The problem for FB stems from the fact that it was never designed for this behavior. A vast majority of deaths on FB are never reported, and those profiles stay in the viral loop. Those that are reported put FB in the awkward position of being referee of the deceased's identity.

Ultimately, I was dissatisfied with allowing FB determine how we, as friends and family, were going to remember my friend's life. I don't like that his jokey profile photo is now stuck there forever. I think we can do better. But beyond FB, the reason I'm excited about building a platform for memories is that the Internet in general does a bad job speaking in the past tense. We are focused on providing tools to help people tell and preserve the stories of their loved one's lives.

<comment><AUTHOR> Rudy,

I think what your team is doing is very noble. You are absolutely correct, Facebook was not tailored to this kind of activity. We need a real platform to memorialize others and - more importantly to the average user - a place where we can memorialize OURSELVES. Mortality is a big issue and we all want to be preserved for eternity in a positive light. Facebook does not give us control over this postmortem public image.

As others have pointed out, 1000memories sits outside of mainstream networks. It asks users to go out of their way to memorialize and interact with the deceased on a platform beyond their daily traffic. Perhaps not a barrier to entry issue, my guess is that 1000memories has a retention issue. How do you keep people coming back after their first visit?

Is there a way 1000memories could better tie in social media and their networks? An obviously example would be to pull a user's Twitter feed into the mix of memories. Family and friends could rank the most memorable tweets and promote certain aspects of the deceased's published life. The same concept applies to blogs, Flickr photos, and Facebook updates.

To win social media integration, the deceased party would need to give 1000memories permission to connect with social media accounts pre-passing and regulate the content that can be shared. 1000memories could market to the living to set up an account that "gives you control of the memories you leave behind." A "will" for postmortem digital image.

I would use a service that offered me the ability to control my image postmortem.

<comment><USER4> Agreed, it gets really interesting when we bring it around to ourselves. It's difficult to import digital property without expressed permission, but I think the space will evolve a lot over the next few years and create a lot of opportunities for story-telling and narrative.

</comment></comment></comment></comment><comment><USER5> My exgirlfriend passed away a year and half ago. My last words to her a year before she died were cruel and ugly and consequently I didn't hear from her family about her death, but learned of it from an online obituary of a small town paper.

I was not friends with her on facebook (our relationship predated it), and so I cannot tease out our memories there. But the loss remains and I find myself returning to her obituary every few weeks.

</comment><comment><USER6> The tone of this article is pretty strange. Is Facebook really a great thing for dead people? Or is it simply ironic to have dead people's profiles being more active than living people's?

Anyway, I agree that Facebook shouldn't suspended dead people's profiles (and they can't technically). Social networking account is a part of one's life, just like a diary, a photo, or anything memorial. It shouldn't be cleared away.

<comment><USER7> Don't want to be spammy here but we recently asked quite a few folks "what do you want to have happen to your on-line accounts when you pass away" and there was a split for many of them. Some people want their accounts suspended/deleted when they pass, some want them left alone, some want them passed on to their families.

[http://www.blog.lifeensured.com/account-handling-after- death...](http://www.blog.lifeensured.com/account-handling-after-death-survey- results)

<comment><AUTHOR> Mike - very transparent and noble of you to post this survey! Thanks.

</comment></comment></comment><comment><USER8> >In a world that hardly prays anymore, Facebook may be the next best thing.

That's a scary thought.

<comment><AUTHOR> I agree, a very scary thought indeed.

I'm sure there's website out there where you can write prayers out and publish them into the abyss. But when it comes to speaking to ancestors or the deceased, why not?

</comment></comment><comment><USER9> Did you notice the image of the gravestone? The inscription has a (not so) subliminal message.

<comment><AUTHOR> Clever observation, though I'm not sure I intended the subliminal message. I have mixed feelings about Facebook's longterm value and scope, but I cannot deny how powerful this network is. Read more of my thoughts on it here: [http://www.craigormiston.com/post/4910698242/will- facebook-r...](http://www.craigormiston.com/post/4910698242/will-facebook- rule-the-future-of-social)

<comment><USER9> The "subliminal message" remark was somewhat tongue-in-cheek. Calling the inscription a Freudian slip might have been more accurate...

</comment></comment></comment><comment><USER10> Interestingly Facebook will let relatives 'memorialise' a persons profile after they are deceased. It's not well known, but I wonder if the features it removes undermine the point this article is making.

<comment><AUTHOR> I am not aware of this feature. Do you have a link or any info to share on it?

<comment><USER11> Help Center > Privacy > Privacy: Deactivating, Deleting, and Memorializing Accounts <https://www.facebook.com/help/?page=842>

</comment></comment></comment><comment><USER12> I find the whole affair disgusting. The internet is not the right medium for every aspect of your life.

<comment><AUTHOR> I agree, the internet is not the right medium for every aspect. But I do think it has a certain level of accessibility and mobility real life cannot offer.

My grandfather's gravestone is in Pueblo, Colorado a thousand miles away in a town I have few reasons to visit. If I had a digital library of memories, I could connect with him more regularly than every 15 years. But he was pre- internet and I have no access to these things. I am stuck with memories in my head, which fade in time (one of the tribulations of being human).

</comment></comment><comment><USER13> True, you can't care about privacy when you're dead!

<comment><AUTHOR> But you can certainly care about it BEFORE you are dead :)

<comment><USER13> my point exactly!

</comment></comment></comment><comment><USER14> I noticed that there isn't an app where you can make your "digital will" i.e. describe how you want your profile to look once you 've kicked the bucket. Isn't there a market for that? Also, dead people's profiles may be ok for the first months, but after a long time it becomes kind of creepy seeing people long dead. Do they keep the profiles forever?

<comment><USER7> Hi, I'm the founder of LifeEnsured. We're that app. We let you do define what you would like to have happen to your on-line accounts after you pass. <http://www.lifeensured.com>.

<comment><USER14> Love your "social security death detection" feature. Hope it doesn't have any false positives

<comment><USER7> Thanks. We hope so too. End of life validation is a tricky problem we think we've become good at solving by combining several validations. [http://www.blog.lifeensured.com/end-of-life-validation-on- li...](http://www.blog.lifeensured.com/end-of-life-validation-on-line-a- tricky-problem)</comment></comment></comment></comment>


New Arduino released: the 32bit Due - <AUTHOR> http://www.wired.com/design/2012/10/arduino-due/ <comment><USER2> I think the G120 and/or G120HDR boards are so much better. Much faster, a ton more memory, full tcp stack and you get to program in a high level language: C#. This also means that there are a ton of libraries, should you need any. And the price is the same.

<https://www.ghielectronics.com/catalog/product/373> <https://www.ghielectronics.com/catalog/product/388>

<comment><USER3> ...but can you then rebuild the board on your own PCB so that each project doesn't use up the expense of a whole G120?

<comment><USER4> Yes, but there are some annoying licensing issues, especially with GHI's proprietary libraries (for microchips wifi module, for example).

</comment></comment></comment><comment><USER5> 12-bit ADCs! Woohoo!

If they ever reach 16-bits (or beyond!), our lab may buy them weekly. The 10-bit resolution of the Arduino ADC is the biggest single thing that's kept us from switching to them for general-purpose DAQ.

The speed with which Physics undergrads can pick up the Arduino environment trumps anything else we use. Yes, we can implement I2C/SPI communication to another ADC chip, but the development environment isn't nearly as stable (chips and vendor-specific serial protocols change). The knowledge that the Arduino codebase should remain moderately stable over time makes it worth our trouble to develop for it.

Thank you, Arduino-folk!

<comment><USER6> The PSoC chip in the freeSoC board (<http://freesoc.net/>) can do upto 20-bits but the IDE isn't as simple to use as Arduino's.

</comment></comment><comment><USER7> Interesting, I tried to read the article and my browser (Chrome V22) went nuts. I don't know what sort of JS wired is pushing these days but from this side of the screen it made their site unusable. Sad really. So I went to the <http://arduino.cc> site ('official Arduino site') and it has no news about a 32 bit Arduino. Ther is an article on Hackaday though : [http://hackaday.com/2012/10/03/finally-an-arm-powered- arduin...](http://hackaday.com/2012/10/03/finally-an-arm-powered-arduino/)

Having recently read about the Sony Nexus fake I find myself even more distrustful of tech journalists.

From the hackaday article it looks like it could be an improvement on the original concept.

<comment><USER8> The arduino.cc link is: <http://arduino.cc/en/Main/ArduinoDUE>

It says the page will be available on Monday, when you can buy it.

</comment></comment><comment><USER9> Two DACs. Brilliant!

</comment><comment><USER10> Looks like a lot of cool stuff on-board..anyone know if the Due will be compatible with older shields?

<comment><USER11> from the article: The Due will continue to work with all Arduino shields [..] that conform to the official Arduino Revision 3 layout. However, the Due operates at 3.3V whereas AVR-based Arduinos operate at 5V, meaning some third- party shields that don't follow the R3 specs to the letter may not be compatible [..]

</comment></comment><comment><USER12> So it does in 2012 what the Parallax Propeller did in 2006. Will the wonders ever cease?

<comment><USER13> The value in the Arduino is not necessarily in the actual processor, but in the tools, documentation, tutorials and library supplied with the boards. And of course in its large community of users and addons ('shields').

<comment><USER3> Plus the whole "open source hardware" thing...

</comment></comment></comment><comment><USER14> How does this compare to the Teensy 3.0?</comment>


When You Fall in Love, This Is What Facebook Sees (2014) - <AUTHOR> http://www.theatlantic.com/technology/archive/2014/02/when-you-fall-in-love-this-is-what-facebook-sees/283865/ <comment><USER3> When I see posts like these, they creep me out. Another reminder that Facebook and all other online services, ad companies know way too much about their users and can deduce more information about you (in truly creative ways) than you imagined would be possible.

When I show these posts to people who resemble the average internet user, they just don't care. This is something that is saddening and baffling for me. I think it's just that most people don't understand why this is an issue to begin with and people need to be made aware of what is going on. The easiest way to start would be to discuss privacy issues with your closest family and friends.

Worth mentioning RMS' blog post: [https://stallman.org/facebook.html](https://stallman.org/facebook.html)

<comment><USER4> Why should I care? For most of human history, most people lived in villages. Everyone they met would know this kind of information about them ("it's obvious to everyone except the couple themselves that x and y are into each other" is a literary trope for a reason). People survived, thrived even - <USER2> fact there's a strong argument our psychological welfare was much better in those conditions than it is in anonymous cities.

<comment><USER5> > Everyone they met would know this kind of information about them

Everyone in the village would have had detailed spending habits, knowledge of every website browsed, media preferences, political views, porn habits, location history and more all backed up in giant databases that are easily search-able?

<comment><USER6> Spending Habits: Yes - you could easily observe what your village neighbors are buying.

Websites Browsed/Media Prefs/Porn Habits: No, these things didnt exist for most of human history. I guess "Media Preferences" would be known if we're talking books vs. newspaper.

Political views: Yes, assuming the person isn't a hermit.

Location History: Yes, presumably you'd know where village-mates are/have been.

Backed up in a database that's easily searchable? I guess that depends on how good your brain is at remembering things.

Not trying to nitpick too hard, but all your viable examples listed are in fact things your fellow villagers would know about you in a close-knit community.

<comment><USER5> I'm going to _highly_ disagree with you.

Yeah, you might have a rough idea of spending habits for a handful of people, but not everyone, and not to the same detail.

Books/newspapers/etc sure, you'd know some of it, but you wouldn't have a record of what they consumed in their home unless they told you.

You might know some of their political views, but only the ones they share openly.

If a co-villager wandered off for a week, you wouldn't likely have any idea where they went. You'd have to rely on what they told you, and where others said they went, which isn't necessarily accurate. Additionally, you wouldn't get the level of detail FB can.

No brain is capable of storing even half of the location that FB's DB can. I don't care how good someone's memory is.

They would know some of the above sure, but it'd be at a high level, and not nearly as accurate.

<comment><USER7> You make some good points, but i have to slightly disagree with some of them :)

"Yeah, you might have a rough idea of spending habits for a handful of people, but not everyone, and not to the same detail."

The shopkeeper does. And as someone who grew up in a small town, the gossip is real.

"ou might know some of their political views, but only the ones they share openly"

Facebook doesnt know my political views? Though I dont openly share them on fb. More to avoid flamewars.

"If a co-villager wandered off for a week, you wouldn't likely have any idea where they went"

Depends how well you know them. You might know they like that spot by the lake for example. And if I leave my phone at home, no one would know where I went.(which sounds like a good reason to carry my phone, safety)

</comment></comment></comment></comment></comment></comment><comment><USER8> Just to be pedantic:

Each of the data points is an average of many users (and potentially have huge errorbars they do not show). Therefore for single individuals, this result may not be very predictive.

In fact, given that the distribution of number of posts likely follows power law, the average is hugely influenced by a small number of heavy facebook users. I would say this figure summaries more about top facebook users rather than the majority of users. So when "you" (an average facebook user) fall in love, this is probably not what facebook sees.

<comment><USER9> I thought the same thing (and wrote a bit about it: [http://www.win- vector.com/blog/2014/02/the-gap-between-data-...](http://www.win- vector.com/blog/2014/02/the-gap-between-data-mining-and-predictive-models/) ).

</comment></comment><comment><USER10> This is from 2014. I'm not sure whether that is a selection bias, but very few of my friends "share timeline posts" with their partners (or future partners, or anyone really) nowadays.

<comment><USER11> I think they just worded it poorly, but what I think they mean is just any posts that involve both people. So this would include photos where both people are tagged, mentioning the other person, posting on the other person's wall, and things like that.

</comment></comment><comment><USER12> Somewhat related, my son showed me a program from Britain called "Black Mirror" ([https://en.wikipedia.org/wiki/Black_Mirror](https://en.wikipedia.org/wiki/Black_Mirror)) The shows are an hour or so long, and it's not a series with a long-running plot, but each episode is separate (à la _The Twilight Zone_ ). We watched one called "Nosedive" ([https://en.wikipedia.org/wiki/Nosedive](https://en.wikipedia.org/wiki/Nosedive)), which was set in a future dystopia where everybody was socially rated and that rating was visible to others. Remember the almost-startup from last year, Peeple.com? Yikes!

</comment><comment><USER13> The graphics are misleading, the Y axis is stretched to make the change look big.

<comment><USER14> If it's significant then what's wrong with using a scale that reflects that, even if the numbers are small in absolute terms?

<comment><USER15> In this case at least, they're going from a peak of ~1.66 posts/day to a valley of ~1.54. As an aggregate that looks like a compelling pattern, but consider it on an individual level: we're talking about one post every ten days. Initially looking at the graph made me think the difference was _way_ greater than a post every ten days, so I'd agree with the GP that it is misleading; I was mislead.

<comment><USER14> Yeah, but if one post every ten days really is predictive, then showing it at a larger scale isn't helpful.

<comment><USER15> Looking at the graph, it seems like we're talking about ~16.5 posts in the ten days before a relationship versions versus ~15.7 in the ten days after. In the context of an individual, Facebook might be able to combine this with a bunch of other data to make guesses about people entering relationships, but a quick look at the graph makes it look like they could do it from just this one piece of data.

FWIW, I think people are often too quick to dismiss graphs as misleading due to the y-scale and I do think the graph is interesting. That said, I think a lot depends on the context, and being directly below "When _You_ [emphasis added] Fall in Love, This Is What Facebook Sees" makes this seem misleading.

</comment></comment></comment></comment></comment><comment><USER16> So facebook has an incentive to keep you out of a relationship, it seems.

<comment><USER17> Lonely people tend to hang our more on social media, I guess. Lonely people are also better consumers.

<comment><USER18> Imagine the firms that would fail if all the pretext and games of dating and sex went away. How many less exotic and luxury cars would be sold? Designer clothes, body spray, make-up, hunks of carbon attached to gold bands? Culture would be dramatically different.

<comment><USER5> I've been wondering what our society will look like once androids look and act comparable to humans. The prevalence of online porn has likely changed social norms drastically in ways I can't imagine, but I'd guess realistic androids even more so.

</comment></comment></comment></comment><comment><USER19> Courtship over Facebook is so unromantic. In my day, we used email, SMS and phone.

<comment><USER20> Don't forget AIM!

</comment></comment><comment><USER21> For reference, previously on HN 3 years ago: [https://news.ycombinator.com/item?id=7250009](https://news.ycombinator.com/item?id=7250009)

</comment><comment><USER22> "We only considered couples who declared an anniversary date (as opposed to just changing their relationship status)"

Seems like a strange choice; most dating couples I know don't seem to set an anniversary date.

<comment><USER23> Many of my facebook friends (n~400, ~19 years old) are in a 'relationship' with their friends as a joke. Enough to skew any results.

<comment><USER24> I'm sure FB can distinguish between real and joke relationships with high accuracy.

</comment></comment></comment><comment><USER25> Here's the full series run by Facebook (2014):

[https://www.facebook.com/data/posts/10152217010993415](https://www.facebook.com/data/posts/10152217010993415)

</comment><comment><USER26> What about private messages? I mean we know Facebook tracks those too. Do they just not want to admit as much publicly?

<comment><USER27> They admit looking at those as well, in the original post they posted on Facebook[1]:

"Relationships start with a period of courtship: on Facebook, messages are exchanged, profiles are visited, posts are shared on each other's timelines."

They didn't produce any graphs of the amount of messages exchanged though.

[1] [https://www.facebook.com/notes/facebook-data-science/the- for...](https://www.facebook.com/notes/facebook-data-science/the-formation-of- love/10152064609253859)

</comment></comment><comment><USER28> I think I'd like to see a law that requires all companies to share all information they have on you in an easily understandable format. I think that'd be a great start.

Any saved derived information, like what kind of person they think you are, or what they think you'd be likely to buy, must also be shared.

Basically a dump of all their databases under your user ID.

</comment><comment><USER29> Original link should have been to Facebook's blog, [https://www.facebook.com/notes/facebook-data-science/the- for...](https://www.facebook.com/notes/facebook-data-science/the-formation-of- love/10152064609253859)

</comment><comment><USER30> So facebook, which needs us to spend as much time on the website as possible to maintain advertising revenue, has the perverse incentive of keeping everyone single. So this could be the answer to the Fermi Paradox

<comment><USER11> You seem to be mis-interpreting.. or inferring data that is not there.

All this is showing is that prior to being in a relationship there is a lot of online activity between two people, and less after being in a relationship. Presumably because prior to being in a relationship, you are spending more time communicating at a distance, and when in a relationship you tend to spend more of your time physically together rather than communicating online.

This data says nothing about how much these people are using the website in general.

</comment></comment><comment><USER31> 1.61-1.65 down to 1.4 something? What is the actual average amount of posts made a day, shared to the timeline?

small scale Y axis make it look quite large, but that doesn't seem a seismic shift.

</comment><comment><USER32> Given the dubious statistical significance and the fact that predicting who's "courting" each other is really pretty easy, this article seems like a "the suit is back" adverticle for Facebook, touting its predictive abilities to advertisers like Jared, Netflix, Bed Bath & Beyond etc.

</comment><comment><USER33> I'm curious to know if this type of data could be used to track infidelity. I'm thinking further than domestic trouble and more towards blackmail, espionage, etc. I can see how this type of information could be useful for the wrong reasons.

</comment><comment><USER34> I'm surprised Facebook hasn't specifically monetized this type of behavior. It seems to me that this is a very clear signal of intent and can be monetized.

</comment><comment><USER35> It sounds like Facebook has a vested interest in keeping people single and to break up existing couples.

I do think that relationships these days are more fleeting than they used to be. I think that Facebook (Instagram, particularly) tends to encourage this type of behaviour.

Constantly being exposed to other couples' luxurious lifestyles and holiday trips is not particularly good for relationships (particularly new relationships).

I do think that most people these days tend to value earning potential and physical appearance far more than personality when it comes to finding a match and I think that social media is probably responsible for much of this.

<comment><USER36> > So basically, Facebook has a vested interest in keeping people single and to > break up existing couples.

Not really. Married couples (particularly those with a baby) are incredibly valuable advertising targets.

> I do think that most people these days tend to value earning potential and > physical appearance far more than personality

There is absolutely nothing new about this.

<comment><USER35> >> There is absolutely nothing new about this.

Not new, but it's getting worse.

<comment><USER37> I care about earning potential and physical appearance. Why shouldn't I? I also happen to care about many other things in a potential relationship but those things you mentioned are definitely high up there for me.

Who are you to tell me what value system I should have? All of this is independent of social media, by the way.

</comment></comment></comment></comment><comment><USER38> (2014)

</comment><comment><USER39> Three full months of courtship? Really?

</comment><comment><USER40> Pardon the crassness but this statistic as well as the study behind is is utterly useless.</comment>


jq - like sed for JSON data - <AUTHOR> http://stedolan.github.io/jq/ <comment><USER2> Considering the amount of json around, wouldn't it be time to settle on a standard for :

\- schema definition ( using something like typescript ? )

\- data path ( similar to xpath )

\- data transformation ( similar to xslt )

i'm pretty sure many people have done their things but i'm still waiting to see a tool like wsdl - based stub generators that would work on many json api.

ps : i know that things like jsonpath and wsdl2 exist, but i really feel like those technologies haven't caught widespread adoption.

<comment><USER3> if you need all these , you should use XML. Crockford said he is not interested in any JSON standard extension, at all. not even to put comments in it. That's what XML is for.

Dont use JSON if you should use XML.

<comment><USER4> Crockford's refusal to extend JSON is genius IMO. Totally agree with you: if the task you're trying to accomplish sounds like a fit for XML, then please use XML. JSON is what it is, and not everything needs to be gradually extended into six mutually incompatible versions of increasing bloat.

[http://geekandpoke.typepad.com/geekandpoke/2012/03/thank- god...](http://geekandpoke.typepad.com/geekandpoke/2012/03/thank-god-not- everything-is-software.html)

</comment></comment></comment><comment><USER5> I've been using jq for a few months now while working with big GeoJSON blobs and it's a big help. The query language takes a bit of doing at first, but it's really pretty great once you learn it.

curl -s [http://tile.openstreetmap.us/vectiles- skeletron/12/656/1582....](http://tile.openstreetmap.us/vectiles- skeletron/12/656/1582.json) | jq '.features[].properties.name'

<comment><USER6> Personally jgrep seems a better choice here

curl foo/bar.json | jgrep -n -s 'property'

Or even better

curl foo/bar.json | jgrep 'status=500' -n -s 'path'

<http://jgrep.org/>

</comment></comment><comment><USER7> For readers who are unfamiliar with sed: don't dimiss jq as yet another json pretty printer. From the jq tutorial "Data in jq is represented as streams of JSON values - every jq expression runs for each value in its input stream, and can produce any number of values to its output stream.".

I've played around with jq and it will definitely become a tool in my toolbox. Kudos to the author.

</comment><comment><USER8> Also available is the brilliant underscore-cli[1], which brings the functional programming of _.js to the command line.

There's an examples page, if you're struggling to see how it might be useful[2], but I find myself using 'extract', 'find', and 'pluck' on a daily basis.

[1]: <https://github.com/ddopson/underscore-cli>

[2]: [https://github.com/ddopson/underscore- cli/blob/master/Exampl...](https://github.com/ddopson/underscore- cli/blob/master/Examples.md)

</comment><comment><USER9> recs has this functionality, and _way_ more; worth giving a shot: <https://github.com/benbernard/RecordStream>

<comment><USER10> I'm a big fan of RecordStream. Everyone should definitely give it a look.

However, is it true to say that recs has this functionality? RecordStream is great with sequences of relatively simple JSON structures, where you want to count, group, collate, etc. the sequence. It looks like jq is better able to handle large, complex arbitrary JSON objects, pulling out deeply nested fields and transforming them. For example, how would you implement jq's "recurse" in RecordStream?

Now what would be really cool is using the two tools together :-)

</comment></comment><comment> guard-of-terra And for xml I highly recommend xmlstarlet.

query/transform xml documents from the command line.

</comment><comment> iso8859-1 Previously on HN (141 days ago): <https://news.ycombinator.com/item?id=4985250>

<comment><USER11> Clearly, this is new to enough people that it got to the front page. Merely linking to a previous submission adds nothing here; if you thing the discussion there was informative (clearly not the case here), then say so.

<comment> iso8859-1 Yeah, I think the previous discussion was informative, it has alternatives and links to ever older relevant posts. Let the karma decide what HN likes. Like you said yourself: <https://news.ycombinator.com/item?id=5664116>

</comment></comment></comment><comment><USER12> I like the concept, but how does it compare to using node + underscore?

Isn't the whole idea of JSON that it exactly serializes most JavaScript objects, and the "right way" to do stuff with it is to just use it like a normal object in your program?

<comment><USER13> This assumes you're using JSON in Javascript. JSON is a pretty widely spread human-readable data format now outside of javascript, and it's nice to have good tools that aren't tied to a particular language.

</comment></comment><comment><USER14> Some time ago I made this: <https://github.com/fabriceleal/Jsoned> using node.js.

It allows you to use Javascript expressions using a map/filter/reduce paradigm to edit the Json stream, like this:

echo "[1,2,3]" | jsoned -m "' _ * 2 '"

which will print "[2,4,6]"

you can even put the javascript code in files.

Check the wiki (incomplete) for documentation: <https://github.com/fabriceleal/Jsoned/wiki/Usage>

</comment><comment><USER15> For a more unix-y version of this, check out Jshon.

<http://kmkeen.com/jshon/>

</comment><comment><USER16> Completely unrelated, but I love the typeface he's used on his site. In fact the whole site is laid out really nicely (pretty, yet none of the aesthetics gets in the way of usability. a balance that most sites fail to strike).

Anyhow, tangents aside, this looks a really handy tool. Thanks :)

</comment><comment><USER17> This is super cool, the language seems both powerful and easy enough to learn. Nicely done.</comment>


Show HN: Peek'n'Poke - <AUTHOR>

Peek'n'Poke is a simple way to ask for feedback on unpublished text content (blog posts, essays, etc).
I made it over the weekend as a way to make it easy to improve with my writing/grammar.
All modifications that people make on texts are aggregated in the interface visible in a simple diff like interface.
http://www.peeknpoke.com <comment><USER2> The color scheme is really good, and design layout is very fluid and clean. Simple to use and simple to understand.

I would suggest allowing people to send invites through Facebook or twitter so friends and followers can review work as well.

Good job though!

<comment><AUTHOR> Thanks, means a lot.

I'll get FB/Twitter up and running by tomorrow.

<comment><USER3> Great design. What did you use for the front end?

</comment></comment></comment><comment><USER4> Have you got a sample of some modified text we can look at?

Do you have to be the original author to see the modifications?

<comment><AUTHOR> Editing: <http://www.peeknpoke.com/look_over/d8acd97d9e6dd30545effe7d>

To see the result login with: email@example.com / ihearthn

As it's implemented right now you have to be the owner.

<comment><USER4> Thanks for making me laugh with the writing sample.

This is a polished piece of work. The transition between each revision is nicely handled, the red/green colouring for the diff view is well though out. You should really be proud of this.

</comment></comment></comment><comment><USER5> Clickity: <http://www.peeknpoke.com></comment>


HN Tools: Search HN user tweets/trends with Sparse.ly - <AUTHOR> http://sparse.ly/list/buzzlists/hacker-news

<comment><USER2> The home page is confusing for people who don't know what it is.

What is it? What does it do? I realize it searches Twitter, but what does it search for?

<comment><AUTHOR> The site lets you search twitter lists or the people you follow on twitter, rather than all of twitter.

see the homepage <http://sparse.ly></comment></comment>


Instagram building standalone shopping app - <AUTHOR> https://www.theverge.com/2018/9/4/17819766/instagram-shopping-app-e-commerce <comment><USER2> So now that Amazon is a bigger threat in the ads space, Facebook enters e-commerce. It's going to be an uphill battle if they want to catch up anytime soon.

<comment><AUTHOR> Amazon does not create intent well. I go to Amazon when I want to buy a particular item. Instagram's shopping app may be able to create intent.</comment></comment>


Tesla Q4 2018 Earnings Letter - <AUTHOR> http://ir.tesla.com/static-files/0b913415-467d-4c0d-be4c-9225c2cb0ae0 <comment><USER2> Pre:

\- Will they give guidance on Model 3, Geographic breakdown(most other peers do)?

\- how many sales are high end cars vs low end?

\- TSLA down 11% on year vs market up 5ish

\- convertable debt issue, we're now in the 20 day average window, option implied vols show market doesn't think they'll make it

\- Elon commented that they could pay bond w cash and didn't seem concerned, cash balance is up, though so are short term liabilities

Car Notes:

\- current lowest price car sold is Model 3 $44,000

\- reported to sustain 7,000 per week, a pretty big miss from the 10,000 that elon claimed they'd be at by mid 2018 but expected

\- Est. 2019 delivered 360,000 to 400,000

\- Est. Q1 delivers expected to be down vs 2018

\- Est. Q1 deliveries expected to be down in North America

\- still no funding secured for China plant

Solar:

\- why even track anymore, installations fell, yawn

Numbers:

\- cash balance is 3.7B

\- cash flow positive again!!, beat last quarter, though capital spending fell alot( down to $300ish million)

\- 4Q Adj EPS $1.93, Est. $2.14 (Big misses, what happend here?)

\- layoffs could cut $400M annually from budget

\- Customer Deposits $792.6M vs $908.5M in 3Q

\- Bonds unmoved, yield still 8%

\- Elon expects to be profitable each quarter in 2019

Misc:

Lots of analysts congratualting Tesla saying they've got the supply side sovled.

Shrinking sales on the high end indicate taht their next challenge will be on the demand side, especially on the high end that allowed them to survive for so long

Can they be profitable with $1000 in margin per car instead of 5,000-10,000?

Interesting note by Bloomberg

> Services reported a loss of almost $500M

"So basically, Tesla is taking a $500 million hit just to save customers from the dealership experience."

<comment><USER3> For me the next couple of years are going to make or break Tesla.

You have pretty much every car manufacturer releasing flagship electric cars in 2019/2020\. And most of them are either cheaper than Tesla or offer far more value i.e. premium branding and better interiors. And whilst they have their Supercharger network I suspect that pretty soon we will governments intervening to regulate standards and build out infrastructure.

So they better get ready quickly before the onslaught comes and like you said be profitable if margins drop.

<comment><USER4> > For me the next couple of years are going to make or break Tesla.

Maybe, but haven't people been saying that about Tesla for 15 years now?

<comment><USER5> That's what happens when you aggressively scale a business. They could have throttled back and been a profitable niche manufacturer with the Roadster, and again (albeit a bigger niche) with the Model S, and again now with the Model 3. But that's not their goal, so they're continuing to expand as fast as possible, and that carries risks.

</comment></comment></comment></comment><comment><USER6> These numbers are unreal. From 100 cars in 2011 to 140,000 Model 3s (+100K Model X and S) in 2019. And all electric. I personally am not interested in minor changes in financials, and want to just sit back and marvel the machine that Tesla and Elon Musk has created. Kudos to everyone at Tesla.

<comment><USER7>
      $TSLA annual revenue (billions)
      2018: $21.46
      2017: $11.76
      2016: $7.00
      2015: $4.05
      2014: $3.20
      2013: $2.01
      2012: $0.41
      2011: $0.20
      2010: $0.12
      2009: $0.11
      2008: $0.01

      CAGR:
      2 yr: 75%
      5 yr: 60%
      10 yr: 107%

source: [https://twitter.com/TeslaPodcast/status/1090719629922746374](https://twitter.com/TeslaPodcast/status/1090719629922746374)

It's interesting to overlay Tesla 2014 - 2019 with Apple 2002 - 2007. Now what Apple did from there will be tough to match (not to mention the margins): [https://ark-invest.com/research/tesla-through-the-lens-of-ap...](https://ark- invest.com/research/tesla-through-the-lens-of-apple)

<comment><USER6> Thank you for putting these numbers out. Also, kind of reminds me of Facebook revenue growth

</comment></comment></comment><comment><USER8> Looks like everyone is worried about a few percentage points up and down in financial numbers. There are three keys points I took away:

1) We expect the capital spend per unit of capacity for this factory to be less than half of that of our Model 3 line in Fremont.

2) Since Model Y will be built on the Model 3 platform and is designed to share about 75% of its components with Model 3, the cost of the Model Y production line should be substantially lower than the Model 3 line in Fremont, and the production ramp should also be faster.

3) This year, we will continue to implement more automation projects, and our ongoing cost reduction efforts will also make an impact.

In the end if Tesla wants to achieve its mission of moving to world to sustainable energy, costs need to go down. Every quarter they are making improvements and the three points above show significantly potential to further improve costs. The global auto market at $2 trillion is gigantic and good fraction of it is going to be electric - it is really a race to improve costs and Tesla is significantly ahead of the pack so far.

<comment><USER3> > Tesla is significantly ahead of the pack so far

Not really.

Audi e-tron, Jaguar iPace and BMW i3 are all well reviewed.

And 2019/2020 will see a wave of car companies e.g. Hyundai, Mercedes entering the market including plenty of interesting startups coming out of China. So Tesla is going to need far more to differentiate themselves than just range.

<comment><USER9> Audi e-tron and Jaguar iPace compete with the Model S; BMW i3's small range puts it in another category entirely. There's no competition for the Model 3 yet, and I don't think there will be for a while.

<comment><USER10> Well, given that the Model 3 is going to cost around 60k EUR in Europe according to recently announced prices, they aren't going to sell many of them.

That's a price you can have a huge (gasoline) SUV for. Or a luxury Mercedes. Or two smaller BMWs ... People aren't going to buy the Model 3 (which is supposed to be a "budget" car) only for Musk's puppy eyes at such price.

The other reason is a horrible lack of charging infrastructure. Europe isn't US where everyone has a house, garage and a driveway so plugging a car in to charge overnight isn't a problem. Here a huge amount of people live in apartments and the shared parkings simply don't have charging spaces. And a token one/two slow charging poles at some larger supermarkets are really not a solution. Even Tesla has a very poor coverage with their superchargers here.

That's why the electric cars aren't exactly selling like hot cakes here. That Model 3 doesn't have an equivalent electric competitor is very much irrelevant when everyone is going to compare it with a gasoline (or diesel) powered car - which cost half the price and have higher utility value.

</comment></comment></comment></comment><comment><USER11> I think Tesla may start to face issues when the other automakers start coming out with electric cars within the next few years. I sold off all my stock in Tesla, although I think they may still be a good short term bet. The supercharger network and cool factor is their big advantage at the moment, but I think in the future they won't do as well when there's more competition.

<comment><USER12> _I think Tesla may start to face issues when the other automakers start coming out with electric cars within the next few years._

Perhaps, but people have been saying exactly this about Tesla for the last ten years now. Hasn't been an issue for them yet, despite much improvement in the EV offerings from other manufacturers.

<comment><USER13> Not only has it not been an issue...Tesla's share of plug-in sales has been _increasing_ dramatically year over year.

Cadillac ELR, BMW i3, Chevy Volt, Chevy Bolt were all billed and hyped as "Tesla Killers" upon launch.

[https://insideevs.com/monthly-plug-in-sales- scorecard/](https://insideevs.com/monthly-plug-in-sales-scorecard/)

To date, no other manufacture has announced any plans to procure enough battery supply to produce cars in comparable volume to Tesla.

<comment><USER14> To date. But more and more manufacturers are biting at their heels for the chance. With better and better cars.

<comment><USER15> that aren't selling nearly as well...

[https://insideevs.com/monthly-plug-in-sales- scorecard/](https://insideevs.com/monthly-plug-in-sales-scorecard/)

</comment></comment></comment></comment></comment><comment><USER16> What Tesla has achieved in the past decade is a miracle. Their success is a combination of luck and the resolve of everyone at Tesla.

But given the tall odds and strong pessimism from multiple groups, it's a marvel they continue to thrive. Kudos Team Tesla!

</comment><comment><USER17> Looks like shares dropped a bit in after-hours. Likely related to the CFO planning to leave during 2019, which was not in the earnings letter.

[https://www.cnbc.com/2019/01/30/musk-says-tesla-cfo- deepak-a...](https://www.cnbc.com/2019/01/30/musk-says-tesla-cfo-deepak-ahuja- is-leaving-the-company.html)

</comment><comment><AUTHOR> Interesting set of charts here:

[https://www.zerohedge.com/news/2019-01-30/tesla-shares- viole...](https://www.zerohedge.com/news/2019-01-30/tesla-shares-violently- unchanged-investors-try-figure-out-what-musk-just-reported)

</comment><comment><USER18> Model Y. So Tesla will have four models: S 3 X Y

<comment><USER19> That isn't an accident. The Model 3 was meant to be called Model E but Ford owns the rights to that name even though they haven't marketed a Model E in forever.

</comment></comment><comment><USER20> Is there another vehicle company that is close to Tesla in self-driving capabilities?

If Teslas drive themselves and other EVs don't, are they a comparable car?

<comment><USER21> Tesla's _don 't_ drive themselves though. Tesla has removed the FSD option from all it's model's since October. Most self driving engineers don't actual believe the strategy of using only standard cameras installed in the vehicle is going to work out long term, and their self driving unit has less funding than BMW's, GM's Cruise division, Google or Uber.

It's hard to see how they win the self driving game, and if they do, it will be years off.</comment></comment>


A Beginner's Guide to PPC: Introduction - <AUTHOR> http://blog.octoclick.com/ppc/beginners-guide-to-ppc/

<comment><USER2> I clicked on this with interest, thinking it was an introductory article on the PPC architecture and instruction set.

Perhaps the title should use pay-per-click instead of PPC? :)

<comment><USER3> You weren't the only one. I remember learning a bit about powerpc assembly several years ago, but never did anything with it. I've since learned a fair bit about sparc, and was eager to compare. Oh well!

<comment><USER4> Yes, I was looking forward to an interesting article on the PowerPC. I am disappointed that it's some marketing thungummy. :)

<comment><USER5> on a page with a broken overlay so yhat you cant even read the text....

<comment><USER6> Doesn't seem broken here.</comment></comment></comment></comment></comment>


The Search for Methods of Group Instruction as Effective as Tutoring (1984) [pdf] - <AUTHOR> http://web.mit.edu/5.95/readings/bloom-two-sigma.pdf <comment><AUTHOR> Also: [https://en.wikipedia.org/wiki/Bloom%27s_2_Sigma_Problem](https://en.wikipedia.org/wiki/Bloom%27s_2_Sigma_Problem)</comment>


Four Day Workweek Tech Jobs? - <AUTHOR> I'm interested in joining a tech company that is experimenting with a four day workweek. Do you know of any organizations running on a four day workweek or similarly structured schedules?<p>And, for larger discussion: do you think the four day workweek will become more prevalent in the tech sector? <comment><USER2> Why not start consulting/contracting on a project basis? You can set your own hours and try out the four day work week.

In my experience, not much work is done on Fridays...so most people work 4 days and are paid for 5.

</comment><comment><USER3> Basecamp during the summer months... but good luck getting a job there.

> And, for larger discussion: do you think the four day workweek will become > more prevalent in the tech sector?

No way - a five day work week with eight hours per day (minimum) is too ingrained into our working culture. "Tech" has nothing to do with it.

</comment><comment><USER4> 30 Hour Jobs[1] is a pretty good resource, I'm subbed to the newsletter as well. [1][https://30hourjobs.com/](https://30hourjobs.com/)

</comment><comment><USER5> Dolby has a 9 hour workday and a Friday off every other week IIRC.

<comment><USER6> Raytheon does this as well. At least they did 15 years ago.

</comment></comment><comment><USER7> How do I apply?</comment>


Key-based device unlocking - <AUTHOR> http://continuations.com/post/139510663785/key-based-device-unlocking-questionidea-re-apple <comment><USER2> This simply isn't feasible.

Yes you can do key escrow via PGP (so you could, in theory, have a sound encryption algorithm at work) but you have multiple sets of issues with this idea.

First the way this escrow works is you encrypt the file system with a randomly generated key. This key is then encrypted by using this escrow key the manufacturer keeps along with another copy of the key being encrypted by some random, initial password which is kept in plaintext or somehow already known to the user (gotta be able to decrypt the encrypted filesystem when the user first gets the phone). At that point the default password is used, the user re-encrypted the key for the data using their own passcode. Now you have two keys that can unlock the phone. This ignores the whole issue of being able to access that data's initial key the entire time during transport but how else are you going to give the user the ability to decrypt the data key so they can even access the device?

Then what do you do with the keys? Phones are made in China so there would be zero stopping China from forcing all companies to provide this escrow key. But let's say you got around that, how do you respond to lawful requests for these keys? You either need to move them all to the United States or keep them in China; either way they're sitting somewhere, at least, where attackers could attempt to gain access to.

Second, and honestly the biggest issue here, is the UX of this flat out sucks. So now if you forget your password you either have to format the phone (thusly destroying the original escrow key) or the phone is simply bricked and there is no way to recover.

There is simply not a mathamatically way to provide multiple parties access to an encrypted piece of data / volume without decreasing its security.</comment>


Decades of neoliberal austerity will make it harder to fight the Covid pandemic - <AUTHOR> https://bostonreview.net/class-inequality-science-nature/amy-kapczynski-gregg-gonsalves-alone-against-virus <comment><USER2> It's not neoliberal austerity. Austerity is a neocon cornerstone.</comment>


Gmail banned in Indian government offices - <AUTHOR> http://timesofindia.indiatimes.com/tech/tech-news/Gmail-banned-in-government-offices/articleshow/46407967.cms?utm_source=facebook.com&utm_medium=referral&utm_campaign=TOI

<comment><USER2> _" The e-mail services provided by other service providers shall not be used for any official communication," the notification says..."_

The U.S. government also mandates that official government e-mail accounts be used for official business, since we have laws that require the archiving of official communications (to avoid corruption, etc.).

For example, there was a controversy a few years ago when the head of the EPA, Lisa Jackson, was found to have been doing EPA business using the e-mail alias of "Richard Windsor", in an apparent violation of federal records laws.[1]

There are also industries like finance where laws mandate that e-mails must be archived for a certain period of time to make them accessible to regulators in case improprieties are alleged. Many other businesses also want to keep archives of employee e-mails in cases patent disputes or other lawsuits arise.

[1] [https://en.wikipedia.org/wiki/Lisa_P._Jackson#EPA_Administra...](https://en.wikipedia.org/wiki/Lisa_P._Jackson#EPA_Administrator)

</comment><comment><USER3> No, they don't ban Gmail in particular, they banned all non-NIC email providers in general. This is not a focused ban, it's a general ban. Obviously they want to have editorial control over government employee generated information.

</comment><comment><USER4> Why is this even sensational when all governments around the world do sth of this sort one way or another ?

<comment><USER5> Exactly what I was wondering. Governments and private companies. This isn't news, it's not even linkbait.

<comment><USER4> When dealing with a lot of sensitive government data a lot of things can happen. It makes so much sense to ban widely used email services where appropriate. Indian media houses just try to make an issue out of nothing just to grab attention instead of concentrating on things that matter. These days media houses like times of India are becoming bad examples of responsible journalism. It's a shame.

</comment></comment></comment><comment><USER6> Oh no, not again. The amount of times this news has been circulated over the last 1-year is amazing.

August 30, 2013 : Cyberspying: Government may ban Gmail for official communication [0]

October 29, 2013 : Govt. mulls ban of official usage of Gmail, Yahoo [1]

March 27, 2014 : Centre asks for ban on Gmail, Yahoo and Facebook sites in government offices [2]

September 12, 2014 : Email services like Gmail and Yahoo! may be banned for government's official use [3]

I looked around for an official notification but could not find any. Usually, it is the DeitY (Department of Electronics & Information Technology) that issues such notifications and the latest one still makes NIC mails a "preferable" option over a mandatory ones [4].

[0] [http://timesofindia.indiatimes.com/tech/tech- news/internet/C...](http://timesofindia.indiatimes.com/tech/tech- news/internet/Cyberspying-Government-may-ban-Gmail-for-official- communication/articleshow/22156529.cms)

[1] [http://www.thehindu.com/news/national/govt-mulls-ban-of- offi...](http://www.thehindu.com/news/national/govt-mulls-ban-of-official- usage-of-gmail-yahoo/article5284904.ece)

[2] [http://indiatoday.intoday.in/story/centre-ban-yahoo-gmail- so...](http://indiatoday.intoday.in/story/centre-ban-yahoo-gmail-social- networking-sites-k-n-govindacharya/1/351399.html)

[3] [http://articles.economictimes.indiatimes.com/2014-09-12/news...](http://articles.economictimes.indiatimes.com/2014-09-12/news/53851235_1_sensitive- government-data-government-communication-government-offices)

[4] [http://dot.gov.in/sites/default/files/2014%2012%2008%20Posti...](http://dot.gov.in/sites/default/files/2014%2012%2008%20Posting%20Information%20on%20Social%20Media.pdf)

</comment><comment><USER7> > monitoring online activities ... on official computers, block content which > it feels is adversely affecting productivity ... and also have a right to > delete e-mails or internet history

Is this unusual in India? In the US this is normal.

</comment><comment><USER8> When I was working in American financial institution we got the same, all Google services (expect Youtube) were banned. I had a little experience in working for big companies, but I think it's pretty standard, isn't it?

</comment><comment><USER9> could be additional reasons such as the fear of sensitive data being transmitted to a company notorious for it's close relationship with the US surveillance apparatus

</comment><comment><USER10> Doing business in India is, apparently, really hard.

[http://www.doingbusiness.org/data/exploreeconomies/india](http://www.doingbusiness.org/data/exploreeconomies/india)

They come 142nd out of 189 countries!

The Indian railway ticketing system is bafflingly terrible.

I'd love to hear from people who live there about what it's like, and if there's any chance it can turn around. It's kind of surprising to me that India is not much more powerful than it is.

<comment><USER11> The Indian railway ticketing system was recently improved and it looks like its vastly better now.

<comment><USER12> Yes, true. Here are the technical details: [http://pivotal.io/big-data/case- study/distributed-in-memory-...](http://pivotal.io/big-data/case- study/distributed-in-memory-data-management-solution) \- a worthy HN submission by itself</comment></comment></comment>


Conquering Hadoop with Haskell [pdf] - <AUTHOR> https://dl.dropboxusercontent.com/u/58525/NY%20Haskell%20Meetup%20-%20Conquering%20Hadoop%20with%20Haskell.pdf

<comment><AUTHOR> Also, there's a full video of the talk here:

[http://vimeo.com/90189610](http://vimeo.com/90189610)

</comment><comment><USER2> This is great. I've been learning Scalding, which is a vaguely similar Scala library for doing map-reduce jobs (where you're writing higher level workflow, and it handles the map/reduce/map/reduce.... cycle to get it done).

I'm going to have to give this a try :)</comment>


The importance of stupidity in scientific research - <AUTHOR> http://jcs.biologists.org/content/121/11/1771.full

<comment><USER2> I would like to counter the opinions that claim that the author should use the term "ignorance" rather than "stupidity". IMHO, the inadequacy that one feels during research does not arise from not knowing the answer, it is a result of not being able to figure out how to approach the problem and reaching an answer. More than this, most of the time you do not know what the appropriate question is.

This is not ignorance, sometimes you simply cannot weave the threads of knowledge you have to reach a pattern. Of course if you have more threads, your job becomes easier; and at some low level of knowledge you will have nothing interesting to ask, but this is not the major difficulty in completing a Ph.D. thesis, at least in my experience (which is in physics/astronomy).

This is what the author is talking about, it is not "knowledge" that is lacking, at least not knowledge in the sense that something you can learn from a book. It is the quality that is mentioned in an essay by pg: <http://www.paulgraham.com/wisdom.html>

I would not call this problem ignorance. Even though stupidity is kinda harsh, it captures the feeling well.

<comment><USER3> the idea that stupidity is harsh, and that we have to rename this feeling as ignorance is all part of the problem. everyone is so hung up on not being stupid. even after reading the article, it's what people care about...

you can argue all you like that the author should use another word. but what you have to get over when you're doing research is exactly that _feeling_. the feeling that people are trying to hide from by playing with words.

not sure i am making my point well - what i am trying to say is that the people who are complaining about the term "stupid" are not helping anyone. they're just exhibiting the same instinctive reaction that you feel in research. whatever name you give it, you feel the same.

</comment></comment><comment><USER4> In my limited experience with academia, I would argue this article is more about recognizing one's ignorance in front of a problem, which is indeed one of the most important attributes of a good scientist : "i dont know how to solve this problem, yet".

This is in fact quite the opposite of a stupid position - which would handle the issue of being faced with a challenging problem with either militant ignorance "i don't know and I don't care" or uninformed arrogance "This? of course I know : <insert wrong answer here>"

<comment><USER5> Very true, but for the first 15 or so years of schooling, knowing the answer to a question is a sign that you aren't stupid, and so the inverse _must_ be true: not knowing is a sign you _are_ stupid. It takes a while to get out of this habit.

<comment><USER6> Yes, that's a huge problem with education — if it's geared towards scoring well on tests, and collaboration and research is often "cheating", then it promotes facile understanding.

For a job, tests are fine; you want to know if someone has a basic foundation of ability before they cut into you. But that's different from actual education, the kind that people go through for the first decades of their lives. That should be about something deeper than reliably scoring 90% on mindnumbing series of tests. Like critical thinking and self-directed learning.

</comment></comment></comment><comment><USER3> note that the author is male; the person who dropped out is female. for some reason, women, on average, seem to take "being stupid" to heart, while men can ignore it. i have no idea why this happens, but i've seen it again and again.

[no criticism of either sex intended. i simply read the article, then thought "i wonder if the author was male and the other person female?", went back, checked...]

<comment><USER7> Anecdotally speaking, that reminds me of a woman's comment about why she found raising a VC round so exhausting and difficult -- it was much harder for her to shake off the criticism and doubt constantly lobbed at her by potential investors, whereas male CEOs did not seem to take it to heart the way she did.

</comment></comment><comment><USER8> When you operate in uncharted waters, independent thinking makes for a good compass. As someone who studied comp sci in undergrad, I felt I didn't develop these skills as much as folks in the humanities.

<comment><USER9> I don't know if it's independent thinking so much as synthetic thinking. In comp sci, you get _a lot_ of mileage out of analytic thinking. It maps nicely to linear processes in a computer, and to the rigor of programming languages.

So much mileage, in fact, that your non-analytic faculties atrophy from disuse.

</comment></comment><comment><USER10> I think I understand the author's point and repeated use of the term "stupidity" keeps the article on a theme and sounds somewhat nove.

However, I think his point would be a lot clearer if he would stop using "stupidity" when he means "ignorance".

<comment><USER11> I think his point is that most smart people throughout schooling have not encountered true ignorance, so it _feels_ like stupidity.</comment></comment>


Ask HN: Why aren't botnet-fodder vendors' names (XiongMai, Dahua) in the news? - <AUTHOR> I had to dig into a few articles to find the names of the devices involved, but they're not in the headlines like Samsung has been. And for all the Note fires, Samsung hasn't even taken down the internet. (Yet.) Among all the "it's going to get worse before it gets better" talk about IoT and embedded security, where's the pressure to get better? I'm sure a few technical folks who made these devices are ashamed, but they won't get additional resources to implement better security or to update old products until the CEOs feel the heat. How can this best be achieved? <comment><USER2> I suppose these vendors are not major household names like Samsung, so it doesn't make as good of a story. I also wonder if perhaps the general public views attacks as inevitable -- sort of a force of nature -- whereas we view it as a problem that can be greatly reduced with specific actions.

As a software developer in the consumer electronics industry, I am quite concerned about these vulnerabilities and the industry's casual acceptance of them. :(

<comment><AUTHOR> > inevitable -- sort of a force of nature

Yeah, sort of! The attempts are inevitable, but I don't think the vulnerabilities are.

I've long used the Three Little Pigs story when explaining security: Yes, the wolf is always there, don't act shocked when you build a house of straw and suddenly a horrible bad nasty despicable and thoroughly evil (just trying to feed his family) wolf shows up.

The pig who hauled the brick did more work than the pigs who hauled sticks or straw, but ended up not getting eaten. Everyone from the breakroom to the boardroom should understand this analogy. I've found it helpful, anyway.

<comment><USER3> True, but today's steel is tomorrow's sticks when it comes to vulnerabilities like this one. Over time, old security protocols are cracked and new vulnerabilities are found in existing devices.

</comment></comment></comment><comment><USER4> I work in this industry, but I might have some facts incorrect.

My understanding was that the devices that were hijacked are older (2-3 years minimum) model NVR's and IP Cameras. The firmware for these devices is built on top of Busy box. From what I can tell, this firmware comes with telnet enabled and after many years of exploits, these companies fixed the firmware and removed the exploit in newer releases (within the last 2 years).

The hijacked devices shared the same characteristics: telnet can be abused and credentials to log into the device were set to default (admin/admin).

Add in a global registry of these devices (shodan.io) and you can essentially tap into these devices fairly easily.

Whoever was behind the attacks using this firmware exploit must have a very intricate understanding of the firmware IMO.

If it was the Chinese gov't, they would be impacting one of the largest providers of CCTV from China (Dahua). The Chinese gov't favors another company (Hikvision) who has raised 6 billion dollars to expand in the US; some if this money came from the Chinese gov't. If China is behind these attacks, it might be to mess with the US and protect their investment (which sounds like a bit of a stretch).

Should these companies have their names out in the open? Probably not; firmware in the last 2 years has removed telnet. You might be able to do some damage with their HTTP API, but the device has to use default credentials. Putting their name out might also encourage others to attempt the same type of attacks.

<comment><USER5> I have reviewed the code and indeed, it is fairly well written. Definitely a professional coder. Not sure if they would have to have a deep understanding of the firmware though. Busybox is linux based and that's all they needed to know. I also think the problem would be fairly easy to fix albeit, through unconventional channels. If it can be hacked, it can be patched and locked down, probably using the same code that the attacks are based on.

<comment><USER4> Can you share the code? I am curious to see the what kind of attack vectors were applied.

Having enough product knowledge to be able to exploit the specific firmwares is what I meant by a deep understanding of the firmware.

My hypothesis is that either the person spent some time working out which cameras to attack, they had previous experience within the surveillance industry, or they did research on common network recorder exploits.

</comment></comment></comment><comment><USER6> Pure speculation, but maybe it's actually the Chinese?

We know it's Chinese equipment, we also know that industry there has strong ties to the Government, see the Huawei investigation as an example ( [https://intelligence.house.gov/sites/intelligence.house.gov/...](https://intelligence.house.gov/sites/intelligence.house.gov/files/documents/Huawei- ZTE%20Investigative%20Report%20\(FINAL\).pdf) )

Maybe the Chinese Government encouraged their suppliers to pump out unsecured crap, knowing they'd be in a position to flood the market and take advantage of it later?

<comment><AUTHOR> The "movie plot" part of my brain loves this theory, but Hanlon's razor suggests otherwise.

<comment><USER7> For those unfamiliar, a geeky etymology [1] of Hanlon's Razor.

[1] [https://en.m.wikipedia.org/wiki/Hanlon%27s_razor#Origins_and...](https://en.m.wikipedia.org/wiki/Hanlon%27s_razor#Origins_and_etymology)

</comment></comment></comment><comment><USER8> Why should they be in the news? We're talking about a few hundred thousand compromised devices at best. Windows malware regularly hits millions.

</comment><comment><USER9> You were just one day too early, the names are beginning to drop now

</comment><comment><USER10> Why aren't news in the news? I see no coverage at al

<comment><USER11> The attack was on the front page of the NYT website yesterday

here's the article: [http://www.nytimes.com/2016/10/22/business/internet- problems...](http://www.nytimes.com/2016/10/22/business/internet-problems- attack.html?ref=technology&_r=0)</comment></comment>


Xamarin Acquires Petzold - <AUTHOR> http://blog.xamarin.com/xamarin-acquires-petzold/ <comment><USER2> Programming Windows 5th edition (1998, 1400pages) still stands in my bookshelf. It's still relevant for Win32 development.

</comment><comment><USER3> I could not thank Petzold enough for writing Annotated Turing book. If you ever wondered what Alan Turing's seminal paper meant _in depth_ but with friendliness warmer than "for dummies" book, give it a try.

</comment><comment><USER4> It's a shame their Indie plan doesn't include Visual Studio support, without that there is way less value as a developer imo (and at $1k/yr still steep for their business plan).

<comment><USER5> One day. One day I hope Xamarin will find a different financing model, or like you said include VS support in their indie plan. It's just far too expensive right now.

<comment><USER6> Yeah. Also, I find it very sad that you cannot even try the Forms API before buying the suite. Their model only works for companies / individuals that don't care about the pricing.

<comment><USER7> Did they get rid of the eval period (30 day money back guarantee is not the same as an eval period) when they switched to a monthly subscription model?

Edit: the trial is mentioned in the extended FAQ [http://xamarin.com/faq#pricing](http://xamarin.com/faq#pricing)

One thing I didn't realize was that it any app built with the trial version can only be used for a 24 period. My plan was to build a LOB app, get my company to see the value in it and then have them purchase the full license. The 24 hour window makes that really difficult.

</comment></comment></comment></comment><comment><USER8> Xamarin Studio is a really awesome product that doesn't get enough love because of anti-MS sentiment.

<comment><USER9> Well not having a linux working version ( at least of xamarin.android) is not helping to fight back that sentiment

<comment><USER10> That is true. I have a Xamarin plan for use with Visual Studio for developing mobile apps in F#.

However I'm just keep running into stumbling blocks and general frustration developing in Windows, having switched from 10yrs of Linux purely for VS. With the general improvements in Mono and the open sourcing of F# I'd love to use them all together on Linux and take advantage of the greater OSS developer ecosystem there.

Monodevelop/Xamarin Studio is fine for development, more so with the newly- added split-view, and the F# support is pretty much on a par with VS these days, but without Xamarin.Android/iOS support I simply can't move.

Maybe I should suck it up and buy a Mac instead.

<comment><USER11> > Maybe I should suck it up and buy a Mac instead.

That is definitely the more comfortable migration path for users coming from a Linux background.

</comment></comment></comment></comment><comment><USER12> Code is a great, great book. Thank you Petzold.</comment>


Show HN: Falcon – A markdown based note-taking app for Mac - <AUTHOR> https://itunes.apple.com/us/app/falcon/id1086447938?ls=1&mt=12 <comment><AUTHOR> Received this question a couple of times, about the app being native or not, so clarifying it. The only web view based component in the app is the preview, everything else is made of basic native elements, & the whole app is coded in swift. I think the doubt rose from the point that you could theme it with hex- color codes. But how that works is that, whenever you change the theme, the app reads the light / dark plist files and assigns color to app components after converting the hex-colors to NSColors. I hope this resolves the doubt.

</comment><comment><USER2> This looks awesome, but I can't use anything for notes without mobile support. I'm excited for when that day comes. :)

<comment><AUTHOR> Thank you. It is coming soon :)

</comment></comment><comment><USER3> Ulysses Look-alike, with added preview. Nice app ;)

<comment><AUTHOR> Thank you. Most note-taking apps follow the same 3-paned split-view format, so you can say that ;)

</comment></comment><comment><USER4> Howdy, is this a native app?

<comment><AUTHOR> Yeah this is a native app, written in swift.</comment></comment>


Rust Traits: Defining Behavior - <AUTHOR> https://oswalt.dev/2020/07/rust-traits-defining-behavior/ <comment><AUTHOR> Hello all - I write technical blogs as a way of learning new subjects, and lately I've been learning Rust. Hope it's useful.</comment>


WeChat is building a Slack killer - <AUTHOR> https://www.techinasia.com/wechat-slack-work-office-chat <comment><USER2> This has got to be a joke! Software built in China cannot even come close to Slack in quality or novelty. Period.

</comment><comment><USER3> Given concerns about Chinese IP theft it would be hard to see this taking hold amongst major US companies.</comment>


Build your own dev tools. Now. - <AUTHOR> http://blog.chaoscollective.org/post/23151828110/build-your-own-dev-tools-now <comment><USER2> Now just to get designers in the loop for some full on design/build action.</comment>


Postgres Count Performance - <AUTHOR> https://www.citusdata.com/blog/2016/10/12/count-performance?source=pg_weekly <comment><USER2> Even if counts could be made faster, at scale you'd probably still want to avoid counting anything that can be pre-calculated.

We use something similar to the trigger-based method they describe, tho have found that a lot of updates to count table inevitably ends with deadlocks. So instead of updating a count value, we always insert a new count of 1 or -1, and use summing to calculate the total count as needed. A background task is responsible for continually squashing the count values.

<comment><USER3> Great solution, thanks for sharing.

At this point, I wonder: with a huge dataset, wouldn't you have better time leaving the count field out of postgres altogether and use something like redis with its INCR/DECR instructions instead? This would prevent having deadlocks as well.

EDIT: that is, if you don't need to use the count field in other queries.

<comment><USER2> We've found that Redis works great for counting some things, but when you need a count which is guaranteed to match what is in a db table, it's preferable to manage counts at a db level. Often we're making counts based on the state of a record, and it's easy for the application level view of things to end up stale.

</comment></comment></comment><comment><USER4> A few comments regarding the count_distinct extension (author here):

> Note that custom extensions written in C like count_distinct are not bound > by the value of work_mem. The array constructed in this extension can exceed > your memory expectations.

This is slightly inaccurate, as this is not specific to custom aggregates. Custom aggregates are estimated just like any other (built-in) aggregates - number of expected groups times memory per group. The trouble is that (a) for aggregates with variable-length per-group state, we don't have a good size estimate, and (b) if the planner decides to use HashAggregate, we're unable to do anything when reaching work_mem. But this has nothing to do with the aggregate being custom - array_agg() and string_agg() have the same issue, for example.

FWIW, the extension was written quite a long time ago - before the various sort optimizations made by Peter Geoghegan. I wonder whether that made count_distinct obsolete.

<comment><USER5> > for aggregates with variable-length per-group state, we don't have a good > size estimate

Can you say a bit more why?

<comment><USER4> No one implemented a better solution ;-)

Jokes aside, it seems simple but is fairly tricky, as it depends both on input data and various other parameters. For example for array_agg() or string_agg() it might be estimated from number of entries / average length. For hll it also depends on the accuracy and expected number of values to track, etc. I was considering adding another method to the API, providing a better estimate, but never got to that.

So the current code simply assumes 1KB (IIRC) per group in those cases, or something like that.

Of course, the memory estimate also depends on the number of groups, but that's an orthogonal issue.

</comment></comment></comment><comment><USER6> No mention in the article of the new parallel query support in 9.6 that can speed up COUNT(*)?

[https://www.postgresql.org/docs/current/static/parallel- plan...](https://www.postgresql.org/docs/current/static/parallel-plans.html)

<comment><USER7> We were doing some benchmarks recently on different databases. Most of the queries were heavy aggregations (>1billion of rows) and, to be honest, we were a bit disappointed with the new parallel query support in pg, we were expecting much better performance.

While doing the benchmarks, we could see that citus was always taking full advantage of all the cores in the cluster, while postgres parallelization was not.

Disclaimer: I'm not a db expert and I don't have any relationship with either citus or postgres.

</comment></comment><comment><USER8> > On the other hand count(1) takes an argument and PostgreSQL has to check at > every row to see that ts argument, 1, is indeed still not NULL.

I expect that's the same for all function calls like this? Surely pg has a concept of constants and doesn't needlessly re-check parameters?

<comment><USER9> It checks that the first column of each row is not null, that's what the '1' stands for.

</comment></comment><comment><USER3> Super in-depth analysis, thanks to the author for writing it.

Past the recommendation of counting based on an indexed column, I wonder if this should really be user's concern. This paragraph especially triggers a "this should be fixed upstream" feeling in me:

> A word of warning. When work_mem is high enough to hold the whole relation > PostgreSQL will choose HashAggregate even when an index exists. > Paradoxically, giving the database more memory resources can lead to a worse > plan. You can force the index-only scan by setting SET enable_hashagg=false; > but remember to set it true again afterward or other query plans will get > messed up.

But worst case scenario, this article will be useful until this is fixed, so thanks again :)

<comment><USER10> If you think "needs upstream fix", then no database is suitable for use, because they all choose poor plans in many edge cases. This is why most databases have extensions that let you hint or force index use (not Postgres, however; a mistake with mitigation), or support parenthesising your joins to force an evaluation order (this is an indirect way of forcing index use or join order, a mitigation), etc.

Query planning is something where a poor choice can have serious performance ramifications, because n is usually much larger than in most programs. Analyzing the algorithmic complexity of a piece of SQL takes some experience and experimentation, and with different table stats the query planner may make different decisions. It can be worthwhile limiting the planner's discretion to get more predictable performance.

(I work on a product where many of the features can be expressed in terms of relational algebra. Often, both the best performing and quickest to write and test implementation logic is a bunch of SQL, and not the kind of CRUD that is easily wrapped with an ORM. What would make my life easier is a SQL linter that, given a model of costs, would prevent people writing queries that scale only linearly with specific table sizes. I've accumulated sufficient intuition that I could do this for MySQL at this point.)

<comment><USER3> Optimizing queries manually is fine by me, what troubled me was more the fact that it was recommended to change a configuration setting on the fly, which sounded like asking for problems regarding concurrent requests.

But actually, I've just made a test, and it appears changing this setting only impacts the current connection, so provided it's toggled back after the request, this should not be a problem.

<comment><USER11> Though I wonder how it would work with pg-bouncer.

</comment></comment></comment></comment><comment><USER12> It's a bit odd there's no mention of the PG columnar store in this article ([https://www.citusdata.com/blog/2014/04/03/columnar-store- for...](https://www.citusdata.com/blog/2014/04/03/columnar-store-for- analytics/)) - especially since it's from the same company.

It would be interesting to see how much the performances improve once you use cstore_fdw (especially since 1M records is quite small when talking about OLAP workloads).

disclaimer: I've never used cstore_fdw, but I have evaluated a number of columnar databases in the past.

<comment><USER13> (Ozgun from Citus Data)

We find that the primary motivation for using cstore is reducing disk I/O / storage footprint. cstore_fdw keeps a columnar layout on disk in compressed form and reads only relevant columns. For example, it's commonly used for data archival purposes.

That said, cstore_fdw doesn't yet make optimizations related to query planning and execution. We made experiments in that direction ([https://news.ycombinator.com/item?id=8423825](https://news.ycombinator.com/item?id=8423825)), but making those changes production ready is no small effort.

Since all benchmarks in this blog post are for in-memory data, I don't know how much they would benefit from cstore. If I have the time, I'll give it a try and update this comment with the results.

</comment></comment><comment><USER14> The counting seems really slow compared to MSSQL? I'm not familiar with the reference hardware used here, but just running a similar test on my desktop with sql server it can count distinct a million strings in under a second (with parallelisation, around 2.5s cpu time).

Or am I missing the fact that these benchmarks are run on a reference spec which is comparatively old?

<comment><USER15> I'm not familiar with mssql, does it use mvcc? That is the reason PG counting is slow (really slow)

<comment><USER14> I'm not an expert, this is my layman's understands.

It has different isolation levels, some involving snapshots and some not. I think most concurrency issues are (by default) dealt with by locks, which start at row level and can escalate to page and table level (with significant slowdown seen when lock escalation happens in contentious places).

But that only has an effect if it's under write.

<comment><USER16> This is historically correct, but I believe later versions of mssql server now default to their mvcc implementation. I think this switch was circa 2005; not 100% sure.

I managed an enterprise applications group that primarily used mssql for data in 2007. I can't recall which mvcc implementation they use. Our servers were MSSQL 2000 and I remember being a bit more than surprised when the DBAs told me the root of the performance problems we had were due to lock escalation; having come from the Oracle and PostgreSQL worlds, I was naive enough to have thought that lock escalation implementations like this were historical curiosities rather than something I'd actually run into... live and learn I guess.

<comment><USER17> I believe it still has to be chosen, and the MS term is 'Read Committed Snapshot'. Everything is a tradeoff, read committed snapshot makes your tempdb busier (since it is involved in versioning), and balloons up the default size of a row in a table. With RCS you end up adding 14 bytes of overhead to each row, and which otherwise wouldn't be there.

So imagine a table with a single integer column and 1 billion rows. Instead of the 9 byte per row overhead (7 bytes for row metadata and 2 bytes for the page offset), you instead have 23 bytes, plus the 4 bytes to hold the int32. Without RCS on, that row would only have a 9 byte overhead and the rowsize would be 13 bytes.

so 1billion * 27bytes = 25.14 GB 1billion * 13bytes = 12.11 GB

There are some other performance tradeoffs (walking row version values, pagesplits when updating records without rowversions after converting the database, et cetera).

Ultimately, MVCC is great for contention, but it stinks if you're trying to efficiently pack in data.

For an alternative perspective, I sometimes bemoan the size of tables in postgres because of the mandatory overhead and versioning.

<comment><USER16> Agreed. Table bloat is no fun either. Though, I would say under the most common workloads, it's easier to manage table bloat than it is lock escalation... I can pick my battleground for table bloat whereas lock escalation is immediate.

Oracle's MVCC method doesn't have that problem... but then you get the imfamous ORA-01555, "Snapshot too Old" from time to time.

Oh well, no perfect worlds I suppose.</comment></comment></comment></comment></comment></comment>