Privacy, Security and Your Dropbox - <AUTHOR> http://blog.dropbox.com/?p=735

<comment><USER2> These are weasel words. Miguel de Icaza's post explains the issue:

> My problem is that for as long as I have tried to figure out, Dropbox made > some bold claims about how your files were encrypted and how nobody had > access to them, with statements like:…

> \- Dropbox employees aren't able to access user files, and when > troubleshooting an account they only have access to file metadata > (filenames, file sizes, etc., not the file contents)…

> But this announcement that they are able to decrypt the files on behalf of > the government contradicts their prior public statements.

Dropbox are avoiding answering this. If they derived the encryption key from your password, as JWZ explained [http://www.jwz.org/blog/2011/04/dropbox- doesnt-actually-encr...](http://www.jwz.org/blog/2011/04/dropbox-doesnt- actually-encrypt-your-files/) and as occurred to any halfway decent hacker, they wouldn't need to have the ability for their employees to decrypt your files — an ability which they are still claiming to not have.

And this is just insulting:

> We understand that many of you have been confused by this situation — and > some folks even felt like we misled them

The problem isn't that we were _confused_ or what we _felt_ or that we were unintentionally "misled". The problem is that, apparently, _you lied to us in order to defraud your customers_.

<comment><USER3> Sorry. I just don't get it. Why are people so surprised by this? I think the people that are shocked are people that just don't bother to think about the devices they use. This is the danger of this whole "ooooh it's magic" shellac marketing craze everyone seems caught up in.

All of this should have been obvious to anyone that bothered to think about what was going on before they stored their precious secrets in Dropbox. Obviously they are holding the encryption keys. Hell, that much was obvious to me when I started using/testing Dropbox just over a year ago and saw my files served over a web page. I can't help but muster a "meh".

The value I saw in their use of encryption is this: when they or amazon or whoever dispose of their drives it will be harder to harvest data off of them. That's all I've come to expect from them and that's all any of their information has ever warranted.

I still derive great value from their service.

Now, I will admit the one that I personally didn't think through back then was the key file thing, but really that's just the same as having an ssh private key. I was aware that it wasn't constantly prompting me for a password so at some level I knew it was going on. Maybe I would prefer some sort of passphrase at login using a keyring for that file. Minor issue.

I do trust them that the run-of-the-mill support personnel are barred from accessing the keystore. Its entirely feasible that there is no mechanism for them to just go browse around your files.

However, it should be blatantly obvious to anyone that understands anything about encryption that someone somewhere at dropbox can access your files. This has always been the case and obvious.

I guess I'm mostly amused by people that are surprised by any of this. The smarter they pretend to think they are, the more amusing they "fell" for something this obvious. It's like the people that fall for mint.com's attempts to obscure the fact that they store your bank account usernames and passwords (oh, sorry, I forgot, they pay a third party to do that for them instead). People just don't think through the mechanisms of their shiny trinkets. You get to just point and laugh.

<comment><USER4> It is completely reasonable for a layman to believe that his files are protected by a password, and therefore that nobody can access those files without the password. You don't have to be an idiot to have that mental model, especially when Dropbox reinforce it with explicit statements like "Dropbox employees aren't able to access user files".

It's a consumer service, and consumers don't know or care how encryption works. They just need to be told the consequences of whatever technical decisions have been made on their behalf, so it's important for Dropbox to be honest in explaining them.

<comment><USER3> It's also completely reasonable for laymen to assume that the government can gain access to their Dropbox files. I don't recall Dropbox ever claiming anywhere that they would be technologically incapable of complying with a subpoena.

<comment><USER2> When they said, "Dropbox employees aren't able to access user files", they were saying they were incapable of complying with a subpoena.

<comment><USER3> Perhaps. I don't think it's appropriate to edit out the context of that statement. What was actually said was "Dropbox employees aren't able to access user files, and when troubleshooting an account they only have access to file metadata (filenames, file sizes, etc., not the file contents)". I don't think that statement as a whole necessarily responds to their capabilities to answer a subpoena. I don't think the lay public would conflate the two contexts.

</comment></comment></comment></comment></comment></comment><comment><AUTHOR> Personally, I can't see that they can do much more than this. If they're going to supply access via the website, or allow you to share with others then they need to be able to decrypt the file.

I never assumed that it was impossible for them to get access to my files. I just assumed that only certain people had access to the keys, and they only gave them out to third parties when forced to by law.

<comment><USER2> > If they're going to supply access via the website, or allow you to share > with others then they need to be able to decrypt the file.

They need to be able to decrypt the file _right then_ to give you access via the website. They don't need to be able to decrypt the file the rest of the time. If the encryption key depends on your password, this is easy to implement. In modern browsers, this could actually be done in the browser, so that the keys for decrypting the file never reach their servers.

To allow you to share individual files with others, they could encrypt each file with a randomly-generated session key, which is encrypted with your user key. When you give access to the file to a recipient, you would be encrypting a copy of that session key with the recipient's public key.

In actual fact, it appears that anyone who knows the secure hash of the file's content is able to discover that Dropbox is storing it: [http://paranoia.dubfire.net/2011/04/how-dropbox- sacrifices-u...](http://paranoia.dubfire.net/2011/04/how-dropbox-sacrifices- user-privacy-for.html) — and there's at least well-founded speculation, if not proof, that this means that knowing the secure hash is sufficient to read the file: <http://news.ycombinator.com/item?id=2440074>

For their previous statements to be true — that their employees have no access to your files — they would _have_ to be doing something like this.

<comment><AUTHOR> Making it password dependent means that they have to decrypt/encrypt when you change password.

And also that you losing your password is apocalyptically bad.

<comment><USER2> There are some ways around that, but basically you're correct. (For example, you could store the user key, or an additional randomly-generated password, persistently on client machines, so that you can usually recover from password loss; and you could use a tree of session keys to minimize the amount you have to re-encrypt. But basically you're right.)

It's understandable that they chose a less-secure system design. It's not understandable that they chose to lie to their customers about it.

<comment><AUTHOR> Oh, I totally agree. Honesty up-front would have been a much, much better policy.

</comment></comment></comment></comment></comment><comment><USER5> I love that they openly in this post discuss that the end-user can use TrueCrypt if they desire:

"For users who feel more comfortable managing their own encryption keys, we recommend using products like TrueCrypt to store encrypted volumes within their Dropboxes. Those users will unfortunately lose access to some functionality, but we leave this decision to the user."

It's the sign of a good company in my opinion that they don't treat their users as unable to understand such things.

<comment><USER6> Except that when you use TrueCrypt most of the functionality of DropBox stops working; you can't access your files through the web interface anymore (duh), but most importantly, you can't use the volume on two machines at the same time.

<comment><USER7> I've used TrueCrypt volumes with Dropbox before, and when I encountered a situation where I wanted to access the volume from two places, I took the sensitive data from Dropbox and just exposed it on my private network where I have more control over access. As far as I can tell, there isn't a great way to keep high security and flexibility when using sensitive data with Dropbox, but keeping sensitive data on the cloud is always a risky affair anyway.

</comment></comment></comment><comment><USER8> This isn't legal advice, but this seems to meet the same legal requirements that big companies must adhere to when using outsourcing providers that will have access to sensitive client data.

The magic words: "Like most major online services, we have a small number of employees who must be able to access user data when legally required to do so. But that's the exception, not the rule. We have strict policy and technical access controls that prohibit employee access except in these rare circumstances. In addition, we employ a number of physical and electronic security measures to protect user information from unauthorized access."

<comment><USER9> Its not about the fact that they comply with court orders, of course they're going to, it is in their ability to give the courts unencrypted data that they shed light on the real issue to do something they said was impossible.

If it were impossible for them to get at the unencrypted data, the encrypted data would be all that that had to had over.

</comment></comment><comment><USER10> This sounds as if they are doing users a favor:

 _We manage the encryption keys on our users' behalf._

To be completely clear to users, they should instead write:

 _We hold the decryption keys and can decrypt your data if compelled to do so. If you don't want us to be able to decrypt your data, encrypt it yourself with GPG or OpenSSL before sending your data to our site._

Edit: Spelling & clarity.

<comment><USER11> No, they shouldn't write that.

Dropbox is a consumer file storage company. Statements like that will confuse the majority of their target market and stall adoption of the service.

What Dropbox is doing in relation to security/encryption is about the best possible solution that balances features and function with data protection for files that might be leaked or stolen.

If you have suggestions for Dropbox, lay out concise technical suggestions. Writing new legalese for them is not solving any of the problems that people are getting so worked up about.

<comment><USER12> Either way, those users will not understand the security of the service they are using. At least with the long version, they'll be aware of their lack of understaning.

</comment></comment></comment><comment><USER13> If anyone wants to vote for a feature of client side encryption there is a votebox here: <https://www.dropbox.com/votebox/21/client-side-encryption>

</comment><comment><USER14> I actually really like that they did a point by point response to their critics. To me this is an important step for today's companies. The internet gives a voice to customers to raise concerns, companies are able to see these concerns immediately, and companies have a great way to respond quickly. I feel like it this type of dialog creates a win-win situation for everybody involved.

</comment><comment><USER15> I think it's impossible to have deduplication (a system to prevent duplication) in encrypted data with different key for each user (the same file from different user will be encrypted differently) without dropbox storing the key (to decrypt back). Which means they have access to the files. We should know this from the very beginning. If it wasn't, how to check the files for deduplication? Enlighten me please.

</comment><comment><USER16> If Dropbox has one key per user, then how do they do deduplication? We know they do deduplication (<http://tinyurl.com/4276pxu>) so this suggests a very small number of keys.

<comment><USER17> Storing unencrypted file hashes is the easiest way I can see to do it; then, for deduplication, the file is decrypted using one user's key, and a copy is made and encrypted under the second user's key.

</comment></comment><comment><USER18> Holy cow, one request per month is _low_? I know that's less than one account in a million, but that's still way higher than I expected.</comment>


Stopping killer robots and other future threats - <AUTHOR> http://thebulletin.org/stopping-killer-robots-and-other-future-threats8012

<comment><AUTHOR> I used to agree with the article's viewpoint wholeheartedly. It involved this romantic notion that the mutual risk shared by both parties in warfare served to act as a deterrent towards future conflict, thereby ensuring peace. Add robots and that mechanism is destroyed.

Moreover, is something really worth fighting for if nations wouldn't otherwise have the will to commit human troops?

Well, yes. Modern political reality is such that large military occupations for humanitarian purposes are incredibly unpopular, and suggesting the mere prospect of such operations is tantamount to political suicide.

Bad actors know this dynamic, and they exploit it with knowledge that western countries will almost certainly not intervene, allowing them to operate with impunity. Recent history points to genocide in Rwanda and Darfur as being prime examples.

Robotic infantry, built and deployed to any reasonable standard, tend to solve this problem.

The article cites concerns over increased civilian casualties if we were to deploy autonomous weapons platforms. On the contrary, like self-driving cars, it's likely that machine control would result in safer outcomes. In the case of robotic infantry, you're talking sensors and weapons that far exceed human capability in their capacity to be surgical. Combined with the fact that self- preservation could easily take lower precedence to preserving civilian life, and it seems clear that the net result would be a decrease in civilian casualties.

To be fair, the article's spirit likely was intended primarily in the context of aerial drones, and I admit turning drones into autonomous killing machines is almost certainly a bad idea, unless drones significantly ramp up their sensor capabilities and start using bullets instead of bombs.

If autonomous robots with the authority to kill are what it takes to make genocide a thing of the past and bring social change to countries where women are still considered property, and where not subscribing to the mandated faith results in death, then so be it.

Banning domestic use of such robots in developed countries would be a fantastic idea, however. Even any less-than-lethal forms.</comment>


I've seen the best minds of my generation destroyed by Matlab… (2013) - <AUTHOR> http://slendermeans.org/julia-loops.html

<comment><USER2> Why does anyone use matlab?

If I needed to solve differential equations I would use C++.

Many embedded jobs require matlab expertise. I always figured it must be cool but didnt care to buy it because comoutational physics is one of my specialties.

Then I git a look at its language. Worse than the cruel shoes.

<comment><USER3> It's incredibly easy/fast to build a working prototype. Much like how LaTeX just lets you write and the processor takes care of the rest, MATLAB just lets you build while the software mostly takes care of the rest.

<comment><USER2> Funny that. From time to time since 2008 or so I have tried to learn LaTex but always I get Lost while attempting yetbstrictly speaking failing to read the fine manual.

I was very excited to find abLaatex resume template, thats the main reason I want to use LaTex. mBut it didnt make muchnsense to me.

Its not like I cant read instructions or am unwilling to, its that the instructions for some things are so burdensome to read that I often choose to do without.</comment></comment></comment>


MIGS(mastercard gateway) 3D secure security flaw - <AUTHOR> http://tinyhack.com/2017/09/05/mastercard-internet-gateway-service-hashing-design-flaw/ <comment><AUTHOR> MIGS provides the 3D secure authentication for most banks in India.</comment>


How Successful Valedictorians Are After High School - <AUTHOR> http://time.com/money/4779223/valedictorian-success-research-barking-up-wrong/ <comment><USER2> There's a bit of a grades-achievement paradox:

1) Studies show that higher grades lead to higher achievements,[0]

2) Studies of very high achievers show mixed or mediocre grades on average

This article alludes to both of those and then just dismisses the 1st one as not relevant to "true" success, which I think sidesteps a more interesting statistical problem.

Imagine dots plotted along an x-axis of "achievement," and a y-axis of GPA. Can you have a strong positive correlation, but make it so if you sample the far right end of the graph, you get mediocre GPAs?

Sure, there are lots of ways to arrange the data for that effect.

Imagine grades put a floor on your achievement. So you have people with low grades and low achievement, low grades and high achievement, but very rarely high grades and low achievement. So the graph is like a semaphore flag with the lower left corner shaded. There's a correlation, but the right have the graph is still randomly distributed.

Or maybe the correlation is clear, but gradually sloping and the variance is extremely high. So all parts of the graph will tend towards mean GPAs, with the averages creeping up each quadrant.

Or maybe at the very highest levels of success other factors start to dominate (connections, location, luck).

The research gives no real hints as to which of these is true, but I think it's a little early for the article's "just-so" story explanations of high grades leading to docile individuals poorly suited for leadership or greatness. This sounds like the myths about Einstein failed at math in school.[1] There are several millionaires that did fine in school, the data hardly tell us it's impossible to succeed after getting good grades, that's reading too much into the averages. For counterexamples, Gates and Zuckerberg were both admitted to Harvard, ie, probably had above average HS GPAs. Buffett talks up how he was getting low grades at a young age (and stealing!), but then "straightened up" after his dad talked to him, presumably getting better grades in college. Any one of these should be sufficient to disprove the notion that high grades are somehow helpful for most success, but suddenly an obstacle to the highest levels of success.

A more thorough investigation of the 700 surveyed "successful millionaires" would probably reveal more. Maybe a lot of millionaires are harbor pilots, idly drawing an inheritance, or worked their way up from the mailroom, no idea without a closer look.

[0] Aside from passing acknowledgement of this research in the article, it's also covered in more detail here: [https://www.washingtonpost.com/news/wonk/wp/2014/05/20/heres...](https://www.washingtonpost.com/news/wonk/wp/2014/05/20/heres- how-much-your-high-school-grades-predict-how-much-you-make-today/)

[1] [http://www.todayifoundout.com/index.php/2011/12/albert- einst...](http://www.todayifoundout.com/index.php/2011/12/albert-einstein-did- not-fail-at-mathematics-in-school/)</comment>


Take action against Internet restrictions in the Trans-Pacific Partnership - <AUTHOR> http://www.democraticunderground.com/1002678173

<comment><USER2> The worst things about the demise of SOPA is the FUD and scare mongering that it has given way to.

The linked "article" is about the Trans Pacific Partnership. It is a trade regulation not an act of congress. The EFF's page is here:

<https://www.eff.org/issues/tpp>

And the EFF action page is:

[https://action.eff.org/o/9042/p/dia/action/public/?action_KE...](https://action.eff.org/o/9042/p/dia/action/public/?action_KEY=8229)

It is interesting to note that the neither the link nor the EFF's page compare TPP to SOPA. They do compare it to ACTA a trade treaty. It seems the submitter is either unaware of the difference between a trade agreement/treaty/regulation and an act of congress, caught up in the anti-SOPA hysterical rage or just looking for a ticket on the anti-SOPA karma train.

</comment><comment><USER3> This article makes no mention of SOPA. The title doesn't correspond to the article. ACTA is a more appropriate analogue, according to the story.

</comment><comment><USER4> Please, stop with the SOPA sensationalism. Yes, SOPA was a big deal. Yes, more recent attempts to regulate the Internet are also a big deal. But not every new law applying to the Internet is the "new SOPA," and the only outcome of calling it that is sensationalism and a loss of legitimate credibility. Regard all of them on their own terms and make a comparison that is appropriate, if necessary.</comment>


Blame the Weather for Last Year's Rise in U.S. Carbon Dioxide Emissions - <AUTHOR> http://www.technologyreview.com/news/532076/blame-the-weather-for-last-years-rise-in-us-carbon-dioxide-emissions/

<comment><USER2> Cool article it sounds counterintuitive when you're thinking of the term "Global Warming". Unfortunately the majority of people equate "Global Warming" to the Earth becoming "Globally Warmed" which is the opposite of actuality. Global Warming is linked to more extreme weather overall and this article presents an interesting point.</comment>


Show HN: Tools for containerizing external services for dev - <AUTHOR> https://github.com/pinglamb/stocker <comment><USER2> Have you seen DevLab? [https://github.com/TechnologyAdvice/DevLab](https://github.com/TechnologyAdvice/DevLab)

Built it to get around some issues we had with Docker-Compose.

<comment><AUTHOR> wow, that's great. I really like the before-task/after-task concept, which makes that a more complete workflow.</comment></comment>


Spike Lee responds to 'stolen' poster designer - <AUTHOR> http://uk.movies.yahoo.com/spike-lee-responds-to--stolen--posters-designer-072840512.html <comment><AUTHOR> Looks like Spike Lee has just made the mother of all hypocritical "its OK to pirate my movie" responses:

 _"Why Should I Pay Someone Who I Never Met Nor Had Any Contact With Ever? He Never Made Any Deal With Me. Why Don't You Pay Me For Your Stupid Text On Thanksgiving Day?"_

<comment><USER2> So you think the guy's an idiot, but you still want to see his movies?

<comment><USER3> Someone can be both an idiot and a good cook/director/whatever...

Not to mention he doesn't think he's an idiot: just a jackass.

<comment><USER2> Rationalizing piracy is no trivial thing, although it seems to be popular in places like Hacker News.

<comment><USER4> You miss the point. Spike Lee's argument is that he never employed the artist, even though his film company ultimately decided on the agent who promoted the film.

If he complains about piracy, all we have to do now is state on twitter:

"I Never Heard Of This Spike Lee,If He Has A Beef It's Not With Me.I Did Not Buy Him,Do Not Know Him.Cheap Trick Writing To Me.YO"

Looks like, to me, that unless Lee backs down, he'll be being mocked for quite some time to come :-)

<comment><USER2> It is still possible to respect intellectual property rights, authorship, be against piracy, support the allegedly-wronged artist, AND find Spike Lee to be an idiot.

The problem with piracy advocates is that they will seize any opportunity to justify piracy, and this is sadly no different. Too many people unfortunately see this as a black-and-white situation where you either support Spike Lee or support pirating his work.

By all means mock him, that's completely deserved.

<comment><USER4> I'm not saying that you should commit piracy. I'm saying that Spike Lee has put himself in an interesting position where he is now advocating piracy, and this may well be the response he starts getting if he complains his work is stolen in the future.

It's more ironic than right.

P.S. I've voted you up, I agree with you.

<comment><USER2> I am totally on board with you in terms of Spike Lee coming across as a total knob, but, unfortunately, and as evidenced by this thread, there are people who will never fail to latch on to these cases and use strawmen like Spike Lee as an excuse for some eye-for-an-eye Internet justice by means of piracy.

As long as someone can get downvoted to -4 for making that very basic point on Hacker News, the point apparently still has to be made, even though it's a waste of time for people like you who see the nuance of it. :)

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER5> > Why Should I Pay Someone Who I Never Met Nor Had Any Contact With Ever? He > Never Made Any Deal With Me.

So if you use an agent, no responsibility?

Comes off a bit like an "I didn't steal your bike, I'm just riding this nice bike my friend found for me" excuse.

<comment><USER6> Well, it _is_ Hollywood. When it comes to financial scheming they give Wall Street a real run for their money.

</comment></comment><comment><USER7> It's not Spike's issue, but to be blunt this is a really stupid response. If he really doesn't give a damn he could just say we're looking into it, and if he wants to use this as an amazing opportunity for a bunch of free publicity he could roll with this and take advantage.

<comment><USER8> How is it not his issue? You don't get to use something stolen just because someone else stole it for you.

<comment><USER9> If the CMO steals someone's logo, do you blame the CTO?

<comment><USER10> If the CTO is the face of the company, then sure. it's not as if shaming the ad agency is going have nearly the same impact. It's Spike Lee's movie, he's the beneficiary, and he's not some fledgling director with no influence on the studio.

</comment></comment></comment></comment><comment><USER11> I Wonder Why Spike Lee Writes His Tweets With So Many Capitals. Does Everything He Writes Look Like This? Now I'm Imagining His Love Letters.

<comment><USER12> Maybe he thinks everything he says is a headline :P

</comment></comment><comment><USER3> From somebody who hired the crammy agency AND used the end (STOLEN) result in his poster, he comes of as a total jerk.

I don't care if some minion did the bad deed for him. When he employs someone (I'm talking of the agency) he has to man up and take responsibility for what they did in his name, especially if AFTER the wrongdoing happened and was pointed to them, he still profits from it.

I'd sue his ass to oblivion if I was the designer.

</comment><comment><USER13> As a 10 year freelancer, doing a ton of agency work, it's not Spike's issue. His production company hired an agency to do ad promotion for a movie he "helped" make. The design guy was star struck and fairly new in the freelance realm and didn't do a proper job with his business logistics.

It sucks, but it's not Spike Lee's issue. It's like blaming Obama for how a vendor screwed some engineer over while setting up the version control system during the build of the ACA site.

<comment><AUTHOR> The original letter didn't blame Spike, it asked him to intervene. You are definitely correct in the technical sense, but Spike's responses have been extremely tone-deaf. Instead of saying "not my problem" he should have said, "I'll have someone look into it."

<comment><USER14> I admire Spike Lee's movies, but from public appearances, he's never struck me as the sort of person who gives much of a shit about what anybody but himself thinks of anything.

<comment><USER4> I have to ask then, what informs his opinions?

</comment></comment></comment></comment><comment><USER15> Spike Lee is very much the right addressee.

He posted Garcia's original comps (not the final poster) on his Facebook. The images had "©2013 Spike Lee" slapped on them. These images are not in this article, but were on JG's website yesterday.

Garcia kindly asked him for help, presenting both of themselves as victims of a shady middleman, leaving plenty of room for Lee not only to come out on top and save his face, but even to turn this into great publicity for the movie.

Apparently, Lee decided otherwise.

</comment><comment><USER16> Um...here are _our_ top comments on that thread:

"

Seems weird he mentioned Spike Lee (who did nothing wrong, as far as I can tell) so many times, but didn't name the agency. Seems to hurt the wrong party's reputation.

Additionally, he has no apparent way to contact him."

and "... What! I think his post does no such thing. Spike Lee did nothing wrong and the post shows nothing but respect for him." Source: [https://news.ycombinator.com/item?id=6811060](https://news.ycombinator.com/item?id=6811060)

Spike Lee literally just said the same thing we said. but when we said, this doesn't seem to have anything to do with spike lee, it's like, cool. But when he says, "this has nothing to do with me" suddenly he's the bad guy.

Just goes to show, if someone accuses you of something, you should apologize for it!! I guess he should have written "I've never met this person in my life...but I'm really sorry about what happened!"

live and learn...

By the way. If anyone here has any problems with anyone...I'm really sorry!

<comment><USER8> I don't think the top comment says what you think it says. It is possible that:

        1. Spike Lee did nothing wrong, AND
        2. Spike Lee bears responsibility

You can complain to a CEO that one of their employees did something wrong, in the course of the employee's duties. The CEO then bears responsibility, even though they may personally have had no part in it.

</comment></comment><comment><USER17> Ask HN: Why Does Spike Lee Like To Write Like This?

[Hint: Capitalising Every Word In A Sentence Shows Everyone How Cool And Big You Are]

<comment><USER18> buT you'vE neveR seeN iT donE backwardS, righT?

<comment><USER4> My eyes!

</comment></comment></comment><comment><USER19> FYI -- this seems to be a response to the " _Dear Spike Lee_ " posting @ [https://news.ycombinator.com/item?id=6811060](https://news.ycombinator.com/item?id=6811060)

</comment><comment><USER20> Never was much for boycotts, but Spike is making me reconsider.

</comment><comment><USER21> Can someone quote this response for those of us that have an overly protective firewall at work?

<comment><USER12> "I Never Heard Of This Guy Juan Luis Garcia,If He Has A Beef It's Not With Me.I Did Not Hire Him,Do Not Know Him.Cheap Trick Writing To Me.YO"

"Why Should I Pay Someone Who I Never Met Nor Had Any Contact With Ever? He Never Made Any Deal With Me. Why Don't You Pay Me For Your Stupid Text On Thanksgiving Day?"

</comment></comment><comment><USER22> Am I missing something? The two posters don't look much alike, aside from having a fairly similar trunk in the foreground at the same angle - and I'm guessing that this has something to do with the imagery in the movie.

<comment><USER23> Yes, the designer's site is down right now but the issue wasn't that the two posters looked alike. It was that Spike Lee posted the actual, unpaid for comps to his Facebook page. Without a contract or even payment the designer should still have exclusive rights to those designs. The Yahoo news article got that part of the story wrong.

<comment><USER22> Thanks for explaining that. So it's a clear-cut copyright violation.

I hesitate to pile on, but Spike Lee has consistentlty been an ass in public - something I first noticed in an interview many years ago when he was expressing revulsion at the idea that his sister might marry a white man, because black-white marriages were such a terrible idea, in his opinion. Very sad, because I think he's very talented as a filmmaker.

</comment></comment></comment><comment><USER24> The least he could do is look into what happened, he has the power to easily do so.

</comment><comment><USER25> Whatever you might think Spike Lee is, he's certainly predictable.</comment>


Curiosity Rover Finds Rock Type That's Never Been Seen on Mars - <AUTHOR> http://www.wired.com/wiredscience/2012/10/curiosity-strange-matijevic/ <comment><USER2> I'm all for what they are doing. And I'm for releasing articles to try and get press and awareness about the Mars rover and the efforts at NASA. But, yea, its supposed to be discovering new things, and its not like there has been some exhaustive study done on all the rocks of Mars (so I believe). They keep it up with these non-news news releases, people are going to get saturated or just plain fatigued.</comment>


Peter Shor's (Shor's Algorithm) Profile on the English Language Stackexchange - <AUTHOR> http://english.stackexchange.com/users/5754/peter-shor

<comment><AUTHOR> It boggled my mind to see that Shor is an expert in English language and history, in addition to being one of the more important computer scientists of the century.</comment>


Don't be a Glasshole: Use FaceBlock - <AUTHOR> http://face-block.me

<comment><USER2> So to protect my privacy, I should create a algorithmic representation of my face, allowing it to be identified in other people's pictures? And my phone/wearable should broadcast this via bluetooth to every camera in my proximity.

I'm sorry, but this sounds like a huge opportunity for exploitation.

<comment><AUTHOR> You only have to take a picture of yourself using FaceBlock. It will automatically discover Google Glass users near you and share the algorithmic representation (which is also automatically generated) with them.

It is a first step towards protecting personal privacy in public settings. We hope more projects will follow up soon.

<comment><USER3> >>It will automatically discover Google Glass users near you.

Assuming, the Google glass's bluetooth is ON (discoverable). Otherwise it won't work. Right?

</comment></comment></comment><comment><USER4> How creepy would it be if everyone changed their social media avatars to be their eigenface ... I'mma do that for Halloween!

</comment><comment><USER5> Awesome job guys!!</comment>


Show HN: ChartMogul SaaS Metrics API - <AUTHOR> https://chartmogul.com/blog/2015/07/launching-the-chartmogul-metrics-api/

<comment><USER2> This seems to address an area (revenue analytics) that the likes of Mixpanel and Kissmetrics don't do out of box as they don't integrate with payment systems. Its a good starting point. As a next step you may need to cover user metics (invade their territory)</comment>


Kissmetrics has either sold or lost control of their domain, with no warning - <AUTHOR> Precursor: I don't like marketing. Not my choice to use Kissmetrics.
So kissmetrics.com is now the SEO blog of someone called Neil Patel. This was apparently announced yesterday (https://twitter.com/Kissmetrics/status/1012074824611459073), albeit only on Twitter and with no prior warning.
I was assuming their domain has been stolen, as well as access to their Twitter account, as this might be the most stupid thing I've seen a company do in a while.
Should we be pulling the KM integration from our software as a security risk? <comment><USER2> Hi,

Currently pinned tweet ([https://twitter.com/Kissmetrics/status/1012087917727006720](https://twitter.com/Kissmetrics/status/1012087917727006720)) says:

> 1/ _Important Kissmetrics Update_

> We have moved the Kissmetrics website to the Space Pencil domain. Space > Pencil is - and always was - the parent company of Kissmetrics. You can > visit here:

> > [http://www.spacepencil.com/kissmetrics](http://www.spacepencil.com/kissmetrics)

\---

About your security concerns, you better check them with KM. It seems they are just in a middle of an organizational re-structure or something. This doesn't mean that they are hacked or lost control of something.

<comment><AUTHOR> I find it utterly bizarre that they would sell off their primary domain for their software to some shithawk SEO guy. What an incredible way to ruin your credibility with customers :\

Honestly it just looks like they're hoping to rake in some cash by dumping loads of traffic onto their SEO buddy, rather than maintaining their business credibility.

<comment><USER3> It's probably not a security concern yet, but definitely take it as a sign that it's a disfunctional organization. Likewise, stop thinking of Kissmetrics/Space Pencil as a long term partner. The only other times I've seen something similar is when the domain was fairly valuable and was sold to an unrelated company (the lithium php framework selling lithium.com comes to mind)

</comment></comment></comment><comment><USER4> His website is a horror show of dark patterns. I'm not even sure if there _is_ any content buried behind all these popups: [https://imgur.com/a/TII6tpT](https://imgur.com/a/TII6tpT)

</comment><comment><USER5> Neil Patel was one of the founders of Kissmetrics, so at least it's not like they sold off the domain to some random person.</comment>


New in PostgreSQL 10 - <AUTHOR> https://wiki.postgresql.org/wiki/New_in_postgres_10 <comment><USER2> If anyone even remotely involved with the maintenance and development of pg reads this thread - Thank you! - for all your efforts in building and improving a first class product that keeps me amazed at the strides it takes with each major. release.

<comment><USER3> Truly. We've recently moved from Oracle (after using it for 15 years) to Postgresql. It's like a breath of fresh air. The documentation for Postgres is unbelievably superior to Oracle. So far its performance is equal to or better than Oracle. We had to go through and rewrite thousands of queries, but the sql syntax of Postgres was always simpler and more logical than the equivalent in Oracle (I think Oracle has too much baggage from being around too long). All in all I'm so impressed by Postgres. I'm sure there are features in Oracle that Postgres doesn't have that keep people on Oracle, but I would imagine that the vast majority of Oracle installs could be moved to Postgres.

<comment><USER4> Do you plan to write more about the migration, like a blog post? That would be very interesting to read.

<comment><USER5> This needs to become a thing. Where people produce writeups on Oracle To Postgres, and how much better it is; under something like #RunsMuchBetterWithPostgres ...

It is strange that there aren't more writeups on Postgresql migrations.

Yandex had a good one. Posted here a long while ago: [https://news.ycombinator.com/item?id=12489055](https://news.ycombinator.com/item?id=12489055)

<comment><USER6> > It is strange that there aren't more writeups on Postgresql migrations.

It's probably because switching databases is very painful and rare.

<comment><USER5> I would think that not having to pay the Oracle/MS/Sybase/Teradata licenses are a good enough incentive...

Speaking of economic incentives.

That could be a thesis topic for my hereto unwritten PhD in Behavioral Economics:

"Choosing to pay" What are the behavioral aspects of top leadership - Why do people still chose proprietary" A quantitative case study."

<comment><USER7> If I had to wager, it's support. Oracle/MS generally provide much better support that open source software.

<comment><USER8> Oracle is definitely not known for their good support, and there are plenty of companies which provide really good PostgreSQL support.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER9> Just...wow.

* Native partitioning

* Parallel query

Honestly, some very welcome quality of life improvements for use cases even outside of what I would consider "Big Data".

> This means that users no longer need to create triggers for routing data; > it's all handled by the system.

Trigger routing has always been a performance foot gun...to the point that it's sometimes better to handle table routing in application logic. I wonder what the performance is comparatively between this new "native" partitioning and existing methods (triggers and rules) and whether or not this makes application routing always inferior.

<comment><USER10> I worry that having things "all handled by the system" can be even more of a performance footgun. I mean that users will be able to do things that are allowed by the docs, but which result in, expensive and hard-to-see things are happening under the hood.

<comment><USER11> This is a valid concern, I've seen simple automatic optimizations backfire before. An example I remember from MS Sql Server (older versions) of a very simple optimization of using indexes can backfire with the dynamic filter pattern:

where ((@foo is null) or (foo = @foo))

and ((@bar is null) or (bar = @bar))

Depending on statistics, order of execution, indexes in place and phase of the moon this could produce fast results or have to revert to a full table scan (because it uses a cached query plan with the wrong index).

Devs still have some awareness of how queries translate to actual running instructions and to measure that the results are in line with what is expected.

<comment><USER12> Modern pg libraries only cache the plan with the query params involved as well so this generally doesn't happen.

(I made it happen with an older version of things and ... yes, argh)

</comment></comment></comment></comment><comment><USER13> I noticed hash indexes are now crash proof and replicated -- this seems to make them actually usable in production. In other words, this release effectively "adds" a new index type. That seems like a much bigger deal than is being talked about, is there any reason to believe that new databases shouldn't be using hash indexes for columns that won't be supporting range queries? (In other words, pretty much all keys.) I've seen mixed reports about hash indexes being beneficial.

<comment><USER14> I think it's a neat feature. But note that orderedness isn't just useful for range queries, it also helps to satisfy ORDER BY and to allow for merge joins without a sort steps. There's also no yet support for index-only scans (probably never), no constraints, and no multi-column index support.

There's also still some performance kinks to work out with the current hash index performance - large when growing the index quickly or under very high concurrency.

<comment><USER15> Stop it! You're giving away all possible answers to one of my favorite interview questions.

 _Edit: not seriously asking you to stop, but the interview question part is true_

<comment><USER16> I hope those are interview questions for dbas because it's way outside common knowledge for backend developers.

<comment><USER17> Why are back-end developers not supposed to have a deep understanding of their tools? I hear this sort of statement all the time, it's infuriating. I once even had a dev team lead claim that it was not a dev's responsibility at all if the app generates bad SQL; the DBA is supposed to make the queries fast (this person no longer works for me, but I consistently hear similar claims from consultants and others in the dev community).

DBAs are expected to understand the crappy inefficient row-at-a-time ORM in use by the devs, which implies understanding a particular programming language, but not the other way around? There's nothing magical about SQL databases that a decent developer can't understand.

<comment><USER18> > Why are back-end developers not supposed to have a deep understanding of > their tools?

You mean all their tools? Like how does modern processors handle branch predictions in the face of out of order operations, or what differences in server cooling will affect throughput or peak performance?

Do you have a deep understanding of all the tools you use? Because I honestly doubt so.

</comment></comment></comment></comment></comment></comment><comment><USER19> > ICU Collation Support

This tiny little mention is a _massive_ win! We now have access to the "und-x- icu" collation. As the postgres docs[1] put it: "ICU root collation. Use this to get a reasonable language-agnostic sort order."

I've been waiting on this _forever_. Columns containing strings from any/all languages can now be sorted well enough to use postgres in truly multi- language applications. If your application has users writing in English, German, Russian, Japanese, et al, using the "und-x-icu" collation should make your application much more user-friendly (as opposed to using the en_US or C locale).

The ICU root collation is far from a perfect system, but it's the best option available today. I've been tied to MySQL purely for its similarly-purposed utf8mb4_unicode_ci collation. Now I can give postgres another very serious look.

[1] [https://www.postgresql.org/docs/10/static/collation.html](https://www.postgresql.org/docs/10/static/collation.html)

</comment><comment><USER20> PgSQL more and more looks like the "redis of databases". Whatever the problem, you can almost always find a good reason to use it :)

You get great performance for the simple "dumb" use case (relational data in DB term, key/value store in redis case), and lots of awesome additional feature on top of it for more complex situations.

I love it.

<comment><USER21> > the "redis of databases"

That's like calling Swift "the Ruby of programming languages".

<comment><USER22> and it says more about redis and ruby than postgres and swift :p

</comment></comment></comment><comment><USER23> Floating point timestamps - the only item which affects my work is also one of the few in this wiki page with no description. These were removed with no breaking compatibility.

For anyone else using them, I just did a bit of reading [1] and it looks like they are inferior to default 8 byte timestamps. FP timestamps have microsecond precision around the year 2000, but it drops off as it moves away from there.

I was using them to be the same as some GPS timing code, but it seems this isn't needed or desirable.

[1] [https://www.postgresql.org/docs/9.0/static/datatype- datetime...](https://www.postgresql.org/docs/9.0/static/datatype- datetime.html#DATATYPE-DATETIME-TABLE)

<comment><USER24> Yeah.. You should never use floating point when your use case wants uniform precision across the full range. That's the opposite of what floating point gives you.

</comment></comment><comment><USER25> > Cross-column Statistics

Holy crap I didn't even know this was feasible much less in development! Can't wait to test this out, as significant amount of my data sets have these kinds of relationships.

Parallel query will also be great for certain queries I do regularly. Been looking forward to this!

Thank you Thank you to all the devs that had a part of this!

<comment><USER26> Cross-column statistics are one of my favorites. The way Postgres 10 handles it is really a _huge_ advancement that addresses a painful edge case that has been around for ages.

[https://blog.2ndquadrant.com/pg-phriday-crazy-correlated- col...](https://blog.2ndquadrant.com/pg-phriday-crazy-correlated-column- crusade/)

<comment><USER27> To be honest, skewed data problem can be addressed with partitioning in many cases.

Cross-column statistics is a great feature nevertheless.

<comment><USER28> Yes, but partitioning also has a number of limitations that may be quite undesirable. Like inability to create UNIQUE constraints or primary keys, for example.

<comment><USER14> More importantly it doesn't help at all with cross-column correlations. It can help with skew in between various rows though.

</comment></comment></comment></comment></comment><comment><USER29> Full text search in JSON & JSONB looks exciting. [https://wiki.postgresql.org/wiki/New_in_postgres_10#Full_Tex...](https://wiki.postgresql.org/wiki/New_in_postgres_10#Full_Text_Search_support_for_JSON_and_JSONB)

<comment><USER30> As someone working with json output from the gmail api I'm curious to see how people smarter than me take advantage of this new functionality so I can adopt it as well.

</comment></comment><comment><USER29> SCRAM authentication looks like a nice security improvement also. No description on the site but [http://paquier.xyz/postgresql-2/postgres-10-scram- authentica...](http://paquier.xyz/postgresql-2/postgres-10-scram- authentication/) gives a good overview.

<comment><USER31> It's an improvement for sure, but I am curious -- does anyone situate a Postgres instance where it is publicly accessible? Who was asking for this feature?

<comment><USER32> Lots of cloud managed dbs are/can be publicly accessible. Eg all heroku hosted Postgres instances

<comment><USER33> are they? even without ssl? by default?

<comment><USER34> Not sure about SSL, but in the past customers of mine have copy-pasted full Heroku PG URLs to me and I was able to get in via `psql` immediately.

So yes they're public but their addresses are basically impossible to guess.

<comment><USER35> Known as "Security through Obscurity" [0]

[0] [https://en.wikipedia.org/wiki/Security_through_obscurity](https://en.wikipedia.org/wiki/Security_through_obscurity)

<comment><USER36> Debatable. If the address is really unguessable, the address acts like a regular key.

It's still not good practice, since most systems treat addresses with far less care than passwords and often save and/or transmit them unencrypted.

</comment></comment></comment></comment></comment></comment></comment><comment><USER37> For anyone using postgres on a daily basis I would highly recommend pgcli. It has a very good auto-complete, and can even suggest JOIN statements (parts of it actually).

<comment><USER38> do you find that it's significantly slower than just using psql?

<comment><USER39> Yes, in many cases (until I stopped using it), it messed up my benchmarks. I was testing some queries, and often it was: \- pgserver fast, returns 10000 rows, pgcli takes time to parse -> result: total time 10s \- pgserver slow, returns 1 row, pgcli instant -> result: total time 10 sec.

As we are working in efficient C/C++, our internal time to parse the query set is closer to psql than pgcli, so we tend to prefer the pgserver fast, but the benchmark using pgcli did not clearly show the best time.

Also, when you are testing a query on one machine, you have to carefully monitor the CPU usage to differentiate the CPU pegging by postgres from the one by pgcli.

All those issues were gone when I got back to standard psql.

<comment><USER40> For bench marking and optimising queries you almost certainly want `EXPLAIN ANALYZE`. That will give you what you want about what Postgres thinks and the time actually taken for the query.

<comment><USER6> `EXPLAIN ANALYZE` - I read that in a Dalek's voice.

</comment></comment></comment></comment></comment><comment><USER41> Postgresql keeps getting better and better everytime! It's amazing the type of features you can pull off once you have a solid foundation :) (P.S. Have been using MySQL in past and MySQL now feels like stoneage database)

</comment><comment><USER42> What is the current best option for Postgres failover? I looked at this at the start of the year, and found lots of options, but all of them seemed to have various drawbacks, and none were natively supported or built-in to Postgres.

<comment><USER43> I would recommend repmgr[1] over solutions like DRBD suggested. The simple reason is that DRBD replicates on block level and doesn't understand postgres data structures, while repmgr relies on mechanisms provided by Postgres.

So if there is a failure, you're far less likely to learn that all your data is gone, because some important block on disk was not replicated and the database is not readable. With repmgr the worst what could happen is that you might lost few latest transactions, but your data should always be consistent and in working state.

[1] [http://repmgr.org/](http://repmgr.org/)

<comment><USER44> Repmgr is relatively easy to setup and supports automatic failover with repmgrd. Resyncing a failed node is also just a few commands. The only limitation I have encountered was that I had to run it behind pgbouncer so the clients can always connect to the primary instance. I think that will change with 10 as it will support client side discovery of the primary node.

</comment></comment></comment><comment><USER45> Shouldn't that be PostgreSQL X?

Seriously good work. I'm not sure why my firm still buys Oracle licenses.

<comment><USER46> Well, this Postgres release introduces automatic partitioning; in Oracle, it has been available for like 20 years.

Also, some shops are heavily invested in PL/SQL code.

The price of migration may far outweigh the license savings.

<comment><USER43> My previous company migrated to Postgres by using EnterpriseDB[1]. This allowed them to run existing Oracle data with PL/SQL as they were working on migrating it to the raw PostgreSQL. So where there is a will there's a way.

[1] [https://www.enterprisedb.com](https://www.enterprisedb.com)

</comment></comment></comment><comment><USER47> Postgres isn't just my favorite database. Postgres is an example for all people about how a project should be run. There are lots of stakeholders who want different things. The dev team looks at what's being asked for and what is reasonable within a certain timeline. And then the team delivers features and supports them. And then on top of that there is best-in-class documentation.

Thank you to all of the team members who put your time and effort into this project. It's not only a wonderful tool that I use every day, it's also a model of how to manage a project.

<comment><USER48> "best-in-class documentation"

?? what am I missing. I dread having to lookup how to do things in postgres because their documentation is so horrible to find things in.

<comment><USER49> Can you provide an example? In my experience I find the index[0] particularly useful, as well as the SQL command reference[1], the datatype reference[2], the function reference[3], and the server configuration section[4]. I know other people have different usage patterns and different amounts of familiarity with the docs, so I'm aware that my experience may not match yours.

[0]: [https://www.postgresql.org/docs/current/static/bookindex.htm...](https://www.postgresql.org/docs/current/static/bookindex.html)

[1]: [https://www.postgresql.org/docs/current/static/sql- commands....](https://www.postgresql.org/docs/current/static/sql- commands.html)

[2]: [https://www.postgresql.org/docs/current/static/datatype.html](https://www.postgresql.org/docs/current/static/datatype.html)

[3]: [https://www.postgresql.org/docs/current/static/functions.htm...](https://www.postgresql.org/docs/current/static/functions.html)

[4]: [https://www.postgresql.org/docs/current/static/runtime- confi...](https://www.postgresql.org/docs/current/static/runtime-config.html)

</comment></comment></comment><comment><USER50> SQLSERVER admin and dev here: these features look really compelling. Though the best part is a team of devs could run these in production and do some self support and basically pay nothing. Not so in SQLSERVER land.

</comment><comment><USER51> Does anyone have any use cases where PostgreSQL falls down/loses to other DB systems? I know sharding/replication has long been a sticking point, but what else is there?

Why do people still choose MySQL/MariaDB/Oracle over PostgreSQL at all?

<comment><USER52> > Why do people still choose MySQL/MariaDB/Oracle over PostgreSQL at all?

Oracle: because the enterprise is its own unique beast. Companies frequently stick to what they know and have invested into, so long as it still works. Enterprise customers buy/adopt software in a very different way than everyone else.

MySQL: if you've got a history with the product and it works well for what you're doing, there is absolutely no necessity-based reason to switch to PostgreSQL. You should switch from MySQL to PostgreSQL if there's a very good reason/s to, a meaningful gain to be had, it's that simple. Otherwise, it's unnecessary optimization. There are a vast number of considerations when operating a business, upgrading to the latest & greatest (speaking very broadly) just to do it, is not a good reason most of the time. There should be a specific prompt to action based on a gain/s or capability that PostgreSQL provides you, in/for your use case, over eg MySQL.

<comment><USER53> Aside from those reasons, are there technical reasons for Oracle over Postgres? My coworkers and I want to move our data warehouse over to Postgres and my boss has been warming up to the idea. (I know switching databases with that much data can be complex and troublesome, but those concerns are outside the scope of this question)

<comment><USER52> Oracle is still a _very_ good option for data warehousing. When well tuned it's extremely performant and reliable for that purpose. If I were operating an EDW today, I'd choose Oracle over Postgres 10 times out of 10 when it comes to very large scale operations. You'll still see plenty of credible arguments around that Oracle can be optimized to be faster than every major alternative. And you'll still see arguments around replication that are legitimate.

Obviously I don't know what your cost or performance considerations are. If your organization can save a meaningful amount of money and operational headache by switching to Postgres, that can be a big deal for any business. If cost isn't much of a consideration in your operation (that is, what sticking with Oracle is costing you), stick to Oracle for data warehousing over Postgres - at least for now imo.

With Oracle it's the same story it has been for a long time (and probably will remain), it's expensive and a pain in the ass to get up and running the way you want it. Postgres has everything to gain vs Oracle from here forward, it seems very likely to bite into some of Oracle's well guarded high value territory in the coming years.

</comment></comment></comment></comment><comment><USER54> Does anyone know whatever happened to the columnar storage work that 2nd quadrant posted on their blog: [https://blog.2ndquadrant.com/postgresql-10-roadmap/](https://blog.2ndquadrant.com/postgresql-10-roadmap/)

Is this essentially a pipe dream?

<comment><USER55> Not a priority for 2Q at the moment. Still general interest in the community, but hard to predict right now.

</comment></comment><comment><USER56> How does PostgreSQL 10 compare with Cassandra for BigData requirements?

<comment><USER57> Define "big data"? You can buy normal tier 1 server vendor hardware with 6T of memory.

If it fits into 1 machine or a few, it isn't "big data".

<comment><USER58> I've heard this 6TB number thrown around a bit recently, but no one can actually provide me with links to hardware I can buy - has anyone successfully setup a machine with this much ram who can post links to what they used?

<comment><USER59> Don't know about hardware you can buy, but AWS will rent you a machine with 3TB Ram by the hour :) [https://aws.amazon.com/ec2/instance- types/](https://aws.amazon.com/ec2/instance-types/)

</comment></comment></comment></comment><comment><USER60> Very exciting. Still badly miss (bi-)temporal tables and native query support [1] without using extensions [2].

[1][https://docs.microsoft.com/en-us/sql/relational- databases/ta...](https://docs.microsoft.com/en-us/sql/relational- databases/tables/temporal-tables)

[2][https://pgxn.org/dist/temporal_tables/](https://pgxn.org/dist/temporal_tables/)

</comment><comment><USER61> Does anyone know if the partitioning can enforce a uniqueness constraint across child tables?

I couldn't tell from the write up if this is one of the things addressed with Native Partitioning

<comment><USER62> No it can't currently. There has been some discussion on how to do that for pg 11 though.

</comment></comment><comment><USER63> To those working on and contributing to the goat DB PostreSQL thanks a lot

</comment><comment><USER64> Every time I see stuff about Postgress it sounds so much better than MySQL.

Every time I get a new job, the existing system is using MySQL.(Every time it causes problems of some sort of other - it's not a bad database, but has too many quirks like JavaScript that will end catching you out at some point).

Last time I got to build something new the time pressure was so great I had to use MySQL, because it was there, set up and working and I knew it. I wanted to use Postgres, but at the end of the day the extra time setting it up wouldn't really have been much benefit as it was a fairly standard Django app, and didn't need any of Postgres's extra features. (I quit that job shortly afterwards).

</comment><comment><USER65> What happens if I try to insert into a master partition table, and there are no child tables that can store my data?

For example, data is partitioned by month, and I have no table for the month in the data I'm trying to insert.

<comment><USER8> It will throw an error. Partitioning support in PostgreSQL 10 only solves a few simple use cases well, but the next version should get much more generally useful partitioning. This is how PostgreSQL usually releases major features.

</comment></comment><comment><USER66> I recently started using PostgreSQL for a project and have been pleasantly surprised on more than one occasion. I also have been hearing more and more people speak highly of it, especially for its security.

</comment><comment><USER67> Can anyone recommend a good resource for learning PostresQL? I have a decent understanding of SQL, I've used MySQL in the past, but every time I see the latest features of Postres I'm left thinking I'll need to buy a book and take a pretty deep dive to figure out how to apply the new features.

<comment><USER49> You'll likely find the PostgreSQL docs[0] very useful. They're organized differently than the MySQL reference making it easier (in my experience) for reading stand-alone.† They're quite well-written and up-to-date.

[0]: [https://www.postgresql.org/docs/](https://www.postgresql.org/docs/)

† The MySQL reference is very useful particularly when you're looking to compare feature differences between versions.

</comment></comment><comment><USER68> Completely off topic... but did anybody notice how they used partitioning example column names were in Spanish?

</comment><comment><USER69> What I miss the most from Postgree is the compression feature in the same level as RocksDB or Tokudb. The actual compression is very bad compared to them.

Do anyone know if they have something planned in this department ?

<comment><USER70> Advanced compression of data is really something that should be looked at separately from database design. Look into using ZFS beneath Postgres for some pretty good compression.

</comment></comment><comment><USER71> Would love to know about FDW push down. That could be really good for specialized applications where you want to use a relational db and wrap a specialized data store outside postgres with a FDW.

</comment><comment><USER72> The only thing I'm upset about is the fact I'll need to wait 3-6 months after it's released for AWS RDS to support it...

</comment><comment><USER73> Can the xml table accept any xml document? What if it's a bunch of unrelated elements?

Can you get back to xml from the table?

<comment><USER55> Any XML document that you can address via XPath in a sensible way. If your XML document is a mess, you'll only be able to get out a mess. :)

To get XML from a table, functions already existed in previous releases (XMLELEMENT etc.).

</comment></comment><comment><USER73> What does native partitioning do?

<comment><USER8> It basically just adds a convenient syntax for creating partitions plus adds automatic tuple routing (routing inserts on the partitioned table to the right partition). This is for most users not a huge improvement over using PostgreSQL's table inheritance to implement partitioning, but it is a very important stepping stone for implementing competitive partitioning in PostgreQL. There are lots of promising patches in the PostgreSQL 11 release cycle which improve on the native partitioning, making it useful for more users.

<comment><USER73> Thanks. I guess I meant to ask what a partition is used for? When is it useful to use one and why?

<comment><USER8> Oh, you meant partitioning in general. It is useful when you have huge tables with many rows, and works by splitting a table into multiple smaller tables which acts like it was one big table to the outside. Some advantages:

\- You can partition by for example month and cheaply delete or archive old data by instead of deleting all rows just dropping the entire partition which is much cheaper.

\- You can have different indexes on different partitions. This is commonly used when some indexes are only used by queries which operate on recent data.

\- You can put different partitions and their indexes in different table spaces. E.g. you can put your older and less frequently accessed data on cheaper and slower disks.

\- Random inserts into B-tree indexes are expensive and when you for example have time series data on the format (device_id, timestamp, v1, v2, ...) it can be useful to partition on the device id so all rows are inserted in strictly increasing timestamp order in each partition. This way you can get more write performance out of your hardware.

\- Most databases have a maximum table size (PostgreSQL's maximum is 32 TB) and you can get around that by splitting your huge table into multiple smaller partitions which all fit in the maximum table size.

\- The partitioning condition can be used as a very coarse index to entirely skip looking at some of the partitions (even at plan time for some queries). This does not always give you much over if you just had had one big table with one index over all of it, but in some cases this can be a big win.

\- For PostgreSQL specifically it looks like you can have some of your partitions be foreign tables. Meaning some of your data can be on an entirely different server, not necessarily PostgreSQL or even a relational database system at all. I am not sure how useful this will be in practice.

Partitioning has a maintenance overhead and adds a bit of extra work to query execution and planning so it is not always worth it.

</comment></comment></comment></comment><comment><USER74> How does the native partitioning compare to something like Timescale's hypertables?

</comment><comment><USER75> Hey, HN! Would you recommend me a practical introduction to data modelling with Postres?

<comment><USER12> Learn normalisation from wikipedia, then read "The Art of SQL" to get a feeling for the shape of the weirdnesses, then read the postgresql documentation from end to end.

(disclaimer: worked for me, may not for anybody else)

<comment><USER76> I intend to read the PostgreSQL docs end to end at some point, but last time I checked the full PDF comes out to something like 3600 pages! I am astounded by the quality of everything about PostgreSQL, the community included.

</comment></comment></comment><comment><USER77> Nice !</comment>


The Infinite Cat Project - <AUTHOR> http://www.infinitecat.com/infinite/cat-html/1-100/1.html

<comment><USER2> Spoiler: for small values of _infinite_ apparently.. 1720 in this case</comment>


Ask HN: Cost of a daily newsletter - <AUTHOR> Given 1000+ subscribers, how much will the newsletter cost (sent daily, provider agnostic)? As subscriber base grows, can I take advantage of economies of scale? Can I do it for free, using Python? <comment><USER2> Hi,

I suggest to use MailChimp, I launch my own[1] recenly and it's very reliable. As long as the list are under 2000 subscribers, it's free to send under 12000 emails.

If you go beyond that, then it means you should have some revenus from sponsor to cover the cost of MailChimp. MailChimp is very reliable and easier to get started IMHO. They can easily import email from a given URL.

\---

[1] [https://betterdev.links](https://betterdev.links)

</comment><comment><USER3> > Given 1000+ subscribers, how much will the newsletter cost (sent daily, > provider agnostic)?

You can check MailChimp or Amazon SES pricing for instance:

[https://mailchimp.com/](https://mailchimp.com/)

[https://aws.amazon.com/ses/](https://aws.amazon.com/ses/)

> As subscriber base grows, can I take advantage of economies of scale?

30,000 / month is not enough to get any real advantage, but if you grow large and going to have reputation of your domain then you can certainly start saving. When you already have working solution and send million of emails each month it's easier to setup your own infrastructure that going to be cheaper.

> Can I do it for free, using Python?

Unfortunately 1000 daily emails is way above the line where you can use some public SMTP provider and expect no trouble. You can of course try to set your own server, but that would mean that some of subscribers certainly going to miss your mail for quite some time and it's doesn't scale too well.

Gmail / G Suite have usual limit of 2000 messages per day and you can also try other large providers like russian Yandex, but they also have about 2000 messages limit. I used both GMail and Yandex on multiple small projects for notification / email verification and never had issues, but I only ever sent below 500 messages / day.

If you still decide to try run your own server or use public SMTP make sure to use and follow best practices before you start sending anything: sent from your project public domain, add mandatory unsubscribe link and headers, only add links to trusted domains and avoid link shorteners, make sure that you handle rejected mail and not sent anything to deleted accounts, etc. So it's would take weeks of work at best to setup it properly.</comment>


Mousers vs. Keyboardists - <AUTHOR> http://whydoeseverythingsuck.com/2008/08/mousers-vs-keyboardists.html

<comment><USER2> Keyboards aren't just about memory, they are a lot slower then the mouse if you don't have almost instinctive muscle memory. If you have to think what the key combo is, use the mouse.

Every so often I'll start thinking that maybe a keyboard + mouse combo is the fastest most efficient way to go... on Windows.

And then I see a windows developer for whom the keyboard has become an extension of his hands and he never touches the mouse. It's crushing to see that kind of speed and power.

The speed and efficiency of the keyboard isn't just a lot faster, it is so much faster that it changes the way you work.

But it's hard to get there, you have to find the commands, practice them, practice, practice, and all the while it would be easier to just reach for the mouse.

And yet I've made it a goal of mine to always get better at using the keyboard. A little bit a time, I figure I'll have no need for a mouse when I'm 80 or so.

<comment><USER3> _Keyboards aren't just about memory, they are a lot slower then the mouse if you don't have almost instinctive muscle memory._

Muscle memory is always instinctive, it's hardwired in the human brain. Everybody can teach his/her muscles to act "on their own" for a task as simple as this without that much practice. (Remapping your keys in your keyboard helps a lot for comfort.)

 _But it's hard to get there, you have to find the commands, practice them, practice, practice, and all the while it would be easier to just reach for the mouse._

I think the trick is to forget about "practising" and just do it. If the command is really helpful, you'll end up using a lot anyway, but the mindset is different (forced practise vs need).

<comment><USER4> _Muscle memory is always instinctive, it's hardwired in the human brain._

Indeed. It's like walking. You don't think you are walking, you just walk.

 _Everybody can teach his/her muscles to act "on their own" for a task as simple as this without that much practice._

I'm not sure how you can teach one's muscles to act "on their own" without too much practice. Maybe if one were trained by injection, Matrix style. :)

<comment><USER5> I remember when I was jockeying my Dad's Apple II+ in grade school. I got to typing "catalog" very quickly. So quickly, that I doubt I was conscious of placing each individual finger.

I wonder if mouse + keyboard input could be augmented by head gestures? NaturalPoint sells an economic mouse that can be operated by placing a reflective dot on a pair of glasses or a hat. What if instead of being a mouse, this could be an entirely alternate input device? You could read head gestures in the same way that the Wiimote detects different movements. These gestures would likely become "muscle memory."

<comment><USER6> One thing that is pretty annoying about GUIs is the fuzzy factor: there are often a few pixels that _could_ be the button, and _could_ be the background, but you can't tell which it is for sure -- so they're effectively off limits. A keyboard doesn't have this issue, so it's a lot easier to turn commands into reflexes -- your reflexes are less likely to be wrong.

This is also why I switched from T9 to whatever the other alphabet-entering system is for cell phones: it was just too frustrating not to know exactly what the output would be if I typed quickly.

</comment></comment></comment></comment></comment><comment><USER7> The difference between keyboard friendliness on OSX and Windows has always been fascinating to me. On OSX, even being able to tab between buttons in a modal dialogue is off-by-default. Opening folders in the Finder is cmd+O instead of just enter. I still have no idea to this day how to access the menu bar from the keyboard.

Meanwhile on Windows, I spend 99% of my time on the keyboard, since both Explorer, Visual Studio, and all the other apps I use couldn't possibly be friendlier to shortcut keys. The consistency gets better with every release as well.

<comment><USER8> Maybe helpful info:

With dialog boxes on OS X you tap Return to use the highlighted option and Esc to select Cancel. (Notice how the keys' left-right position on the keyboard matches the dialog?)

For the common "Don't Save | Cancel | Save..." dialog you can type Command-D to choose Don't Save.

To get to the menu bar type Ctrl-F2.

I think OS X definitely directs you toward using the mouse, but I've found that keyboard shortcuts do exist for most everything.

</comment></comment><comment><USER9> The real problem with keyboarding is the TERRIBLE lack of consistency. Apart from a very few commands, most applications seem to just randomly select keyboard letters to map to tasks based on the first letter of the english word. When I switch to a german keyboard, the key stroke commands change again! I use my apple, and things are different again.

It does not work. A UI has to be consistent, and keyboard commands are extremely inconsistent. Look at the vim style commands and compare them to the windows style commands.

And another problem with keyboard commands is that there is no hint. How many people have gotten stuck in vim because the interface is not acting as expected?

Using a gui to shut down an App, I look for an X and press it. Using a keyboard, it could be ctrl+q, ctrl+c, ESC, :q!, Alt+F4, Alt+B (German Systems), Alt+Q (English Systems) or exit().

It's TERRIBLE user interface design to have so many different ways of performing the same action on the same operating system.

F1 works because it's always Help. The arrow keys work because they always mean the same thing. That's how keystroke commands should work: One combination, one meaning - all the time and everywhere! So when you learn that stuff, you know it.

I don't want to invest my time in learning app specific shortcuts for applications that I don't use that often. And that are liable to change anytime.

The state of keyboard control of software is broken, and has been broken for a long time.

My suggestion for the fix - an OLED keyboard that switches to "command mode" when a button is pressed. So the commands are always in the same position with all apps, all layouts, all languages and stay consistent.

<comment><USER10> That's nothing to do with the keyboard and everything to do with bad UI.

I could just as easily come up with a poorly thought out rant about how my options panel moves from "tools->options" on Windows to "edit->preferences" on Linux, how tab close buttons are placed wherever the user feels like, how tooltips are haphazardly added to an app, how sometimes the close button exits the app, other times it minimizes to the system tray, etc.

A badly designed user interface sucks. There are lots of them out there.

<comment><USER9> So which of the examples above (with the exit keystroke) is well designed, and which is not?

<comment><USER10> <rhetorical question> So which of the tab button locations is well designed, and which is not? </rhetorical question>

It doesn't matter _which_ one is used, so long as the application sticks to the platform conventions, and for the actions that don't have platform conventions, picks bindings that don't need you to be a finger contortionist.

It doesn't matter if the 'ok' button is to the left or right of the cancel button, but if a UI designer swaps them around relative to what the platform normally does, he's just designed the UI poorly. If 'q' exits a command line app normally, but the developer makes 'esc' exit his app, then you just made a poorly designed UI.

</comment></comment></comment></comment><comment><USER11> I was working on a project with another programmer, and there was this little "test" window and an icon that only showed up when you alt-tabbed to our app (it is a system-tray-only app). I simply assumed that the other guy knew about it and was going to take it out before release. Well, it stayed there for a long time, and the release got closer and closer. It finally got entered as a bug, and I asked the other developer "so you weren't going to remove it?"

He said "I didn't know it was there. I don't really use alt-tab."

</comment><comment><USER12> Working in financial models all day in Excel, I can't imagine having to use the mouse for any more than 10% of my work. Keyboard shortcuts are so much faster that they basically render a mouse useless.

It takes some getting used to and a bit of practice, but once you get comfortable with the keyboard, there's no turning back. I really only use the mouse for casual web browsing and solitaire.

</comment><comment><USER13> If you can't remember keyboard commands, programming idioms, etc. will be a bear.

</comment><comment><USER14> Where do mouse gestures fit in? It seems like they're in the "remembering lots of stuff" category (though there is less possible variation to the mouse signal, it only has so many buttons and relative movement/timing).

</comment><comment><USER15> Oddly enough, I find that I have to put my hands on the keyboard to figure out what a particular emacs command is. My muscle memory has outlasted my 'brain' memory.

</comment><comment><USER16> Thanks, Hank, for bringing up a very important topic that we don't talk about enough.

Interestingly, of all the things that retard adoption of new technologies, I think that the mouse is still the biggest one.

I have customers with call centers and power users still on green screens and command lines. The switch to GUI has always been too expensive. When you have 4000 users, the cost of average call times going from 2 to 4 minutes adds up quickly.

Make no mistake about it, for repetitve tasks, the keyboard is always faster than the mouse. "Alt" and "Ctrl" commands are cool and geeky, but like Hank says, who can remember them all?

We need web apps with intuitive keyboard interfaces. Fkeys (they're on every keyboard) and single letter commands are easily adoptable. (Oh, you mean "C" brings up the Customer record, cool.)

The ability to do this is as old as javascript. Once we developers take the GUI pain away from the power users, the big migrations can continue.

<comment><USER17> _Fkeys (they're on every keyboard) and single letter commands are easily adoptable._

I dig this one. On my mac, I always set it to let the F keys behave normally. I hit fn + an F key to change volume, etc.

I map all my f keys in emacs to something useful and save plenty of time because of it.

</comment></comment><comment><USER18> The My Favorite Text Editor debate might change its spots, but it just never gets old.

</comment><comment><USER19> as expertise goes, keyboard > mouse

</comment><comment><USER20> I tend to go to the mouse as soon as I have to work with more than one window, but I think keyboarding wins long-term.

In my opinion, the advantage of keyboarding is not due to its being faster, but the fact that, once learned, it doesn't break flow. The time lost due to mousing is trivial, but the compromise to flow is significant, at least for me. This is why hackers and gamers alike prefer to keyboard.

<comment><USER21> "I tend to go to the mouse as soon as I have to work with more than one window"

The particular window manager you're using might have something to do with that. I'm looking forward to upgrading to a tiling window manager, myself. When I'm done procrastinating.</comment></comment>


The world's worst toaster - <AUTHOR> http://sethgodin.typepad.com/seths_blog/2008/03/the-worlds-wors.html <comment><USER2> The GE microwave that came with my house is worse than I thought was possible. It includes an appointment scheduler (!) and has some sort of message button I've never pressed for fear of being reminded of what my voice sounds like. Apparently the designers thought there was a large underserved market of families whose entire lives revolve around their microwaves. All the buttons to support these extra features mainly make it less convenient to use for heating up food.

The worst feature by far is its insistence on having its clock set. Every time I lose electricity, which happens every few weeks in my neighborhood, it refuses to cook anything until I:

      1. Press "clock" to set the time (even though all other functions are disabled).
      2. Enter the time in HHMM format.
      3. Press "start" (but don't get too excited about actually "starting" anything yet).
      4. Locate the single-purpose "AM/PM" button and press that either once or twice.
      5. Press "start" again.
      6. Enter the date in MMDDYY format.
      7. And again press "start".

What this means is that you can always calculate how long it has been since the first time I zapped my morning instant oatmeal after the last power outage by subtracting from the current time and date displayed by the microwave: 2011-11-11 11:11 a.m.

So, yeah, don't design a web site like that.

<comment><USER3> If it's a toaster, it probably can run NetBSD.

Since you don't report any crashes, it's a plausible idea.

I don't think it's possible to design a webite as cruel as this microwave.

<comment><USER4> You, sir, have clearly never dealt with web-based course management software :).

</comment></comment></comment><comment><USER5> As a counter-point, I have the world's best microwave. There is only one button that I use - the add 1 minute button.

Put the food in, hit the "add one" button once or twice, and that is it. The microwave starts automatically. As far as I can tell, it took 40 years of microwave UI development to get to this point.

<comment><USER6> I had a microwave once that only had one nob that you turned and that was it. I've never seen another setup that was that simple and effective on a microwave.

<comment><USER4> That's how my current microwave is, and it's great. Not only that, but the numbers on the knob no longer have any bearing on actual time increments, so I don't have to think at all. Just turn the knob an arbitrary amount, and hopefully I'll take it out on time.

Brilliant.

</comment></comment></comment><comment><USER7> So was this "star" marketer fooled by marketing? "It's pretty fancy and shiny and microprocessor controlled. And it makes toast." Shouldn't the criteria have been related to function: like 'I only have to insert toast and remove toast... it's one step shorter than old toasters'.

As an aside, I'd love to post this on his blog, but that doesn't appear to be an option. Is accepting critical feedback in an open forum a risk not worth taking when marketing oneself?

</comment><comment><USER8> I guess it's to prevent phishing. If you have to take all the steps yourself instead of clicking on a link from the e-mail, the chances of someone being able to hijack a link are smaller.

If it's always (1) click link, (2) enter credit card details. You are teaching your users to become victims of fraud.

</comment><comment><USER9> Irony is that company that sold that Toaster made money. How ironic (and encouraging) is the fact that companies like eBay, Microsoft makes loads of money ;-) Feature-laden products sell. At the day end, we all want to acquire powerful things and you'll find very few _powerful_ products in this world that were made keeping in mind the _less is more_ philosophy.

</comment><comment><USER10> Reminds me of the Plantronics desk phone headsets. I don't know where they get the engineers to design these. Just a few examples of what's wrong:

\- the base station has a "configuration dial" that requires a flat object to insert into the slit to turn it

\- There are 2, yes 2, pairs of volume buttons on the base itself, AND one on the headset..

\- The mute button on the headset has a sharp protrusion that literally pokes you in the finger when you try to push it..

Seth's post and experiences with such systems make me appreciate sites like tumblr and slinkset quite a lot.

PDF user guide: [http://www.plantronics.com/media/media_resources/literature/...](http://www.plantronics.com/media/media_resources/literature/user_guides/Voyager510S_userguide_en.pdf)).

</comment><comment><USER11> There are plenty of bad toasters out there. Many products are designed (destroyed) as this toaster was with lots of extra super duper features in mind, thinking that users absolutely have to have the ability to do X...and y, a,b,and c.

Then they hit you with all the options all at once. Yes build in that must have functionality but don't forsake intuitiveness. Don't mistake functionality with usability.

By making a simple task more complex you do not necessarily increase customer safety (or satisfaction) and as henning notes so aptly, "Making it easy for people to give you more money on a regular basis is a good thing."

In my experience air travel is more akin to the 11 click ebay example than a 2 step toaster.

</comment><comment><USER12> Man, Seth Godin can't even do a good job of reviewing a bad toaster. When writing a critical review, the bar is clearly set at what Amazon.com reviewers come up with. Godin should see where that bar is set for bad toasters:

[http://www.amazon.com/review/product/B00004R92D/ref=cm_cr_dp...](http://www.amazon.com/review/product/B00004R92D/ref=cm_cr_dp_hist_1?%5Fencoding=UTF8&filterBy=addOneStar)

I note, with some smugness, that "DANGEROUS --- SHAME ON TOASTMASTER" is also a catchier title than "World's Worst Toaster".

</comment><comment><USER13> This sort of interface problem is common with clocks too. We have a clock at work that signals break time; it has four alarms but you have to manually turn them off. One time I had to be the one to turn it off, and I looked at the back. Four identical, round buttons - simply labelled "A", "B", "C", and "D". WTF? I need the manual to look up what the buttons do!

</comment><comment><USER14> "It took me more than 11 clicks to send them $6."

Upgrading an account in Backpack, 37Signals' web-based calendar/PIM/organizer thing, takes 4 clicks.

Making it easy for people to give you more money on a regular basis is a good thing.

Note that it takes roughly the same amount of effort to downgrade or cancel your account altogether.

</comment><comment><USER15> Even better than that toaster, when I tried to reinstall my wireless driver on Windows a while back (since then, I have seen the light and switched to Ubuntu), the new driver wouldn't install because a few files were missing. "Please click here for help [online]."

I kid you not.

</comment><comment><USER16> The worst designed telephone: [http://www.jgc.org/blog/2008/02/worst-designed- telephone-in-...](http://www.jgc.org/blog/2008/02/worst-designed-telephone-in- world.html)

</comment><comment><USER17> What's the deal with toasters...

And would you believe that the process of air travel is like...two steps:

1\. Go to airport 2\. Get on plane.

My god, if human heavier-than-air flight is simpler than applying heat to bread, then we are _all boned_.

<comment><USER18> True, but it takes a couple of minutes to make toast and the whole house-to- plane process takes forever.

The airlines are still trying out different plane boarding techniques, with mixed degrees of success.</comment></comment>


Ask HN: How do you keep track of life's events and get togethers? - <AUTHOR> I often find myself trying to keep track of too many events in my head, leading to double booked plans or even worse, forgetting plans.
How do you help manage this?
Is it the use of calendar apps, meet up, eventbrite, Excel, better memory skills, or an entirely different system or thought process?
Thanks HN MG <comment><USER2> Usually a calendar app will do tbe trick, but I try to only have one engagement maximum per day.

I'd rather have a few friends I can really focus deeply on, not a wide network of superficial connections.

Otherwise, the invites pile up, and the complexity becomes torturous for me personally.

</comment><comment><USER3> Just put it in my calendar app, always have my phone on me so it's convenient.</comment>