Poor Man's Scalability - <AUTHOR> http://codypowell.com/taods/2011/10/poor-mans-scalability.html <comment> captain-asshat Cross posting my reply from another site:

> We were now rendering our site index in 1.5 seconds or less, compared to 3 > seconds before.

This is the real WTF. What kind of queries take three seconds to run, that is ridiculous. Even with indexes 1.5 seconds is insane.

Deciding against premature optimisation isn't permission to throw away good, efficient design. It's clear from the rest of the article that these guys really have no idea what they're doing. Quoting the results of all their optimisations:

> In the end, we did not go down. The last round of load tests with httperf > showed us handling 200 GETs a second indefinitely, with an average response > time of about 1.2 seconds. We served thousands of requests without even a > hiccup, and we did it without spending any money.

This is really poor performance, and it shows in the site when you change page or select a different category/filter. I'm guessing that they also chose a nosql solution for the wrong reasons, as it seems everything there could be done much faster/better by using an ACID compliant database.

</comment><comment><USER2> Thanks to the author for taking time to write up his experience. That said, this is really basic stuff. How in the world could a home/index page have so many queries that it took 9 seconds? Oh, no indices... Then how could you have built a DB-backed web app and not considered indexing? A better title for the post might be "Scaling Mistakes I Made While Building Our Website and How I Fixed Them".

<comment> eric-hu I'm very very reluctant to call those mistakes. From the author:

> I didn't get too upset here; I expected us to do poorly. After all, we'd > spent all of our development time in actually building something people > want, not on scaling up for thousands of imaginary users.

Perhaps it would be relevant to the conversation if the author could chime in with how many projects or ideas they went through that failed. Perhaps they had to do consulting on the side for income.

Premature optimization is a mistake. It's time wasted on delivering minimal value to you or anyone else.

<comment><USER3> Indices on a database aren't "premature optimization". They're part of understanding your data model, how it will be requested, and--more in a SQL database than a MonogDB database, but still--constraints on the data stored in your database.

Claiming basic development practices to be "premature optimization" is a fantastic way to paint yourself into a corner because of stupid decisions in your haste to "get it out the door." Your MVP isn't V if it still takes a second to render your homepage to users, because they'll _leave_. (Site responsiveness is a huge factor in bounce rate, even for sites that people actually want to look at.)

<comment><USER4> I consider Indices to be premature optimisation. Moreover, they're very low level, potentially dangerous premature optimisation. Imho, they should be the very last trick you use to optimise.

First, you try and figure out if you can reduce the number of repeated queries. Then you try and figure out if you can get rid of chunks of code that spawn lots of queries altogether, by tweaking the algorithm to do things differently, without needing that data. Then, finally, once you've used every trick to make it all faster, then you apply indices to speed up that handful of remaining queries.

If you apply indices first, then you won't spot the other potential optimisations, and when your indices stop covering up for your poor coding it will be much later, and much harder to fix.

<comment><USER5> No, caching should be the last trick you use to optimize.

Indices are standard way to achieve basic performance levels of a database. They may have their downsides but "potentially dangerous" is dramatically overstating the case. Furthermore, the dangers of premature optimization are about taking extra time or adding complexity to something that ultimately doesn't matter, not about using very basic features in a sane way.

The correct optimizations to make first are ones that make the biggest impact percentage-wise as well as being the most elegant in the code. Indices typically fit both categories very well. Unless you are doing a lot of stupid things, there's not going to be much lower-hanging fruit, but even if there is, after you apply sane indices that will be when your profiling will start to reveal the real interesting possibilities for optimization. The idea that indices are a good final optimization does not show much interest in real performance.

<comment><USER6> I would take swombat's advice to look at query patterns, eliminate repeated queries, and tweak your algorithm to require less data well before I thought about adding indices (or caching, which I agree is the last thing you should do).

You want to make the riskiest, most invasive changes first, because those are the ones that the rest of your codebase builds upon. If you've changed your query patterns and the app _still_ isn't fast enough, it's relatively trivial to add indices on top of that. The speed benefits are cumulative, and none of the work you've done examining your data-access patterns has been undone by making that data access faster.

If you add indices first, however, and it still isn't fast enough, you have to examine your query patterns anyway. And this time, the work you did _is_ undone by your further optimizations. The benefits of indices depend a lot on how you access your data: they slow down writes to speed up certain queries. If it turns out that the queries you're speeding up don't occur frequently anymore, your index is counterproductive.

(That said, since adding indices is often a task that takes 5 minutes of your time, you might want to give it a try and see what sort of results you get before investing a lot of effort in other stuff. If it doesn't work, you can back out the change and then start looking at your query patterns.)

<comment><USER7> Disagree on caching as the last thing you do.

If you're building an app that will hopefully be bigger than you expect, build cache into your data layer, it's not complex and will pay off sooner than later.

</comment></comment></comment></comment></comment></comment></comment><comment><USER8> It's fairly basic stuff this, but important none the less.

Kudos for recommending Yslow (or PageSpeed) - it's indeed brilliant. You didn't mention much about http caching, which I think is probably as important as the other things you mentioned; Not only does it improve performance for the user, but it also reduces load on your server and it enables you to put up an edge side cache for extra performance.

As for the problems mentioned with the load balancer - You could have simply provisioned a new instance and installed your own load balancer. There's dedicated packages like haproxy, but you could also just put up nginx or lighthttpd. This machine can later double as your edge cache (Squid).

Btw. 1.5s to _render_ the front page? That's way beyond acceptable, in my book. But I suppose it depends a bit on what the web site does.

<comment><USER5> > _Btw. 1.5s to render the front page? That's way beyond acceptable, in my > book. But I suppose it depends a bit on what the web site does._

Agreed. This will have an extremely negative impact on your user engagement. Now if this is measuring end user response time rather than server-side page generation it might not be quite so bad, but generally I think most common pages should target < 150ms.

<comment><USER9> So what would you both consider "the next steps" in page speed optimizations?

<comment><USER10> If you really do have a legitimate 1500 msec process occurring on the home page, try to flush as much of the HTML to the browser before you start the heavy lifting. Ideally, you could have the entire HTML shell delivered immediately, and the content elements slotted in at the very end (using DOM manipulation).

If you can put all the content inside a

</comment></comment></comment></comment><comment><USER11> Because i'm in a particularly pedantic mood: I don't think ELB (Amazon's load balancing) is free, which is a little misleading.

In fact:

* $0.025 per Elastic Load Balancer-hour (or partial hour)

* $0.008 per GB of data processed by an Elastic Load Balancer

Source: <http://aws.amazon.com/ec2/pricing/>

Obviously these are problems loads of people face / hope to face, so it would have been nice to get some more 'meat' in this post.

Edit: formatting

<comment><USER12> Neither is a second EC2 instance, or were they running that already? Was it just not running a web server?

</comment></comment><comment><USER13> No CDN? Adding CloudFront is really simple and the costs are insignificant.

<comment><USER14> CloudFlare is easier. And free.

<comment><USER15> Paying for CloudFront doesn't leave me wondering what their revenue model is and how it might affect how they serve my content.

</comment></comment></comment><comment><USER16> Your story closely resembles the challenges we had scaling sommelierwine.ca before a media event at our latest launch location. I knew that our site would receive an increase in traffic, and the increase would only be temporary.

As a rails site hosted on Heroku, this gave us the ability to scale our site using the gambit of widgets that Heroku offers; but, we didn't want to spend the money and cheat ourselves from the satisfaction of scaling our site.

Using New Relic (a gem available on Heroku) I identified our first performance bottleneck - database queries. Certain queries were taking over 30 seconds to complete! These database queries all involved a table of BLOBs (on the order of 20MB) that needed to be searched and sorted. I tried adding indexes but that only marginally reduced the query time. The solution we found included moving the BLOBs to their own table while keeping an index to them in a table of their attributes [1]. Doing this, we were able to reduce query times down to less than a 100ms.

The remaining bottlenecks we found using YSlow has helped us reduce the overhead in loading pages substantially.

Even though we were able to weather the storm of visitors to our site, we did leave some tasks for the next one including implementing caching, background processing, and remote file storage. All in time.

Does anyone have other wallet-friendly Rails/Heroku scaling stories share?

[1] This is database 101 stuff - a class I never took.

<comment><USER17> > [1] This is database 101 stuff - a class I never took.

Yes, yes, a thousand times yes.

The best and most important way to improve database performance is to have a sensible schema. Query planners can do magic with stuff that's sensibly normalised and with things that are sensibly _de_ normalised.

Where they choke is on schemata where everything is just squished into a bunch of tables with fields holding internal datastructures.

</comment></comment><comment><USER18> The next step after ensuring you're doing the basics (cdn, minimizing data over the wire, etc) for us was to throw the entire site behind varnish. Set a "don't cache" cookie if users login, add a comment, etc. so that their changes are visible immediately.

We took a slow site running on hardware, put it on EC2 (slower hardware obviously), and with varnish are saving about 60% cost of hosting, and decreased page load time on some of the popular pages by upwards of 50%, all for minimal engineering effort.

</comment><comment><USER19> This seems like basic stuff. I'm more interested in the story about how you got this to the top of the front page.

 _Edit: Less obnoxious._

<comment><USER20> Basic stuff is valuable to people who need to learn the basics. Not everyone on HN is an experienced programmer, much less an experienced web programmer.

<comment><USER21> Thank you.

As I was scanning the comments on this post, I was a bit surprised by the all the "This post was too basic. You are clearly an amateur and wasting my time" comments. Well, to those authors, not everyone on HN is an l337 hax0r such as yourselves.

Yes, the author still has some work to do. Maybe if he spent another couple weeks just on this, he could get response times down. Maybe he should use a reverse proxy or maybe he shouldn't. As with most problems of this type, the answer is, it depends.

To a young programmer with a few years experience who may be working on his/her first high traffice website, I thought the blog post was fantastically written. It was clear, concise, explained well the low-hanging fruit of optimization, explained the difference between performance and throughput, and decisions-making/tradeoffs made when preparing for a traffic spike. Well done.

</comment></comment></comment><comment><USER22> Cody, just one note on your website - I couldn't tell at all what Famigo offers without clicking on something. Instead of having "Famigo helps families find and manage mobile games and apps" in a hover-over pop-up, have a similar tagline somewhere people can see easily.

<comment><USER23> Thanks for pointing this out. We JUST pushed a change this week where we removed a huge banner saying "We help families find and manage blahblahblah" and replaced it with a few other things. Maybe we need to revert that!

</comment></comment><comment><USER24> "Moving the Javascript to the bottom" - wouldn't the best thing to do would be to invoke the JS from onLoad? And put the code in a separate minified JS file, so it can be cached?

<comment><USER25> Yes, but moving the actual script tag that loads that minified JS file to the bottom of the HTML can help in some situations. Less so now that most UAs have decent speculative parsers, but it used to be that seeing the script tag would block any resources below it from being requested until the script was done loading.

And being cached won't help for first-time visitors to the site, of course.

</comment></comment><comment><USER26> Nice writeup and valuable experience. Just curious why no extra money was spent? Spinning up more EC2 instances costs money, right?

</comment><comment><USER27> With the rise of cloud services, and their relative inability to serve naked domains, there are a couple of easy fixes for this.

The first is to create a vhost in your webserver that serves an index.htm page on your NON-www url. The contents of that page would simply redirect users to the www.yourdomain url.

If that seems like too much work, an even easier fix (truncated for brevity) is to do something like this:

      var url = window.location.href;
      if(url.indexOf("http://www") != 0) {
        window.location.replace("http://www.yourdomain/");
      }

You'd obviously want to capture the remainder of the querystring and add it in, but that's the general idea.

<comment><USER28> Why not plain old HTTP 301/302 redirect?

<comment><USER12> Please, please use a redirect. The GP's suggestion is not something you want to do.

Even better, just use <http://wwwizer.com/> (look for the free redirect service banner on top).</comment></comment></comment>


Fair Analytics: A fair and distributed alternative to Google Analytics - <AUTHOR> https://github.com/vesparny/fair-analytics <comment><USER2> If this gains widespread use then the transparency points will make it far worse for privacy than using Google.

Why? Google have access control over the data or view it in anonymised form when crunching. But this data will open to anyone and would allow people to correlate visitor sessions not just on a site but across other sites that use this, to the degree that you could start deanonymising the internet activity of people.

It's a strange world in which we can say, "At least today only Google has the data and they've made it hard to get to".

<comment><USER3> I am trying to figure out what the problems are with Google Analytics anyway.

<comment><USER4> Some of us just don't want to live under Google's umbrella.

Personally, I have no interest in sacrificing my privacy for "free" services. I'm not important, nothing I do is controversial or of any significance, but having my activities logged and monitored by a giant all-seeing eye is still uncomfortable. It's disturbing, to me, how accepting of these things people have become.

</comment></comment></comment><comment><USER5> I'm not sure I'd go with the idea (yet). The power of Google Analytics is in the "Analytics" pieces. This seems to be only the data gathering piece. Granted that is also important. But this would need a ETL & BI system or something like that in-front of it to be useful.

It is a great start.

I also really enjoy looking through the source code for projects like this and seeing how other coders do things.

<comment><AUTHOR> As you said, this is just a starting point :) It's meant to be the engine gathering data.

It's possible to build any type of chart/fancy dashboards on top of it

</comment></comment><comment><USER6> At least for me, the real value that GA provides is being up all the time and giving me peace of mind that hits won't get dropped. Running my own analytics infrastructure is possible, but would get in the way of what my site was actually trying to do.

What the open source analytics world needs is better infra, not better collection.

</comment><comment><USER7> "Fair" aka "Missing features".

Is there an OS alternative to Google Analytics that's comparable in featureset?

<comment><USER8> Maybe something like Piwik is what you're looking for. Open Source, self hosted analytics.

[https://piwik.org](https://piwik.org)

</comment></comment><comment><USER9> It seems that users will be very tempted to write additional "session" information, for example some kind of visitor id to the logs. They would need that to, for example, correlate which people visit the homepage with which referrer then buy a product. That's a core use of most analytics products.

Considering the raw logs are timestamped and public, that then sounds like a rather large privacy hole for the users.

I would imagine some kind of pre-aggregation happening, and then making available aggregated data instead of raw logs.

<comment><AUTHOR> Writing additional "session" information without explicitly asking for user consent often leads to threatening user's privacy.

This is one of the main reasons for fair analytics to exist.

<comment><USER10> I think you're missing the point. People are going to add-on the missing features anyway.

</comment></comment></comment><comment> irq-1 Advertising needs to filter out fake views, or they'd end up paying for click farms. I don't see how this protects against this.

<comment><AUTHOR> At the moment there is configuration option to only accept requests from a specific host. This should help avoiding fake clicks.

I'd love to have suggestions on how improve this particular aspect :)

<comment> irq-1 I don't think there is an answer. Fake clicks can only be determined by a 3rd party -- or else sites could host the ads and report whatever numbers they want.

The only way forward that I can see is anonymous aggregation by a 3rd party that is not the advertiser or the website, and which has incentives to protect the users.

Imagine a system with dozens of 3rd party collectors that anonymize browsing data. The data is submitted by users who have a plugin for their collector. The collectors then sell the anonymous data to the advertisers. Advertisers can then compare data from different collectors to determine fake clicks, and what the level of payment should be.

There are two big problems with a scheme like this. First, anonymous data is much less valuable to advertisers. I think this can only be solved by technical means; making anonymous data the only information available. Second, why should users have a plugin that tracks them? Access to content seems like the obvious answer; users with a plugin (being tracked by a collector) could be allowed to read the NYT and the NYT website could be paid by the collector.

Realistically, I don't see either of the problems being resolved.

</comment></comment></comment><comment><USER11> If it meets the guidelines, this might make a good 'Show HN'. Show HN guidelines: [https://news.ycombinator.com/showhn.html](https://news.ycombinator.com/showhn.html)

</comment><comment> awinter-py re: privacy.

Important to distinguish between a blogger's utility from getting session information and the huge privacy risk of sending everyone's reading to a central provider (GA).

I use piwik and I have it set to respect do-not-track; I assume that means if someone visits from a private tab in FF, I won't even see a hit (piwik experts -- correct if wrong).

But I also benefit greatly from the analytics from non-private browsers -- seeing that users are, for example, clicking on the footnotes incentivizes me to deliver thorough arguments in the future.

</comment><comment><USER12> I find the auditability most interesting, I bet this could be useful for some blockchain based decentralized applications.</comment>


Ask HN: Reliable sources for US news? - <AUTHOR>

With the mainstream going increasingly celebrity/scandal obsessed, and the fringe well becoming more,, fringe, I'm finding it increasingly hard to find quality national reporting. Where can one actually find quality national news? (NB: I am primarily interested in current events, and not longer-time period publications such the New Yorker or Economist) <comment><USER2> <http://www.bbc.co.uk/news/></comment>


Malaysia Airlines flight MH370 Infographics - <AUTHOR> http://graphics.thomsonreuters.com/14/MH370/index.html <comment><USER2> These are well-done graphics and Reuters Graphics has an excellent stylebook...but just to be a nag...this strikes me as a good example of how custom Web navigation can be counterproductive for the reader.

I'm sure most people can figure out that the top three images are meant to be buttons, but I'll admit that I didn't notice, at first, that when you click each subhed-button, a "Next" option pops up in the top right, along with page- boxes to the top left. It's obvious once you've noticed them, _but the trick is noticing them in the first place_ , which you might not because your eyes immediately jump down to the graphic. How many users, after seeing the initial graphic, immediately click on one of the other subhead buttons, thus seeing only a total of 3 graphics?

An equally problematic issue is that there's no way to deep-link any of the items. I can't easily discuss a specific graphic (and they're hard to locate via rightclicking, as they're embedded via svg tag) here, or even one of the subnav-topics, without fully describing the actual graphic...which is a real inconvenience. The use of clickable gray boxes as pagination only adds to the inconvenience, because now I have to count which box I'm on.

I think this would've been better served by having each sub-topic be a long, scrollable webpage...as it is now, you _already_ have to scroll to read most of these graphics. Once you get done with one graphic, you have to scroll back to the top, click "Next", and then read through...at least have a "Next" at the bottom.

The main argument for having this compact nav is to invite quicker comparison between slides...I don't think that's a great argument even in the best of cases, but here, because of the vertically-long slides, it's not even applicable. It'd be far easier to flip back between sequential graphics by simply scrolling.

OK, that was a lot of nagging, but I hate to see great work obfuscated by "gee-whiz" navigational design that is user-unfriendly. I would discuss the graphics but like I said, it's kind of a pain to specify and describe each particular slide without deeplinks.

</comment><comment><USER3> tl;dr: Designers, please think about ordinary-sized screens!

This presentation may have good design _within_ it, but it is not actually _presented_ well. It is intended to be a series of slides which appear on single screens. However between its thick header and insanely thick header, there is not enough real estate on the screen to view a full slide on my 13" MacBook Pro. So I have to scroll down to view the bottom of the slide, then scroll up again to click the navigation element, then scroll down again to view the full slide, etc. It's incredibly aggravating.

So I've just hacked the CSS to set the header and footer to display:none. Not an auspicious way to begin an interaction!

<comment><USER4> Generally, I just right click on the component, select _inspect element_ , and then press delete (after possibly making sure the root element of the offending feature was actually selected).

<comment><USER3> How is it that I never figured out that was possible?!? Thank you!

</comment></comment></comment><comment><USER5> Excellent presentation.

On the presentation, it does feel like a stew with all the right ingredients but without seasoning to fully appreciate the taste. Seasoning not meaning editorial content, but a glue to hold the how and why together, and understand the combination of the taste.

</comment><comment><USER6> I'm not a designer, so i speak as a consumer.

I disagree with complements on the look of this. I kind of feel it's misisng something, which i can't quite pin down. perhaps it's the look of the maps, that they don't quite match the rest of the feel. The sources text in FF28 W7x64 doesn't look particuarly good, not very ledgible.

maybe it's just an overall static and clinical look that i don't quite like (which is odd as i do like industrial design).

I think it possibly needs a bit more art direction. but i'm a consumer so what do i know.

<comment><USER7> Agreed. The fonts render terribly on Chrome Windows 7 on my machine.

Also, I kept expecting to be able to scroll and it took a little while to figure out how to navigate the pages. The nav bar at the top could serve dual- purpose: allow the user to click on a particular box while also allowing scrolling and showing the user's current position in the content.

<comment><USER8> Identical experience here. The fonts looked so bad in Chrome that I opened FF (which I very rarely do) to see how they compared, and it was far more readable in FF.</comment></comment></comment>


The Raspberry Pi you'll be buying in January - <AUTHOR> http://www.raspberrypi.org/archives/422

<comment><USER2> Connectors are becoming huge. Are there any alternatives to HDMI/RJ45 ports with a lower profile, similar to the mini-USB one?

</comment><comment><USER3> I really hope they don't sell out in two minutes; I can think of about twenty needs my Pis are going to fill.

</comment><comment><USER4> I am really looking forward to this. Handing it to my son and saying 'go at it - hack around'.</comment>


Ask HN: Is it still relevant to send you website to DMOZ - <AUTHOR>

I rarely if ever hear people suggesting dmoz.org anymore. It used to be the craze when I was getting started in web development and SEO<p>Did dmoz become irrelevant? Do you even go to dmoz ever? Are search engines not considering its weight anymore? <comment><USER2> I would vote that it is not relevant. Maybe someone has information that I don't which would sway my view, but I doubt search engines rely on it anymore and nobody uses web directories anymore.

</comment><comment><USER3> I'm not so sure it's actually irrelevant, however it had been two or three months since I submitted a site that I'm working on, and it hasn't been added yet.</comment>


NewCalendar – is it too late to fix a Babylonyan mistake? (2009) - <AUTHOR> http://newcal.org <comment><USER2> I'm all in for a calendar reform, but I prefer the Cotsworth plan: [https://en.wikipedia.org/wiki/International_Fixed_Calendar](https://en.wikipedia.org/wiki/International_Fixed_Calendar)

<comment><AUTHOR> It resembles [http://www.newearthcalendar.com/](http://www.newearthcalendar.com/) a bit.

</comment></comment><comment><USER3> >same algorithm for determining which years are leap years as the Gregorian calendar

It's ridiculous to propose overhauling the calendar without addressing what is IMO one of the most annoying parts about dealing with calendar dates.

<comment><USER4> And what alternative do you have in mind? We could redefine how long a year is. But that would require some tricky orbital engineering and a very large rocket.

<comment><USER3> I don't have a serious suggestion. It would indeed necessarily involve decoupling the calendar year from the seasonal year. Three 487-day "years" lines up with 4x365 + 1 leap day, and the seasons are still regular (over a longer period).

</comment></comment></comment><comment><USER5> Here's a bit less radical calendar reform, to maximize ease of use: [http://www.symcal.info/](http://www.symcal.info/)

</comment><comment><USER6> Too much free time, dude :)</comment>


Amazon is Down - <AUTHOR> http://www.amazon.com <comment><AUTHOR> This is the text on their homepage: "We're sorry! An error occurred when we tried to process your request. Rest assured, we're working to resolve the problem as soon as possible. If you were trying to make a purchase, please check Your Account to confirm that the order was placed. We apologize for the inconvenience. "

<comment><USER2> screenshot: [http://imgur.com/yDtWSkU](http://imgur.com/yDtWSkU)

</comment></comment><comment><USER3> I look forward to learning more about Amazon as a company by reading later about how they handle this situation.

</comment><comment><USER4> Up for me right now

<comment><AUTHOR> Yes they just went back up...

</comment></comment><comment><USER5> International versions (tried .de and .es) seem to be up, interestingly enough.

</comment><comment><USER6> As is IMDB.com.

<comment><USER2> Amazon owns IMDB.com , are any other Amazon services down ?

</comment></comment><comment><USER7> It's back up now. That's quick. What's their uptime SLA number? 99.999%? That is a 5minute 15s per year downtime.

</comment><comment><USER8> Other services like seller central and amazon canada is working normally.

<comment><USER9> There's a notification in SC

"Seller Central was down from 11:20 to 11:38 PST today. We have corrected this issue, and sellers are now able to access Seller Central. We apologize for any inconvenience."

I didn't notice it down, though.

</comment></comment><comment><AUTHOR> Just came back for a quick second, and than right back to being down.

<comment><USER2> this happened a couple of minutes before as well

</comment></comment><comment><USER10> Wow, every minute lost is so damaging.

<comment><AUTHOR> Millions of dollars per minute!

<comment><USER9> 100 billion yearly turns into around 200K per minute.

</comment></comment></comment><comment><USER11> Just when I need to print up receipts

</comment><comment><AUTHOR> Looks like they are back up now...

</comment><comment><USER12> So much for the cloud.

<comment><USER13> That's not even the cloud. It's a website. Amazon is more than just AWS, this is impacting Amazon.com.

<comment><USER14> While true, amazon.com does run on AWS now.

</comment></comment></comment><comment><USER9> Huh, looks like capitalizing the DOWN made this get 4 times as many votes than [https://news.ycombinator.com/item?id=11261484](https://news.ycombinator.com/item?id=11261484) as of now

Edit: and now title has been edited to down. Was capital when I posted.

<comment><USER2> the other one did not appear when I sorted by new , only this one

<comment><USER9> I see it in incognito, and in [https://archive.is/w3glR](https://archive.is/w3glR)

Does HN hide some links selectively?</comment></comment></comment>


Cassini sniffs oxygen on Saturnian moon - <AUTHOR> http://www.abc.net.au/science/articles/2010/11/26/3077376.htm

<comment><AUTHOR> citation is here "Cassini Finds an Oxygen—Carbon Dioxide Atmosphere at Saturn's Icy Moon Rhea" ~ [http://www.sciencemag.org/content/early/2010/11/24/science.1...](http://www.sciencemag.org/content/early/2010/11/24/science.1198366/suppl/DC1)</comment>


Facebook Stored Millions of Passwords for Instagram Users in Plaintext - <AUTHOR> https://malwaretechno.blogspot.com/2019/04/facebook-stored-millions-of-passwords.html <comment><USER2> I feel like people shouldn't be waiting until there's more bad news before they consider changing their passwords - as soon as you hear that a password breach was likely, go and change your password.</comment>


Are smart people overrated? - <AUTHOR> http://www.gladwell.com/2002/2002_07_22_a_talent.htm <comment><USER3> God, this is classic Gladwell. Skip critical details to come out as a "holier than thou" contrarian.

 _Kitchin's qualification for running EnronOnline, it should be pointed out, was not that she was good at it. It was that she wanted to do it, and Enron was a place where stars did whatever they wanted._

EnronOnline was one of the most successful units of Enron. In fact, UBS picked up the platform and now many of the techniques developed at EnronOnline was picked up by the electronic energy trading platforms.

Kitchin is still considered one of the foremost authorities in the world of energy trading and finance. I believe she is a MD at Deutsche Bank in the field of energy structuring and got her development of the business turned into a Harvard Business School case study.

Clearly not only was she good at organizing the business, she succeeded at it. This is exactly what I really dislike about reading Gladwell's stuff. His job is to find a previously successful story-line that has hit snags, in order to act as an armchair quarterback. He is just like most other journalist and historian - seemingly wise because of post-hoc analysis but incapable of actual prediction.

<comment><AUTHOR> I don't see him as coming out as "holier than thou", although admittedly a bit contrarian. Plus, the point about Kitchin wasn't that she was unsuccessful. Quite the opposite in fact. The point was that these decisions affected the whole company.

And for the record, I think that Enron goes well beyond having "hit snags".

<comment><USER3> Definition of holier than thou: _Exhibiting an attitude of superior virtue; self-righteously pious._

Gladwell tries to come off from a moral high ground where he preaches about certain concepts (in this example the benefits of talent) by self-selecting evidence.

I really find it funny how the entire employee grading process has become a huge area of contention amongst arm chair quarterbacks. Recently, there was another article where it talked about Wall Street's excessive competitiveness as having caused the crisis. Gladwell and similar authors forget the one company (and the one manager) credited with popularizing this entire concept: General Electric and Jack Welch. I like the off-handed way Gladwell mentions a specific quote from a G.E. exec without talking about the success G.E. had with this approach I can use Welch and G.E. in order to argue against this entire article. In fact, Welch's biographies do the job. Not only did he grade employees into three tiers, he graded entire businesses into tiers. If a business wasn't #1 or #2, its shuttered. I will bet I can actually construct a better argument for supporting the quest for talent.

Enron blew up because upper management was a bunch of liars. What does talent have to do with it? In fact, Enron failed at following the "promote talent" concept. If they did take this philosophy to heart, they would have fired the international heads that bled billions of dollars from Enron's bottom line. In fact, Skilling and the top protected those sub-par managers and execs leading to the shady accounting to hide the carnage. The talent they picked up in their energy trading department was (and is) great. Go into any of the nation's energy trading floors and Enron alumni enjoy great reputations.

As for the record, I think Gladwell goes well beyond being "a bit contrarian". ;)

</comment></comment></comment><comment><USER4> I like Alan Kay's saying that "a point of view is worth 80 IQ points" ([http://billkerr2.blogspot.com/2006/12/point-of-view-is- worth...](http://billkerr2.blogspot.com/2006/12/point-of-view-is-worth-80-iq- points.html)).

In the end I don't think that it's either mostly about intelligence or mostly about the system but a balance, where good system will multiply the effectiveness of intelligent people.

We can find examples of few intelligent people in the right place creating astonishing breakthroughs (see Xerox Parc). I would think that if they were not above-average smart, they wouldn't have done what they did, so we shouldn't under-rate being smart.

On the other hand there are also examples of undoubtedly smart people creating over-complex, broken designs (e.g. SOAP) so pure intelligence is not all there is.

<comment><USER5> Richard Benson (master printer, MacArthur genius grant winner, former dean of Yale School of Art) has a saying (approx.), "Smart people are a dime a dozen. Show me someone who can make something and I'll be impressed."

Wise people are also, of course, impressive.

</comment></comment><comment><USER6> "Among the most damning facts about Enron, in the end, was something its managers were proudest of. They had what, in McKinsey terminology, is called an "open market" for hiring. In the open-market system"

Perhaps I missed this in the scandal, but I thought the most damming facts about Enron were wholesale accounting & auditing fraud with participation from Anderson. How does 'hiring smart people' explain the fact that their auditors conspired to commit fraud? If Anderson also heavily engaged in this HR practice Gladwell may have a point. But it's still heavily anecdotal. Like most business analysis it's backward looking instead of making predictions based on the theory and reporting the results.

Also, the article seems to conflate having an MBA with being smart. I wonder if Gladwell has ever thought that the reason Southwest has a vastly more efficient organization than its competitors is because it has a bunch of smart people who figure out how to do things more efficiently. People so smart in fact that they realize people on the ground can make much more informed decisions than people in a boardroom. An MBA is not a proxy for being smart, an MBA is a proxy for doing well in school. Unfortunately, school has almost nothing to do with the real world.

When I think about 'smart' people, I don't think about IQ or degrees. I think about people I've seen do the 'impossible'. A 'smart' person for customer service might be someone who is very friendly and personable. A 'smart' person for engineering may not be very personable at all but can come up with a working design in days instead of months.

<comment><USER7> While I agree your sentiment that degrees and IQ have next to nothing to do with whether or not someone is actually smart, have a look at current culture and at corporate america.

At amazon, everyone was obsessed with how "smart" amazon employees were, and yet the place was a showcase of terrible engineering -- rife with accidental complexity and tight coupling that spanned not just systems within a group, but across entire divisions -- and some what I read in this article I saw there first-hand. The management complained constantly about how bad our tools were, yet the people that they promoted and took care of were the ones that designed and implemented those same tools.

A great many managers haven't the faintest idea as to how to determine what constitutes a "smart" employee, so they zero in on things like degrees and number of hours worked because they don't have any rational standard for identifying the good ones from the bad, so the guy with the degree who's working 80 hours a week is obviously the best guy on the team! Look how hard he's working! He must be REALLY smart!

"Also, the article seems to conflate having an MBA with being smart. I wonder if Gladwell has ever thought that the reason Southwest has a vastly more efficient organization than its competitors is because it has a bunch of smart people who figure out how to do things more efficiently. People so smart in fact that they realize people on the ground can make much more informed decisions than people in a boardroom."

I think that was precisely his point. Enron used MBAs as a proxy for "smartness" and turned out to be a spectacular disaster. Southwest doesn't, and is one of the few US-based airlines that isn't relying on the US government to keep it alive.

<comment><USER3> Could it be that once you reach a certain level of size in any organization bureaucracy and complexity turns up? The reason why Amazon might have "terrible engineering" while the sleekest app startup doesn't is because Amazon is really, really big.

Look I understand degrees and IQ really don't mean much by themselves. Most of my friends attend "elite" colleges (as society deems it). Some are brilliant. Some aren't. However, the numbers show that "smartness" and particularly an "obsession" with smartness matters. Amazon is really successful. Google is really successful. D.E. Shaw is notoriously successful. Facebook is successful. Teach for America is super successful. The defining theme to all of these organizations is a focus on hiring the best talent possible. And often times the best talent happens to be at those top schools.

Think of the timeline of hiring. These companies often target people in the 22-27 age bracket. That is just 4 to 9 years away from their high school when college decisions were made. Work ethic, ambition, and a determination to get things done is pretty damn autocorrelating in its nature. Someone mentioned how PG noted in an essay that he was underwhelmed by MIT, Harvard, Stanford kids. But look at the YC startups. The number of startups funded by YC by alumni from MIT, CMU, Stanford is far more than what it would be if there was no correlation-causation.

I think drawing broad conclusion from a criminal enterprise like Enron is a poor way of doing things. Just because Enron touted their smartness and failed doesn't mean that is the norm. On a large enough sample set, I will still bet you that the McKinsey conclusion is right in that "talent" and the drive to get the top talent is still a big part of the most successful organizations.

<comment><USER7> "Could it be that once you reach a certain level of size in any organization bureaucracy and complexity turns up?"

I would tend to agree with that, although I've seen startups run by bad managers install a pointlessly complex hierarchy for no reason other than that companies have hierarchies.

"The reason why Amazon might have "terrible engineering" while the sleekest app startup doesn't is because Amazon is really, really big."

No. Their abysmal engineering is by far the worst I've ever seen... well, not by far, what I ran into at Disney gave Amazon a run for its idiocy. The reason for the terrible engineering is that amazon is a Gen-Y shop. The company is notorious for abusing employees, and very clearly shows preferential treatment to interns rather than seniors. They have a _lot_ of trouble hiring experienced engineers, and since everything they do is an emergency, they put however they can on the job... in other words, most of Amazon's mission critical software was designed and implemented by people who either just graduated from college, or hadn't yet (interns).

And then they hire seniors to "fix" it... the culture is such that they think "smart" is the ticket, and don't value the wisdom and knowledge that come with experience.

Now count the number of successful companies to the number that aren't. And take into account the number of unsuccessful projects compared to the number of successful ones, even at successful companies. What I suspect that you'll find is that in most companies, even successful ones, the primary reason that projects "succeed" is that the management changes its definition of "success" in order to avoid taking blame for mis-managing a project to its demise.

</comment></comment></comment></comment><comment><USER8> "Smart" has too many meanings to people. But successful (by western consumerist societal standards) people are totally overrated.

Happy people and being happy in general are totally underrated.

<comment><USER9> Yes, pg commented about "smart" people from top colleges applying to YC in it's earlier days, and he was surprised at how underwhelming they were. Then he goes onto say that just because they appeared impressive to admissions officers when they were 17 doesn't predict future success.

Some quotes:

 _"Success is decided by the market: you only succeed if users like what you've built. And users don't care where you went to college."_

 _"Y Combinator has been an unprecedented opportunity for learning how to pick winners. One of the most surprising things we've learned is how little it matters where people went to college."_

 _"There's nothing like going to grad school at Harvard to cure you of any illusions you might have about the average Harvard undergrad. And yet Y Combinator showed us we were still overestimating people who'd been to elite colleges. We'd interview people from MIT or Harvard or Stanford and sometimes find ourselves thinking: they must be smarter than they seem. It took us a few iterations to learn to trust our senses."_

 _"We're talking about a decision made by admissions officers—basically, HR people—based on a cursory examination of a huge pile of depressingly similar applications submitted by seventeen year olds. And what do they have to go on? An easily gamed standardized test; a short essay telling you what the kid thinks you want to hear; an interview with a random alum; a high school record that's largely an index of obedience. Who would rely on such a test?"_

\- from "News from the Front" <http://www.paulgraham.com/colleges.html>

EDIT: grammar

</comment></comment><comment><USER10> I'm becoming more and more convinced that the politics of intelligence is one of the most potent and unspoken. It's far more of a touchy subject than gender, class, or race. In fact, it precedes them: people seem to get the most upset when groups of people are called intellectually inferior (they don't really seem to care about what you have to say about cultural vs genetic roots). It seems to rule everyday interactions and organizational decisions.

In America, I can't think of something more insulting to say to someone than to call their heritage intellectually stupid.

</comment><comment><USER11> No, but Gladwell is.

</comment><comment><USER12> Gladwell talking about others being overrated

 _Riiiiiiiiiiiiiiiiiiiiiiiiiiiight_

</comment><comment><USER13> I do think that Gladwell is being hypocritical here. The people who wrote the survey in the first place were doing a study to find things in common among successful companies. He identifies things that have made individual organisations successful in some places, and then generalizes to say that this falsifies the argument that recruiting the best talent isn't important for all organisations.

This is in fact he opposite of the methodology he advocates in 'outliers' - where he suggests that you should look for things in common between successful organisations.

</comment><comment><USER14> (2002)

</comment><comment><USER15> Are stupid people overrated?

<comment><USER16> Some are.

</comment></comment><comment><USER17> If smart is pursued to the detriment of other positive characteristics - <USER2> 'smart' can be considered overrated.

<comment><USER7> One must take under consideration the definition of "smart" first.

I had a co-worker who had memorized a raftload of facts without actually understanding many of them. She frequently mis-diagnosed problems with customer computers because although she had a huge list of facts in her head, she couldn't think her way through them to make sense of what was really going on.

Due to her ability to spew facts on demand however, the management thought she was exceptionally smart, even though she couldn't actually figure anything out for herself.

The company's hierarchy reinforced this sort of thing, because the "engineering" team was a bunch of college dropouts who didn't have the faintest idea as to how things like UNIX networking worked. They said that total memory = physical + virtual, so my attempts to explain that UNIX doesn't work like that went unheard, and customer machines continued to suffer because they didn't have enough memory to serve their buyers' needs.

The last I heard the company had something like a 4% customer satisfaction rate.</comment></comment>


Mobile revolution in Vietnam - <AUTHOR> http://www.wsj.com/article_email/vietnams-mobile-revolution-catapults-millions-into-the-digital-age-1434085300-lMyQjAxMTI1MjE2MjgxODI0Wj <comment><USER2> I was in Vietnam in May for about a month, so some anecdotes:

I was amazed by the amount of iPhones there. It seemed to dominate in usage over Android. I was expecting to see a lot of 80 USD ShenZhen Androids, but no... either you have a dumb phone or an iPhone in Vietnam.

And iPhones are not just for the rich kids. People having about a dollar or two of disposable income per day are using (second hand?) iPhone 5+ models. No laptop, just a phone. I was traveling with my brother who still uses his iPhone 4 with iOS6 which was drawing some laughs :)

Social media wise: the people I interacted with all used Facebook a lot. Both Gmail and Yahoo for email. Twitter/Instagram, not so much. Snapchat was unheard of. I expected WhatsApp to be popular, but not so much.

<comment><USER3> The iPhone would be a great "single computer" for all of those people .. if only it were more open. I wonder if there are any Vietnamese hackers out there in some village, trying to work out how to write their own apps for the iPhone5 .. what a real pity that such devices, quite capable of supporting their own developer/user/owner, have been neutered of the task.

<comment><USER4> Now that (as of last week) you can get a certificate to compile apps for your device from Xcode for free, it's totally possible. Someone would have to write an app that bundles an IDE, clang, codesign, etc. Fetching that certificate could be done directly by the app or through a proxy; this is the only sticky part, because I'm not sure whether Apple would approve an App Store app that uses undocumented web service or Xcode to do so, but the thing is, if Apple doesn't like it, there is now the "nuclear option" of staying off the App Store and having a web service that fetches said certificate, signs the IDE itself with it, and sends it to you. And it may not be necessary to go that far, perhaps depending on how much is being done on the client side versus the server.

If you add a Bluetooth keyboard and a TV, an iPhone starts to seem like a totally reasonable development platform...

Edit: By the way, if HTML dev (including local) suits one's needs, which for a beginning programmer it well might, that's been possible for a while. C.f. "Procoding".

</comment></comment></comment><comment><USER5> There are some key enablers that they did right:

1\. Internet connection (Landline and 3G) is so cheap relative to income. There is no locked in contract.

2\. Free wifi everywhere and all businesses have free wifi (coffee shop, pho, book store,...)

3\. Huge number of iPhone are used devices. They can get it for cheap because they are either stolen (in US) and hand carry to Vietnam.

4\. Most people have family in US that would buy the phone for them or bring used ones back.

5\. iPhone have very high resale value. So they use it for awhile and resell to get new ones. It's not that much of a big cost.

6\. Smart phone in Vietnam is like car in US. You gotta have one or otherwise, you're considered poor!

The following is a lie (from the article):

>> A Vietnamese government agency forecasts the market for e-commerce will generate revenue of $4 billion this year compared with $700 million in 2012.

There are high percentage of mobile users but they don't spend a lot. As a matter of fact, most ecommerce revenue are cannibalizing from people who would buy these things by driving a bike to the store anyways. Most shops do offer delivery for free or small fee. Reason: it's like 2-3 miles trip max. Everything is super close together.

Disclaimer: I am Vietnamese, spent most my life in Vietnam, tried to do startup in Vietnam, failed to compete with Facebook when it started with only 1M users in Vietnam.

</comment><comment><USER6> "Internet penetration has grown to 44% of the communist state's 90 million people from 12% a decade ago. Much of that is driven by smartphones, which are used by more than a third of the population"

Basically, like cell phones brought voice to many who could skip landlines, and the associated expensive infrastructure, smart Internet phones are bring the web to people. Local businesses are taking advantage of this.

</comment><comment><USER7> I spent 2 months in Vietnam (HCMC) last Nov-Dec, and that was the first thing that surprised me. Not the iPhone5s but everyone seemed to have an iPhone6, and yep I couldn't figure out the economics, but that's the reality (thanks to posts below the economics now make sense).

</comment><comment><USER8> > Men read news from a smartphone in downtown Hanoi, Vietnam, on Dec. 5. > Photo: Agence France-Presse/Getty Images

TIL: Vietnamese have amazing eyesight

</comment><comment><USER9> 500</comment>


Declarative Templates: A Better JavaScript View Layer - <AUTHOR> https://medium.com/@julienetienne_/declarative-templates-a-better-javascript-view-layer-f6a6ab5a7aa7 <comment><USER2> Or just use something like twig where the whole point is to have your view data sorted BEFORE you get to templating, which IMO, is the real source of clutter. Template systems should be boring, stupid, and clean. Making all of the things JavaScript has never been necessary. Definitely better than React or most anything else that's popular though.

</comment><comment><USER3> > It's not only pleasantly readable

I find JSX to be more readable. "Tags as first class functions" means I have to parse the functions into HTML as an extra mental step when trying to understand the structure and elements that make up the view.

<comment><AUTHOR> there({class:'is-no'},extra('Mental Step'));

<there class="is-no"><extra>Mental Step</extra></there>

<comment><USER3> Spent 10 seconds looking for the closing tags and trying to understand hierarchy on the first version. YMMV. I've been reading HTML for 20 years now.

<comment><AUTHOR> I hear you. Maybe that wasn't a good attempt of an example how about this [https://github.com/julienetie/hypertext#3-case-create- html-e...](https://github.com/julienetie/hypertext#3-case-create-html- elements-without-a-virtual-dom)

Currently aside from JSX and things like mustache some devs are using h [https://github.com/snabbdom/snabbdom#inline- example](https://github.com/snabbdom/snabbdom#inline-example) and Cycle has a syntax not far off from what I'm proposing [http://widdersh.in/tricycle/](http://widdersh.in/tricycle/) . But the syntax is not actually the big deal, the biggest benefit is the complete separation of logic from semantics.

I don't expect everyone to like it but I don't think it's horrible or difficult to look at when used in a real project.

<comment><USER3> > the biggest benefit is the complete separation of logic from semantics.

Then maybe allow JSX? The way I write my React components, the view is what you find in the `render()` method, and there is little to no logic there. The big selling point for me with React is that I no longer have to learn _yet another templating language_. Which seems to be somewhat where you are leaning as well. But being able to read HTML as, well HTML (or something very close like JSX), has a lot of benefits. Just going between code and the browser debugger for example. The mental step of translating between what I see in the browser and what's in my code, is in fact a step.

<comment><AUTHOR> "The big selling point for me with React is that I no longer have to learn yet another templating language."

Hmm, I get the impression you're tired of having to learn something new and you're happy to stick with what just works for you. . I understand but I'm not sure everyone feel that way, but I would be honest enough to say I think most people in FE development are on your wavelength.

Here's the entire syntax documentation: \- Tags are functions \- Attributes + props are object literals \- Strings are text nodes, \- Remaining parameters are child elements

It's nothing but basic JavaScript.

The separation is not enforced by API it's a discipline that can't be forced into React via JSX as far as I've tried.

Here you have: \- No transpiler \- No logic mixed in semantics \- No vague component separation \- No wrapping everything in a div (use ...spread) \- No new syntax to learn

Maybe it's not for you, but the advantages are more than obvious.

</comment></comment></comment></comment></comment></comment><comment><USER4> I love it. Please build this so I can use it immediately.

<comment><AUTHOR> Will do, eventually ;)</comment></comment>


Ubuntu shows off a hybrid handset - <AUTHOR> http://www.bbc.com/news/technology-35655342 <comment><USER2> So they are still showing it off on a Nexus 4?

Meizu are just about to release Meizu Pro 5 Ubuntu Edition that have top of the line hardware, but it's missing MHL so it can't be used for convergence... WHY DO THAT!?

</comment><comment><USER3> > BBC's Chris Foxx asked Ubuntu's founder Mark Shuttleworth whether the firm > would be playing catch-up with HP which announced a hybrid handset running > Windows 10 at the Barcelona tech show.

"Playing catch-up" because... they were announced at the same show?

<comment><USER4> HP aren't using their own OS. MSFT demoed their continuum technology at the Surface event back in October, so there's a good chance HP will beat Canonical to the market.

<comment><USER5> Is the HP one running x86? I'd imagine Linux would be hugely preferred to Windows on ARM since so much more of the software ecosystem is available.

<comment><USER4> It's ARM, using a Qualcomm Snapdragon 820. You're effectively limited to Windows store universal apps (which are usually pretty terrible), plus MS Office.

</comment></comment></comment></comment><comment><USER6> This page is approximately 10% content, 90% other garbage

</comment><comment><USER7> The whole world moves to putting out different devices for different usecases and integrating them with protocols and APIs. Why is Canonical moving in the other direction?

<comment><USER8> Because the whole world is trying to push you to buy different devices for the sake of selling new trinkets, and what all that I really want is to have one personal computer that I can carry and is powerful enough to be useful on its own.

</comment></comment><comment><USER9> How hard is compiling an Ubuntu phone kernel compared to android?</comment>


HN Build This: iPad iPod Apps Data Syncing - <AUTHOR>

My dad has a problem that I think many others will have also. He has bought an iPad after buying an iPod touch. His problem is, he has shopping lists data in some app on iPod touch but it won't get moved to his new iPad's same shopping list app.
He wants them to sync, or at least, if you've edited the same shopping list on both devices, take the newest version an sync time. I'm surprised Apple hasn't built this in; it sounds trivial enough that I think if I was an iPhone + Mac developer I'd be able to do this in an afternoon.
Since solving this problem will help many users I think this is an opportunity for someone/company to make some money. What do you think, will anyone be interested? <comment><USER2> Unfortunately, it is impossible to modify the way a locked iPad/iPhone syncs...

Even assuming you can trust that the internal data structures of iPad/iPhone versions are identical - you can't implement this as a 3rd party app you install, as 3rd party apps are sandboxed and can only access their own data. And the only sync supported is via iTunes, which again can't be modified or replaced without jailbreaking the device.</comment>


Postmortem: Every Frame a Painting (2017) - <AUTHOR> https://medium.com/@tonyszhou/postmortem-1b338537fabc <comment><USER2> > "Nearly every stylistic decision you see about the channel — the length of > the clips, the number of examples, which studios' films we chose, the way > narration and clip audio weave together, the reordering and flipping of > shots, the remixing of 5.1 audio, the rhythm and pacing of the overall video > — all of that was reverse-engineered from YouTube's Copyright ID."

Copyright and the rabid defense of IP by corporations shapes so much of what we're allowed to see and talk about and _how_ we have to address those things. There's no telling how much better these already great videos would have been if they weren't shackled by fear of censorship or attack by media cartels.

These guys had the time, skill, and energy to figure out how to work around the problem, but I can't help but wonder how much stuff we're missing out on because a system designed to encourage the creation and propagation of creative works has been perverted to control and suppress it instead.

<comment><USER3> Only if you're using other people's work. Make your own original content and there isn't nearly as much of a problem. Who says everyone should have the right to profit off somebody else's work? especially one that's only considered good because of all the money spent on marketing it.

<comment><USER4> If you don't consider Every Frame A Painting to be original work, then I think this is just one of those situations where there is so little common ground between two sides that there can never be any agreement.

<comment><USER5> I'm not sure you should even bother framing things in terms of "common ground" or "two sides" when it's clear that one party is trying to define _original work_ so narrowly that it's doubtful anything could qualify.

</comment></comment></comment></comment><comment><USER6> >A huge percentage of the Internet is the same information, repeated over and over again. This is especially apparent on film websites; they call it aggregation but it's really just a nicer way to say regurgitation.

Man, what an apt way to explain my frustration with the monoculture of the web. But it's creeping into the world of books, where fewer and fewer writers know how to research anything. I cannot tell you the number of books I have picked up lately that cite things they saw on Twitter or gleaned from a Google search.

<comment><USER7> Ironically, even this very article has been posted on HN twice before.

<comment><USER8> HN aggregates, it has no illusions of creating original works. Occasional high-quality reposts are a virtue of HN, and I don't think HN suffers from the same regurgitation issues as film sites. The film websites often come with analysis or reviews of films where the regurgitation lies, whereas Hacker News provides no editorialized comment on the articles, only discussion.

"Echo Chambers" etc... could be merely a distributed version regurgitated content, however I think the HN community tends to self-police groupthink

<comment><USER9> The comments on HN are quite often regurgitated, especially for the political topics rehashed every week.

<comment><USER10> Comments are not the main content that HN aggregates. They facilitate discussions around the main content, which are just either links or at best a question.

</comment></comment></comment></comment></comment><comment><USER11> Even as a non-cinemaphile, this channel, and the post-mortem essay, had resonated with me. Both are excellent, both provide far more than insights into a century of cinema, although if that's all that you take from it it's more than worth the time.

I'd commented on the postmortem essay shortly after it was published with some additional thoughts of relevance to my own (very non-cinematic) work:

[https://old.reddit.com/r/dredmorbius/comments/7hgb24/brillia...](https://old.reddit.com/r/dredmorbius/comments/7hgb24/brilliant_failure_every_frame_a_painting/)

(I'd also submitted the same link, a minute after smacktoward did, full credit to them it's a very worthwhile submission.)

<comment><USER12> Same here. As a... let's say, cinema-newbie, I found the Every Frame a Painting channel simply _fascinating_. I devoured every chapter in a short time. To me this kind of channels is a triumph, what YouTube can be and should strive to be.

I think they did overreach with some conclusions. For example, his "obvious" conclusion about the story told by the edition in the clip with the father and the two young daughters wasn't at all obvious to me (and judging by the comments on that video, a lot of other viewers didn't get it either). I suppose it's part of the subconscious knowledge someone in the industry has which someone outside doesn't.

<comment><USER13> Is anyone aware of other content like it on YouTube? Long-form , artistic, very high quality. Channels like 3Blue1Brown and Kurzgesagt are usually great, but I don't think they're the same kind of thing.

<comment><USER14> Nerdwriter: [https://www.youtube.com/channel/UCJkMlOu7faDgqh4PfzbpLdg](https://www.youtube.com/channel/UCJkMlOu7faDgqh4PfzbpLdg)

<comment><USER13> I've come to feel nearly all of his content is just blogspam in video form.

</comment></comment></comment></comment></comment><comment> open-source-ux What a shame this YouTube channel ended. The videos were insightful and so well put-together. Making a video that is both informative and entertaining at the same time is much harder than it seems.

Are there are other film-related YouTube channels people recommend?

Two I recently discovered:

 _Mystery Clock Cinema_ : Film director Alex Proyas ('Dark City', 'I, Robot') talks on camera about independent filmmaking

[https://www.youtube.com/channel/UCYO7h2SOZjRah8bhyOXtGHw/vid...](https://www.youtube.com/channel/UCYO7h2SOZjRah8bhyOXtGHw/videos)

 _Ponysmasher_ : Film director David Sandberg ('Annabelle Creation', 'Shazam!') has a few behind-the-scenes videos and thoughts on filmmaking. The channel is not updated very often, but his most recent video 'Random Lessons Learned from Making Films' is very watchable and has a surprising (and very honest) revelation near the end of the video:

[https://www.youtube.com/user/ponysmasher/videos](https://www.youtube.com/user/ponysmasher/videos)

<comment><AUTHOR> Red Letter Media: [https://www.youtube.com/channel/UCrTNhL_yO3tPTdQ5XgmmWjA](https://www.youtube.com/channel/UCrTNhL_yO3tPTdQ5XgmmWjA)

Lessons from the Screenplay: [https://www.youtube.com/channel/UCErSSa3CaP_GJxmFpdjG9Jw](https://www.youtube.com/channel/UCErSSa3CaP_GJxmFpdjG9Jw)

<comment><USER15> RLM is great for more casual and humorous views on film. Their "Best of the Worst" series is one of my favorite things to watch

</comment></comment></comment><comment><USER16> _> Nearly every stylistic decision you see about the channel — the length of the clips, the number of examples, which studios' films we chose, the way narration and clip audio weave together, the reordering and flipping of shots, the remixing of 5.1 audio, the rhythm and pacing of the overall video — all of that was reverse-engineered from YouTube's Copyright ID._

Wow. Talk about "the medium is the message".

</comment><comment><USER17> What I love about Every Frame a Painting is the depth of its research. It's quite clear that the creators had an extensive knowledge of film. The videos always include wonderfully selected interviews with the filmmakers that emphasize the point far more than just a person talking. The Deakins interview in Shot | Reverse Shot taken from Cinematographer Style is a nice example^[1]. Or Sidney Lumet's interview in the Kurosawa video^[2]. Taking from film criticism solidifies the video's arguments, creates an amazing rhythm to the videos (Tony and Taylor are clearly great editors) and also prevents the "annoying person endlessly lecturing to me about film" feeling that I get with some other channels.

I've thought about doing videos of my own on various topics (the usage of eyes and glances in Il Posto, Kieslowski's poetry, the reflexivity of In the Mood for Love, etc.) but the YouTube film criticism industry is pretty saturated. Perhaps a different format?

[1]: [https://www.youtube.com/watch?v=5UE3jz_O_EM](https://www.youtube.com/watch?v=5UE3jz_O_EM) [2]: [https://www.youtube.com/watch?v=doaQC-S8de8](https://www.youtube.com/watch?v=doaQC-S8de8)

</comment><comment><USER18> Discussed at the time: [https://news.ycombinator.com/item?id=15836782](https://news.ycombinator.com/item?id=15836782)

</comment><comment><USER19> Every Frame a Painting is what truly turned me on to Fincher, after I'd enjoyed Seven, Fight Club, The Social Network, Gone Girl, Zodiac.

He's on the same level as Lynch for me; they both found their favourite perversion.

As for Fincher, Mindhunter is the spiritual successor to Zodiac. It's fantastic.

</comment><comment><USER20> I've always seen this as a workflow/business failure.

Obviously they had a great product -- everyone I've shown EFaP to loves it. They could have spun it into something of a personal brand (like, say, CGP Grey or Nerdwriter did). But instead killed it off because it felt like more of a burden than a reward.

Might have been on monetization, or personal perspective. In the essay they seem uncompromising on quality, but there are probably ways to get similar quality output with fewer hours by improving the workflow. I imagine if they made tens of thousands of dollars per video, or built a large brand of the video's success, they wouldn't have felt like such a burden either.

<comment><USER21> This was no failure.

"A thing isn't beautiful because it lasts".

<comment><USER22> Except that it is. They say so themselves and also mention that is totally OK with failing.

<comment><USER21> Except that it isn't. They say so themselves and also mention that "...Every Frame a Painting ended up being both a personal and a professional success."

Edit: and because "Vancouver never Plays Itself" (the phrase, the concept, the idea) has transcended.

</comment></comment></comment></comment><comment><USER9> Trying to read that made me appreciate Tony's work to create video versions of his essays.

I watched every EFaP video and never noticed Taylor existed. :-(

</comment><comment><USER23> >There is no such thing as free content on the Internet

[...]

>Everything costs something to make. If a person is putting out content for free, that means they're not getting paid for their time.

First of all, remember when people posted things on the internet just out of our desire to share things? I certainly support Tony's decision to shut this down - it is his decision to make, and no one should feel FORCED to create content for free.

But second of all, doesn't the second commend contradict the first?

<comment><USER24> the delightful ambiguity of English allows us to say such things as "Things that are simple should be simple" and in context of describing design of systems have it understood as Things that are simple to describe should be simple to do.

In this case the ambiguity has allowed them to say That which is free is not free, meaning that which is free to consume is not free to make.

I certainly remember people posting stuff on the internet for free out of their desire to share. I also remember lots of those people over the years shutting down their delightful sites because not having the time to keep doing things for free, nor the money to pay for the hosting or other things like that.

<comment><USER23> >meaning that which is free to consume is not free to make.

I guess it depends on what cost you want to include? I create content "for free" using my free time. Would you consider my mortgage, the cost of my computer, etc, to be part of the cost, so it is not "free" ? I wouldn't, because I would incur those costs whether or not I was making content "for free".

>I certainly remember people posting stuff on the internet for free out of their desire to share. I also remember lots of those people over the years shutting down their delightful sites because not having the time to keep doing things for free, nor the money to pay for the hosting or other things like that.

Yes, more and more all the time - exactly my point. And like I said in my original post, it is certainly there prerogative. If they don't enjoy it, they should stop.</comment></comment></comment>


Lawrence Journal-World gets out of the CMS business, losing WordPress, etc - <AUTHOR> http://www.niemanlab.org/2012/06/the-lawrence-journal-world-gets-out-of-the-cms-business-losing-out-to-freebies-like-wordpress/ <comment><USER2> Every news organization that I've talked to that's used Ellington loves it. If they want to seriously make a change for the better in an industry dominated by antiquated CMSs that users hate, they need to open-source it. Hopefully they make the right choice here.

</comment><comment><USER3> Open sourcing Ellington might help grow the userbase which could lead to maintenance contracts, contracts for adding features, etc.

<comment><USER4> Ellington is built on top of Django, so the generic parts have already been open-sourced.

<comment><USER3> Django is a framework and Ellington is an application. Both frameworks and applications can be open sourced.

</comment></comment></comment><comment><USER5> "We think a lot of our Ellington product and think it's one of the tops in our industry"

Frankly, I don't think so, and that's probably the problem.

I could be wrong but most companies management that got a baked CMS think they got the most powerful, frikking awesome, and it general it's just a soup of unmanageable legacy code.

Of course there is a lot of other problems that could spring like a bad sales teams, but if the platform awesome ido not see why it should not be striving since it has been online for so long.

<comment><USER6> Honestly if Djang is any indication of the development skills of the team I highly doubt the codebase is a huge mess. There are not many organizations where the development of an in-house tool spins off a high quality framework that becomes a mainstream tool.

</comment></comment><comment><USER7> A few years ago, I used to work for a large local newspaper/TV media holding company with about 70 web properties.

In the 1.5 years I was there, they had about 6 CMS in action. IPS(i think?), ExpressionEngine, Day CQ, Wordpress and Ellington.

Smaller TV affiliates and publications are moving to Wordpress because of its price and how easy it is to find support/developers with experience.

If I remember correctly, the company I was previously with left Ellington to develop their own CMS features on top of Django.

<comment><USER8> I have a couple questions about this; could you email me at mail at [my- username] dot com? I'm just really interested in the local web media industry.

</comment></comment><comment><USER9> The issue with Ellington has always been it's closed source nature. You make a lot of money from people who install, setup, and use your CMS/app/whatever who need a support contract later on.

If you have to pay up front, you're much more likely to go with something like Wordpress.

</comment><comment><USER10> This was an earlier conversation on the related subject of using a CMS for a big site:

<http://news.ycombinator.com/item?id=2764850></comment>


Yahoo stock soars on new buzz of Microsoft deal - <AUTHOR> http://timesofindia.indiatimes.com/business/international-business/Yahoo-stock-soars-on-new-buzz-of-Microsoft-deal/articleshow/10251171.cms <comment><USER2> Here we go again. Last time we saw a similar headline, it did not end well for either comany. Hope the rumor dies away soon.</comment>


How much do startups pay their developers? - <AUTHOR>

i always see threads for a developer needed on some startup here on HN. How much can a startup developer expect to make (even though they are most probably joining for the thrill and passion for the company)? <comment><USER2> I try to get 6 figures, but its hard to find startups that can afford to pay that much for long. What you make totally depends on where you live/work.

When it comes to stock, I view it as a useless bonus, many times I was promised stock and it was never issued, and most of the time it would be worthless anyway.

</comment><comment><USER3> I naively was looking for and got a low sum for a company here in Orlando, approximately 35k and I ended up resenting the company and others. I was fairly sure I was the lowest paid member on the staff and that the intern who was a son of one of the investors might have made more than me.

My current job is not a startup, but I got a much more reasonable figure in line with what I need to happily live my life.

</comment><comment><USER4> Does the startup have funding? Does it have appreciable revenue? Are either of these things on the horizon?

Does the developer have applicable experience? Is he willing to take increased equity in exchange for a decreased salary? How many other people on the job market have the same skill set?

Where is the startup located? What is the cost of living there? What is the median salary for the area?

<comment><AUTHOR> well a ballpark average for a software engineer right out of college is around the tune of $60K.

<comment><USER5> depends on your level of skill and the school you came out of too. if you're reading news.ycombinator, please don't shortchange yourself and accept $60k, which btw, just happens to be exactly what I did when I got out. my current company offers $100k for friggin QA positions if you know C++/C (i.e. you're actually capable of being a developer). of course, my company is not a startup anymore, so adjust your figures appropriately.

what I've seen is that anything goes for a startup. it depends a lot on how well funded they are and how in tune with the establishment the founder(s) are. but if you ask me, you should really not be assessing a startup based on salary. doing a startup is a passion. why would you do one otherwise? if salary is important to you, then just stick with the small company doling out $100k for a QA position sort of gig.

</comment></comment></comment><comment><USER6> Well, recently out of college in west Texas I managed 40K a year plus 16K shares at 3 dollars a share. Wasn't really sure if that was any good.

</comment><comment><USER7> The real question is how much equity you'll receive. Without a share in the upside, working as an employee at a startup is a sucker's game.</comment>


Vault 0.10 released - <AUTHOR> https://www.hashicorp.com/blog/vault-0-10 <comment><USER2> What happened to the old Unix philosophy about doing nothing on success (a.k.a. "Rule of Silence")?

That is:

      $ vault kv undelete -versions=2 secret/my-secret
      Success! Data written to: secret/undelete/my-secret

should instead be:

      $ vault kv undelete -versions=2 secret/my-secret
      $

Also, the screenshots on the Vault OSS UI Introduction page [0] are completely unintelligible.

[0]: [https://www.hashicorp.com/resources/vault-oss-ui- introductio...](https://www.hashicorp.com/resources/vault-oss-ui-introduction)

<comment><USER3> For your average user I think this is confusing and not good UX.

Since a lot of people using this tool will be developers with MacBooks and not hardcore *nix users, it's probably a good choice to ignore that convention.

<comment><USER4> > For your average user I think this is confusing and not good UX.

I disagree, as the average vault user is likely someone familiar with Unix (though there is a Windows binary). Here is an example that I just ran on my machine.. very silent.

      $ mkdir source
      $ cd source
      $ touch file
      $ rm file
      $

<comment> onmai-xyz

      $ rm file
      $ echo $?
      $ 0

Silent, unless you go looking. The return value of the last executed command is available as $?.

I picked this up at [http://www.rsync.net/resources/howto/remote_commands.html](http://www.rsync.net/resources/howto/remote_commands.html) for running commands over ssh and not seeing the console output.

</comment></comment></comment></comment><comment><USER5> Vault secrets versioning, Enterprise UI open sourced, root DB creds rotation, Azure Auth method and GCP secrets engine and GCP cloud spanner as a storage backend, really great release!

</comment><comment><USER6> A simple thing that's been missing for a long time now is a way to patch existing secrets in the KV store. Since the secret itself is composed of keys and values it quickly becomes cumbersome to just update one of those. I guess now that versioning is enabled it will be easier to build a tool on top.

        $ vault kv put secret/my-secret my-value=itsasecret
        $ vault kv patch secret/my-secret new-value=othersecret
        ^--- this doesn't exist
        $ vault kv get secret/my-secret

        </comment><comment> Data </comment><comment>
        Key         Value
        ---         -----
        my-value    itsasecret
        new-value   othersecret

</comment><comment><USER4> A while back I was using Vault, and the standard logging didn't give me enough to setup any sort of useful detection & alerting capabilities. I tried to increase verbosity in Vault's logging, as the CLI would accept integer parameters, and the documentation made it seem this was possible.

However, the source code was hardcoded to only one logging level (they didn't actually have multiple logging levels). The homepage marketing-speak was disingenuous on this front. Coupled with other poor architecture choices (wasn't actually HA, forced you to use other Hashicorp products, etc), I lost interest in the product.

Seemed they were mostly focused on unique features than standard enterprise requirements. Have things improved?

</comment><comment><USER7> The web-ui and Google Cloud IAM secrets engine plugin are awesome new features. I've been playing and setting up Vault today on GCP and overall really happy.

Also, I believe using Google Storage as a backed now works with clusters meaning Consul is now not required for clustering.</comment>


Ask HN: Is anyone working on Googles original mission statement? - <AUTHOR> Perhaps the Internet Archive? <comment><USER2> We have a different mission statement: Universal Access to All Knowledge.</comment>


Star Trek Copyright Settlement Allows Fan Film to Proceed - <AUTHOR> https://www.nytimes.com/2017/01/21/movies/star-trek-axanar-fan-film-paramount-cbs-settlement.html <comment><USER2> "Though the suit was derided by some fans, The Hollywood Reporter noted that the studios were using it to signal that they would no longer tolerate professional-quality derivatives of its movies and television series."

I would be very interested in finding out how they intend to define "professional-quality". Particularly these days.

Would you not quickly end up needing to "sabotage" your output in order to skirt the rules of such a definition, in what amounts to a strange new dogma format?

<comment><USER3> > I would be very interested in finding out how they intend to define > "professional-quality"

They tried to: [http://www.startrek.com/fan- films](http://www.startrek.com/fan-films)

<comment><USER2> Thanks! It seems that the key here is "creators, actors and all other participants must be amateurs".

So can you ever ship two Star Trek fan productions? Or does the first production render you sufficiently experienced to no longer qualify as an amateur? ;)

</comment></comment></comment><comment><USER2> Could definitely have hoped for a more useful settlement. Like requiring that Axanar be monetised and all proceeds go to Paramount - without imposing any creative or production limitations.

The need to send a clear message about their strategy and relationship with fan fiction going forward makes sense, but surely there's a useful compromise to be found which would also increase the value of their IP?

</comment><comment><USER4> It's kind of sad that companies spend so much time squashing ants. Instead of seeing an opportunity to encourage more excellent film-making, and allowing talented people to keep improving their skills and one day produce even more awesome projects, all they see is a chance to go to court.</comment>


Ask HN: Why isn't there a power-user web browser? - <AUTHOR> I'm really fond of the idea that a web developer only needs three tools to do their job: a terminal, a text editor, and a web browser. But while terminals and text editors are specialized tools for developers, for a web browser, developers are just using the same tool as everyone else.
Text editors and terminals have a bunch of similarities: They're completely customizable, you can change and add key bindings, and they're scriptable. Those two features make it really easy to add my own features to my terminal and text editor. But it's really difficult to add features to my web browser.
Here are some examples of features I want to add keyboard shortcuts to in my web browser:
- Opening the source code for the current web page in my text editor.
- Copying the URL that's open in each tab to the clipboard.
- Move a tab to a new window.
- Start a new blog post for my Jekyll blog with the current URL.
Text editors and terminals make adding these kinds of features easy, but they're not easy to add to web browsers.
If there are web browsers that make these kinds of features easy to add then I'd love to hear about them! But that still leaves the question: Why are these types of features important to web developers in their terminal and text editor, but they aren't important in their web browser? <comment><USER2> You could try looking into Qutebrowser, which is a keyboard driven browser designed with vim controls and is configured with python, so there is some room for extendability and scripting.

Without doing any work, I can:

(yy) Yank the URL of the current tab to the clipboard

(gD) Move current tab to a new window

With some work, it's not impossible to get the features you want, or something close.

<comment><AUTHOR> Thanks yeah, I've never tried Qutebrowser but I've been curious whether it can do some the things I'm looking for. It sounds like it has some nice built-in bindings, but do you know if it can be scripted? E.g., Emacs can be scripted in Emacs Lisp, Vim can be scripted in Vimscript, and VS Code can be scripted in JavaScript. And all three can call-out to shell commands, including further processing the results of commands. If you know off-hand whether Qutebrowser has these features I'd love to know. (I've done a bit of research myself but haven't found much.)

<comment><USER2> Yes, it supports some level of scripting that I have not personally explored yet. See here:

[https://www.qutebrowser.org/doc/userscripts.html](https://www.qutebrowser.org/doc/userscripts.html)

Looks like you totally can set up a shortcut to open stuff(like URLs for instance) in other programs. If you can find some time for it, you should definitely give it a try. For the stuff you mentioned, looking at this tells me it shouldn't be impossible or even that hard.

<comment><AUTHOR> Nice yeah, it does look promising. Doesn't look like full-blown scripting support like in the aforementioned text editors, more like the limited support that Mutt has. But this is definitely enough to pique my interest, and it looks better than anything I've seen in any other browser.

</comment></comment></comment></comment><comment><USER3> to some extent, curl and wget are off-the-shelf power user web browsers. And lots of folks script up custom spiders for headless web browsing.

<comment><AUTHOR> Agreed, but that's similar to doing a find-and-replace with `sed` versus doing it in your text editor. Even with a CLI that can do find-and-replace there's still a lot of times you'd want to do it within your GUI text editor. It's the same for me with the web browser and the uses cases I listed in the post. I'm curious why other programmers aren't as interested in these workflows as I am.

</comment></comment><comment><USER4> > I'm really fond of the idea that a programmer only needs three tools to do > their job: a terminal, a text editor, and a web browser.

What about a compiler, linker, and other related tools? Also, I don't consider a web browser to be an essential programmer's tool at all. I often go days without using one in the course of my work.

But maybe I'm being a little unfair here. From the rest of your post, I think that perhaps you're not actually talking about all programmers, but rather developers working on web-related things.

<comment><AUTHOR> Totally fair. I updated the post to say "web developer" instead of programmer, you're right that that's more accurate.</comment></comment>


Ask HN: What if your OPT card hasn't arrived after 60 days? - <AUTHOR>

Graduated and waiting on the OPT card. International counselor says I will be fine, but I wanna double check and verify. <comment><USER2> It generally takes more than 60 days. I think it should be within 90 days ideally but I have seen it go till 100+ so I don't think you need to panic.

Just keep checking the status online and you should see it change in about 2 weeks.</comment>


Women Did Everything Right. Then Work Got 'Greedy.' - <AUTHOR> https://www.nytimes.com/2019/04/26/upshot/women-long-hours-greedy-professions.html <comment><USER2> It pains me to see an article where we are asked to sympathize with the plight of a successful pair of NYC lawyers. What is going through the mind of the author of such an article? Are these at risk people who need our help?

<comment><AUTHOR> I think you are missing the point. But I agree, a pair of big city lawyers was not the best example the author could have used to make his argument.

<comment><USER3> Her argument.</comment></comment></comment>


Kari Byron, Tory Belleci and Grant Imahara Netflix Trailer - <AUTHOR> https://www.youtube.com/watch?v=ohb5k3_vQcE <comment><USER2> I am happy for them! They are a great team.This White Rabbit Project looks very promising.</comment>


Augmented Reality in OpenCL (2011) - <AUTHOR> http://kineme.net/composition/MOostrik/AugmentedRealityinOpenCL

<comment><USER2> Very cool implementation. We are looking at real time AR video filtering using this implementation:

[http://www.learnopencv.com/non-photorealistic-rendering- usin...](http://www.learnopencv.com/non-photorealistic-rendering-using-opencv- python-c/)</comment>