CppCMS – High Performance C++ Web Framework - <AUTHOR> http://cppcms.com/wikipp/en/page/main <comment><USER2> Some of the negative comments here really anger me, and I feel a need to speak up.

Who gives a flying _fuck_ if this tool is the right one for whatever is your individual definition of web development? The fact is that someone had a need for a particular solution and they were kind enough to build it, document it, and show it off here for _free_. Turning your nose up at this based on language choice just makes you a close-minded fool.

Applications for this do in fact exist, and I'm quite pleased that the author was kind enough to post this.

On a past project I've had to write my own asynchronous web framework in C based on the wonderful libmicrohttpd. I decided on C because this was a resource limited device on which I was already using every available CPU cycle on a realtime computer vision algorithm and it all was just barely performing within acceptable limits. I needed as tiny a footprint as I could get.

I'm in the early stages of a different project which will probably face a similar situation, except on this massive concurrency will be key. I will absolutely be looking into CppCMS when the time comes.

<comment><USER3> You know, every time I see one of these types of frameworks the first thing that comes to mind is "Some people are just determined to make C++ look cool again". I mean sure, sometimes you run into a case where you are running computer vision algorithms on an embedded device and have to choose C or C++.

But how common are those use-cases _really_? And why on earth would you want to worry about buffer overflows and memory leaks when you're already worrying about XSS and CSRF attacks?

As fast as modern Java is, it just seems unnecessary.

<comment><USER4> > And why on earth would you want to worry about buffer overflows and memory > leaks

Have you written a line of code in modern C++? Know what RAII is? If you use C arrays and char*s in C++, that's your problem - the language provides idioms that are "as safe as" Java.

<comment><USER5> RAII doesn't save you from buffer overflows, or memory leaks when you need an object to live beyond the current scope.

C arrays and pointers can still be used on the stack, and are sometimes simply the right tool for the job.

While RAII greatly reduces memory leaks and enhances resource-management, it doesn't save you in all cases, which means there still is an overhead and you will still have to worry about it.

Java doesn't have Undefined Behaviour, buffer overflows, or memory leaks (not completely true, I'm not counting space leaks here but I hear it's possible to introduce a memory leak by writing a faulty class loader).

While C++ does provide idioms for writing safe code, C++ can't remove them, which means you will always have to worry about it. Sometimes the problem isn't actually your code, but the library you're using, which is the case at my job, which again made me miss a safer language :(

<comment><USER6> > RAII doesn't save you from buffer overflows

How do other languages solve this problem?

> or memory leaks when you need an object to live beyond the current scope.

Have you ever heard about smart pointers? They take most pain from dealing with pointers. Managing circular references remains a bit complex though.

<comment><USER5> > How do other languages solve this problem?

Reading or writing past the allocated parts of a buffer is treated as a runtime error, wheras in C/C++ you just end up with corrupted memory.

> Have you ever heard about smart pointers?

Ahh, completely forgot about these. You're right, they're great.

</comment></comment></comment></comment></comment></comment><comment><USER7> Disclaimer: I used to write Web apps in C++ in late 90s/early 00s for the largest web company of the time.

1) DB is often a bottleneck for web apps. Not always. But often. Very often. Optimizing pages rendering will improve the performance for pages that are already rendered pretty fast. It will do nothing for pages that are loaded slow due to DB access. A better/smarter cache will give you much bigger bang for the buck.

2) For the annual salary of a _good_ C++ programmer (i.e. one who can write safe and _readable_ code with STL and Boost), I can run 50 large AWS instances for a year.

3) Writing safe C++ code is hard. Even with STL and Boost you still have to understand the little details of objects ownership (see boost::shared_from_this<> as an example).

Overall, I would invest in distributed system with smart and efficient caching instead of trying to optimize single server performance. At the end, you will run out of the single box solution (if you are successful). Going from 1 server to 2 servers is really hard. Going from 10 to 100 is not too bad.

<comment><USER8> Disclaimer: I'm not a C++ guy; I generally write in Ruby and PHP.

I agree with your overall point — C++ may not be a good fit for web development, and there's a lot to be said for being able to scale across multiple servers (and choice of platform will determine how quickly you have to solve that problem).

That said, my experience is that for most projects the performance bottleneck is the framework, not the DB. On big PHP projects, loading lots of code on every page load takes a huge amount of time, and many frameworks do things like parse multiple XML files on every request.

Similarly, Rails is slow… just take a look at how much time is spent outside of the database for a typical request. It's crazy how long it takes, considering that most requests boil down to a DB query and then some string concatenation.

Another angle on his is the common pattern of having a bunch of app servers and just a few DB servers. That implies that the DB is not the main bottleneck.

Again, I agree with your overall thesis, I just have to disagree with the common wisdom that is point 1.

<comment><USER7> In my statement I kind of assumed that all the "cheap" optimizations in the framework are already done (e.g. PHP APC cache is enabled, lazy classes/configs loading is implemented, etc.) so we compare "apples-to- apples": a highly optimized C++ framework to a highly optimized, say, PHP framework (can't comment about Ruby - didn't have experience building/running large-scale apps on it). And of course, it all depends on the application itself. Fetching single "narrow" rows by primary key is obviously cheap and nobody cares. However, you have to dig through a table larger than 500G with multiple indexes is not so cheap.

And the reason why you don't see a lot of DB servers is that it is HARD to scale DB by just adding servers (I am ignoring for a second non-SQL servers w/o joins as well as high-end Oracle and new MySQL-Galera solutions). An ACID compliant SQL DB is a single-server affair unless you invest heavily into the DB itself since it is really hard to run DB cluster even from pure operational standpoint.

<comment><USER9> > In my statement I kind of assumed that all the "cheap" optimizations in the > framework are already done (e.g. PHP APC cache is enabled, lazy > classes/configs loading is implemented, etc.) so we compare "apples-to- > apples": a highly optimized C++ framework to a highly optimized, say, PHP > framework

I'm kind of skeptical, to be honest. I suspect a simple, unoptimized C++ application wouldn't have a lot of problems keeping up with a highly optimized PHP framework. Interpreters do so much extra work.

Not that I would use C++ for web stuff- that's 99% string munging, and string munging in C++ is how you wind up with your desperate last words mockingly quoted on seclists.org.

<comment><USER7> Assume your DB query takes 10 secs. It's irrelevant if C++ code takes 0.01 sec when your PHP code takes 0.1 sec.

<comment><USER10> Now why would I assume that? There are times when you need to generate a big report, and just the SQL can take that long or more. But typical web requests for typical applications are not like that. DB queries shouldn't take more than a couple hundred milliseconds for "typical" application requests.

Even counting that, the comparison is also not necessarily correct. In some cases, perhaps the C++ code would have taken 0.1s, where the ruby code takes 5s. In fact, 50x is about the slowdown you used to see for ruby in the programming languages shootout, and it is still pretty bad. [1]

[1] [http://benchmarksgame.alioth.debian.org/u32/benchmark.php?te...](http://benchmarksgame.alioth.debian.org/u32/benchmark.php?test=all&lang=yarv&lang2=gpp&data=u32)

</comment></comment></comment></comment></comment></comment><comment><USER11> I must say, I made a beeline for the security page and was not disappointed. While I think they underplay the risk of memory corruption flaws (namely by emphasizing backups to be critical, rather than isolating user data to minimize compromise in the case of the site being owned), they otherwise do a fantastic job. I can nitpick a bit (and will probably send some recommendations to improve the guide a little) but I really have to commend them on that.

<comment><USER12> I knew I would find a statement like this in the security page: "So using modern language techniques programming in C++ is not more dangerous then programming in Java, Python or Ruby."

This is a belief that many C++ programmers have, and having spent much of my life in the past few years finding ways to break a C++-like memory model (Rust) I cannot emphasize enough how dangerously wrong it is. Safer than C, sure. As safe as a memory-safe language like Java, absolutely not.

<comment><USER13> What are you talking about?

The safety you mentioned comes from only two things: type safety and runtime array boundary checks.

There is no problem in implementing runtime boundary check in C++.

What else in C++ is less safe than in Java? Is there some kind of magic?

<comment><USER12> Iterator invalidation, returning references that outlive their referent, storing references in data structures that outlive their referent, invalidation of the "this" pointer, etc.

<comment><USER13> Basically those are examples of bare pointer manipulations (as references are essentially syntactically sugared bare pointers).

Although it can be mitigated with specially written collections (incompatible with the standard collections) and strict coding guidelines, I have to agree with you that the C++ memory model itself is insanely dangerous.

</comment></comment></comment></comment></comment><comment><USER14> I would like to see an implementation of our test suite [1] for CppCMS in order to see the CppCMS vs CPoll-Cppsp cage match. Any C++ experts who would be so kind to send us a pull request? :)

[1] [http://www.techempower.com/benchmarks/](http://www.techempower.com/benchmarks/)

<comment><USER15> I love that the code for the fastest database backed web server[0] found in this test suite, beating golang by 40% and node.js by 13x comes from a repository[1] that is filled with "all the code i've ever written since grade 11; also includes some forward-ported code from grades 8 and 9." including random class projects and a netcat replacement.

[0] [http://www.techempower.com/benchmarks/#test=query](http://www.techempower.com/benchmarks/#test=query)

[1] [https://github.com/xaxaxa/workspace/](https://github.com/xaxaxa/workspace/)

<comment><USER14> Yeah. I've considered adding a popularity attribute of some sort to use in the filters. E.g., "mainstream," "challenger," and "fringe." But to date, I've not put a whole lot of thought into how exactly I could objectively measure popularity. Perhaps Google search hits for "[name] web application framework."

<comment><USER16> That would be biased towards the frameworks named as some common word (like "go") being mistakenly marked as more popular than they are.

<comment><USER14> Indeed. I'm not really sold on the idea, and addressing this is not a priority.

I just wanted to point out that I'm aware that it's a little funny to see fringe frameworks compared head-to-head with mainstreamers. But the charts are but one data point to consider in evaluating options. :)

</comment></comment></comment></comment></comment><comment><USER17> Haha, I feel like there is a language bubble, everyone is using their interpreted/bytecode language, but people never realize how performance can be and is still always an issue. Even with the fastest computer, or cloud, or cluster, computer resources are always finite, and it's easier to reach that limit than anything else.

People relying too much on inteepreted or bytecode languages end up making abstraction on what happens on the metal, forget the basics of good enough performance, and bam, even the fastest server looks like a piece of garbage.

I'll always be conservative regarding language choices, because if a language makes things easier, it should also be taken with a grain of salt.

</comment><comment><USER18> CppCMS license is LGPLv3 - not too company friendly.

CppDB on the other hand is MIT or Boost. It is the best DB library I found so far for C++.

<comment><USER19> > CppDB on the other hand is MIT or Boost. It is the best DB library I found > so far for C++.

Have you evaluated Code Synthesis's ODB?

<comment><USER20> Are these key-value DBs like LevelDB / BerkeleyDB / etc? or maybe something more sophisticated?

<comment><USER18> Both CppDB and ODB are abstraction layers between your code and SQL (Postgres, mySQL etc). ODB is somewhat closer to ORM.

CppDB uses MIT or BSD license (you pick), ODB is under LGPL or commercial license.

</comment></comment></comment></comment><comment><USER21> "CppCMS is a Free High Performance Web Development Framework (not a CMS)"

This is so dumb. Why it is called CppCMS then?

<comment><USER22> [http://cppcms.com/wikipp/en/page/faq#If.CppCMS.is.not.a.CMS....](http://cppcms.com/wikipp/en/page/faq#If.CppCMS.is.not.a.CMS..where.does.this.name.come.from).

Quote from that page:

The original idea was: "Write your own CMS in C++".

The lead developer asked in 2008 for a better name but didn't get any suggestions, so the name stuck.

It was decided later to keep this name as it was already well known name and all the software around used cppcms as the namespace.

</comment></comment><comment><USER23> I looked into CppCMS a few months ago, I would love to use it but the lack of SSL/HTTPS is a huge detriment. If I have to stick nginx in front of it for SSL I may as well keep using fastcgi.

</comment><comment><USER24> I would be interested in how this perfoms compared to the web server implemented in Go

<comment><USER25> Or Erlang.

</comment></comment><comment><USER9> They've got translations in their example: [http://cppcms.svn.sourceforge.net/viewvc/cppcms/blog/trunk/p...](http://cppcms.svn.sourceforge.net/viewvc/cppcms/blog/trunk/po/)

Something more web frameworks should do.

</comment><comment><USER26> If the only benchmark is going to be against PHP + Wordpress, you might as well just say it's better than zero.

</comment><comment><USER27> It seems there's no documentation?

<comment><USER28> [http://cppcms.com/wikipp/en/page/cppcms_1x](http://cppcms.com/wikipp/en/page/cppcms_1x)

</comment></comment><comment><USER29> Odd wiki. Unlike Wikipedia, the front page is completely open to public editing.

</comment><comment><USER30> C++ isn't suitable for Web programming.

<comment><USER31> You do realize that the most widely used Ruby, PHP, Python, Perl, JavaScript and Java implementations all rely very heavily on C and/or C++ in one way or another, right?

Even when using JRuby, for example, it's still running on a JVM that's very likely implemented using one or both of C and C++.

Don't forget that the major web servers and web browsers are all implemented in C and/or C++, or in one of the many scripting language or other runtime implementations implemented using C and/or C++.

And that's ignoring all of the other infrastructure, like server operating systems, router software, and so forth that's implemented using C, or C++, or both.

Every line of code executing in your non-C or non-C++ web development language of choice likely depends on many thousands of lines of C and C++ code, even as you malign them here.

<comment><USER32> Yes, of course in the end everything runs on machine code. That does not imply though that it is reasonable to write your web applications in machine code or assembly or C or C++.

I'm not arguing whether or not C++ is actually suitable for web development, I just don't think your line of argumentation makes sense.

<comment><USER31> The majority of that C and C++ code is hand-written, and not just the output of a compiler or an assembler. They have directly been used to craft some of the most important, and often most difficult to implement, parts of the web application stack.

It's absurd to claim that C and C++ aren't suitable for web development when a huge part of basically every web app today is written in one or both of them. They aren't just suitable for web development; they are critical for it.

<comment><USER33> You're arguing semantics so let me continue the pattern.

In any non-trivial (jobs/money are on the line), web development != web programming. C/C++ is suitable for web programming, sure, but web DEVELOPMENT is an entirely different matter and is highly dependent on the availability of support, both community and professional, staff, and the competency threshold (or "learning curve") which is the knowledge you need to contribute meaningfully. The latter is one of the most important, especially since the vast majority of C/C++ programmers are not web developers, and doing something wrong in C++ is a lot easier than in memory managed, interpreted languages.

</comment></comment></comment></comment></comment><comment><USER34> 1997 called. They want their framework back.

<comment><USER34> While I still stand by what I said, I want to add a clarification. Some languages are good at some things while others are good at some other.

C/C++/etc are good for job intensive stuff that doesn't need fast maintenance cycle.

Dynamic language are good as web server because of their flexibility and their fast response time (In term of maintenance)

I wouldn't run an OS on Javascript and I wouldn't use C++ as a Javascript replacement in the browser.

<comment><USER13> Don't you think that first off you need to be an expert in all those languages, in web-technologies, have vast experience and up to date knowledge in all of that, to make such claims?

I'm pretty sure you just have wrong impression of C++, or to put it simply, you don't know C++.

In what way does C++ have worse response time in terms of maintenance? What do you mean exactly?

In general, statically typed compiled languages are better at refactoring than dynamically typed interpreted ones. Changing programs in the latter without fair test coverage means teetering on the brink of a catastrophe. Just recently I have seen an article on HN claiming unmaintainability of dynamic languages.

The truth is that every general purpose language is general purpose, and the claim about language suitability is a commonplace and an overgeneralization. You can do 3D games in JavaScript and client web apps in C++. Different languages have different drawbacks and that _may_ limit their applications, but C++ is _not_ targeted for OS development, and JavaScript is _not_ targeted for web development, both are just general purpose languages.

Speaking of "low-levelness" of C++, with all the modern features and libraries it is as high level as other languages, and in some aspects is even more high level, e.g. JavaScript or Python have no corresponding means as the template metaprogramming which is the high high level. C++ just _allows_ you to do low- level manipulations, you are not required to.

There are many myths and prejudices against C++ caused by ignorance and frustration of those who have not cracked it at a time.

<comment><USER35> Exactly! C++, well used, is largely a _vertical_ language, not a low-level or high-level one. That's why it's easy to create a fairly big system entirely in C++. And having worked with such a system, I can say that it's quite comfortable, in some ways that most so-called "higher-level" languages don't offer. For example, your comment about refactoring is absolutely spot-on.

Most criticisms of C++ rely on outdated prejudices. Some people still think that modern C++ code is rife with pointers to void and reinterpret_casts.

<comment><USER36> To be fair, bad code still is, and it's more common than most of us would like to admit.

</comment></comment></comment></comment></comment><comment><USER37> What's next ? A OS written in Ruby ? _snark_

<comment><USER38> It's not a Content Management System. Read the article.</comment></comment>


Ex-Google, Yahoo, Facebook employees start company to support Hadoop - <AUTHOR> http://venturebeat.com/2008/10/14/ex-google-yahoo-facebook-employees-start-company-to-support-hadoop-data-processing-platform/

<comment><USER2> Really cool. Hadoop is pretty mature and Yahoo's shown it works well on large clusters, but it can be a bear to install, maintain, and learn. I'm sure many companies wouldn't hesitate to pay a few bucks to get another company to help them get up and running faster.

</comment><comment><USER3> I do not quite understand why people build hadoop and lucene with java. Is a python & c++ implementation of mapreduce a better choice?

<comment><USER4> Because Java is an incredibly powerful tool that works across a large selection of platforms with minimal effort.

And, Hadoop is only the infrastructure, they have APIs for C, C++ and shell for the applications. <http://wiki.apache.org/hadoop/FAQ>

<comment><USER2> Python: [http://www.michael- noll.com/wiki/Writing_An_Hadoop_MapReduce...](http://www.michael- noll.com/wiki/Writing_An_Hadoop_MapReduce_Program_In_Python)

Ruby: <http://blog.rapleaf.com/dev/?p=16>

It's pretty hokey, either running through the shell or calling a REST service through a proxy, but it's been done if you're allergic to non-scripting languages.

</comment></comment></comment><comment><USER5> Is this the result of that cyprus trip?</comment>


Watercress tops list of 'powerhouse fruits and vegetables.' Who knew? - <AUTHOR> http://www.washingtonpost.com/news/to-your-health/wp/2014/06/05/finally-a-list-of-powerhouse-fruits-and-vegetables-ranked-by-how-much-nutrition-they-contain/

<comment><USER2> All of the Brassicaceae / Cruciferae [1] contain a group compound called Indole-3-carbinol [2] which is purported to have cancer protective properties.

1\. [http://en.wikipedia.org/wiki/Cruciferous_vegetables](http://en.wikipedia.org/wiki/Cruciferous_vegetables)

2\. [http://en.wikipedia.org/wiki/Indole-3-carbinol](http://en.wikipedia.org/wiki/Indole-3-carbinol)</comment>


Not Google Cardboard - <AUTHOR> http://moron4hire.tumblr.com/post/91360230105/who-needs-google-cardboard-a-couple-of-weeks

<comment><USER2> "Show HN" is for something you've made that people can play with. A blog post doesn't count as a Show HN, so we've taken that out of the title here. For more information, see the Show HN guidelines at [https://news.ycombinator.com/showhn.html](https://news.ycombinator.com/showhn.html).

</comment><comment><AUTHOR> I've been going through the demos that Google provides on the Cardboard site and it's pretty incredible. There were some fleeting moments of forgetting what I was doing. If you can get the side light blocked out, get it comfortable on your face, let go with your hands, and just relax, it feels like you're standing in a completely different room.

Input is nigh impossible. They have a magnetic clicker in their design. I reach under the bottom of mine and tap the screen. It's kind of weird to see a super large finger come in and poke at an invisible glass wall between me and the thing I'm trying to touch. Regardless, I don't think I'd like the clicker. Touching the contraption cements it into reality. The best effect for me came from letting go. Voice commands coupled with radial heads-up menus are probably the best bet. Perhaps nod-to-confirm, shake-to-cancel could be implemented.

The demos that had flat wall surfaces seemed the most convincing. There was a tour of Versailles that at first seemed really bad, while it was still outdoors. Once it got indoors, the photo quality didn't actually improve at all, but the notion of the walls really helped sell it. I stood in front of a painting of Marie Antoinette and surprised myself when I tried to step closer, only to bump into a table in meatspace.

There was a demo of YouTube where it simulated a large screen TV in front of you, and you peered around the TV to select the video, like they were on shelves 360 degrees around you. That one also worked really well, other than being stuck with someone else's selection of videos, it really felt like sitting in front of a big TV in a very dark room.

A video from a helicopter flying over the Great Barrier Reef further supported my experience that constrained rooms are better.

There were some other things, but not a lot to write home about. I'm surprised there aren't any telepresence demos yet.</comment>


Cows painted with zebra-like striping can avoid biting fly attack - <AUTHOR> https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0223447 <comment><USER2> The fact that zebra's stripes evolved to avoid getting bit by flies (and specifically, the tsetse fly, which transmits a deadly) has been known since at least the 80s. I wonder why these researchers are doing these experiments now.

In certain parts of Africa zebras have lost most of their stripes, and the map of stripeless zebra is strongly correlated with the map of where the tsetse fly are less active.

</comment><comment><USER3> Academic Literature:

> Six Japanese Black cows were assigned to treatments using a 3 × 3 Latin- > square design. The treatments were black-and-white painted stripes, black > painted stripes, and no stripes (all-black body surface). Recorded fly- > repelling behaviors were head throw, ear beat, leg stamp, skin twitch, and > tail flick. Photo images of the right side of each cow were taken using a > commercial digital camera after every observation and biting flies on the > body and each leg were counted from the photo images. Here we show that the > numbers of biting flies on Japanese Black cows painted with black-and-white > stripes were significantly lower than those on non-painted cows and cows > painted only with black stripes. The frequencies of fly-repelling behaviors > in cows painted with black-and-white stripes were also lower than those in > the non-painted and black-striped cows.

2020 netspeak:

        Biting Fly: Bzzzzzz
        Cow: *Disguises as zebra*
        Biting Fly:
        Biting Fly:
        Biting Fly: Bzzzzzz *flies away*</comment>


Ask HN: Why do most sites still not track state well? - <AUTHOR> I have a bit of frontend experience and a general understanding of how backends work, but I guess the "glue" layer between them must be more difficult than I think. Why is it that when I need to reset a password, or log in or out, the site that I'm interacting with is unable to handle even basic state? I'm not blocking cookies, so it can't be that.
Can't the page/endpoint that I left in order to reset my password be encoded into the password-reset URL, so that after my password has been reset I'm brought back to where I was?
Password resets aren't the only manifestation of this, but they're the most obvious. <comment><USER2> I built a user management system circa 2001 that kept state across login and other events using the method you suggest. It even could save POST contents around a login.

I used it for a few web sites that cumulatively had 500,000+ members, open sourced it, didn't get any interest from anyone else. My system was mostly written in PHP, but the actual cookie handling module could be coded up in about 50 lines of code and I ported it to Perl, Java, Cold Fusion and a few other languages.

My viewpoint at the time was that user management was the central service behind a "portal" web site, that is, you would log into Yahoo once and what bound it all together was they you had one login. In contrast to "PHPNuke" which was one huge application that had half-baked implementations of different services, you could pick best of breed applications and tie them into the auth system.

At the time there was a lot of code reuse in terms of applications (WordPress) and there were a few proto-frameworks like Struts, but Ruby-on-Rails was the first web framework that was transformational.

User management libraries didn't catch on until federated login came along.

I think today people are often ignorant and wouldn't care a bit about user experience if somebody wasn't threatening that something awful was going to happen if their site wasn't accessible. Half of the job of the W3C is normalizing deviance by not standing up against the pop-in windows that harass you everywhere on the web the same way those shoes you saw on Zappos five years ago are still following you around.

On the other hand, my priorities have always been different from other people. I've never bolted pagination onto a user interface after the fact. I have tried to parallelize something without chunking the work (to reduce overhead so it actually speeds up) but that was a n00b mistake 20 years ago -- people write these blog posts where they make it sound like Actors or some other shiny new thing is the key thing and no, you'll have to chunk no matter how you do it. (e.g. is more fundamental!)

<comment><AUTHOR> Thanks for the great response. If I understand you, the problem is that this sort of state handling works at a low(ish?) level, and the frameworks that most people use nowadays didn't prioritize it, so in order to do it properly you'd have to write custom code at a lower level of abstraction than most devs are comfortable with? Basically, it was handled poorly in libraries that are now universally used? Please correct me if I'm wrong.</comment></comment>


A warning from South Korea: the 'fantasy' of returning to normal life - <AUTHOR> https://www.ft.com/content/d68d6292-0486-4bfc-bf5c-54ce850a3f7a <comment><USER2> _"When a person tests positive, a team of two investigators immediately begins contact tracing. We go to the scene and check CCTV footage and look into who the patient was in contact with. We place those contacts under self-quarantine and test them."_

A warning indeed.</comment>


So you want to start a startup? - <AUTHOR> https://medium.com/me/stats/post/5c077556ddb5 <comment><USER2> The link doesn't seem to work :(</comment>


Does business school teach people how to assert power before they're ready? - <AUTHOR> http://sachin.posterous.com/does-business-school-teach-people-how-to-asse

<comment><USER2> """Business school should stick to teaching the mechanics of running a successful business as much as possible, and leave the psychology out of it."""

This is an interesting thought that I don't agree with. Management is simply the art and science of achieving results through people. If you broke management down into its major components, I believe that you would find that management is mostly rooted in psychology with a sliver of economics. If you leave the psychology out of it, there isn't much left.

(X-posted to blog comments)

<comment><USER3> Agreed, if you want to know how things really work one of the best articles I've ever read on the subject is by Venkat from ribbonfarm.com. Read the Gervais Principle it's long but well worth it, lots of fresh thinking, and if you watch the office you'll see it in a whole new light.

Part 1 [http://www.ribbonfarm.com/2009/10/07/the-gervais- principle-o...](http://www.ribbonfarm.com/2009/10/07/the-gervais-principle-or- the-office-according-to-the-office/)

Part 2 [http://www.ribbonfarm.com/2009/11/11/the-gervais- principle-i...](http://www.ribbonfarm.com/2009/11/11/the-gervais-principle-ii- posturetalk-powertalk-babytalk-and-gametalk/)

Part 3 [http://www.ribbonfarm.com/2010/04/14/the-gervais- principle-i...](http://www.ribbonfarm.com/2010/04/14/the-gervais-principle- iii-the-curse-of-development/)

Part 4 [http://www.ribbonfarm.com/2010/10/14/the-gervais- principle-i...](http://www.ribbonfarm.com/2010/10/14/the-gervais-principle-iv- wonderful-human-beings/)

<comment><USER2> I'll read them all over the next couple of hours. (I am sick in bed anyhow :)) Readings that I recommend are "The Great Jackass Fallacy" (I got my copy <$10 on Amazon) and "Drive" by Daniel Pink. There's also a Youtube/TED video that sums up "Drive" very well. <http://www.youtube.com/watch?v=rrkrvAUbU9Y>

</comment></comment></comment><comment><USER4> Sachin's conflating the teaching of techniques he finds distateful with the general notion of teaching management skills in general.

Trying to teach the mechanics of running a business without going into psychology is like trying to teach the piano with only a book and a metronome. I cannot tell you how many times an MBA2 student (I'm currently an MBA1 at Michigan, which is awesome and amazing) has made a comment like "I was in [x] situation that was eerily just like a roleplay situation in my MO class."

The mechanics of running a business are exactly the sort of thing you can learn in a book (or a blog); dealing with people to get optimal outcomes takes a classroom environment with an excellent teacher and groups of other students. You have to bang the keys to really learn how to be an effective manager of - and hopefully, leader of - people.

<comment><USER5> Weird, I always thought you were OP. Guess I should click those usernames more often.

<comment><USER4> That pisses off the other Sachin a lot more than it bothers me. :)

</comment></comment></comment><comment><USER6> In my experience with business school and engineering school, EVERYONE asserts power before they earn it. Others may call it "fake it till you make it", but I would say that over-confidence and power games are rewarded in social constructs which make me (and other techies I know) very uncomfortable.

</comment><comment><USER7> Does tech school teach people to assert competence before they're ready?

<comment><AUTHOR> you can't compare technical skills with teaching attitude

</comment></comment><comment><USER8> "Business school should stick to teaching the mechanics of running a successful business as much as possible, and leave the psychology out of it."

I disagree. At least 85% of success is knowing how to put on a good show. Knowing what behaviors/signals/designs will get what reactions is key, and it's something that can really only be learned from a competent teacher.</comment>


Startup school 2007 videos? - <AUTHOR>

<comment><USER2> Hopefully the videos will be out soon. I'll add them to this feed when they are. In the meantime, here's the podcast feed: <http://feeds.feedburner.com/Ycombinator-StartupSchool>

Note that currently the 2007 talks are from my lower quality recordings, but I'll add the official ones when they come out.

<comment><USER3> _'... 2007 talks are from my lower quality recordings ...'_

though some of the comments, additions, cues over the talks are pretty funny

</comment></comment><comment><AUTHOR> It would be nice to have access to the videos for the event.

</comment><comment><USER4> Did you try searching? <http://wiki.startupschool.org/doku.php>

<comment><AUTHOR> Yes, but the videos are not there; only the audio recordings are.</comment></comment>


Braess' paradox - <AUTHOR> https://en.wikipedia.org/wiki/Braess%27_paradox <comment><USER2> Another example of the Nash equilibrium being worse than the group optimum is some examples of Tragedy of the Commons, things like overfishing. It's possible to create a model that shows that a more regulated environment leads to more fish _per individual_ than the Nash equilibrium.

I don't think it's appreciated enough that some forms of regulation actually do make things better for everyone, and that acting only in one's own self- interest can lead to results that are provably worse for your own self- interest.

<comment><USER3> "I don't think it's appreciated enough that some forms of regulation actually do make things better for everyone, and that acting only in one's own self- interest can lead to results that are provably worse for your own self- interest."

As someone who is libertarian-ish, I find that a funny thing to say which doesn't jive with reality. Western countries usually have _huge_ governments with _huge_ amounts of regulation, and the default assumption of almost everyone around me seems to be that regulation is the way to deal with almost anything we don't like.

So saying that it's not appreciated that some regulation makes things better for everyone sounds to me like saying "it's not appreciated enough that clothes make us warm".

(Unless, of course, you're referring only to specific circles, e.g. libertarian economists).

<comment><USER4> This contributor to the talk page will appreciate no such thing:

this whole article is poorly veiled propaganda by the current communist administration to justify government ownership of roads. After all, who but the government could remove a link to benefit "everyone"? By generating this absurd fake propaganda, it hopes to ply public sympathy toward state involvement and ownership. Naturally the actual proposed "facts" are totally rubbish: it is obvious that you could never decrease traffic congestion in an area by removing already congested roads. The article is an obvious hoax from the first sentence: "Braess's paradox, credited to the mathematician Dietrich Braess" redlinks. I've added a deletion tag. 18.104.22.168 (talk) 01:27, 31 March 2010 (UTC)

<comment><USER5> I am always amazed that people with such thought processes are not locked in padded cells, or at least taken to some provincial farmhouse to rest for a while before returning to society.

<comment><USER6> The Wikipedia talk pages are a more effective prison than any mere set of walls.

</comment></comment></comment></comment></comment><comment><USER7> I had a hard time understanding how this could be true, until I read the simple math example presented. The problem seems to be one of externalities. Each individual driver is selfishly optimizing for his own travel time, but by choosing to drive in a short-congested route, instead of a longer-sparse route, he's negatively impacting the travel times of everyone else around him.

It's been well proven in economics that externalities, if unregulated, will produce sub-optimal outcomes. That the way to restore optimal outcomes is to impose a tax that's proportional to the negative externality imposed on others. In an ideal high-tech world, this can be achieved by having differential prices that each driver has to pay, to use each road/highway. In a low tech world, narrowing/closing a road is akin to imposing a tax on certain routes, which explains why it can counter-intuitively help us get to a more optimal outcome.

<comment><USER8> So if I understand it, adding road signs before the new route adding estimated travel times (that take congestion into account) will fix things? The paradox arises from considering only route length, and not congestion?

<comment><USER7> No, the problem comes from considering only your travel time, and not how your choices impact other drivers. The example given in the wiki demonstrates that even road signs with estimated travel times, will still not fix the above problem.

<comment><USER9> However the example have a road that takes 45 minutes regardless of how many drivers go through it, that is not very realistic is it. nowdays we have apps that can tell us in real time how long each path will take according to current traffic, and the time is never constant.

<comment><USER7> It's actually pretty realistic for very lightly congested roads. At those levels of congestion, the primary bottleneck is the distance and speed-limit. Marginally increased traffic will have minimal impact on travel time.

In contrast, for heavily congested roads, the primary bottleneck is the congestion itself. Hence, marginally increased traffic will have a proportional impact on travel time.

</comment></comment></comment></comment></comment><comment><USER10> The german wiki article has an paragraph about an analogue effect in mechanics in which adding an additional spring to support a hanging weight can causes the weight to hang lower than before. Which I found quite interesting.

<comment><USER11> I don't speak German, can you provide a link to the paragraph and/or a translation?

<comment><USER12> [https://de.wikipedia.org/wiki/Braess- Paradoxon#Mechanisches_...](https://de.wikipedia.org/wiki/Braess- Paradoxon#Mechanisches_Analogon)

<comment><USER13> The article describes a weight initially hanging from two strong springs (blue) and two weak springs (yellow). The subsequent addition of the strong red spring in the indicated position will, according to the article's calculations, make the weight hang lower overall. I also find that pretty counterintuitive!

Edit: the explanation given is that the red spring redistributes forces so that more force is applied to the weak springs and less force to the strong springs, causing an overall greater total extension.

</comment></comment></comment></comment><comment><USER14> >if each driver is making the optimal self-interested decision as to which route is quickest, a shortcut could be chosen too often for drivers to have the shortest travel times

Does this just mean people are making bad decisions based on incomplete information?

For example, would this still be a problem if everyone was using Google Maps routing which changes routes based on congestion patterns?

<comment><USER15> This applies even with perfect information, as long as drivers are completely selfish.

<comment><USER16> How's that work? Is the idea that an individual driver would have to take a penalty in order for everyone else to have better road use?

(Also, is this specified somewhere? It seems like the paradox relies on people making their minds up ahead of time (hence imperfect information) and then not changing their minds halfway.)

<comment><USER17> Exactly. In other words, in order to globally optimize for throughput, you would have to deoptimize some people's routes.

<comment><USER18> Normally you would do this through prices, but most roads don't work that way.

<comment><USER19> Hwy 635 in Dallas has a variable-toll expressway lower level.

<comment><USER18> There are definitely many variable-tolled roads. The trouble is that frequently they are embedded in a system of non-tolled (or, really, not- allowed-to-be-tolled) roads, and that the tolling is not well-focused on congestion. Basically, the Braess "paradox" emerges if you try to make a road network that's optimal for travel patterns over an entire day, without using pricing to ration scarce space.

</comment></comment></comment></comment></comment></comment></comment><comment><USER19> I learned about this in a course titled Dynamics of Complex Networks and Systems taught by Mark Spong, a robotics control theorist active in dynamical systems research. Ever since learning about Braess' Paradox in that course, I have noticed examples of it in the highways of DFW. The simplest case to spot is where an off-ramp feeds directly to the next on-ramp, creating a "shortcut" for drivers who exit and re-enter the highway.

I've tried to make the case that by closing "key" exits on the highway, we could reduce congestion. But who would be responsible for making that likely unpopular decision?

<comment><USER20> They do this on some exits. Speak with your county commissioners, the police, etc. [https://www.youtube.com/watch?v=9Ia4pzskae8&list=PLEEC4A6D0A...](https://www.youtube.com/watch?v=9Ia4pzskae8&list=PLEEC4A6D0A6CD0C5B&index=9)

</comment></comment><comment><USER21> Yeah. Good road planning in the future should avoid situations where a driver can save 5 minutes by delaying 10 others for 1 minute. Adding new roads can certainly create such situations where they weren't before. You need serious effort to figure it out though, because adding a new road can have lots of other effects as well.

</comment><comment><USER22> This reminds me of the example of the placement of a barrier at an exit to reduce the number of interactions to improve pedestrian flow

[http://www.gkstill.com/CV/PhD/Chapter3.html](http://www.gkstill.com/CV/PhD/Chapter3.html)

Scroll down to 3.5.3 Flow through a door.

I can't find the original article I read years ago but it showed a large round barrier to be the most effective, and more than one barrier for larger numbers of people

<comment><USER23> Dang you hit it before me - I was wondering the same thing, and posted about it to another comment here...

</comment></comment><comment><USER24> That's where variable toll expressways are a great idea. If you add a faster way and equip it with variable tolls, the paradox shouldn't appear. The new way will not be congested (if the toll has no upper limit) and drivers taking the traditional way will experience less congestion.

</comment><comment><USER25> The Braess paradox applies to all networks, including the Internet, the financial system, FaceBook, etc.

It seems crazy, but interconnecting more nodes to improve efficiency (as measured by travel time, costs, etc.) can paradoxically make the whole network less efficient!

</comment><comment><USER26> this wiki article seems to get posted every few months... [https://hn.algolia.com/?query=%09Braess'%20paradox&type=stor...](https://hn.algolia.com/?query=%09Braess'%20paradox&type=story)

</comment><comment><USER27> See also, the Price of Anarchy [1] and How Bad is Selfish Routing [2].

1 [https://en.wikipedia.org/wiki/Price_of_anarchy](https://en.wikipedia.org/wiki/Price_of_anarchy)

2 [http://theory.stanford.edu/~tim/papers/routing.pdf](http://theory.stanford.edu/~tim/papers/routing.pdf)

</comment><comment><USER28> Hm, can't this be circumvented by pushing towards a correlated equilibrium ([https://en.wikipedia.org/wiki/Correlated_equilibrium](https://en.wikipedia.org/wiki/Correlated_equilibrium)) instead of a Nash equilibrium?

</comment><comment><USER29> Is this a function of turbulence? When I think about 'perfect' traffic, I think about a pipe with water flowing through it without eddies. Branch and merge a pipe along its length and you'd introduce all sorts of ripples and turbulence.

<comment><USER30> IT doesn't have to

[https://en.wikipedia.org/wiki/Braess%27_paradox#Example](https://en.wikipedia.org/wiki/Braess%27_paradox#Example)

</comment></comment><comment><USER31> Someone who is perhaps better versed in machine learning:

Is there some sort of intuition here that would connect this paradox to the effectiveness of dropout in deep neural networks, or am I drawing connections where there aren't any?

<comment><USER15> Having studied both topics (just coursework), I don't think so.

<comment><USER31> That's what I assumed, it feels like there should be an intuitive connection (both are kind of situations where a local optima is avoided by removing the greedily optimal path on a graph structure), but I may be wrong, i dunno.

<comment><USER32> I'm not sure dropout has anything to do with local optima or removing greedily optimal paths, since it is random.

The original dropout paper's handwavy justification for dropout is that it prevents co-adaptation. It prevents individual units (nodes/neurons) in the network from relying on specific units in the previous layer firing as well. This is a bad thing because it's fragile (if one unit is off). I say handwavy because this is just intuition; there is not really any proof that this is actually what is happening.

Another commonly cited motivation is that dropout is like learning an ensemble of multiple networks.

The only paper I've seen that theoretically analyzes dropout is: [https://arxiv.org/pdf/1506.02142v6.pdf](https://arxiv.org/pdf/1506.02142v6.pdf), which proves it's equivalent to approximating gaussian processes (this is beyond me).

<comment><USER31> I think that you could consider the co-adaption as a form of greedy local optima (I think this is a very handwavy explanation, though, and I'm having trouble forming it better, so take with that what you will). Dropout prevents that by not allowing the 'greedily optimal paths' to form, since sometimes they don't exist, so you can't rely on them too much.

<comment><USER32> Sure, but dropout actually increases training error (makes you _less_ likely to find the globally optimal training error), with (sometimes) a decrease in generalization error (test error). So any connection is very thin.

<comment><USER31> Oh actually that's a really good point and I think makes the difference clear (there isn't such a train/test divide in the driving example).

Thank you :)

</comment></comment></comment></comment></comment></comment></comment><comment><USER33> It will be interesting to see a case where all drivers use real-time traffic updates from services like Google Maps

</comment><comment><USER34> HN might as well pin this article to the top since it gets reposted so much.

<comment><USER35> I benefited from this repost.

</comment></comment><comment><USER36> "Perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away."

\- Antoine de Saint-Exupery</comment>


Ask HN: Does anyone hire average developers? - <AUTHOR> Obviously most companies do by definition. <comment><USER2> Depends on how you define average. I work on a software team that's about twice as productive as any other team our size that I've previously encountered, regardless of skill. But the technical abilities of everyone on the team (with one possible exception out of ten) fall well within the fat part of the Gaussian distribution. We're all "average" technically.

But I wouldn't call anyone here "average". We excel as a team because everyone's "soft" skills are well above average. Self management, in particular. But also pride in one's work. Ability to collaborate effectively. Willingness to learn from one another. Lack of ego. Interest in solving business problems rather than "sexy" technical problems. Empathy.

We're building what's 95% a CRUD app anyway, though with some additional complexity to handle scale. Engineers with top 10% technical skills would be bored out of their skulls here. But if we hired "average" engineers without those exceptional "soft" skills we'd be accomplishing less than half as much as we do now.

</comment><comment><USER3> :) average is probably worse than you think. I've followed the LinkedIn status of couple guys I knew as "developer interns" that didn't make our cut to get an offer at the end of their internship. We always told them we only had a certain number of spots to make them feel a little better. The truth was some of us devs would get together and talk about who wasn't "getting it" or was generally clueless.

They were nice guys for the most part and I follow them mostly hoping they will find a more fitting dev job somewhere else. This isn't always the case. Maybe half the new devs that didn't make our cut end up leaving software completely, or finding a job on the periphery like analyst, tester, or UI mockup designer.

Software development requires a very logical mind and the ability to dream up the idea of a thing and make it real. I think the creative side of software development is downplayed, it's definitely the most fertile field for inventor types.

Just like making pretty paintings, programming feels natural to those that can do it. And just like art, everybody can draw but the quality of the artwork varies wildly.

Again with the art analogy, some people are just not capable of being taught how to program above a basic level of understanding. In the beginning I sunk tons of time into helping some of our more clueless interns become productive. I'm kinda jaded now, more in the "sink or swim" camp, because I saw just how much the innate ability to grasp programming concepts affected outcomes. A few of our interns that seemed to be charity cases at the beginning became semi competent developers within a few months. Others came in with a CS101 level understanding and never advanced beyond that.

Eventually I decided my time was better spent helping those who were catching on quickly.

So the main reason everyone strives to get good developers is because so many are terrible. The freedom in software design means the quality depends a lot more on creativity than in branches of engineering with strong physical constraints. It's a lot easier to decide if someone designs roads properly than if they write "good" software. The terrible developers end up doing menial jobs like maintaining a library's website or being the "tech guy" at a small business.

Another example is that extremely successful software like Slack or Whatsapp was designed by just a couple very creative and good engineers. As powerful, smart, and huge as a company like Google is, they can't manage to outcompete the couple guys behind Slack or Whatsapp.</comment>


Firefox to ban unsigned extensions and will not allow local user override - <AUTHOR> https://blog.mozilla.org/addons/2015/02/10/extension-signing-safer-experience <comment><USER2> Asking developers to user a developers version of Firefox is quite smart. Developers are technically adept enough to understand the difference between branded and non-branded Firefox and might even want all sorts of add-ons that help developments but otherwise are annoying to browsing.

I hope Mozilla starts making it easier to have and run multiple Firefox browsers with different profiles at the same time. It is possible now, but cumbersome.

</comment><comment><USER3> > This gives developers great power and flexibility, but it also gives bad > actors too much freedom to take advantage of our users.

Ask.com toolbar? Also bites into their bottom line as a whores to search engine money</comment>


Rand's running (obv) and he's accepting Bitcoin - <AUTHOR> http://www.washingtonpost.com/blogs/the-switch/wp/2015/04/07/rand-pauls-campaign-takes-bitcoins-heres-why-thats-a-genius-political-move/

<comment><AUTHOR> ... bring on the down-votes!</comment>


The Hardest Problems - <AUTHOR> http://allenc.com/2014/01/hardest-problems/ <comment><USER2> Requiring lots of effort != Hard (Engineering). Anything that requires a lot of money instead of ingenuity is particularly unattractive to engineers, I believe.</comment>


Ask HN: How can I help digital privacy and activism by slinging code? - <AUTHOR> Writing code seems like my highest leverage activity. Where can I help? In light of the recent US election, I'd like to contribute. Ideally it'd be around secure communications, digital privacy, and helping activists spread their message and organize. <comment><USER2> Welcome to the fray.

First: decide what your skills are, and where you can contribute.

Try the Tor Project first: [https://www.torproject.org/getinvolved/volunteer.html.en](https://www.torproject.org/getinvolved/volunteer.html.en)

The community is great there and very accepting of newbies that have the skills.

Signal / open whisper systems also are open source and accept pull requests:

[https://github.com/whispersystems/](https://github.com/whispersystems/)

Read the anonbib:

[http://freehaven.net/anonbib/](http://freehaven.net/anonbib/)

Also, there are other mixnet/darknet projects that could use help, such as i2p:

[https://geti2p.net/en/get-involved](https://geti2p.net/en/get-involved)

And if you want decentralized, mesh communications, cjdns and the Hyperboria experiment:

[https://hyperboria.net/](https://hyperboria.net/)

Don't forget: if you are a developer, you actually make money, too. Vote with your dollars as well - donate to these persons. Many are 501c3's and it is tax deductible for you.</comment>


Quora's first acquisition is Arab Spring instigator's Q&A site Parlio - <AUTHOR> http://techcrunch.com/2016/03/30/quora-parlio/ <comment><USER2> Please, cut it out with the bullshit. This guy played a negligible role in the Arab Spring. The only reason he received the media attention he did was because he held a prominent position at an American company.

Calling him the Arab Spring's instigator is an insult to Egyptian activists and organizers, not to mention the revolutionaries in Tunisia.

<comment><USER3> Take a fucking look around.

How many people in prominent positions in well known companies open their mouths when there is a obvious stink in the air?

I have massive respect for the guy.

<comment><USER4> The actual instigator: [https://en.wikipedia.org/wiki/Mohamed_Bouazizi](https://en.wikipedia.org/wiki/Mohamed_Bouazizi)

Fuck a prominent position. He doesn't have a prominent position, and every single large exec would open their mouth when their entire country is being destroyed.

Fuckin' hell HN. As an Arab, I'm severely disappointed that everyone forgot the reason the Arab Spring started was because a 26 year old burned himself alive because he couldn't afford to live. Not some social media savvy guy from google.

<comment><USER3> So whats the lesson? To me it's pretty simple. You want change learn to do it the right way. Gandhi, MLK and Mandela didn't burn themselves and get change over night. And guess what, being media savvy is how you make change happen. There are to this day people who will call those three characters the most media savvy men of their time.

If marketing can sell Coke, Donald Trump and Kim Kardashian it can be used to sell other things too. Positive things. Constructive things. To pretend without media savvy your message will get out and have influence is highly misguided in this day and age.

<comment><USER5> What fucking media and marketing are you even talking about? I am a Tunisian citizen and I have never heard of this guy until these last couple of days.

<comment><USER3> Well I have. And I deeply respect what he is trying to do. It's solution oriented unlike cheap talk.

[https://www.youtube.com/watch?v=HiwJ0hNl1Fw](https://www.youtube.com/watch?v=HiwJ0hNl1Fw)

<comment><USER2> Give it up. I left out an important part in my original post: this guy only got media attention in America. No media attention in the Arab world, because being a Google employee didn't make his sacrifice any more significant. The European media had enough sense to not focus on this guy either. Only the insufferable mainstream American media is self-centered enough to push the narrative that an employee of an American company was instrumental in the Arab Spring.

<comment><USER6> I am Arab. Please speak for yourself. Nobody appointed you spokesperson for Arab opinion.

I think Wael has played and continues to play a very important role.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER7> Does anyone know how Quora plans to make money?

<comment><USER8> Advertising. They're loaded with search traffic. Loyal direct visits not so much. Anyone in Quora, refute me if you want.

<comment><USER9> Their personalised email digest is outstanding so I would expect this to drive a lot of return visits.</comment></comment></comment>


Where Did App.net Go Wrong? - <AUTHOR> http://arielmichaeli.com/2014/07/where-did-app-net-go-wrong/ <comment><USER2> > _If Dalton's idea of the platform was really to create a Twitter-like > messaging service he should have made it easier to access._

This is actually the complete opposite of what happened, and I think the key to where they went wrong.

Dalton created a Twitter-like messaging service as a way of showcasing what App.Net could do. But he seemed very intent on downplaying the service and focusing on App.Net as a platform. This is what caused all the confusion; most of the actual _users_ of App.Net were primarily users of Alpha, and anyone who was creating a product with the App.Net platform were facing the confusion of people thinking that's like creating a product with Twitter. I believe the reason Dalton did this was because his dream was for him to solely care about the platform and for other developers to provide all the apps. But nobody else was going to provide an Alpha clone, nor should they.

What Dalton really should have done was immediately spun off Alpha into its own full-fledged product. Give it a snappy name (heck, Alpha isn't too bad, certainly better than App.Net), give it its own domain, and make it very clearly a distinct product from App.Net. Then he could brand it all he wants as "built on top of App.Net" or whatever, and having login, etc. go through the app.net domain, but ultimately it should have been immediately recognizable as what it was; a Twitter-like messaging service (with much better features), that was built on top of App.Net. Just like a third-party app would have been.

This would have made it much easier to understand. People talking about App.Net would be obviously talking about the platform, people talking about Alpha would be talking about the messaging service, and trying to encourage people to build new apps on top of the App.Net platform would have made far more sense than convincing people to build new apps on top of what they think is basically just a Twitter clone.

<comment><AUTHOR> I fully agree. Separation would have made it easier to understand it's not the goal but rather an implementation.

</comment></comment><comment><USER3> To me, it was two things:

1\. As a Twitter replacement, it solved the wrong problem. They seemed to think that the flaw with Twitter is that it's free. In my opinion, the flaw with Twitter is that it's centralized.

2\. Every time I mention how the Twitter replacement isn't compelling, people would talk about how that's not what it is, it's a general platform for Stuff that just happens to be something you can use to build a Twitter replacement. But they never really explained what it _was_ , just that it wasn't merely a Twitter replacement.

<comment><USER4> "...They seemed to think that the flaw with Twitter is that it's free. In my opinion, the flaw with Twitter is that it's centralized..."

The flaw that made me leave Twatter for App.net was the ever-increasing amount of stinking advertising [laughingly known as 'Sponsored Tweets' ] littering my stream, combined with the relentless barrage of spam emails from Twitter themselves.

Alpha offered everything Twatter did, as well as double length posts and without spamverts. Unfortunately [and I agree with previous comments on App.net's piss-poor marketting] few people even knew it existed.

</comment></comment><comment><USER5> Where did App.net go right? I generally like the vision that App.net was promoting, but when you are trying to build something that relies on network effects for success you need to remove every impediment to attracting users. They've had several glaring impediments since day 1.

<comment><AUTHOR> I think trying to monitize the way they did was a big mistake. A social network isn't really a business tool and mass-market consumers aren't ready to shell our a monthly fee for a new and unnecessary way to communicate.

Twitter, the giant that's integrated into pretty much everywhere, is still fighting for engagement and new signups. So for a company to come and challenge them with such a vague value proposition is plain bad.

<comment><USER6> I think that makes sense. I wonder though if WhatsApp is an outlier - did they always charge a flat fee to use their messenger client, or is that a recent thing (after they attracted immense numbers of users)?

<comment><AUTHOR> That's a somewhat different use-case though because the alternative (text- messages) always had some sort of price tag, and iMessage wasn't around when WhatsApp started.

<comment><USER7> There were free mobile clients for email, Skype, Google Talk, AIM, Yahoo, ICQ, etc.

<comment><AUTHOR> True, but they're only extending an existing service you have to be a part of.

</comment></comment></comment></comment></comment></comment><comment><USER8> App.net was born out of a visceral, snap reaction to Twitter's API decisions that, really, the mainstream didn't really care about. That's not enough to build a product on top of. What's more, while I sympathize with the developers who were negatively impacted by Twitter's decisions, one could argue the choices they made were _right_ for the product. If you believe that, App.net's decision, the inverse, was _wrong_ for the product (reducing quality and focus).

Or in short, most people just don't care.

<comment><AUTHOR> That's definitely what it feels like, a visceral reaction, but from everything I could piece together it was about much more. Dalton tried to create a platform for communication that _could_ power a twitter-like service.

But... he never really explained it and in the absence of a message everyone had to figure out the message on their own using what's most obvious.

That's how it became a twitter-clone/wannabe.

<comment><USER8> Yes, I agree. Selling a platform is different from selling a product - especially when you're speaking directly to the consumers. Consumers want the product, not the platform.

</comment></comment></comment><comment><USER9> The only thing that went wrong with App.net is that it felt like you had to be part of some inner circle to understand what they were actually doing. Their communication and marketing was, and still is, diabolical. I still have no idea what they are selling.

<comment><AUTHOR> I totally agree, a clear message can make it or break it for a service, especially one with such a strong competitor.

</comment></comment><comment><USER10> I think it really boiled down to the fact the market for "paid Twitter" was too small to support the kind of network effects required to create an entire ecosystem.

<comment><USER11> I have never heard even one compelling argument for why paying for app.net makes sense.

I pay money and...? I honestly have no idea.

<comment><USER12> You pay money, Dalton becomes a partner at YC.

<comment><USER13> Everybody wins!

Oh. Wait.

</comment></comment></comment></comment><comment><USER14> Site appears to be down, Google cache version: [http://webcache.googleusercontent.com/search?q=cache:RxMTeUF...](http://webcache.googleusercontent.com/search?q=cache:RxMTeUF5-CkJ:arielmichaeli.com/2014/07/where- did-app-net-go-wrong/&hl=en&gl=us&strip=1)

<comment><AUTHOR> Thanks for sharing the cached version! The blog is back up and is fortified with a few layers of caching :)

</comment></comment><comment><USER15> App.net's problem was that it was a philosophy, not something that people would actually want to use.

</comment><comment><USER16> I'm the first person to criticize free online media, but they should have remained free until gaining a critical mass of users and THEN charged them to stay on.

That's how Twitter should work...think Lady Gaga or Barack Obama are going to leave Twitter over a $200 annual fee now? Nope, but they're sure as hell not joining an unproven network with an upfront $20 annual fee.

Of course, you'd have to be clear about this in advance. Nobody likes an unexpected bait-and-switch.

<comment><AUTHOR> Or use some sort of a freemium model much like the one they ended up doing anyway.

<comment><USER16> I should have been clearer...I think that model is effective only when users who have disproportionally significant influence are charged.

So Gaga and Obama should actually be charged an annual fee of something like $20,000. Maybe it's tiered, so someone like Robert Scoble pays $2,000.

Going by number of followers, Gaga's influence is an entire order of magnitude higher than Scoble's, so it makes sense for her fee to be an order of magnitude higher.

Everyone else below a certain threshold is free. I wouldn't (and shouldn't) pay for Twitter...it gives me little influence.

</comment></comment></comment><comment><USER17> Main problem was that Twitter wasn't quite bad enough for people to stop using it. Even if the website / official app was a mess, most of the App.net early adopters use third-party Twitter clients anyways.

If even the people who paid for App.net were still using both systems, and cross-posting, that wasn't really sustainable once the initial interest was gone.

</comment><comment><USER18> I still don't know what it is...

</comment><comment><USER19> At least part of the problem was the resentment that people felt about _needing_ to sign up to paid service to keep their identity and follow their friends. Their was implied social proof in the very beginning but with a pretty big arm twist.

</comment><comment><USER20> I still don't know what app.net is about after reading their About section. Seems like a mish mash of products. Their messaging is very confusing. I couldn't figure out why, me as a user, would ever download and use their app(s)?

</comment><comment><USER21> Charging money. Centralized. Personal data stored in the clear.

</comment><comment><USER22> It didn't solve the problem of centralized control.

</comment><comment><USER23> Seems like a lot of beanplating around "it wasn't actually compelling to enough people to make money".

<comment><AUTHOR> As a twitter-clone only, sure. But I think it was supposed to be about much more than that.

</comment></comment><comment><USER24> They just dont have a sustainable business model. That's all.

<comment><AUTHOR> I think if their concept was clearer the business model would have evolved to be effective. It's hard to get it right the first time but with a strong enough base they could have iterated.

</comment></comment><comment><USER25> You really can tell when people have websites not meant for any significant traffic. Site is down.</comment>


Deploying Django on Nginx + Gunicorn over TLS - <AUTHOR> http://ydevel.tumblr.com/post/22850778860/deploying-django-on-nginx-gunicorn-on-linux

<comment><USER2> Good post, but I think the title is a bit misleading - "configuring an HTTPS site with Django and Nginx" might be more appropriate.

"Deploying Django... over TLS" suggests (to me, at least) some method of deploying Django over a secured connection, whereas this post describes how to set up Django/Nginx on a single server, with a self-signed cert.</comment>


Building Analytics at 500px - <AUTHOR> https://medium.com/@samson_hu/building-analytics-at-500px-92e9a7005c83?hn <comment><USER5> A free (as in speech and beer) alternative to Periscopre.io would be [http://redash.io](http://redash.io). You'll have to host it yourself, but it supports more than just SQL, and is centered around the idea of sharing queries, not reports, which helps shifts the discussion from "how did you get those numbers" to "what do those numbers mean".

<comment><USER6> re:dash author here if anyone has any questions.

Our company (EverythingMe) had a very similar journey to the one Samson describes (Splunk, MySQL, logs, ... -> Redshift). At the time Persicope wasn't there (or we didn't find it), so we built re:dash instead.

</comment></comment><comment><USER7> Airflow by airbnb is a nice alternative to luigi. We've been using it for our ETL and it's been working greatly so far. [http://airbnb.io/projects/airflow/](http://airbnb.io/projects/airflow/)

<comment><USER8> Can you write a little more about your evaluation of the two?

We've been using the python celery task queue library to coordinate python ETL-esque jobs, but as our setup gets more heterogenous language and job wise we're looking for something built more explicitly for the purpose. Luigi and Airflow are both contenders.

<comment><USER7> I think Luigi doesn't have a central scheduler which allows you to run tasks hourly/daily. In the docs it suggests that you can use cron to trigger tasks periodically. This was the major reason why we didn't pick Luigi. Airflow provides that capability and also has a nice UI where you can manage the DAGs (visualize the task instances, set task states, etc). It's a young project and the codebase is clean and easy to understand. Airbnb is also actively developing on it as far as I know.

<comment><USER8> Thank you for your response!

</comment></comment></comment></comment><comment><USER9> One of my main takeaways from this article is that there is no such thing as easy analytics. In particular, you have to put though and energy into designing your analytics data stores and your ETL process.

It's a bit of a relief to see this after wondering why I've been unable to find an analytics solution that felt complete.

</comment><comment><USER10> For anyone setting up a similar system, our product [https://fivetran.com](https://fivetran.com) automates the most annoying part of this: getting the data into Redshift! We support MySQL, Postgres, Salesforce, and lots of other data sources.

<comment><USER11> Sorry if I'm getting this mixed up.

Snowplow is shown on your pricing page. Doesn't Snowplow already connect directly with Redshift? Or was Fivetran built before Snowplow finished their new setup?

<comment><USER10> Yes this is a common point of confusion. There are three options:

1\. Run your own Snowplow collector that writes to your Redshift. 2\. Use Snowplow Inc's hosted collector that writes into your Redshift. 3\. Use Fivetran's Snowplow collector that writes to your Redshift.

We have customers running each of these configurations. In the case of #1 & 2, they're just using us to sync other data into the same Redshift cluster as their snowplow collector.

</comment></comment></comment><comment><USER12> Great post.

I didn't have this to read several months ago so I did end up writing my own ETL solution. It has been fun though and now we're into the analytics phase and using Chart.io - did you compare that to Periscope? It's a tad pricey but the charts are pretty easy to generate using their query builder if you don't want to get dirty with SQL (or have team members who don't have SQL skills yet).

I do think that rolling your own ETL can be rewarding though - especially if you are wrapping each attempt at an ETL process in a class and storing long term data about the monthly/daily/hourly/irregular processes for internal analysis, forecasting, bug-reporting, and providing fodder for visuals to sell the rest of the team on what you're doing.

</comment><comment><USER13> Little bit more about luigi would be handy .. I believe same can be achieved using Pentaho Kettle as that has a complete workflow structure and its Opensource as well .. however any more examples or use cases of luigi?

</comment><comment><USER14> This seems to echo something I said in another thread; the hackers are further away from their users:

 _Even for technical people, having access to analytics is helpful. Being informed about product metrics helps developers see the 'why' behind their work — a key ingredient to high performance_

You mean that the technical people who actually implement the software are so alienated from their users that they have to wait for a royal decree to spur them to work?

I wonder how few of their ideas flow back up to the top.

I like the article; it's nice to see one use case of ETL. Too often people directly hit the production database to get reports instead of ETLing into a read-only db that won't affect site performance.

</comment><comment><USER15> Is there a good guide to getting started using logfiles as the basis for analytics? Best practices on using client vs server-side signals (and if both, not double-counting), etc. I'm interested in learning more about it, but a google search doesn't turn up anything relevant. Snowplow seems to be using a setup like this, but not sure how they'd fit in with what the author was discussing.

I'm using Django, but I imagine this kind of system would be mostly platform- agnostic.

</comment><comment><USER16> How much of this could be accomplished using something like Mixpanel? Honestly curious.

I know a true ETL solution like this will be more powerful and flexible, but how do you decide when it's worth the investment? Are there key use-cases or reports that only this type of solution can provide? Is the main issue with external analytics (like Mixpanel) that you need to know what data to collect before you can report on it?

<comment><USER17> MixPanel is nice, but its support for multiple sources of data is not very good. You can slice and dice events, but if you want to join that to your user database, things get awkward/not-possible pretty quickly.

The 2nd thing is that MixPanel is write-once, read-forever. aka if you misspell something or change definitions you end up in a bad place. These ETL processes are much more capable of adapting and normalizing a schema over time.

</comment></comment><comment><USER18> Excellent writeup. I love Splunk for many reasons (built a custom security App for Splunk as a pretty successful fraud detection tool at an enterprise financial firm).

Did you ever looked into Splunk DB connect app?

It allows to use structured DB (such as MySQL and others) as a data source for Splunk. I haven't used it yet but interested in feedback.

</comment><comment><USER19> There's a grammar error in the first sentence. "an quick" should be "a quick".

Great, detailed article though.

</comment><comment><USER20> Great by post by Samson! I sometime wish I could tell stories and explain like he can

</comment><comment><USER13> Can Airflow be used to load data from a file in filesystem( not in Hive ) to MySql .. also instead of calling a Bashscript can we call a PHP script.. Any pointers will be helpful.

<comment><USER21> A PythonOperator using the MySqlHook should do the trick. Oh and you can call a PHP script from BashOperator.

</comment></comment><comment><USER22> Nice post! We went through a very similar process back in Zalora ([http://www.zalora.com](http://www.zalora.com)) building our own Data Warehouse and Analytics function for over 300 internal users at the time.

Back when Redshift was on its initial beta release (2012), there almost wasn't any ETL / charts tool available for us, so we ended up building most of the tools and libraries ourselves. A few of them were open source too:

\- [https://github.com/zalora/redsift](https://github.com/zalora/redsift) A web-interface SQL tool for Redshift, letting the user to query and also export them to S3 (and send an email alert to user once done.)

\- [https://github.com/zalora/postgresql-user- manager](https://github.com/zalora/postgresql-user-manager) for managing user privileges

\- [https://github.com/lenguyenthedat/aws-redshift-to- rds](https://github.com/lenguyenthedat/aws-redshift-to-rds) for copying Tables from Redshift to RDS (postgresql)

\- [https://github.com/zalora/kraken](https://github.com/zalora/kraken) a bit similar to luigi or airflow.

Things has changed quite a lot since then, there are a lot of great solutions to our problems that are either free or very cheap and production-ready:

\- redash.io ([https://github.com/EverythingMe/redash](https://github.com/EverythingMe/redash)) - web-interface SQL tool with visualization - <USER4>

\- redshift_console ([https://github.com/EverythingMe/redshift_console](https://github.com/EverythingMe/redshift_console)) - redshift ops tool - <USER4>

\- flydata ([https://www.flydata.com/resources/flydata-sync/sync-rds- mysq...](https://www.flydata.com/resources/flydata-sync/sync-rds-mysql-to- redshift/)) sync live data from MySQL to Redshift - Subscription base

\- dreamfactory ([https://aws.amazon.com/marketplace/pp/B00GXYDK18?sr=0-3&qid=...](https://aws.amazon.com/marketplace/pp/B00GXYDK18?sr=0-3&qid=1385831342361)) for providing REST API interface to your database (supports redshift and other databases) - <USER4>

We didn't use Tableau in Zalora either (due to pricing and the number of users that we have in-house), and ended up building our own customized data dashboards with d3js and a few other different frameworks.

However, as long as you are ok with the price, Tableau is pretty good. It's being use widely in my current company ([http://commercialize.tv](http://commercialize.tv)) :

\- You can minimize processing from Tableau server by just create another data mart layer from Redshift with your ETL tools / scripts, having Tableau connecting directly to it.

\- Visualizations / Charts creation process is pretty much straightforward. The end-result will look exceptionally comparing to other solutions that we have tried.

\- They also have a really good and active community.

</comment><comment><USER23> I thought this would be about analytics visualization dashboard with little real estate (500px). Had no idea that 500px was a company name.

<comment><USER24> They're an interesting company IMO. They've done a good job of 'gamifying' photo sharing. I have at least one relative that's addicted to it and pays for an annual subscription.

I'm not sure how well they're doing business-wise. I did notice that they recently pivoted one aspect of their business. They used to facilitate users selling prints of their works; now they've dropped that service and are building a stock photo service instead.

<comment><USER25> What do you mean by "gamifying"?

<comment><USER24> Instead of just letting people 'like' or 'favorite' users' photos, 500px assigns a score to each photo, from 0-100. They have an algorithm that factors in all the activity on a user's photo to assign that score. As the score increases the photo moves to different sections of the site (Fresh, Upcoming, Popular).

My relative posts a photo almost everyday and is always trying to figure out how to get a higher score; different photo subjects, posting at different times of the day, etc.

</comment></comment></comment></comment><comment><USER26> Nice Post!

</comment><comment><USER27> For anyone looking for a visualisation solution reasonably priced were you don't need to write SQL [http://www.viurdata.com](http://www.viurdata.com) We will be launching support for Amazon Redshift very very soon.

</comment><comment><USER28> Nice post.

Looker ([http://looker.com](http://looker.com)) is an alternative to Periscope that a ton of venture-backed tech companies are using (mostly in conjunction with Redshift). Here are two posts from Buffer and SeatGeek on their stacks using Redshift, Luigi and Looker. Buffer: [https://overflow.bufferapp.com/2014/10/31/buffers-new- data-a...](https://overflow.bufferapp.com/2014/10/31/buffers-new-data- architecture/). SeatGeek: [http://chairnerd.seatgeek.com/building-out-the- seatgeek-data...](http://chairnerd.seatgeek.com/building-out-the-seatgeek- data-pipeline/)

Looker does not necessitate each member of the team to know SQL to explore the data or create reports. It also has a text-based modeling language that is a thin abstraction layer of SQL. It makes SQL modular and reusable making it far more efficient to support a wide range of analysis. It's more expensive than Periscope, but it's way more powerful.

<comment><USER29> Hey Zack, it's probably a good idea to disclose that you work at Looker if you're going to talk about them on HN. [https://www.linkedin.com/profile/view?id=145490726](https://www.linkedin.com/profile/view?id=145490726)

I run marketing for Periscope. Like Chartio, you guys have a great product. We routinely send people your way when they have more business user needs and aren't a good fit for Periscope.

For those evaluating tools like Periscope/Looker/Chartio:

Whereas Looker focuses on business users, we've built Periscope around serving SQL analysts better than anyone else.

If you don't have someone who knows SQL on your team, Looker (or Chartio) is probably a better fit. (though I should mention that some customers have learned SQL through our customer support :)

We're focused on the SQL analysts who are building dashboards and running reports, day in, day out. With that said, every one of our customers has employees who don't know SQL, but need to dive into their data. So we've built a ton of features to make this easy as well.

If you want to find out more, visit [https://www.periscope.io](https://www.periscope.io). If Periscope looks like a good fit, sign up and we'll get you set up with a free trial.

<comment><USER30> Periscope question: "Periscope plugs directly into your databases...."

As I understand it, Periscope is cloud based, so could you elaborate on how it might plug into large _locally hosted_ databases, or is the caveat that my databases must also be in the cloud?

<comment><USER31> (Periscope co-founder here.)

There needs to be a route from the internet to your DB, yes. Sometimes this works because the DB is in the cloud, or sometimes a port or SSH tunnel is opened into a local network.

<comment><USER30> Ok, so you don't have to physically move you data to the cloud, it can reside locally and just be queried on demand, right?

<comment><USER31> yep</comment></comment></comment></comment></comment></comment>