On the Complexity and Performance of Parsing with Derivatives - <AUTHOR> http://arxiv.org/abs/1604.04695 <comment><USER2> Summary of parsing with derivatives (Brzozowski 1964) for the unfamiliar (like me):

      For example, with respect to the character f, the derivative
      of the language for which [[L]] = {foo, frak, bar} is
      Df (L) = {oo, rak}. Because foo and frak start with the
      character f and bar does not, we keep only foo and frak
      and then remove their initial characters, leaving oo and rak.

      We repeat this process with each character in the input
      until it is exhausted. If, after every derivative has been performed,
      the resulting set of words contains the empty word,
      then there is some word in the original language consisting
      of exactly the input characters, and the language accepts
      the input. All f this processing takes place at parse time, so
      there is no parser-generation phase.

The specific algorithm they build off (Might 2011) has extended this to memoize repeated derivations of the same arguments and to allow for lazy evaluation in the case of recursion.

But this algorithm was implemented in a mind-bendingly-slow way:

       For example, they report
      that a 31-line Python file took three minutes to parse!

They prove that the worst case is actually cubic time and provide a concise cubic-time implementation in Racket: [https://bitbucket.org/ucombinator/derp-3](https://bitbucket.org/ucombinator/derp-3)

Their implementation is orders of magnitude faster of prior PWD implementations, but still orders of magnitude slower than a Bison parser of the same grammar, which they attribute to language choice (C vs Racket).

Please correct me if I'm wrong about any of this! Summarizing research like this helps me understand it, but that doesn't mean I actually understand it.

<comment><USER3> Yeah, that's accurate! I'm glad we finally know the running time for parsing with derivatives.

Let me add some more background, about stuff that I would expect to be covered in this paper's related work section, except that it doesn't have one...

Most well-known parsing algorithms that handle arbitrary CFGs run in cubic time, so this paper's O(n^3) running time is about the best you would expect. Some other parsing algorithms that also handle arbitrary CFGs in cubic time include CYK[1], Early[2], and GLR[3]. I used to be excited about parsing with derivatives because of its simplicity, but CYK and Early parsers are both actually _simpler_ than parsing with derivatives, once you've thrown this paper's optimizations in.

[1] [https://en.wikipedia.org/wiki/CYK_algorithm](https://en.wikipedia.org/wiki/CYK_algorithm) [2] [https://en.wikipedia.org/wiki/Earley_parser](https://en.wikipedia.org/wiki/Earley_parser) [3] [https://en.wikipedia.org/wiki/GLR_parser](https://en.wikipedia.org/wiki/GLR_parser)

[EDIT: Previously, I stated "CFGs, in general, can't be parsed in better than cubic time.", which is incorrect. Thanks, pacala.]

<comment><USER4> Out of curiosity, do any of the other algorithms have the ability to be "paused" arbitrarily, i.e. with inverted control? I'm thinking specifically of parsing input from an asynchronous interface.

Derivatives do that well, since input is essentially pushed into the parser.

<comment><USER5> Pausing is very straightforward with bottom up algorithms like GLR. The entire state of the parse is stored in an explicit stack data structure, as opposed to implicitly in the program's call stack as with top-down algorithms. So resuming the parse is as simple as resuming the parse loop using a previously- stored stack.

</comment></comment></comment></comment><comment><USER6> Note that the complexity is still O(n^3), which is higher than that of other algorithms for most "real-world" grammars.

I read Might's initial paper on derivative parsing a while ago, and I agree that it really seems very elegant at first when compared e.g. to bottom-up parsing. The problem of the approach though is that parse tree generation is more tricky than for other methods and (IMHO) renders the technique considerably less elegant. Of course, if only recognition is needed then this technique is straightforward, but in most practical use cases the user will want to generate a parse tree as well.

Also, the performance of the parser implementations dicsussed in the paper (and in Might's original work) are able to parse a few hundred lines of code per second (for a small language), which is very slow compared to e.g. generated recursive-descent or bottom-up parsers, which can achieve several hundred thousand of lines of code per second for real-world languages.

So although it's an interesting concept I personally think it will not be of very high relevance to practical parser generation, at least for use cases where efficiency and scalability matters.

I have been working on a descriptive parser generator though to see if I can get anywhere near the performance of my currently used PEG parser (which uses conditional memoization -i.e. packrat parsing- to speed up the parsing).

<comment><USER6> Also, like other people already have pointed out, the time complexity calculations in the academic literature are not always relevant for practical parsing:

1) A parser that might have exponential-time complexity might still beat a polynomical/linear-time complexity parser in practice, depending on the grammar and actual code parsed.

2) For real-world grammars, constants (i.e. the c in O(c*n)) will be the determining factor between two alternative parsing techniques with identical time-complexity. As an example, memoization-based (packrat) parsing in combination with PEG achieves the same time complexity as shift-reduce style parsers for many grammars, but in practice the memory allocations and bookkeeping required to do the memoization make the latter approach much faster.

I have implemented several different parsers myself, and while it is pretty straightforward to write a parser for a real-world language (e.g. Python) today, achieving very good parsing speed is not. As an example, on my machine, the Python parser can process about 100-300k lines of code per second (including AST generation) while a comparable packrat PEG parser is slower by a factor of 5-10 (for many use cases, parsing at 10.000 loc / second is still good enough though, but it should not be considered fast)

</comment></comment><comment><USER7> So, if applicable here, what do you parsing experts think of GLL parsing?

[http://dotat.at/tmp/gll.pdf](http://dotat.at/tmp/gll.pdf)

It mentions retaining easy implementation and debugging of LL parsers while knocking out limitations with some LR-style stuff. A hybrid of sorts. Result is cubic time.

<comment><USER8> GLL is fantastic! Unlike Earley and GLR, implementing GLL parser combinators is pretty easy. It's pretty fast and has some other nice features (it can be implemented to produce a lazy stream of possible parse sequences).

It is, IMO, the current state-of-the-art in parsing general CFGs. Derivative parsing is very interesting and may overtake GLL and related algorithms.

<comment><USER7> Appreciate the feedback. My plan was to push some formal methodists or LANGSEC types to try to do a verified GLL if the opportunity arises. So far, there's been SLR [1], LR(1) [2], and PEG [3] verified for correctness. Think verified GLL generator is ideal next target given benefits?

[1] [http://users.cecs.anu.edu.au/~aditi/esop.pdf](http://users.cecs.anu.edu.au/~aditi/esop.pdf)

[2] [http://pauillac.inria.fr/~xleroy/publi/validated- parser.pdf](http://pauillac.inria.fr/~xleroy/publi/validated-parser.pdf)

[3] [https://arxiv.org/pdf/1105.2576.pdf](https://arxiv.org/pdf/1105.2576.pdf)

</comment></comment></comment><comment><USER9> Not trained as a computer scientist I find myself in the position of those six blind men and an elephant.

I have bits and pieces of information but not a coherent whole. I would suspect that this parsing by derivative thing is connected to the fact that there is one to one correspondence between rational polynomials and regular languages. What feels tantalizing to me is the following: given the connection between regular languages and neural networks other (meaning those that are not based on differentiation) fast parsing techniques probably would have imply something about fast back-propagation with automatic differentiation. Would love hearing more on this.

</comment><comment><USER10> I'm currently working on a parser generator with easy of use and performance in mind.

[https://github.com/dragostis/pest](https://github.com/dragostis/pest)

<comment><USER6> Looks interesting! Have you implemented any more complex grammars with this, e.g. Python or Javascript? Would be interested to see how they perform.

Also, does it support multi-stage parsing, i.e. generation of a token tree and then a parse tree from it?

<comment><USER10> There is a WIP php parser developed by the community: [https://github.com/steffengy/pesty- php/blob/master/src/parse...](https://github.com/steffengy/pesty- php/blob/master/src/parser.rs)

And, yes, it does support multi-stage parsing. After the tokens are generated, there is a process! macro which handles them. This is where you can produce an AST.

</comment></comment></comment><comment> crier-io Being a regular developer how does this help me?

<comment><USER11> It doesn't. This is designed for more general context-free developers.

<comment><USER12> Quite the memer, aren't you?

<comment><USER11> No memes involved; that was a 100% artisanal hand-crafted joke. I'm sorry if you didn't like it, though.

</comment></comment></comment></comment><comment><USER13> For me "Figure 6. Performance of various parsers" is unreadable.

</comment><comment><USER14> It's amazing that text parsing -- one of the first problems studied in CS -- is still such a difficult problem (conceptually). I look forward to the day when the average undergrad graduating with a computer science degree will be able to write a context-free-language parser from scratch in a few days.

<comment><USER15> Text parsing for programming languages is NOT a difficult problem. It is very easy actually, much easier than most academics would have you believe.

What they are doing is trying to write theories and build conceptual systems about how to do things. That is their job. But when it comes to practical matters, the best route to take, as someone who wants to build a working compiler that gives good error messages and where the parser does not hamstring the rest of it, is to ignore almost all that stuff and just type the obvious code.

<comment><USER16> > Text parsing for programming languages is NOT a difficult problem.

That depends entirely on 1) what grammar you're parsing, 2) what memory pressure you're in, 3) how fast you want to do it, and 4) how comprehensive you want the error handling. It's actually quite difficult to parse C++ on a low-memory embedded device without resorting to clever techniques.

<comment><USER17> > It's actually quite difficult to parse C++

This is a specific problem of C++ though, given the fact that its grammar is extremely complex.

<comment><USER18> I'm pretty curious what kind of project gave zeroxfe experience with parsing C++ on low-memory embedded devices.</comment></comment></comment></comment></comment>


Show HN: Wysgy, a commandline notes app for power users - <AUTHOR> https://crates.io/crates/wysgy <comment><AUTHOR> Hi HN,

author here. I asked HN ten days back [1] (summarizing here):

\</comment><comment></comment><comment>-

(a) I want to use a Graph Database as a personal todo list, organizer, knowledge repo. Example: say I'm looking for new houses to rent, I want to create nodes and relationships around this. Some nodes could be the available houses themselves (along with their properties/traits). Some nodes could be the criteria for the houses themselves. I want a single place visualize all my activity, todos, knowledge base, to-read links, all interconnected. (b) Optionally, I want to be able to create dashboards that show some specific part of the graph in a very readable way (such as 2D tables, flowcharts, etc.) What I have tried: Neo4J - Cypher covers (b) but inputting nodes and relationships is not a user friendly task.

[1] [https://news.ycombinator.com/item?id=20196008](https://news.ycombinator.com/item?id=20196008)

\</comment><comment>----

Based on comment/s I received, I built one in Rust. Description and usage instructions in link. Eager to receive feedback.</comment>


[Poll] Have been you got married? - <AUTHOR>

<comment><USER2> Yes. Married in 2000 and divorced in 2008.

</comment><comment><USER3> +1</comment>


The Desks Of 23 Successful Tech People - <AUTHOR> http://www.businessinsider.com/desks-of-famous-tech-people-2013-4?op=1

<comment><USER2> A few interesting names, although while the beach with Branson is fun, he likely has something closer to a desk, and the article ends with a slew of Bloomberg people and a few random names that I sure couldn't place.

</comment><comment><USER3> So basically successful tech people don't really work, they let other people work for them and get all the credit.</comment>


The Great Itch? A weekend project with ways to target developers - <AUTHOR> https://medium.com/@jonathangill/the-great-itch-the-project-4e5e4389fe9e

<comment><USER2> Done the right way, as always.</comment>


Flights API - <AUTHOR> Im starting to build a website for searching flight tickets.
Any recommendation for a good, cheap (or even better, free) API to use? <comment><USER2> Not sure what this gives you, but this is the google flights API

[https://developers.google.com/qpx- express/](https://developers.google.com/qpx-express/)

</comment><comment><USER3> Use all of the APIs so you get more coverage.

<comment><AUTHOR> Yeah, i was thinking on doing that. But Ill start with just a couple to try out the concept first</comment></comment>


Ask HN: Can we add tags to submissions?  - <AUTHOR>

It would improve the quality of comments, stories and would increase the accessibility of the site. <comment><USER2> In what way does adding tags do more than

1) searching for stories by keywords via HN Search,

2) scanning the main page and the new page with word search on the page (implemented as CTRL-F in many browsers),

3) searching for best comments on the bestcomments

<http://news.ycombinator.com/bestcomments>

page,

4) searching for active threads on the active

<http://news.ycombinator.com/active>

page,

5) searching for comments by users with high karma averages on their comments by following links from the leaders

<http://news.ycombinator.com/leaders>

page, and

6) doing site-restricted searches of the site with Google

[https://www.google.com/search?sourceid=chrome&ie=UTF-8&#...
to focus on comments with particular keywords? I use all of those methods on different occasions to find good content here. What could we all do with tags (implemented how?) that we could not do by those means?

<comment><USER3> Tagging is useful for a lot more than just finding old stuff on a topic. It's massively valuable for filtering new stuff and community management.

A lot of the problems with the devolution in HN could be resolved by implementing a few common tags like, 'startups', 'programming', 'political', 'current_events', 'design' and then having different thresholds for each tag to reach the front page.

IMHO, the number of votes to reach the front page for a political story should be much higher than one that is about programming. I think most would agree.

With tagging pg would be able to figure out if one tag is getting out of hand and squelch it temporarily or even permanently.

As the community has grown, we're suffering from filter failure ( <http://www.youtube.com/watch?v=LabqeJEOQyI> ). The less HN is about stuff that is interesting to hackers, the more it becomes welcoming for people who aren't hackers, the more non-hackers that join the more up votes that non- hacker focused stories get. It's a vicious cycle and tag squelching is one good way of moderating the community and keeping it focused.</comment></comment>


Should a business be open now? Actual P&L numbers - <AUTHOR> https://www.facebook.com/somerandombar/videos/2566384356954015 <comment><AUTHOR> The name of the Seattle restaurant is actually Some Random Bar ([http://www.somerandombar.com/](http://www.somerandombar.com/)). In this video, he shares all of his costs and revenue, then compares his total costs being open for only takeout against his fixed costs while closed.

Even with $37k in April takeout revenue, he would have lost less money (-$10k vs -$12k) by being closed.</comment>


Pay remote workers more than their counterparts - <AUTHOR> https://maxmautner.com/2020/04/01/remote-work.html <comment><USER2> People are paid more because demand for workers is greater than supply and there is a somewhat functioning market not because of what's fair or nice the argument is just silly.</comment>


Ask HN: Working Prediction Model. How do I share the results? - <AUTHOR> Completed a Prediction Model.
It took me 10 years (side project).
The model is working better than expected with incredible results.
I want to start a service where I can share the results, any ideas ? <comment><USER2> Everyone in quantitative finance has built a back-test that would make them billions on paper. It is highly likely there is over-fitting issues, errors in calculations of returns etc.

You should share the list of trades, rough high level idea of what you're making money from etc.

If you have something, trade it with capital. Nobody seriously buys signals for good reasons.

</comment><comment><USER3> A prediction model about what? Which dataset are you using to train and evaluate it?

<comment><AUTHOR> Data = finance

Results = Stock Market</comment></comment>


Peter Thiel Knows Bubbles, And This Is 'Not A Bubble' - <AUTHOR> http://blogs.wsj.com/venturecapital/2011/08/04/peter-thiel-knows-bubbles-and-this-is-not-a-bubble/

<comment><USER2> This piece is ridiculous. An investor who owns $1.7 billion in Facebook stock is downplaying the idea of a bubble before he has the chance to sell any of it. Shocking. I was rather hoping he'd decide to shave millions-to-hundreds- of-millions of dollars from his net worth by popping the bubble publicly in the Wall Street Journal.

And his reasoning is impeccable; we're not in a bubble because there are no crazy launch parties. Clearly, that's how investors value assets. But last I checked there was a recent bubble in real estate, but I didn't see many crazy launch parties there. So now I'm quite confused.</comment>


Using Infrasonic Sound to Detect an Intruder - <AUTHOR> http://techcrunch.com/2014/11/03/cocoon/ <comment><USER2> A recent art project used ultrasonic waves to jam cellphone microphones. The idea was to prevent eavesdropping from recorder applications or a compromised, or otherwise always-on mic.

A side effect of this, and perhaps these devices as well, as depending on the volume, these effectively create RF interference in a way in the circuit, as in the electromagnetic signal beyond the mic is changed, and could lead to unintended side effects such as the inability to speak to 911/999 emergency services.

<comment><USER3> I interpreted this design as a microphone only, not something that also emits sound. I could be totally wrong though.

<comment><USER2> Yes, I think you're right.

</comment></comment></comment><comment><USER4> My dog does this.

He knows way before I do when someone he knows is walking / driving up the street</comment>


Gource – a software version control visualization tool - <AUTHOR> http://gource.io/ <comment><USER2> Years ago I was a man of many hats at a startup. One of my hats was build and release guy. We had a demo day for investors and I was like "what the heck do I show them?"

A Gource generated movie of the startup's main repo saved the day. :-)

Edit: oh wait, it was probably codeswarm:

[https://news.ycombinator.com/item?id=221601](https://news.ycombinator.com/item?id=221601)

</comment><comment><USER3> The project is a bit undocumented but the configuration offers a lot of possibilities. I'd recommend iterating with various config settings with `--load-config gource.conf`

A gource I configured recently of the Chrome DevTools team's activity: [https://youtu.be/N48q9aiUMjI](https://youtu.be/N48q9aiUMjI)

Another plus for Mac users: it's an easy install on homebrew.

<comment><USER4> It's also on MacPorts.

(I discovered this halfway into writing my own portfile.)

</comment></comment><comment><USER5> My first introduction to Gource was a visualization of early Minecraft development:

[https://www.youtube.com/watch?v=zRjTyRly5WA](https://www.youtube.com/watch?v=zRjTyRly5WA)

<comment><USER6> Same here, would be nice if that included the Client + Server merge..

Related, Forge has one as well: [https://www.youtube.com/watch?v=0goY8bp59wc](https://www.youtube.com/watch?v=0goY8bp59wc)

</comment></comment><comment><USER7> Recently rediscovered tig (Text-mode interface for Git) and have been loving it:

[https://jonas.github.io/tig/](https://jonas.github.io/tig/)

<comment><USER8> I always install tig immediately after git.

<comment><USER9> I use it instead of less, tig is even less.

</comment></comment></comment><comment><USER10> Gource is incredibly powerful, but it does take a bit of playing around with all of the config options available to get your results just right.

I put together [https://www.youtube.com/watch?v=aTYJAWWZ2jQ](https://www.youtube.com/watch?v=aTYJAWWZ2jQ) for OctoberCMS, an open source content management framework that's been steadily gaining in popularity for the past four years.

</comment><comment><USER11> This is so incredibly entertaining to use. Although you have to have a fair number of active daily contributors for it to actually be interesting to watch.

</comment><comment><USER12> ~25,000 of the most starred Github repos have Gource vids on YouTube.

Search "YOUR_REPO_URL gource" or "Evolution of YOUR_PROJECT_NAME gource"

Full command found in recent vid descriptions.

\---

Back in the day when YouTube didn't set reasonable user upload limits...

</comment><comment><USER13> TBH, I was a little disappointed. The animations are neat, but I was hoping for something more practical.

I did recently learn about magit's magit-log-all command, and that's been really useful.

</comment><comment><USER14> While gource is incredible I would love if it was more versatile. It would be incredible if it could be themable.

<comment><USER15> On that note, I once saw it being (ab)used to visualize activity on a phpBB. I think they mapped threads to files and posts to commits or something like that. Interesting idea and I was wondering if you could apply it to other things as well.

</comment></comment><comment><USER16> Here's a script I hacked together that simplifies gource video generation a bit: [https://github.com/dailymuse/gg](https://github.com/dailymuse/gg)

</comment><comment><USER17> Homebrew users:

      brew install gource

</comment><comment><USER18> What's practical application of this?

<comment><USER19> It's pretty.</comment></comment>


In the Shadow of John McCarthy - <AUTHOR> http://blog.fogus.me/2011/11/03/in-the-shadow-of-john-mccarthy/ <comment><USER2> For a lighter side of both Steve Russell and Alan Kotok, you absolutely must read Steven Levy's Hackers.

A large chunk of this book deals with the very early computer scene of MIT in the late 1950s and 1960s. Both of those guys played a big role, especially Russell. He developed a game, Spacewar, that just enraptured the whole PDP crowd and led to this mania of gameplay and hacking, gameplay and hacking, etc.

It is really a delightful book. It's great to see these gentlemen recognized for their more formal contributions.

</comment><comment><USER3> "Even in 1969 the AI lab had a direct news feed from the New York Times. And on-line text chatting between users, with video support was possible"

<comment><USER3> From the comment by David Luckham.</comment></comment>


Advanced Computer Architecture Reading List - <AUTHOR> http://www.cs.ucf.edu/courses/cda5106/summer02/reading.html <comment><USER2> My favorite books on computer architecture:

_Modern Processor Design_ (Shen and Lepasti). Lots of material on superscalar processors, with some nice examples from existing work.

_The Pentium Chronicles_ (Robert Colwell). Much less technical, but a lot about the social nature of getting a significant CPU design done.

</comment><comment><USER3> Very strange to post a reading list from a course given in Summer 2002.

I took this same class at UCF more recently with Dr. Heinrich, who did an amazing job. I don't think his course materials are public, though he used Modern Processor Design[1].

[1] [http://amzn.com/B00HCLUL5O](http://amzn.com/B00HCLUL5O)

<comment><USER4> "Summer 2002."

WRT the Cray article, I think it would be a struggle to find a better ten page "primary-ish" source than the original ACM article for the Cray-I. Bitsavers (and mirrors) has the original hardware ref manual but that's in excess of two hundred pages. The original marketing brochure (see bitsavers again) is basically the linked ACM article with bigger graphics and is 15 pages long, better off just reading the shorter ACM article. Its relevant today because as a marketing tool back then it was the first heavily pipelined vector processor for a lot of otherwise experienced folks which in 2013 makes it an ideal intro to the topic for comp sci students.

I'm having severe trouble at this time of day thinking of anything new in processor design over the past decade to add to a list. New implementations of very old ideas, bigger implementations of very old ideas, but not able to think of any actually new ideas. I'm talking about a decade of substantial engineering-type achievements vs no basic science-type discoveries.

</comment></comment><comment><USER5> A few historically interesting architecture books that are now available for free online are:

Thornton, James E, _Design of a computer: the Control Data 6600_ [http://ygdes.com/CDC/DesignOfAComputer_CDC6600.pdf](http://ygdes.com/CDC/DesignOfAComputer_CDC6600.pdf)

Gordon Bell, _Computer Engineering: A DEC View of Hardware Systems Design_ [http://research.microsoft.com/en- us/um/people/gbell/CGB%20Fi...](http://research.microsoft.com/en- us/um/people/gbell/CGB%20Files/Computer%20Engineering%207809%20c.pdf)

Levy, Henry M., _Capability-Based Computer Systems_ [http://homes.cs.washington.edu/~levy/capabook/](http://homes.cs.washington.edu/~levy/capabook/)

</comment><comment><USER6> For computer architecture readings, I'd start here: [http://www.amazon.com/Readings-Computer-Architecture- Morgan-...](http://www.amazon.com/Readings-Computer-Architecture-Morgan- Kaufmann/dp/1558605398/)

Then follow up with the excellent series here: [http://www.morganclaypool.com/toc/cac/1/1](http://www.morganclaypool.com/toc/cac/1/1)</comment>


SQLite schema boilerplate for user accounts, roles, logins and auth tokens - <AUTHOR> https://github.com/membership/membership.db/tree/master/sqlite <comment><USER2> I hate to be that guy, but why is something as trivial and inflexible as this being voted up?

It's the schema for the user accounts and logins for a specific application, but without the actual application or documentation that would explain what the idea of each table and column is?

Is this meant as a "Here's how you could model a database for doing this", why is it lacking any explanations on how to use it, justifications for the choices, or examples on how the UI flow would look like?

Why is a user's phone number so important that it's part of this, but not a user's last IP address or time of login?

Super confused here.

</comment><comment><USER3> Your table naming convention needs work...

        User model -> User table
        Role model -> UserRole table
        UserRole model -> UserUserRole table

If you consider that User is the module, then your User table should be called UserUser if you want to be consistent.

But that's silly.

Calling your module Auth would be better. Then the table names don't contain RepeatedRepeated parts.

Or even better, don't use the module name in the table name. It's pretty clear what User, Role and UserRole mean just on their own.

And since you're considering using SQLite as the DB, your system will (hopefully) be small enough not to worry about naming collisions, so why bother with module scoping the tables at all?

</comment><comment><USER3> If you want something that is a lot more descriptive and generic, get your ideas from this.

[https://schema.org](https://schema.org)

</comment><comment><USER4> How is this on the front page?

Surely if you are coding something to use this, you would be able to write the 30-odd lines that these sql statements are...

also table names like "UserUserRole" don't look right to me.

</comment><comment><USER5> Is there any documentation for this?

For example is "securityStamp" in User.sql designed to store the salt? Or is it something else entirely? There's security stamps in other frameworks which are just unique values (e.g. UIDs) which just change when a user's password changes so you can invalid old cookies without storing the hash within the cookie.

If there is no salt support at all then this is inappropriate in 2016 as boilerplate.

</comment><comment> vhost- Why the camel cased columns?

<comment><AUTHOR> Good question. This db schema is supposed to be used by web developers. With camelCase columns it might be easier for developers to query the database directly from the code (unless they use an ORM, in that case snake_case should also work well).

<comment><USER6> How so? It doesn't really matter. With uppercase you now have to put quote in SQL, I am not sure other drivers, but in psql you do. Just saying.

</comment></comment></comment><comment><USER7> Please do not use SQLite for multiuser systems. You will have a bad time.

<comment><USER8> In all seriousness, ever since SQLite got WAL support, it's been fine for light multiuser systems. I would define "multiuser" as "around ten or fewer concurrent users", and I think there are some major backup considerations that go with this, but I don't think it's an inherent nonstarter.

<comment><USER7> WAL still requires a table level write lock.

<comment><USER9> SQLite uses only file locking.

<comment><USER10> That was true for a long time, but newer sqlite can do table level locking when in shared cache mode. Section 2.2 here: [https://www.sqlite.org/sharedcache.html](https://www.sqlite.org/sharedcache.html)</comment></comment></comment></comment></comment>


Every color has its beautiful name - <AUTHOR> https://moevis.github.io/color-tale/ <comment><AUTHOR> These days I got names of colors in different languages from this website: [http://htmlcss.jp/color/china.html](http://htmlcss.jp/color/china.html). I surprisingly found that the name of colors are full of beauty. So I wrote a tool to extract dominated colors from an image, and name them.</comment>


Hosting Embedded Rust Apps on Apache Mynewt with STM32 Blue Pill - <AUTHOR> https://firstname.lastname@example.com/hosting-embedded-rust-apps-on-apache-mynewt-with-stm32-blue-pill-c86b119fe5f <comment><USER2> I wonder if Mynewt would consider adding Rust code to the OS core code. It would probably complicate the build process and other things and I'm not sure it would be worth it for this type of programming.

Thanks for the interesting article, I enjoyed reading it.

<comment><USER3> Thanks :-) I'm not speaking on behalf of the terrific Mynewt team, but I feel that Mynewt is already a great embedded RTOS with many well-crafted low-level APIs, done in C.

Like the Mynewt Sensor Framework. It really saves you a lot of trouble when building IoT devices. Just pick a sensor driver from their library, and Mynewt handles all the sensor polling for you...

[https://mynewt.apache.org/latest/os/modules/sensor_framework...](https://mynewt.apache.org/latest/os/modules/sensor_framework/sensor_framework.html)

Adding Rust to the core Mynewt OS may cause "Abstraction Bloat", I think. We will start adding layers of wrappers on top of bare metal to make the OS look tidily object-oriented, to fit all past, present and future microcontrollers, peripherals, data interfaces, ... Then add another layer to make it easy for app developers to call.

No offence to the Embedded Rust community, but I fear that Mynewt with Rust will become a complicated OS with this GPIO code...

[https://rust-embedded.github.io/book/static- guarantees/desig...](https://rust-embedded.github.io/book/static- guarantees/design-contracts.html)

The best way forward I think: Get embedded app developers to move away from C and port their embedded apps to Rust. Build the apps with safe Rust wrappers on top of the OS.

Once we figure out what kind of APIs the embedded Rust apps really need (e.g. Sensor Framework), then we dig deeper and restructure the OS with Rust. This could work with any embedded OS: Mynewt, Zephyr, FreeRTOS, ...

</comment></comment><comment><AUTHOR> While the article is a little over the top with Rust vs C, the how to portion of this artivle is nice, as well as the explanation of some of the pitfalls.

<comment><USER3> Thanks! :-) I'm the author and I have seen my IoT students suffer while learning Arduino programming. So my article was really about the bad C coding in Arduino: pointers, bit-banging, ...

But really... we have suffered with C for so many years, why haven't we moved on to Rust?</comment></comment>


Has the relevance of Scala diminished with the release of Java 8? - <AUTHOR> <comment><USER2> It's been discussed before [http://ahmedsoliman.com/2014/03/26/will- java-8-kill-scala/](http://ahmedsoliman.com/2014/03/26/will-java-8-kill- scala/)

and comments [https://news.ycombinator.com/item?id=7478367](https://news.ycombinator.com/item?id=7478367)

</comment><comment><USER3> No, Java 8 is still far behind Scala.

</comment><comment><USER4> [http://en.wikipedia.org/wiki/Betteridge%27s_law_of_headlines](http://en.wikipedia.org/wiki/Betteridge%27s_law_of_headlines)

</comment><comment><USER5> Scala is too far ahead

</comment><comment><USER6> No</comment>


Andro.js - <AUTHOR> http://maryrosecook.com/post/andro-js

<comment><USER2> Nice post! It's interesting that hardly anyone has chimed in that some programming language features make addressing this problem considerably simpler without forcing the programmer to write excess boilerplate. Objective-C delegation, JavaScript Proxies [https://developer.mozilla.org/en/JavaScript/Reference/Global...](https://developer.mozilla.org/en/JavaScript/Reference/Global_Objects/Proxy), ClojureScript protocols all provide the machinery to make this kind of thing work without making the code too hard to reason about.

</comment><comment><USER3> I really like the loosely coupled approach.

However, I think that sprinkling andro.eventer all over the place is a stumbling block to me - it's essentially a global variable.

I realise that you don't want to polute the objects too much, but why not just add bind and emit methods to the composed object? Either that or pass andro as a parameter to the mixin's setup method?

Also, the bind(this, "touch", function...) to me looks like you're binding to the "touch" event of "this". Is the closure tax really so high that you can't store "this", or use something like jQuery.proxy()?

Like I said, I think this approach has a lot of potential, and seems to fit quite naturally the eventful model that the DOM uses.

<comment><AUTHOR> Thank you for your excellent feedback.

Yes, andro.eventer is there to avoid polluting the owner object. However, your idea of passing andro to the behaviour's setup() method is a great one. I'm going to give it a try.

The reason I require the behaviour to be passed to bind() is so that the passed function can be run with `this` set to the behaviour. I couldn't see another way to let bind know which behaviour should be bound to the event.

</comment></comment><comment><USER4> I wish Node.js had not introduced "evented" as an adjective, leading to "event" as a verb and "eventer" as a noun...

</comment><comment><USER5> string hell squared.

<comment><AUTHOR> I'm very open to critique. Can you elaborate?

<comment><USER6> I cant speak for that guy but my take is that all of the strings except rarrr should statically defined in the object that defines their behaviour. They may even be integers instead of strings to speed things up. Then you would need some way to avoid collision for the event integers shared across many objects.

Repeating strings is hard to refactor and comparing strings is much less efficient than comparing integers. Think about making sure every bit in two 2 * 8 * length string of bits plus overhead is the same vs two 32bit integers. The latter far fewer cycles.

<comment><USER7> I'm not convinced this is useful JavaScript optimization advice. JavaScript doesn't have integers, and string literals are interned, so it shouldn't make a substantial difference if you have the same literal written out 15 times or just defined once and placed in a property. (In fact, using the plain literal everywhere may be faster because it omits the dynamic property access.)

<comment><USER4> I'd rather mistype an identifier than a string literal, because nothing will complain about the wrong string literal.</comment></comment></comment></comment></comment>


JSON is not Javascript Object - <AUTHOR> http://www.fizerkhan.com/blog/posts/JSON-is-not-Javascript-Object.html <comment><USER2> there are also some unicode character differences, see: [http://timelessrepo.com/json-isnt-a-javascript- subset](http://timelessrepo.com/json-isnt-a-javascript-subset)

<comment><AUTHOR> I did not know that. Thanks for useful info.</comment></comment>


Show HN: p.ostero.us - the post posterous blog platform built with Octopress - <AUTHOR> http://p.ostero.us

<comment><AUTHOR> Hi Guys

I'm a long time posterous user and not happy with the recent twitter acquisition.

Twitter is ganna put posterous in maintenance mode and forget about it - <USER2> is not how I want my blog and websites hosted

Instead of doing something stupid like moving to wordpress, I've setup <http://p.ostero.us>. A paid Octopress + more blogging platform aimed squarely at the disenchanted posterous users

Octopress is amazing and I want to make it available to non commandline loving people

I've setup a 'kickstarter' at <http://www.indiegogo.com/postposterous> to get some initial cash in the bank to make this happen asap * Australian resident so no kickstarter.com for me :(

Let me know if you have any queries, I really think the time is right for octopress to shine

Cheers

Justin

<comment><USER3> That seems like a really terrible violation of Posterous' brand and trademarks. There is also a lot more to keeping a blogging service running for posterity than just charging. It is a huge responsibility for people to entrust you with their content, one not to be taken lightly.

<comment><AUTHOR> agreed, with p.ostero.us - the goal is data portability - i dont want to own your content - just a nice way to display it - your content can be from your github repo etc..

\- plus - with 1 click you can download your entire p.ostero.us/octopress install and self host whenever you want ( all the data is yours ) - thats the feature i wish posterous had</comment></comment></comment>


Type-safe Ajax calls with Scala.js/Autowire - <AUTHOR> https://github.com/lihaoyi/autowire#autowire-020

<comment><USER2> Great to see more and more libraries making use of Scala.JS.

Autowire seems like a first step towards a framework like Google Web Toolkit, where client and server code can be written in the same language. The questions is then, what can we learn from GWT and what can be done better?

Maybe Java was the reason why GWT never became widespread (being less flexible than Javascript with closures and stuff), but this should have changed with Java 8?

<comment><USER3> > where client and server code can be written in the same language

While I think Autowire will be a very useful tool, I'm not seeing how it's "a first step" ... I have successfully written and deployed Scala on both client and server without any issues since some time already.

I think the issue with GWT was that it

a) tried to be a framework (not sure why Java people love to build frameworks instead of libraries ... libraries compose, frameworks don't)

b) and required Java which just isn't flexible enough (syntactically and semantically) to interact with JavaScript in a meaningful way.</comment></comment>


Critique my first website [please] - <AUTHOR> http://notezil.la

<comment><AUTHOR> I made it in my final year of Secondary School.

I was hesitant to share because the target auidence isn't exactly HN, but I imagine that they're lots of clever people here who'd be able to emphasize anyways.

I know that it's not the best example of responsive design and that it's basically a sandcaste* so I was hoping for some general design mistakes that I can learn from before I rebuild the site. Particularly things that're unintuitive or out of place.

Thanks, Jean-Le

*I needed to complete a prototype for a competition so I ended up building a sandcaste out of javascript-- Should've used wordpress for the demo... but in the heat of everything and with writing my own final exams, I just kept building it was a single page monster with ALL of the notes being stored in the javascript file >.< Seriously... when you click a new note, it just replaces the contents of an <article>... my logic being that all the extra text in the javascript cost less bandwith than a picture.

I plan to build most of the backend stuff in Clojure, so if anyone has any advice on that it's very welcome.

</comment><comment><USER2> I really love the dinosaur, and the animations are cool.

I'm not a designer in any way, so don't take my opinions very seriously, but here are my thoughts:

I found it non-obvious that the homepage wasn't all there is. Maybe the link text for "adventure" should be something different.

FORUMS looks like a header to me, with the posts underneath it. I see now that it takes you to the full forum, but I'm not sure what's going on with the dyslexia stuff beneath it.

The word 'back' on the navigation on the right could maybe be more descriptive depending on what page you're on.

As for Clojure, if you have any more specific questions, stop by #clojure on freenode on irc, everyone's super helpful. I'd be glad to help with anything more specific as well. I don't know how much you know, but compojure and ring are the goto tools to get started with web dev in clojure.

<comment><AUTHOR> Thanks for taking the time cgag :) The 'non-obvious' aspects was exactly what I'm trying to weed out going forward.

I've built some demos before- enough to appreaciate 'The Clojure Way' of doing things over my experiences with other tools... but what I've really been struggling with was the methodology of development- how do I start a project, how much documentation is appropriate, keeping a tidy source tree, refactoring etc...

I started using IRC just on your reccomendation and, it's one of my favourite things on the internet now :) I'm not sure why I didn't start using it eariler :D</comment></comment>


Bitcoin Mining on Amazon EC2 – don't do it - <AUTHOR> http://andrei.oprisan.com/bitcoin-mining-on-amazon-ec2-dont-do-it/

<comment><USER2> Do you think people would pay to rent mining equipment?

I'm not sure how the pricing model would work, but it would allow more people to contribute that can't afford to spend thousands upfront. Maybe some sort of pooling of funds, buy equipment, split profits.

<comment><USER3> You can buy bonds representing a perpetual 1Mh/s, which amounts to the same thing.

<comment><AUTHOR> where can you do this? it would be nice to see marketplaces develop where people can rent out spare cycles on their GPU</comment></comment></comment>


Why Startup Advice Isn't Meant for You - <AUTHOR> http://blog.microlancer.com/startup-advice-isnt-meant/

<comment><USER2> Interesting ideas. Can you suggest some sources that are focused on boot- strapping rather than producing black swans?</comment>


PCC: Re-architecting Congestion Control for Consistent High Performance [pdf] - <AUTHOR> http://arxiv.org/pdf/1409.7092v1.pdf

<comment><USER2> That's not an unreasonable approach. When I came up with the original approach to congestion control in TCP 30 years ago (see Internet RFCs 896 and 970), TCP behavior under load was so awful that connections would stall out and fail. The ad-hoc solutions we put in back then at least made TCP well behaved under load.

In the early days of TCP/IP, it was clear that there was no known solution to congestion in the middle of a big datagram network. There still isn't. Out near the edges, we can use fair queuing, which I invented and named, and which is offered in most Cisco routers for links below Ethernet speeds. We still don't know what to do in the middle of the network. "Random Early Drop" is an ugly hack, but it's what we have in the big routers. What saved us for years is that fiber backbones had so much more bandwidth than the last mile consumer connections that the congestion stayed near the ends. Then came streaming HD video.

The treatment of congestion went off in a different direction than I'd expected. Originally, ICMP Source Quench messages were used for congestion control. Routers were expected to tell endpoints when to slow down. But they weren't implemented in Berkeley's networking stack, and gradually were phased out. (They would have been a denial of service attack vector, anyway.) Instead, congestion was inferred from packet loss. Even worse, congestion is inferred from not having a packet acknowledged, so you don't know which direction is congested. That's not a particularly good approach, but it's what we've got.

I originally took the position that, as an endpoint, you were entitled to one packet in flight. That is, the network should be provisioned so that every connection can have one packet in transit without packet loss. If you have that much capacity, and you have fair queuing in routers, you only lose packets if you have more than one in flight. You're thus competing with yourself for bandwidth, and an "increase slowly, back off big on packet loss" strategy works fine.

In a world with Random Early Drop, not enough fair queuing, and big FIFO buffers (someone called this "bufferbloat"), you get both packet loss and excessive transit time. It's hard for the endpoints to adapt to that no matter what they do.

Anyway, there's nothing wrong with using a smarter algorithm to calculate optimal window sizes and retransmission times. As long as it evaluates how well it's doing and feeds that back into the control algorithm in a rational way, that's fine. Today, you could probably afford to run machine learning algorithms in routers. So go for it!

If you're going to do that, it's a good time to fix delayed ACK as well. As is well known, the interaction between delayed ACKs and the Nagle algorithm is awful. This is a historical accident; I implemented one, the Berkeley guys implemented the other, and by the time I found out I was out of networking and involved with a small startup called Autodesk. So that never got fixed. Here's how to think about that. A delayed ACK is a bet. You're betting that there will be a reply, upon with an ACK can be piggybacked, before the fixed timer runs out. If the fixed timer runs out, and you have to send an ACK as a separate message, you lost the bet. Current TCP implementations will happily lose that bet forever without turning off the ACK delay. That's just wrong.

The right answer is to track wins and losses on delayed and non-delayed ACKs. Don't turn on ACK delay unless you're sending a lot of non-delayed ACKs closely followed by packets on which the ACK could have been piggybacked. Turn it off when a delayed ACK has to be sent.

I should have pushed for this in the 1980s.

John Nagle

<comment><USER3> John, if you care to get back in touch with Jim Gettys and I, we can update you on where things stand on the whole bufferbloat effort.

FQ on modern cisco routers is generally quite crippled, and even that much is not available elsewhere.

However, on the whole, the fq + aqm situation along the edge of the internet is looking up, with nearly every third party home router firmware incorporating fq_codel (notably openwrt and derivatives), and it being integral to streamboost, and other QoS systems like those in the netgear X4.

Some examples of the kind of results we get now from these new algos:

[http://snapon.lab.bufferbloat.net/~cero2/jimreisert/results....](http://snapon.lab.bufferbloat.net/~cero2/jimreisert/results.html)

[http://burntchrome.blogspot.com/2014/05/fixing- bufferbloat-o...](http://burntchrome.blogspot.com/2014/05/fixing-bufferbloat- on-comcasts-blast.html)

I really liked the PCM paper (wanted code tho) for clearly identifying the parameter space modern tcps need to operate in. FQ everywhere, or nearly everywhere, would make the tcp's job far easier.

There is an aqm/packet scheduling working group in the ietf in the process of standardization, etc.

</comment></comment><comment><USER4> Thank you all for your interests!

We just updated a full-length version of PCC to arXiv, but it takes some time for arXiv to process and post online. The full version discussed our approach in a more detail and, I believe, clearer way. We have more evaluation for more challenging network scenarios and also evaluated PCC with different utility functions expressing different data transfer objectives.

So here is a preview: [http://web.engr.illinois.edu/~modong2/pcc_fullpaper.pdf](http://web.engr.illinois.edu/~modong2/pcc_fullpaper.pdf) During this two weeks, we are writing some simple docs and making the file- transfer SOFTWARE more user-friendly. We also plan to release our experiments/demo CODE so one can easily play with and compare PCC with existing TCP variants.

Code is available at [https://github.com/modong/pcc](https://github.com/modong/pcc), but currently it is under some refactoring and some compile/run instructions are outdated. We will have a more formal release in two weeks.

Thanks

-Mo

<comment><USER5> Thanks for the preview Mo, I am reading through it now. Would there be any further advantage to measuring one-way-delay as opposed to measuring RTT?

</comment></comment><comment><USER6> They lost me when it became obvious that PCC doesn't interoperate very well with "normal" TCP (let's call it reno-tcp[1]). TCP is not the optimal solution and PCC may very well be better but there is just no way to use a new and unfair-to-reno and ignore all consequences. There must be some safeguards to handle all the older systems (be they routers, embedded systems, IoT or just plain old Win XP machines).

It is an interesting approach and an interesting research that drives us forward but it is in no way a practical approach.

They implemented this thing in UDT, if they can come up with a Linux patch that would be awesome for real world testing and implementation verification.

[1] Reno is the original TCP congestion control, New-Reno was the most common adaptation for a long time. These days CUBIC is the default in Linux, Windows has their own algorithm. All of these behave in a very similar way to the old Reno, they all have the basic structure of AIMD - Additional Increase, Multiplicative Decrease (slowly increasing on no signal and fast decrease on loss signal). in the case of CUBIC and H-TCP the slow increase is not very slow and the fast decrease was slowed down to handle very high speed links and high latency (high bandwidth delay product).

<comment><USER4> This actually can lead to a very interesting discussion about TCP friendliness, which I didn't have space to elaborate in paper:

1\. The Trend of Selfish Applications: TCP's poor performance is not only an object of research study. More and more applications have chosen to ignore the notion of TCP friendliness and use selfish application-level optimization techniques such as opening many parallel connections (as in modern web browsers and GridFTP), contacting multiple servers in parallel (in BitTorrent), or modifying TCP to use a very large initial window size. In fact, some CDN providers use selfish optimization of TCP to accelerate data transfer from CDN servers to end users on the public Internet. This trend indicates that often, applications prefer higher performance regardless of potential TCP unfriendliness. Although we recommend use of PCC only when it is isolated from legacy TCP, it is likely that due to PCC's robust high performance, some or many users (especially those suffering in network environments that are challenging for TCP) will use PCC as another selfish performance optimization technique. Legacy TCP users, on the other hand, will unfortunately suffer from performance degradation when competing with PCC and other selfish optimization techniques under FIFO queue based network.

What happens next in this tussle? One possibility is a slow arms race towards selfish high performance rate control. If done through adoption of PCC using our safe utility function, it will actually avoid a ``tragedy of the commons'' situation as we have shown. But more immediately, network owners (ISPs) will likely take action to avoid complaints.

2.ISP response to selfishness:

o maintain legacy TCP users' satisfaction, network operators can respond to selfishness by deploying flow-level or customer-level resource isolation at the point of congestion. Of course this requires hardware support, but it is feasible for two reasons. First, unlike protocols like XCP and RCP, an ISP can deploy resource isolation as a local change at an individual congested device, requiring no signaling or packet header changes or explicit feedback to end- hosts. Second, the belief that per-flow FQ is infeasible for high speed switching hardware is now a misconception, with merchant silicon supporting hundreds of thousands of flows at hundred-gigabit speeds. Even by 1999, early implementations used limited hardware (less than 33Mhz FPGA or CMOS) to support FQ on multi-gigabit links. Today, an off-the-shelf EZchip NP-4 supports 256K flows with 100 Gbit throughput, VSC2202 supports 500K flows with Gigabit Ethernet throughput, and so on. We obtained a quote of $25 for the VSC2202, of which the cost of the FQ logic itself is only a fraction.} In certain cases resource isolation is already deployed; cellular networks for example use separate queues for each user.

3\. Towards a new architecture?

The above trends suggest that in-network resource isolation may be increasingly deployed as a consequence of the continuous tussle between applications optimising selfishly and ISPs protecting their infrastructure with means that are feasible to deploy. We argue that this process is actually (slowly) evolving the Internet towards a fundamentally better architecture: end-hosts focus on optimising their desired performance, while the network's job is to isolate resources across users. This architecture offers a clean separation of concerns. End-hosts in this architecture deal with what they can directly observe and control, leading to simple, clean designs and the flexibility to express their heterogenous objectives and even safely evolve to new end-to-end protocols. Similarly, the network deals with what it directly observes and cares about, i.e., competition between users or flows. Our evaluation has already demonstrated the flexibility that results from this architecture.

<comment><USER6> While there are users of selfish transports and there is space for them in controlled networks it will be hard to get something selfish widely used considering the slow adoption of congestion control.

It would be nice to see quantification of NewReno TCP behavior against PCC, both short and long lived connections and how they fair in such an environment. If you want to convince the world to switch to your approach you'll need to do that.

At the end most people will just use the default of their OS and will never even consider what happens underneath so you'll need to convince the Linux kernel and BSDs that your approach is better overall and viable for the wide internet.

One thing I failed to take account before is that most transfers are affected by the congestion control on the server and less so on the client (besides uploads) so you really need to focus on the OS vendors to implement PCC in their OS and possibly make it a default. Linux has a configurable congestion control mechanism and I added H-TCP to it back at the time. Wasn't too much work either.

I don't really buy the ISP response argument, they will fail to do it early enough and will only get to it when things break severely enough and then their response will be overly aggressive.

It would be useful however to maybe add a mode detection to PCC that will throttle it somewhat to not take full bandwidth in order to leave some space for legacy TCP. It will add complexity to PCC and this needs to be balanced but at the end you'll need to do what it takes to get PCC beyond closed networks and into the internet and you have many gatekeepers to convince.

</comment></comment></comment><comment><USER5> I think this is a really good idea. Where TCP will often just multiplicatively decrease sending rate at every dropped packet (which happens 0.5% to 1% of the time if the network is normal), PCC does not react immediately but rather feeds this into a learning algorithm, which also learns from doing frequent experiments increasing/decreasing the sending rate and measuring the impact on loss rate, throughput and latency.

Essentially, if I understand it correctly, it's just measuring loss rate, throughput and latency and iteratively adjusting sending rate in response. I think it's very similar to LEDBAT and delay-based congestion control except that it's not just taking delay from bufferbloat into account, but also loss rate and throughput, which makes sense if the goal is to optimize these.

<comment><USER4> Thank you!

PCC measures these performance metrics and combine them via a utility function describing the data delivery goal, which can be "high throughput, low loss rate, don't care latency" or "high throughput, low latency" as we evaluated in Sec. 4.4 in the full paper</comment></comment>


Parrying the Latest Predictions of Facebook's Demise - <AUTHOR> http://www.nytimes.com/2015/03/26/technology/debunking-the-latest-predictions-of-facebooks-demise.html?_r=0 <comment><AUTHOR> Facebook seems to me to be very slowly fading - fewer and friends in total take part.

The sad thing is that I see any specific replacement. No other social network has the same "public sphere" quality and whatever Facebook's other qualities, I miss this

</comment><comment><USER2> As Facebook grows and monetizes, it's diluting its core attraction - interaction with friends and their photos and videos

<comment><AUTHOR> Honestly I don't think that's what's happening.

I don't find my Facebook experience more disrupted. There was a time when ads started appearing on my feed. And they stopped again.

I get the impression Facebook is highly attuned to how much engagement it gets and does everything it can to keep that high.

I think the problem is more just the problem you get being at a party with interesting people for a long time. No matter how interesting the people might be, you get tired of them after a while.

Moreover, the process of disenchantment and disengagement happens at a different rate for different people. And because Facebook works hard to engage the people who are engage-able, you get an effect where some people have left, some people at the door hoping for others to shut up so they can leave and some people are happily engaged with the remainder of the party.

And it seems like longer term, the process of disengagement will only increase because the division between those "still at the party" and those who have left will tend to breed antipathy.

Oddly, I think you also have a division between those who post photos of themselves and those who post little comments and links - good looking photos are hard to maintain and so the photo part only happen occasionally. Snark can happen often. And so you the splitting of social groups.

And this is a social experiment, more or less, effecting millions of people today.</comment></comment>


Cambridge man who claims he invented email sues Gawker for $35M - <AUTHOR> http://www.bizjournals.com/boston/news/2016/05/10/cambridge-man-who-claims-he-invented-email-sues.html?ana=twt <comment><USER2> The articles being sued over:

[http://gizmodo.com/5887480/the-inventor-of-email-did-not- inv...](http://gizmodo.com/5887480/the-inventor-of-email-did-not-invent-email)

[http://gizmodo.com/5888702/corruption-lies-and-death- threats...](http://gizmodo.com/5888702/corruption-lies-and-death-threats-the- crazy-story-of-the-man-who-pretended-to-invent-email)</comment>


Ask HN: What are the most popular product roadmap tools? - <AUTHOR> <comment><USER2> Sounds stupid, but if you are talking big milestones and overall vision, Excel works just fine.

<comment><USER3> Yeah I've tried a lot of these and every time I find that I'm not actually doing anything productive, but just linking pieces of text together. They seem like the ultimate busywork tools.

</comment></comment><comment><USER4> Don't know if it is the most popular tool, but I can definitely recommend ProductBoard - [https://www.productboard.com/](https://www.productboard.com/)

It is beautiful app with many integrations(Zendesk, Jira, Trelo, Slack, etc) and many features which helps you to conduct research, prioritize what to build next, communicate plans and progress.

Hubert, the CEO of PB, is also very active on Medium where he writes about product management - [https://medium.com/@hpalan](https://medium.com/@hpalan)

</comment><comment><USER5> I've used Product Plan consistently for the last several years.

[http://productplan.com](http://productplan.com)

Full disclosure: I helped them get started (supplied some early coding time).

<comment><USER6> We use ProductPlan here, at Notion ([http://usenotion.com](http://usenotion.com)) too, and we're really happy with it. We wrote a bit about it, and the ProductPlan team put it on their blog: [https://www.productplan.com/data-driven- roadmap/](https://www.productplan.com/data-driven-roadmap/). At my last company, it was a mix of spreadsheets and Confluence.

</comment></comment><comment><USER7> If it's not already on your radar, be sure to check out ProductPlan. It's intuitive and very visual: www.productplan.com

</comment><comment><USER8> JIRA Agile would cover most things you need, attractive if you are already tied into the Atlassian eco-system.

[https://www.atlassian.com/agile/roadmaps](https://www.atlassian.com/agile/roadmaps)

</comment><comment><USER9> Aha is such a tool - some PMs swear by it. It seems ok. It's super expensive.

<comment><USER10> Aha is an excellent way to visualize and bootstrap a complex product.

JIRA integration is effective and is bidirectional.

When it is necessary to provide timelines that allocate not only dates, but effort in terms of hours that be mapped to cost, it beats MSProject. More efficient, more flexible and the trial period is sufficient to make the leap into a proper Agile tool like Trello,JIRA, orAsana via Zapier.

</comment></comment><comment><USER11> Prodpad is great. Trello is also flexible enough to be used with many different workflows.

</comment><comment><USER12> Trello and Asana are pretty good.</comment>


Why Aren't America's Shipping Ports Automated? - <AUTHOR> http://priceonomics.com/why-arent-americas-shipping-ports-automated/ <comment><USER2> Why compare Rotterdam (Europe's largest) to Oakland?

Let's compare to Los Angeles (America's largest) instead ...

Rotterdam moves about 50% more containers, but services 36,000 ships vs. Los Angeles 2,100. Los Angeles sits on 7,500 acres with a 3,200 acre harbor while Rotterdam sits on 13,000(!) acres with a 13,000 acre harbor.

Dunno, for all that automation, Rotterdam doesn't look that much more efficient--and it uses a lot more land and water to do so.

If we include Long Beach which is effectively next to Los Angeles, the two ports combined move _MORE_ containers in a year than Rotterdam while sitting on less land and harbor space.

So, as we can see, the antiquated, evil longshoreman's union is ... more efficient than the best automation in Europe.

Ummmmmmmmm.

Now, quit reposting this agenda driven hit piece.

<comment><USER3> That's a great point, Rotterdam and Oakland are both not great choices for comparisons. However, there is a general point that US ports such as LA + Long Beach do lag in efficiency compared to Asian ports such as Singapore for example (which is heavily automated but also unionized).

For example in "CONTAINER TERMINAL PRODUCTIVITY: EXPERIENCES AT THE PORTS OF LOS ANGELES AND LONG BEACH" Le and Murphy give some comparisons of LA and Long Beach to other leading terminals.

Taking Singapore as our main example, as of 2004 it moved 2.7 as much container volume as LA but occupied roughly half the land area. It has 6 times the TEU/acre of LA and 64% higher TEU/crane. General trends also hold for Kwai Tsing (HK) and Klang (Malaysia).

To quote Le and Murphy in the above-mentioned paper:

"Further consideration of these factors reveals that the terminal operators at these ports are aware of the technologies and practices used at other world ports that would allow them to achieve a higher level of performance. However, the present operating agreements between terminal operators and port labor prevent the implementation of such technologies and practices."

<comment><USER2> LA and Long Beach may also be hampered somewhat in that we apply pollution regulations to them, do these other ports have those restrictions?

Effectively, "it's complicated".

<comment><USER4> Everything is complicated, and we live in a world of high causal density, but that does not mean we should throw up our hands in dismay and self-pity.

It is clear that the union is exploiting its monopoly on the supply of port labor, to decrease efficiency, and increase costs and union membership. For some reason, most people are less likely to attribute high costs to monopoly unions, but more than happy to do so for monopoly corporations.

</comment></comment></comment></comment><comment><USER5> Imagine my surprise when the answer is "unions"

<comment><USER6> Damn those workers for wanting a living. How dare they. Scum, all of them.

<comment><USER7> "Today, the average member of the union in Oakland makes $147,000 per year in wages, with benefits equal to another $82,000 per year."

\--

That is a huge amount of money, especially considering that includes entry level positions. While I understand these machines are complicated, and don't blame them for wanting to keep their jobs, I didn't expect over 200k a year total to be the average.

This is especially true considering that the average individual income in Oakland is around 32k a year. ([http://www.bestplaces.net/economy/city/california/oakland](http://www.bestplaces.net/economy/city/california/oakland))

<comment><USER8> I find it interesting (if a bit sad) that your reaction to that earnings disparity is, essentially, "Union workers make too much money" rather than "Everyone else makes too little." In the free market, it seems everyone is allowed to negotiate the price of their goods except the wage-workers.

<comment><USER9> Why does HN rally against the big companies that use regulatory capture to squeeze out more profits, but then gives unions a pass. They are doing the same thing: supporting legislation that allows them to squeeze out more profits (as wages).

<comment><USER10> A single employee doesn't have much leverage compared to a company. The only leverage a worker has is the "freedom" to quit. For most companies this is a minor annoyance, but nothing they really worry about.

Compare this to the leverage a company has. It controls wages, work hours, work conditions and can fire a worker. This can have devastating impacts on the life of a worker. Unions are the only realistic option for the average workers to have an impact. It levels the playing field.

Unions can certainly have too much power and make self destructive decisions, but you can say the same about many corporations. This is not a problem with unions or corporations, but with a power balance that is out of control.

The only option to a union is strict regulations from the goverment, but that has problems of its own. At least the union has a more direct interest and domain knowledge about their work place

<comment><USER4> Until recently I would have agreed with many of your points, but I recently read a post to the contrary.[1] Here is a short excerpt:

> _Everyone talks as if bosses have the better end. But talk is very different > from action. If everyone were trying to start their own businesses and hire > workers, that would count as "acting as if bosses have the better end of the > deal." Most workers, however, make no effort to become entrepreneurs. You > could object that most workers don't have the money to open their own > businesses, but most rich workers make no effort to become entrepreneurs > either._

[1] [http://econlog.econlib.org/archives/2015/09/scott_alexander_...](http://econlog.econlib.org/archives/2015/09/scott_alexander_3.html)

</comment></comment></comment></comment></comment></comment></comment><comment><USER11> One thing the article nor earlier short discussions here fail to mention is that a ship costs money every day, also when in port.

Consequently, speeding up unloading means one can raise prices or become more competitive, either with other ports or with other modes of transport. The latter may mean that job losses due to automation get partly offset by job gains due to increases in traffic.

</comment><comment><USER12> Double repost: [https://news.ycombinator.com/item?id=10266191](https://news.ycombinator.com/item?id=10266191) [https://news.ycombinator.com/item?id=10262186](https://news.ycombinator.com/item?id=10262186)

</comment><comment><USER13> What would happen if someone tried to open a new port?

<comment><USER14> Most of the best natural ports are already developed. Also, having a great modern port is not much good if you don't have the transportation infrastructure in place to distribute goods from the port.

</comment></comment><comment><USER15> I would like to point to a very interesting Ask HN from a few months ago. Shipping and logistics came up and was the most up voted discussion. Lots of great discussion there.

"Ask HN: What problem in your industry is a potential startup?"

[https://news.ycombinator.com/item?id=9799007](https://news.ycombinator.com/item?id=9799007)

</comment><comment><USER16> So much value can be unlocked by automating the ports that there would be plenty of room for a negotiated settlement to make the unions happy as well.

<comment><USER2> I don't disagree, but you need some numbers to back that up.

You have to get the containers _out_ of the port as well.

So, can the rail system or trucking transfer point handle another 25% in volume?

To top that off, these ports are some of the biggest pollution sources. What happens when you add 25% to the traffic?

These things aren't simple choices.

<comment><USER16> You don't need a lot of numbers to understand that unloading ships 24/7/52 is better than unloading only during union work hours.

<comment><USER2> Reference please that the longshoremen don't work night shifts.</comment></comment></comment></comment>


Automating a multi-tiered application securely on AWS with Docker and Terraform - <AUTHOR> https://www.airpair.com/aws/posts/ntiered-aws-docker-terraform-guide

<comment><USER2> I was excited to find out recently that Hashicorp has hired Clint Shryock to expand Terraform's support for different AWS services. If you check the git history there's a lot of work going on to move over to aws-sdk-go. Hopefully Terraform will reach or exceed parity with Cloudformation in the next few releases.

[https://hashicorp.com/blog/clint-joins- hashicorp.html](https://hashicorp.com/blog/clint-joins-hashicorp.html)

<comment><AUTHOR> Yeah. They are aggressive migrating to aws-sdk. I managed to fully move to Terraform from CloudFormation and has been amazing so far!

</comment></comment><comment><USER3> Great article to explain terraform to newcomers. I am extremely excited to move all of my infrastructure to terraform once a few more features have been added. Its a great project, by great people, from a great company.

<comment><AUTHOR> Thanks for the kind words. Its an amazing tool indeed, their progress is extremely impressive as well!</comment></comment>


Muting the Mozart effect - <AUTHOR> http://news.harvard.edu/gazette/story/2013/12/muting-the-mozart-effect/

<comment><USER2> Anecdotally, some music helps me to concentrate. Some music helps me to concentrate on programming; other music helps me to concentrate on reading. Some music doesn't help me to concentrate at all (except for on the music itself).

Based purely on my own personal experience, I would not expect any arbitrary music to help someone with any arbitrary cognitive tasks, but I'm not convinced that, at least for some people, there doesn't exist _some_ music that can help them with _some_ cognitive tasks. Admittedly, that's more ambiguous and open-ended than this present study.

[EDIT: typo]</comment>


Selling a Good Reputation on eBay - <AUTHOR> http://cgi.ebay.com/Picture-tree-100-Positive-FB-PF-buyer-ASAP-/180672912258?pt=Art_Photo_Images&hash=item2a10f1db82#ht_500wt_898

<comment><USER2> That will be taken down in no time. It's illegal as per eBay policies.</comment>


Ask HN: When did HN start using recaptcha? - <AUTHOR> I just got a recaptcha prompt while logging in.<p>Has Google been linking my HN profile to my ad profile this whole time? <comment><USER2> I didn't see this question until earlier. That's why the site guidelines ask you to email firstname.lastname@example.com if you want to ask us something: [https://news.ycombinator.com/newsguidelines.html](https://news.ycombinator.com/newsguidelines.html)

We turned it on temporarily because HN was under attack by an account-stealing botnet. Obviously we don't leave it on any longer than we have to. We also are happy to put anyone's username on a whitelist, as the people who emailed us found out.

<comment><USER3> Thank you for clarifying. I wasn't aware you run a whitelist, this sounds like a good approach.

</comment></comment><comment><USER4> I was also made to log in via a recaptcha today. The recaptcha widget appears just below the HN login form so pretty sure this is added by HN, not Cloudflare.

If people are complaining about it on this thread, I would also like to register my complaint with it. I think Recaptcha has two major problems:

\- It's Google. I would want it to be technically impossible for someone in Google to link my HN profile with my Google profile.

\- The Recaptcha widget is plain abusive to Firefox users. If I log in with Chrome, I can get past the Recaptcha hurdle in just one attempt. But if I am Firefox user, I am forced to submit like 5 Recaptchas, then be greeted with a "Please try again" message, then submit like 5 more, and only then be allowed to log in. No I don't want to do this amount of work for Google for free. I urge everyone to please stop supporting Recaptcha.

<comment><USER5> \- Privacy and security enhancing techniques make the captcha harder or impossible. For example, the tracking protection built into firefox blocks the recaptcha.js (in a certain view even rightfully so, because google will use that for tracking)

\- Apps do not work anymore. This cannot be solved. For recaptcha, one needs a working browser steered by humans. Apps necessarily act like robots.

This comes in line with increased usage of "2"-factor authentication. For example, amazon now requires a browser cookie to login. If you don't have such a cookie, you need to enter a code received via e-mail or SMS. If you delete your cookies (maybe automatically), you This comes under the pretense of security but really is used to fight user privacy.

One general point: Businesses uses machines to provide services. Why shouldn't we as consumers be allowed to rely on machines to consume said services? 15 years ago there was this "semantic web" fad with "intelligent agents". This mess is a huge step backwards.

The root-cause of this is two-fold: \- advertising-based business model. If every "bot" needed to pay for the service as well as any other user, revenue will not be hurt by "bots". \- other misuse \- "Dumb users" as more computer- illiterate people are using these services, it gets easy for businesses to dismiss user choice.

If this should not end in a dystopian nightmare, we will need: \- Privacy- preserving login protocols that are stronger than user/password \- Privacy- preserving and low-friction micropayment (e.g. Taler) \- Some privacy- preserving way to fight misuse. I have no idea here. Maybe some crypto-social- network with zero-knowledge-foo?

</comment></comment><comment><USER6> The recaptcha has broken HACK, the iOS app, for me.

</comment><comment><USER7> Might be cloudflare. They use Recaptcha.

<comment><USER5> No. It's a 2nd screen to complete login. It also breaks mobile clients such as materialistic and is plain evil anyway

<comment><USER8> I was wondering why I couldn't upvote anything through the app and then also wondering why the app kept thinking I wasn't signed in. This is poor indeed. I like using Materialistic. I guess I just won't vote, submit or comment until this madness is over.

</comment></comment></comment><comment><USER3> apparently very recently. f __king disgraceful.

<comment><USER2> When HN is under attack, we have to do things we wouldn't otherwise do. Please see [https://news.ycombinator.com/item?id=20794659](https://news.ycombinator.com/item?id=20794659).</comment></comment>


How Do We Get More Students Interested in Math, Science & Tech Careers?  - <AUTHOR> http://mashable.com/2011/09/07/stem-microsoft-infographic/?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+Mashable+%28Mashable%29

<comment><USER2> I can tell you one thing, the answer is NOT increasing tuition and cutting financial aid. 100% certain increasing tuition and cutting student financial aid is the wrong answer.</comment>


Show HN: Simple metro-styled todo app (need feedback) - <AUTHOR> http://besttodoever.com/

<comment><AUTHOR> Yep, yet another todo list web-app. Tailored to my needs, maybe someone else's too. I made it so that it doesn't even require signing up to try to make it easier to understand whether you want it or not. Thanks in advance for any feedback.</comment>


Google starts showing full page previews in search results - <AUTHOR> http://www.blogstorm.co.uk/google-starts-showing-full-page-previews-in-search-results/ <comment><USER4> This is very bad.

People won't have to visit the page they found - meaning no visit, no ads shown, no money. Every real content producer is going to lose BIG with this.

Also this looks like World-Wide Copyright Infringement to me. Possibly, a good reason for class action suit from everyone who created any kind of site. Google now creates and distributes large portions of your web page without any kind permission. (I don't think this (near full-size screenshots with readable text) could pass as "fair use" in any kind of sane trial)

Sadly, this is exactly what I predicted 2 years ago: "What worries me is the amount of text they [Cuil] show on the search page. It's becoming much and much more of a nuisance that search companies think it's okay to massively copy parts of your site and display them. "

<comment><USER5> No, you're thinking small. First, the full page shows including ads. If anything potentially _more_ sites and ads will receive a view as users mouse over results. The difference is a user doesn't have to go to a page and be annoyed with pop-up messages before hitting the back button. When a site with the desired content is found the user will then click through to it.

This appears to me to offer end users a better search/browse experience, and that's always a win for everybody.

<comment><USER4> You can't tell your advertiser: "Hey, your ads, probably, were shown about a hundred thousand times on Google. Well, I can't prove it, users can't click on it and also they see a random frame from your ads (if they see it at all), but still now you can pay me".

Why would I (as a visitor) need to click the site if I see what I searched for right there, on Google screenshot?

<comment><USER5> _Why would I (as a visitor) need to click the site if I see what I searched for right there, on Google screenshot?_

Because that's not how people typically interact with websites. If it was, the World Wide Web would simply be made of small JPEG pic versions of what sites look like. If you have a site with quality content on it, which happens to be ranked high in search results I can pretty much guarantee you are still going to receive your high click-through traffic. You may even receive more. The only people who lose out with this are those with low quality or spam type sites where their traffic is "crap, not what I wanted" hit-the-back-button clicks, which don't provide much value to advertisers anyway.

<comment><USER4> Imagine this: you search for some topic. You hover over one result and get a pretty big chunk of text about that topic, you hover over the next one - <USER2> get one more big chunk of text, you hover over 2-4 more - you've all the information you need. There's no more need to click.

I don't mind "small JPEG pic versions of what sites look like" - these are fine. I'm strongly against the JPGs of site where you can read text from my site. And with a move of mouse - from next site. All cropped around the subject without any ads or any message attached to article that PAID for this article.

<comment><USER5> I'm sorry, but I still think you are thinking small. Let's assume the type of site or query is not shopping or research related where a user must scroll, click, etc. for the site, and it's news type editorial. You're thinking users will prefer to squint and experience the page/article with the preview version instead of simply clicking through to the site? What about comments? Hacker News is a great example of just how much value comments add to content. I think you're guarding user behavior that doesn't need to be guarded. The only sites that have anything to fear from this are ones users don't want to be on. If users want to be on your site, they will be.

<comment><USER4> Don't you USUALLY read editorial type news on the site you know, instead of SEARCHING for it?

Do you seriously think that "research and shopping" doesn't constitute overwhelming majority of searches?

Do you seriously assume that searching for editorial content is what majority of users do instead of "research and shopping"?

The only thing I'm guarding is MY OWN CONTENT. I wrote it. I own it. I don't want Google to display it in readable form in bigger and bigger chunks every year. I'd opt out of "short snippets" they show now if it was possible - <USER3> the titles - that'd be ENOUGH to reference my content.

Google was built on shoulders of people who produce the content. Now Big G. acts like they own this content and are free to do whatever they please with it to satisfy visitors on THEIR site and earn money from THEIR ads, using MY content.

I do understand it's better experience for visitors, but why would I want to produce the content anymore if Google steals my profit by showing the content directly to users? I want people who benefit from my content to be on MY site, not on Google site.

<comment><USER5> _Don't you USUALLY read editorial type news on the site you know, instead of SEARCHING for it?_

Yes, probably.

 _Do you seriously think that "research and shopping" doesn't constitute overwhelming majority of searches?_

Yes, I do think so and that supports my point. Those type searches are not usually satisfied by viewing a small screenshot of a site's homepage.

 _The only thing I'm guarding is MY OWN CONTENT. I wrote it. I own it._

Calm down. I certainly agree content creators own the rights to their content. :)

If this is really a problem I'm sure Google would allow some way for sites to opt out of the preview. However, I'm thinking that Google's priority is to give users the best search experience possible, so sites that don't allow the preview might rank lower in search results. I'd certainly allow it for my sites.

 _Google was built on shoulders of people who produce the content._

I'd argue there is a symbiotic relationship between content creators and search engines/portals. Without each other neither would likely have many visitors.

I totally agree content creators have the right to say how their content is displayed beyond fair use. However, at the same time Google and other search engines are going to strive to give users the best possible experience, and that probably means helping them avoid spammy/low quality sites by previewing past them.

<comment><USER4> I'm glad we agree.

Yes, it was symbiotic, that's why it was "kind of" fair play until NOW.

Now they're pulling the blanket to their side and it starts to get parasitic.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER6> I could see definitely see arguments on both sides of the fence regarding this feature. It may also lead to more work for designers, since the "first impression" of a website would occur before the click-through.

</comment><comment><USER7> I can see the usability benefits, not least when it tells the user the obscure phrase they searched for is found in the third article down the page.

Unless the website is designed to be readable as a scaled down .jpeg and/or the entire useful content of a page is encapsulated in the single paragraph Google chooses to highlight I don't think it's a massive loss for webmasters. If sites have to look halfway decent to get decent click throughs it could be a godsend for designers though.

The implementation is pretty ugly...

</comment><comment><USER8> Should be able to disable it in robots.txt or meta tags, so shouldn't be a big deal for publishers.

<comment><USER9> That's true, but that doesn't make it OK. Google is making copies of other people's work without permission, and displaying them for profit. That is the kind of thing that copyright law was meant to prevent. This should be an opt- in feature, something that publishers can _enable_ in robots.txt, if they think it will help them get more traffic.

<comment><USER10> People have been using extensions to add website thumbnails to search results for years without a peep from web developers screaming copyright infringement.

Why don't I see you complaining about them providing textual context to the term searched for? Oh, right, because every other search engine on the planet does it. So now that Google is adding by default something that extensions have had for a while now, a graphical context in addition to text, you suddenly think it's evil?

Give me a break.

<comment><USER9> The amount being copied makes a big difference, in terms of fair use. If you're copying the entire page, and _showing_ it to people, then that's a more egregious violation than taking a small snippet to provide context.

I'd also say that Google's caching is a copyright violation, and should be opt-in. Google seems to have this notion that because they're above the law when it comes to copyright. They're not. They may be doing this for good reasons, and it may have positive effects for the users of their search engine, but that's irrelevant. The law is the law.

Also, I never said that it was evil. I only said that they shouldn't do it. There is a wide gap between those two assertions.

*Edited to change "provide text" to "provide context" at the end of the first paragraph, and remove an unfortunate use of all caps. Wasn't thinking as clearly as I should have been, sorry.

<comment><USER10> <http://www.blogstorm.co.uk/images/google-new-previews.png>

It's a large thumbnail where the text isn't legible at all outside of a few select words and the extracted surrounding text. This isn't the same as providing a full-page preview with Google wrapping it in an iframe, for example, to get advertising revenue. People still have to go to your site to read anything outside of the contextual text.

</comment></comment></comment></comment></comment><comment><USER11> This experiment illustrates the potential for Google.com to evolve into a cloud-based Chrome.

They can own the browsing experience regardless of the browser or platform used.

Think about the potential this has as browsing from televisions, set-top boxes, mobile etc continues to explode.</comment>


Business success should not trump creativity: Pandora isn't playing fair - <AUTHOR> http://thehill.com/blogs/congress-blog/technology/270817-business-success-should-not-trump-creativity-pandora-isnt-playing-fair

<comment><AUTHOR> I'm submitting this because it appears to be in direct opposition to this that was on the front page of HN a while ago: [http://blog.pandora.com/pandora/archives/2012/10/pandora- and...](http://blog.pandora.com/pandora/archives/2012/10/pandora-and-art.html)

<http://news.ycombinator.com/item?id=4633842></comment>


Gadget Patrol: 21st century phone - <AUTHOR> http://www.antipope.org/charlie/blog-static/2009/12/21st_century_phone.html <comment><USER2> Stross directly explained what my startup (Avecora <http://avecora.com>) is trying to work towards.

The current mobile telcos provide two major things: the a) network infrastructure and b) the network experience. The infrastructure is the physical satellite/cell tower/everything supporting the mobile experience. The mobile experience encompasses the use of the network: phone numbers, SMS, digital downloads, WAP, etc.

However, the aspiration for Google (and my startup Avecora) is that the mobile telcos transform themselves into a strictly infrastructure play, essentially becoming the backbone NOT OF wireless telephony/texting/etc., BUT RATHER OF wireless intercommunication. Indeed, we want them to become completely hands- off in terms of the wireless experience, because they're good at infrastructure and making communication successful, but they're less good at innovating in the experience.

One of the reasons we don't want them controlling the wireless experience is that they have to support so much legacy _stuff_. My startup's big selling point is in the Avecora Network: there are no real "phone numbers," but rather it operates on a more convenient structure: the mobile network is a social network. You call a person, not a phone number.

We both do want the internet to eat the phone system, and I think it'll offer a much more enriching and integrated mobile experience than it is today, especially when so much already works over TCP/IP (VoIP, for example.) However, with phone companies already making an absolute killing in costs (my bill this month is $90) I don't see this happening anytime too soon, especially the idea of VoIP replacing voice. The carriers will take a huge loss and there's no way in hell they'll sign off on that. (I'm skeptical of T-Mobile's $30/mo data-only plan—I can just grab that and hook up my $0.021/min Skype account to it.)

The best idea, and how we're approaching it, is to price it as one Connection Package and have it be on par if not a bit less than current prices. That's the only way it will really work for both the consumer and the telcos.

Google will get there first, perhaps with the killer product, perhaps not. Indeed, we are competing with Google, and we don't have a prototype right now—but in this space, it seems like, at first, it's more about the strategy than what we've got. (Wish me luck.)

<comment><USER3> That flies in the face of conventional thinking, which says ideas are cheap and delivery is everything. Unless conceptually you've figured out such an enormous breakthrough that the solution is literally in your hands right now, you have no competitive edge over anybody, and you're working with a concept simplistic enough a thirteen-year-old could understand it. That's not a good thing, if the concept's all you've got. It means the thirteen-year-olds at Google and Skype have figured it out too.

But you're adept at saying very simple and ugly things with enough words that it sounds almost swallowable. I had to reread what you posted twice to make out what you were saying. Basically, you're admitting that Skype has an alternative that's cheaper and more effective, and that it works perfectly save phone companies don't like it. Your solution is to price gouge the same way phone companies are right now so that people will be tricked into thinking there's a profit. But phrased like that, it sounds more like a concept without an idea, artificially priced at a level beyond its actual value.

I wish you luck not for Avecora but for whatever idea you have after this. The best way to learn quickly is to pursue a wild, failing ambition. Two years from now, if you've learned, you'll have a savvy that's the envy of your peers. Just make sure that for the time being, you're enjoying your work. Then failure really doesn't matter.

<comment><USER2> Thanks for your honesty, brother.

</comment></comment></comment><comment><USER4> Why does anyone believe that a carrier will sell cheap no-contract unlimited data-only service (probably at a loss) in exchange for a non-exclusive on a phone that isn't even the best in the world? I guess it's appropriate that Stross is tackling this topic since it sounds like total fiction. :-)

<comment><USER5> The article suggested that the current structure of the telco industry would have to change "the economic basis of your mobile telephony service in 2019 is going to be unrecognizably different from that of 2009".

I think there is something to this argument. I worked for a telco OEM in 2005. Then it was possible to buy the hardware to build an "NFL" city backbone in the US for about $50 million. Adding fiber, end user access, operations and installation would clearly add much to this but it is likely that within the next decade that a wireless network with a significant nationwide footprint could be built by a company capitalized with a few billion dollars while not carrying the legacy operations costs of the incumbents. This company would likely have no problem in driving network costs to the ground. This almost happened during the first internet bubble and could this time around follow from the current cloud computing trend. Adding compute services on top of network services could help to regain some of the margins lost by commoditization of the network.

<comment><USER4> What you're talking about sounds totally awesome, except these rumors say that one of the _existing_ carriers is going to blow up their whole business model about _two weeks from now_. And that just doesn't make any sense.

</comment></comment></comment><comment><USER6> Promoting this internal link, an interesting analysis of AT&T's possible packet buffering/congestion issues.

[http://blogs.broughturner.com/2009/10/is-att-wireless- data-c...](http://blogs.broughturner.com/2009/10/is-att-wireless-data- congestion-selfinflicted.html)</comment>