Announcing SyntaxNet: The World's Most Accurate Natural Language Parser - <AUTHOR> http://googleresearch.blogspot.com/2016/05/announcing-syntaxnet-worlds-most.html?m=1 <comment><USER2> Evidence that this is the most accurate parser is here; the previous approach mentioned is a March 2016 paper, "Globally Normalized Transition-Based Neural Networks," [http://arxiv.org/abs/1603.06042](http://arxiv.org/abs/1603.06042)

"On a standard benchmark consisting of randomly drawn English newswire sentences (the 20 year old Penn Treebank), Parsey McParseface recovers individual dependencies between words with over 94% accuracy, beating our own previous state-of-the-art results, which were already better than any previous approach."

From the original paper, "Our model achieves state-of-the-art accuracy on all of these tasks, matching or outperforming LSTMs while being significantly faster. In particular for dependency parsing on the Wall Street Journal we achieve the best-ever published unlabeled attachment score of 94.41%."

This seems like a narrower standard than described, specifically being better at parsing the Penn Treebank than the best natural language parser for English on the Wall Street Journal.

The statistics listed on the project GitHub actually contradict these claims by showing the original March 2016 implementation has higher accuracy than Parsey McParseface.

<comment><USER3> spaCy is another active open source (MIT) POS-tagger. In a previous discussion on HN[1] it was well received.

There is a simplified educational 200 lines python version [2] of it. It claims 96.8% for the WSJ corpus.

What am I missing here?

[1] [https://news.ycombinator.com/item?id=8942783](https://news.ycombinator.com/item?id=8942783)

[2] [https://spacy.io/blog/part-of-speech-pos-tagger-in- python](https://spacy.io/blog/part-of-speech-pos-tagger-in-python)

<comment><USER4> spaCy's 96.8% accuracy is for the task of POS tagging while Google's reported 94% accuracy is for dependency parsing, a significantly harder problem.

<comment><USER3> Thanks (also to charlieegan3) this explains the difference nicely.

Actually I found an educational 500 lines python parser from the same author as well [1]. It claims an accuracy of 92.7 for the WSJ corpus.

[1] [https://spacy.io/blog/parsing-english-in- python](https://spacy.io/blog/parsing-english-in-python)

</comment></comment></comment></comment><comment><USER5> This is really cool, and props to Google for making it publicly available.

The blog post says this can be used as a building block for natural language understanding applications. Does anyone have examples of how that might work? Parse trees are cool to look at, but what can I do with them?

For instance, let's say I'm interested in doing text classification. I can imagine that the parse tree would convey more semantic information than just a bag of words. Should I be turning the edges and vertices of the tree into a feature vectors somehow? I can think of a few half-baked ideas off the top of my head, but I'm sure other people have already spent a lot of time thinking about this, and I'm wondering if there are any "best practices".

<comment><USER6> This would be very interesting when applied to Biblical Studies. Any serious academic discussion of biblical texts will involve syntactical breakdown of the text being discussed. Most of the times the ambiguities are clear, but its still quite common for a phrase to have several possible syntactical arrangements that are not immediately clear. These ambiguities are also challenging becuase the languages are dead (at least as used in the biblical texts). So the type of ambiguity of "Alice drove down the street in her car" can lead to some significant scholarly disagreement.

I could see Parsey McParseface helping identifying patterns in literature contemporaneous to the biblical texts. Certain idiomatic uses of syntax, which would have been obvious to the original readers, could be identified much more quickly.

<comment><USER7> I was going to say... my main interest in this project is precisely for Biblical studies... I could talk about analyzing the Bible for hours, but let's just say there's way more depth than many even realize. The Aleph Tav in relation to the Book of Revelation is one such example, many translations omit it, but the Aleph Tav Study Bible explores it in depth. There could be many discoveries made with these kind of projects that are missed by just about anyone only reading a translation.

There are a ton of Jewish Idioms in the Bible that many don't understand at all, including "No man knows the day or the hour" which is a traditional Jewish Wedding Idiom. Lots and lots of things could be explored with enough data and resources.

<comment><USER8> I'd think that the advantage of machine translation is on corpora that are not known up front (i.e. user-supplied text) or corpora that are exceptionally large.

If you have a small (ish), well-known text, I don't think you will get much insight from machine translation. Certainly there are plenty of uses for computer text analysis/mining in biblical studies, but I doubt translation is one of them. And for obscure idioms or hapax legomena, machine translation definitely can't help you because by definition there are no other sources to rely on.

<comment><USER7> Well I wouldn't look for idioms, but it would be interesting to throw in information such as "Strong's Concordance" into the mix, I've yet to really think of an application for this library fully, but it would be fun to play around with it nonetheless. I would be analyzing the Hebrew / Greek / Syriac scripts, seeking verses omitted, or missing, etc. It would make for interesting studying if anything.

<comment><USER9> You might be interested in Andrew Bannister's research on computer analysis of the Quran. He wrote a book on it [1], and there's also this paper which gives a high-level overview [2].

[1] [http://www.amazon.com/Oral-Formulaic-Study-Quran-Andrew- Bann...](http://www.amazon.com/Oral-Formulaic-Study-Quran-Andrew- Bannister/dp/0739183575)

[2] [http://www.academia.edu/9490706/Retelling_the_Tale_A_Compute...](http://www.academia.edu/9490706/Retelling_the_Tale_A_Computerised_Oral- Formulaic_Analysis_of_the_Qur_an)

</comment></comment></comment></comment></comment></comment><comment><USER10> One of the projects I'd love to develop is an automated peer editor for student essays. My wife is an english teacher and a large percentage of her time is taken up by grading papers. A large percentage of that time is then spent marking up grammar and spelling. What I envision is a website that handles that grammar/spelling bit. More importantly, I'd like it as a tool that the students use freely prior to submitting their essays to the teacher. I want them to have immediate feedback on how to improve the grammar in their essays, so they can iterate and learn. By the time the essays reach the teacher, the teacher should only have to grade for content, composition, style, plagiarism, citations, etc. Hopefully this also helps to reduce the amount of grammar that needs to be taught in-class, freeing time for more meaningful discussions.

The problem is that while I have knowledge and experience in the computer vision side of machine learning, I lack experience in NLP. And to the best of my knowledge NLP as a field has not come as far as vision, to the extent that such an automated editor would have too many mistakes. To be student facing it would need to be really accurate. On top of that it wouldn't be dealing with well formed input. The input by definition is adversarial. So unlike SyntaxNet which is built to deal with comprehensible sentences, this tool would need to deal with incomprehensible sentences. According to the link, SyntaxNet only gets 90% accuracy on random sentences from the web.

That said, I might give SyntaxNet a try. The idea would be to use SyntaxNet to extract meaning from a broken sentence, and then work backwards from the meaning to identify how the sentence can be modified to better match that meaning.

Thank you Google for contributing this tool to the community at large.

<comment><USER11> I think this is still risky if used in a context where the student might think that the computer is somehow always right. Great English writers often deliberately use sentence fragments or puns, or use a word with a nonstandard part-of-speech interpretation (especially using a noun as a verb). They may also sometimes use a sentence that's difficult for readers to parse and then explain the ambiguity after the fact.

If a teacher gave students a grammar-checking tool to check their writing, they might assume that the tool knew better than they did, which is only sometimes true.

<comment><USER12> Those great writers that break all the rules still know them.

"Learn the rules like a pro, so you can break them like an artist."

-Picaso

<comment><USER13> This is often repeated, but there's no evidence it's true. Many great writers had no formal training.

"And that quote is almost certainly made up."

\- Willem Shakespere

<comment><USER14> Knowing the rules is not the same thing as having formal training.

With then-innovated art like Cubism, which in some sense broke earlier rules, the point is that those artists, like Picasso, were _able_ to do representative art in correct proportions -- they were just going beyond that.

This is a nontrivial issue, because there are always students who think they can skip learning boring mundane old fashioned art and go straight to something like cubism, but in 99% of cases that doesn't work well at all compared with learning "the rules" first.

With writing, many great writers have broken "the rules" with punctuation, spelling, grammar, etc. But the important thing is that they do so _on_ _purpose_.

Whereas if one doesn't know the rules in the first place, one doesn't have the choice of whether to follow them or break them.

Such a person will always break those rules they don't know (subconscious knowledge counts btw) -- but not for aesthetic reasons, only out of ignorance.

<comment><USER15> My perhaps favourite example of knowing when and how to break the rules is Franz Schuberts "Erlkönig" [1] because it is so stark.

If you listen to it without paying attention to the text (based on a poem by Goethe by the same name; both the German text and an English translation is found at [1]), parts of it sounds like horrible jammering and poor harmonies and it's easy to write it off as not sounding very nice.

Here's [2] a much clearer rendition (two singers, with much stronger delineation of the three different characters) than the one linked from Britannica.

If you _do_ pay attention to the text, it is very clear that the unpleasant parts are very deliberate:

The singer(s) switches between the role of a father, his sick dying child, and the Erl-king that occurs in the hallucinations of the child while the father is riding to bring the child to a doctor.

The big difference between the unpleasant-sounding parts of this song and a bad composer is the clear intent and delineation - Schubert made things sound bad intentionally explicitly at the points he wanted to illustrate pain and fear, rather than because he didn't know how to make things sound pleasant when he wanted to.

The song clearly proves this by setting the childs jammering and the fathers fearful attempts to soothe him up against much more pleasant segments where the Erl-king speaks and tries to seduce the child to come with him.

You only get that clear separation if you know how to evoke each effect precisely. Arguably a bad particularly composer wouldn't even know how to make things sound bad the "right way" \- there's a big difference between random bad sounds and making things evoke a child in pain.

[1] [http://www.britannica.com/topic/Erlkonig](http://www.britannica.com/topic/Erlkonig)

[2] [https://soundcloud.com/sean_contretenor_lee/erlkonig](https://soundcloud.com/sean_contretenor_lee/erlkonig)

</comment></comment></comment></comment></comment></comment><comment><USER16> Now for the buffalo test[1]

`echo 'Buffalo buffalo Buffalo buffalo buffalo buffalo Buffalo buffalo' | syntaxnet/demo.sh'

       buffalo NN ROOT

       +-- buffalo NN nn

       |   +-- Buffalo NNP nn

       |   |   +-- Buffalo NNP nn

       |   |   +-- buffalo NNP nn

       |   +-- buffalo NN nn

       +-- Buffalo NNP nn

            +-- buffalo NNP nn

[1]: [https://en.wikipedia.org/wiki/Buffalo_buffalo_Buffalo_buffal...](https://en.wikipedia.org/wiki/Buffalo_buffalo_Buffalo_buffalo_buffalo_buffalo_Buffalo_buffalo)

<comment><USER17> How does it handle the "translation" from that wiki page?

Bison from Buffalo, which bison from Buffalo bully, themselves bully bison from Buffalo.

<comment><USER16> > Input: Bison from Buffalo , which bison from Buffalo bully , themselves > bully bison from Buffalo . Parse:

       Bison NNP ROOT
        +-- from IN prep
        |   +-- Buffalo NNP pobj
        +-- , , punct
        +-- bison NN ccomp
        |   +-- themselves PRP nsubj
        |   +-- bully RB advmod
        |   +-- from IN prep
        |       +-- Buffalo NNP pobj
        +-- . . punct

<comment><USER17> Interesting.

"bully RB advmod"

I assume this is wrong, and probably triggered by the "ly" ending? Also, odd that it dropped the second clause and didn't see any trouble with the parsed results lacking a single verb. I guess it has to be able to handle fragments, etc.

</comment></comment></comment></comment><comment><USER18> It's really nice to have access to these kinds of tools. I am sure some folks from Google are checking this, so thank you.

Analysis of the structure of a piece of text is the first step to understanding its meaning. IBM are doing some good work in this area. [http://www.alchemyapi.com/products/demo/alchemylanguage](http://www.alchemyapi.com/products/demo/alchemylanguage)

Anything in the pipeline for this project to help with classifying sentiment, emotion etc. from text?

<comment><USER19> Yes, we derive syntactic meaning from grammatical structure. It's one thing getting a machine to understand grammar and another to get a human to understand. If anyone is interested, Doing Grammar by Max Morenberg is an excellent source of knowledge about grammar.[0]He approaches grammar very systematically which is helpful if people want to train machines.

[0] [http://www.amazon.com/Doing-Grammar-Max- Morenberg/dp/0199947...](http://www.amazon.com/Doing-Grammar-Max- Morenberg/dp/0199947333)

</comment></comment><comment><USER20> I'd love to hear Chomsky's reaction to this stuff (or someone in his camp on the Chomsky vs. Norvig debate [0]).

My understanding is that Chomsky was against statistical approaches to AI, as being scientifically un-useful - eventual dead ends, which would reach a certain accuracy, and plateau - as opposed to the purer logic/grammar approaches, which reductionistically/generatively decompose things into constituent parts, in some _interpretable_ way, which is hence more scientifically valuable, and composable - easier to build on.

But now we're seeing these very successful blended approaches, where you've got a grammatical search, which is reductionist, and produces an interpretable factoring of the sentence - but its guided by a massive (comparatively uninterpretable) neural net.

It's like AlphaGo - which is still doing search, in a very structured, rule based, reductionist way - but leveraging the more black-box statistical neural network to make the search actually efficient, and qualitatively more useful. Is this an emerging paradigm?

I used to have a lot of sympathy for the Compsky argument, and thought Norvig et al. [the machine learning community] could be accused of talking up a more prosaic 'applied ML' agenda into being more scientifically worthwhile than it actually was.

But I think systems like this are evidence that gradual, incremental, improvement of working statistical systems, can eventually yield more powerful reductionist/logical systems overall. I'd love to hear an opposing perspective from someone in the Chomsky camp, in the context of systems like this. (Which I am hopefully not strawmanning here.)

[0]Norvig's article: [http://norvig.com/chomsky.html](http://norvig.com/chomsky.html)

<comment><USER21> Not in either camp, quit the nlp field due to disillusionment that it would lead to anything useful or meaningful.

Both rule-based and statistical approaches are fundamentally flawed by not incorporating any real world information. Humans do language going from real world info, mapping to grammar or rules. Computers are trying to go the other way and are not going to succeed other than as mere toys.

Even tech progression-wise, both rule and model/nn approaches are really bad since there is no meaningful sense of iterative progress or getting better step by step, unlike cpu chips or memory speeds. They are more of a random search in a vast space, hit or miss, getting lucky or not, which is very bad no good, as a technology or as a career.

<comment><USER22> How about looking at it this way. We now have multiple implementations that get 94%+ success _without_ knowing anything about the world. Isn't that remarkable?

Now to get to 99.4%+ how about we combine techniques such as spaCy or Parsey McParseFace (love the name Google) with very simple real-world cognitive models. So for the example given "Alice drove down the street in her car." a simple cognitive model would _know_ that streets cannot be in cars and so be able to disambiguate. A cognitive model wouldn't know all the facts about the world, it would know certain things about streets, certain things about cars and be able to infer on the fly whether the relationship between streets and cars matches either the first parse possibility or the second. To me this seems like the obvious next step. If it's obvious to me it must have been obvious to someone else so presumably somebody is working on it.

<comment><USER21> The 94% success rate is in made up, limited tests. Real world, they fail constantly in weird horrific or laughable ways. See any Microsoft AI public demo ever. It's like the self driving car claim of millions of miles without accidents, except that humans took over anytime there was a chance of one.

</comment></comment></comment></comment><comment><USER23> This looks fantastic. I've been fascinated with parsers ever since I got into programming in my teens (almost always centered around programming language parsing).

Curious - The parsing work I've done with programming languages was never done via machine learning, just the usual strict classification rules (which are used to parse ... code written to a strict specification). I'm guessing source code could be fed as data to an engine like this as a training model but I'm not sure what the value would be. Does anyone more experienced/smarter than me have any insights on something like that?

As a side-point:

Parsy McParseface - Well done. They managed to lob a gag over at NERC (Boaty McBoatface) and let them know that the world won't end because a product has a goofy name. Every time Google does things like this they send an unconscious remind us that they're a company that's 'still just a bunch of people like our users'. They've always been good at marketing in a way that keeps that "touchy-feely" sense about them and they've taken a free opportunity to get attention for this product beyond just the small circle of programmers.

As NERC found out, a lot of people paid attention when the winning name was Boaty McBoatface (among other, more obnoxous/less tasteful choices). A story about a new ship isn't going to hit the front page of any general news site normally and I always felt that NERC missed a prime opportunity to continue with that publicity and attention. It became a topic talked about by friends of mine who would otherwise have _never_ paid attention to _anything_ science related. It would have been comical, should the Boaty's mission turn up a major discovery, to hear 'serious newscasters' say the name of the ship in reference to the breakthrough. And it would have been refreshing to see that organization stick to the original name with a "Well, we tried, you spoke, it was a mistake to trust the pranksters on the web but we're not going to invoke the 'we get the final say' clause because that wasn't the spirit of the campaign. Our bad."

<comment><USER24> _Curious - The parsing work I 've done with programming languages was never done via machine learning,_

Artificial languages (such a programming languages) are usually designed to be unambiguous. In other words, there is a 1:1 mapping from a sentence or fragment to its abstract representation.

Natural language is ambiguous, so there is usually 1:N mapping from a sentence to abstract representations. So, at some point you need to decide which of the N readings is the most likely one.

Older rule-based approaches typically constructed all readings of a sentence and used a model to estimate which reading is the most plausible. In newer deterministic, linear-time (transition-based) parsers, such ambiguities (if any) are resolved immediately during each parsing step.

In the end it's a trade-off between having access to global information during disambiguation and having a higher complexity. So, naturally, the rule-based systems have been applying tricks to aggressively prune the search space, while transition-based parsers are gaining more and more tricks to incorporate more global information.

</comment></comment><comment><USER25> For those wondering: the license appears to be Apache 2.0 ([https://github.com/tensorflow/models](https://github.com/tensorflow/models))

</comment><comment><USER26> Dockerized version so you try without installing:

[https://hub.docker.com/r/brianlow/syntaxnet- docker/](https://hub.docker.com/r/brianlow/syntaxnet-docker/)

</comment><comment><USER27> > _Humans do a remarkable job of dealing with ambiguity, almost to the point > where the problem is unnoticeable; the challenge is for computers to do the > same. Multiple ambiguities such as these in longer sentences conspire to > give a combinatorial explosion in the number of possible structures for a > sentence._

Isn't the core observation about natural language that humans _don 't parse it at all_? Grammar is a secondary, derived construct that we use to give language some stability; I doubt anyone reading "Alice drove down the street in her car" actually parsed the grammatical structure of that sentence, either explicitly or implicitly.

Anyway, some impressive results here.

<comment><USER28> Various syntactic theories (HPSG, GPSG, minimalism, construction grammars) from linguistics are certainly derived constructs, but most researchers would agree that they all reflect real abstractions that humans make. I think the NLP community has good a job of harvesting the substantive aspects (which tend to be fairly conventionalized upon across theories) without overfitting on specific cases. "Alice drove down the street in her car" is easy for people to process, "The horse raced past the barn fell" is not, because it requires a pretty drastic reinterpretation of the structure when you get to the last word.

That said, there is some interesting work on "good-enough" language processing, which suggests that people maintain some fuzziness and don't fully resolve the structure when they don't need to. [1]

[1] [http://csjarchive.cogsci.rpi.edu/proceedings/2009/papers/75/...](http://csjarchive.cogsci.rpi.edu/proceedings/2009/papers/75/paper75.pdf)

<comment><USER24> _but most researchers would agree that they all reflect real abstractions that humans make_

They reflect a particular language in its well-written form. However, humans are extremely robust against syntax errors. I am not a linguist, but I think this speaks in favor of lexicalist approaches: we can be very free in word order, as long as our brain can match up e.g. verbs with their expected arguments.

</comment></comment></comment><comment><USER29> I'm sure it's only a matter of time before someone puts this online in a format easily played with. Looking forward to that

<comment><USER2> It's already available here - [https://github.com/tensorflow/models/tree/master/syntaxnet](https://github.com/tensorflow/models/tree/master/syntaxnet)

        echo 'Bob brought the pizza to Alice.' | syntaxnet/demo.sh

        Input: Bob brought the pizza to Alice .
        Parse:
        brought VBD ROOT
         +-- Bob NNP nsubj
         +-- pizza NN dobj
         |   +-- the DT det
         +-- to IN prep
         |   +-- Alice NNP pobj
         +-- . . punct

<comment><USER29> I mean fully online, where I don't have to download and setup tensorflow.

<comment><USER2> Yes, probably a few days, unless you go through the effort to setup a web server.

For other non-Parsey McParseface dependency parsers and POS taggers that are web accessible, see [http://corenlp.run/](http://corenlp.run/) and [http://nlp.stanford.edu:8080/parser/](http://nlp.stanford.edu:8080/parser/).

<comment><USER30> Does Google's have a better sense of humor than the 3 in this thread? They all fail on:

Time flies like an arrow. Fruit flies like a banana.

<comment><USER2> Really, the mechanism of all these parsers, including SyntaxNet, is the same in that they use statistical training data to set up a neural network. Here's a paper on the Stanford CoreNLP parser, which you can compare with Google's paper: [http://cs.stanford.edu/people/danqi/papers/emnlp2014.pdf](http://cs.stanford.edu/people/danqi/papers/emnlp2014.pdf)

So, really all of the above parsers are weak in that they only output a single best parsing, when in reality sentences can have more than one valid structure, the principal example being the second sentence you've provided. I don't think Google's model has a better sense of humor than the others, no. I anticipate that they all have used relatively similar training data.

However, there is probably a trivial way to get the second sentence to parse as

          Subject --- Verb --- Object
         Noun       Verb   Article  Noun
          |   \       |     |        |
        Fruit flies  like   a      banana .

and that is to provide training data with more occurrences of ...

      > N{Fruit flies} V{like} honey.
      > N{Fruit flies} V{like} sugar water.

than occurrences of

      > A plane V{flies} PREP{like} a bird.

The more sentences using simile that the parser finds, the less likely the neural net is to consider 'like' as a verb. It's also impacted by all of the uses of [flies like].

That's the nature of statistical language tools.

The stock parser debuted here gives the same answer as CoreNLP, by the way.

        flies VBZ ROOT
         +-- Fruit NNP nsubj
         +-- like IN prep
         |   +-- banana NN pobj
         |       +-- a DT det
         +-- . . punct

So much for Parsey McParseface's sense of humor.

</comment></comment></comment></comment></comment></comment><comment><USER31> I'm glad they point out that we need to move on from Penn Treebank when measuring the performance of NLP tools. Most communication doesn't sound like the Penn Treebank, and the decisions that annotators made when labeling Penn Treebank shouldn't constrain us forever.

Too many people mistake "we can't make taggers that are better at tagging Penn Treebank" for "we can't make taggers better", when there are so many ways that taggers could be improved in the real world. I look forward to experimenting with Parsey McParseface.

</comment><comment><USER3> Say, I wanted to use this for English text with a large amount of jargon. Do have to train my own model from scratch or is it possible to retrain Parsey McParseface?

How expensive is it to train a model like Parsey McParseface?

<comment><USER32> Training the model is basically free. Obtaining annotated text for your domain can be quite expensive if you hire a trained linguist.

<comment><USER33> >Training the model is basically free if you have a few GPUs and don't pay electricity

<comment><USER34> More like a thousand or so CPUs and a few hundred GPS.

</comment></comment></comment></comment><comment><USER35> I started working on a parser as a side project that could parse simple sentences, create a knowledge graph, and then you could ask questions based on the graph. I used [http://m.newsinlevels.com](http://m.newsinlevels.com) at level 1 to feed it news articles and then you could ask questions.

It worked pretty well but I lost interest once I realized I would have to feed it tons of words. So could I use this to do something similar?

What programming language would I need to use?

<comment><USER36> Is your work available somewhere? I would love to play with something like it.

<comment><USER35> It was a hack in C# with no unit tests it never got past the conceptual stage.

</comment></comment></comment><comment><USER37> As someone who has published work in the NLP area, I always take claimed results with a grain of salt. With that said, I still will have to read the paper to know the implementation details, although my problem with generic linguistic approaches such as this one seems to be is that it is usually hard to "port" to other languages.

For instance, the way they parse sequences of words may or may not be too specific to the English language. It is somewhat similar to what we call "overfitting" in the data-mining area, and it may invalidate this technique for other languages.

When I worked on this area (up to 2014), I worked mainly in language- independent statistical approaches. As with everything, it has its cons as you can extract information from more languages, but, in general, with less certainties.

But in general, it is good to see that the NLP area is still alive somewhere, as I can't seem to find any NLP jobs where I live! :)

Edit: I've read it in the diagonal, and it is based on a Neural Network, so in theory, if it was trained in other languages, it could return good enough results as well. It is normal for English/American authors to include only english datasets, but I would like to see an application to another language.. This is a very specialized domain of knowledge, so I'm quite limited on my analysis..

<comment><USER32> It's not particularly hard to port nlp to other languages when you use these methods. You are mostly limited by tagged corpora.

Nlp is very much alive and well.

</comment></comment><comment><USER38> According to their paper ([http://arxiv.org/pdf/1603.06042v1.pdf](http://arxiv.org/pdf/1603.06042v1.pdf)), the technique can also be applied to sentence compression. It would be cool if Google publishes that example code/training-data as well.

</comment><comment><USER39> Shouldn't the title be renamed for "The World's Most Accurate Natural Language Parser _For English_ "?

It's impressive how Google's natural language features, since the simpler spell check, degrades when it work with languages different from English.

</comment><comment><USER36> > It is not uncommon for moderate length sentences - say 20 or 30 words in > length - to have hundreds, thousands, or even tens of thousands of possible > syntactic structures.

Does "possible" mean "syntactically valid" here? If so I'd be interested in a citation for it.

Also, I wonder what kind of errors it makes wrt to the classification in [http://nlp.cs.berkeley.edu/pubs/Kummerfeld-Hall-Curran- Klein...](http://nlp.cs.berkeley.edu/pubs/Kummerfeld-Hall-Curran- Klein_2012_Analysis_paper.pdf)

</comment><comment><USER40> I don't see how a linguistic parser can cope with all the ambiguities in human speech or writing. It's more than a problem of semantics, you also have to know things about the world in which we live in order to make sense of which syntactic structure is correct.

e.g. take a sentence like "The cat sat on the rug. It meowed." Did the cat meow, or did the rug meow? You can't determine that by semantics, you have to know that cats meow and rugs don't. So to parse language well, you need to know an awful lot about the real world. Simply training your parser on lots of text and throwing neural nets at the code isn't going to fix this problem.

<comment><USER41> That's exactly why it's using a neural net and yes, a lot of text will fix this problem. The only reason why we know cats meow and rugs don't is by learning about cats and rugs. Throw enough training data at it and the parser will figure out what is meowing.

An interesting example of this you can easily try for yourself is playing with Google's voice to text features--if you say silly things like "the rug meowed" you will have terrible results because no matter how clearly it can hear you its training data tells it that makes no sense.

<comment><USER42> This is actually a rather serious limitation of statistical approaches to language: they work best with utterances that have already been said, or with concepts that are already strongly associated in common speech. Such utterances may make up the bulk of what we say and write, but the remainder isn't gobbledygook. It contains most of the intimacy, poetry, and humor of interpersonal communication, all of which trade on surprise and novelty.

<comment><USER43> That's basically Chomsky's argument with the "colorless green ideas" sentence. If you put words together to form a sentence never seen before, supposedly a statistical model cannot help you. The thing is, a paper later showed that a simple Markov model is actually perfectly able to discriminate this grammatical sentence from an ungrammatical one. Novel and surprising sentences are never completely alien. They use familiar structures of the language, and combinations of words and other building blocks that we have seen before, and this is exploited when we analyze such sentences. Surprise and novelty are actually strongly related to statistics (cf. information theory).

</comment></comment></comment></comment><comment> aaron-santos I'd love to see the failure modes especially relating to garden path sentences. [1]

[1] - [https://en.wikipedia.org/wiki/Garden_path_sentence](https://en.wikipedia.org/wiki/Garden_path_sentence)

<comment><USER2> Statistical models for parsing are almost guaranteed to fail at parsing garden path sentences.

</comment></comment><comment><USER44> Anyone planning (or already busy) training Parsey with one of the alternative Treebanks available from Universal Dependencies [1]? Would love to know your results when you have any :)

I am personally looking for a somewhat reliable NLP parser which can handle Dutch at the moment. Preferably one which can handle POS tagging without hacking it in myself.

[1] [http://universaldependencies.org/](http://universaldependencies.org/)

</comment><comment><USER45> > At Google, we spend a lot of time thinking about how computer systems can > read and understand human language in order to process it in intelligent > ways.

There is 6 links in this sentence in the original text. I get it can help to get more context around it, but I think it's actually making the text harder to "human" parse. It also feels they have hired a cheap SEO consultant to do some backlink integrations.

<comment><USER46> Is this a joke? SEO consultant at Google? Backlinks? It's just a simple way to give it context, and I barely noticed the links.

Can't work out if this is a joke.

</comment></comment><comment><USER47> Parsey McParseface is a great name.

<comment><USER48> Yeah I noticed that :) Is this a Simpsons reference? Can't quite place it.

<comment> UK-AL Havent heard about Boaty McBoatface?

<comment><USER49> I kind of feel that it dates further back?

<comment><USER50> Definitely; I think it's originally from Friends (possibly in a slightly different form; Xey McXerson). Haven't tracked down the exact reference, but it's well over a decade old.

[http://www.duetsblog.com/2016/03/articles/trademarks/namey-m...](http://www.duetsblog.com/2016/03/articles/trademarks/namey- mcnamerson-what-happens-when-you-crowdsource-names/)

</comment></comment></comment></comment></comment><comment><USER51> What would it take to build something like "wit.ai" using SyntaxNet ? i.e. to extract "intent" & related attributes from a sentence e.g.

Input : "How's the weather today"

Output : {"intent":"weather", "day":"Use wit-ai/duckling", "location":"..."}

</comment><comment><USER52> How would you feed a sentence to a neural net? As I understand, the inputs are usually just floating point numbers in a small range, so how is the mapping performed? And what if the sentence is longer than the number of input neurons? Can that even happen, and pose a problem?

<comment><USER53> What wodenokoto said, and also look up "Word Embeddings", word2vec is a popular method.

[https://www.tensorflow.org/versions/r0.7/tutorials/word2vec/...](https://www.tensorflow.org/versions/r0.7/tutorials/word2vec/index.html)

There's a bunch of blogs, tutorials, etc, around word2vec and other methods of generating vectors from a training set of words.

Also, in the tensorflow models codebase where this syntaxnet code lives, there is an another tensorflow-using-method of generating word embeddings with demonstration code called Swivel

[https://github.com/tensorflow/models/tree/master/swivel](https://github.com/tensorflow/models/tree/master/swivel)

</comment></comment><comment><USER54> Cool - I reckon I'm going to try to use it to build a "linter" for natural language requirements specifications. (I'm a bit sad like that).

</comment><comment><USER55> Anyone know a tool that does Natural Language to SQL?

<comment><USER33> [http://quepy.machinalis.com/about.html](http://quepy.machinalis.com/about.html) [http://kueri.me/product/](http://kueri.me/product/)

</comment></comment><comment><USER56> This could lead to a fun WordPress plug-in. All postings must be parsable by this parser.

Surprisingly, this thing is written in C++.

</comment><comment><USER57> one interesting use i can think of is new improved readability scores that can take into account words that are common or uncommon depending on part of speech. (e.g. a text that used "effect" as a noun would be lower-level than one that used "effect" as a verb)

</comment><comment><USER58>
        1. WordNet
        2. ImageNet
        3. SyntaxNet

        ...

        n. SkyNet

</comment><comment><USER59> What are some use cases for this for hobbyists?

</comment><comment><USER60> I'd love to let this loose on the comments section of worldstarhiphop or liveleak and see what it comes up with...

</comment><comment><USER61> Parsey McParseface <3

</comment><comment><USER62> Parsey McParseface? Nice touch Google.

[https://github.com/tensorflow/models/tree/master/syntaxnet/s...](https://github.com/tensorflow/models/tree/master/syntaxnet/syntaxnet/models/parsey_mcparseface)

<comment><USER63> Bill, help me name this parser.

What kind of parser is it?

An _English_ parser.

... :)

You know, it's a brilliant name. It's a person's name, like Siri or Cortana. But if they'd named it Jane or John, the mass media would have a fit. So they side-stepped that by making it sound computer-y, while remaining a proper name. Brilliant.

<comment><USER64> Why is everything English only? :( there are other languages too.

#otherlanguagesmatters

<comment><USER27> Because we didn't win the II World War... :).

(INB4 yes, that's the reference to an old joke about Lufthansa pilot in Frankfurt complaining he's forced to speak in English to the control tower)

Honestly though, English is a quite good starting language due to its simplicity, but I do hope more attention will be expended on making the technology like this trainable on other languages as well.

<comment><USER65> Simplicity!?

</comment></comment></comment></comment></comment><comment><USER66> Did I just read it as Skynet ?

</comment><comment><USER67> Meh.

This kind of parser isn't all that useful anyway. Parts of speech are one of those things people use to talk about language with, but you don't actually use them to understand language.

<comment><USER68> You do subconciously. When you read "Dave punched John" you don't need to think "hmmm, is 'John' the object or the subject?" but if your brain hasn't figured out which is the object and which the subject you won't know who is hitting who.

You don't need to be able to name or define parts of speech but you need to be able to parse them, or you won't understand anything.

<comment><USER27> Yeah, except are you sure that the way you parse a sentence maps 1:1 to what you learned as "parts of speech"? I think an equivalent understanding, one that seems to be more intuitive to the way my own brain works (if I can believe introspection), is that "punched" is to be read by default as "\--punched-->", and "was/is punched by" pattern-matches to <\--punched--". Arrow denotes who's punching whom.

<comment><USER68> That's your brain taking a shortcut because you know that "punched" is a verb, and that a verb in that form is likely to be followed by a subject noun, etc. Even if you never study grammar and couldn't even answer the question "which is the verb in this sentence?" your brain has still learned to recognise the different types of word which is why you understand them.

Disclaimer: I'm not a linguist, but by layman's standards I'm pretty confident.</comment></comment></comment></comment>


Apple of 2019 is Linux of 2000 - <AUTHOR> http://nibblestew.blogspot.com/2019/10/apple-of-2019-is-linux-of-2000.html <comment><USER2> I feel like the author didn't run Linux in the 90s or early 2000s. Until you've had to manually write a monitor's mode lines in your X11 config with the warning it could blow up your monitor to do it incorrectly - you didn't really fight with Linux.

Anyone remember winmodems?

Apple is nothing like this.

I'm sorry the reserved space thing happened. I've not used a Mac daily in at least over a year now. I can't stand the laptop keyboards and windows 10 is (so strangely to me) enough now.

All computer systems are complex now and designers struggle with balancing overwhelming users with detail and not providing enough visibility into details that are important. I know this because I've had to make the calls on what is shown to users vs what we document for support engineers.

Simple is worth striving for and incredibly difficult to get right.

</comment><comment><USER3> This is really not a good argument.

OP ran into a random edge case bug, and, being a power user, decided to do what power users do: tinker and google.

What OP could have done, which you most definitely could not do in 2000 with Linux, is go to an Apple store and have them try and sort it out for you. Ehile they might not succeed, I am willing to bet they could at least get some kind of solution to work.

Also, Apple is nothing like Linux in 2000. My mother can use OS X really easily. My mother food never in a million years use Linux in the year 2000. This is a stupid article written by a bitter user who found a random bug.

<comment><USER4> "This analogy works well for the points made, but does not apply in this other case. Therefore your argument is invalid"

That is not how analogies work. Also the apple store help is explicitly discussed in the article.

</comment></comment><comment><USER5> > Linux 2000: it is very difficult to find a laptop with more than two USB > ports.

AFAIK laptops designed for Linux didn't have that problem. Nor did they have glossy screens, misdesigned keyboards, or corporal existence.</comment>


A new study proves parachutes are useless - <AUTHOR> https://bigthink.com/surprising-science/parachute-study <comment><USER2> In 2000 while visiting the top of the World Trade Center I decided if I worked in an extremely tall building I would want a parachute just in case. Unfortunately it seems no one else entertained such worst case thinking.

</comment><comment><USER3> Great title. In fact, I don't even know if I have to flag it to the editors as misleading or not.

</comment><comment><USER4> tl;dr -- it's satire

> _The authors explain that the whole study was designed to highlight the > limitations of randomized trials and the dangers of not reading past the > first paragraph of a study._

</comment><comment><USER5> Yes, but at what p-value?</comment>


The first man at trial over a "gig economy" got dismantled on cross-examination - <AUTHOR> https://arstechnica.com/tech-policy/2017/09/plaintiff-suing-grubhub-admits-lying-on-application-barely-working-his-shifts/ <comment><USER2> Strange. Class action litigators are usually careful to select a plaintiff with clean hands that a jury will find easy to sympathize with. This guy doesn't feel legit and going to trial over such a small amount (as class status was denied) seems like poor legal strategy.</comment>


Show HN: Stencils.io – Graffletopia alternative for free OmniGraffle stencils - <AUTHOR> http://stencils.io/

<comment><USER2> I'm one of the authors. We hope it's a good start of a collection for OmniGraffle users. Requesting everyone to contribute; submit more stencils and make this place great.

<comment><USER3> This is a great idea! I was shocked to see the stunt pulled by graffletopia given the community goodwill it had. Hopefully you can fill the void left by their dickish move.

FYI, clicking on "Submit a free stencil" takes me to a warning:

> We're sorry. This document is not published.

<comment><USER4> "Dickish move" is a strange way to put that. Do you think their service is free to operate?

If you're an Omnigraffle user, you've already paid for a professional-grade application, and the stencils offered by Graffletopia are inarguably huge time savers.

$24/year, even if you only use one stencil in one project, should be an expense not even worth mentioning (15-30 billable minutes?) for most professionals. Certainly when compared to the time required to create all those assets yourself instead.

Anyhow, to each their own, but I don't understand this kind of angry reaction when a free service becomes very popular and decides to start charging a completely reasonable fee for their offering.

Especially from an entrepreneurial community like HN.

<comment><USER5> The problem is that they are charging money for things that are free to distribute. I think most of the Graffle creators would be upset with this development if they were aware of how disingenuous this was. In fact, they (Graffletopia) are taking advantage of the fact that most of the items are not licensed in any way. For example: [https://www.graffletopia.com/stencils/1197](https://www.graffletopia.com/stencils/1197)

This guy could very easily put his graffle on github (as Viget has done here: [https://github.com/vigetlabs/Twitter-Bootstrap-for- Omnigraff...](https://github.com/vigetlabs/Twitter-Bootstrap-for-Omnigraffle) and License it as MIT, GPL, whatever.

$36 is not a reasonable amount to ask for. Nor is $24, especially in the days of Amazon S3 hosting, or even putting a slick static page up and pointing contributors and downloaders to Github. I respect the marketplace aspect, letting posters set a price on their work and taking something off the top. That's not dissimilar at all to what the Tuts+ and Medialoot marketplaces do. But they don't charge you to download their freebies, and charging someone for a membership to download more than one free item a month is total bullshit.

<comment><AUTHOR> stencils.io is actually a static webpage with direct links to the stencil's original locations.

<comment><USER5> Thumbs up to you for doing it the right way.

</comment></comment></comment></comment></comment></comment><comment><USER6> What are the pro/cons as compared to Graffletopia?

Graffletopia seemed to be perfectly functional to me.

<comment><AUTHOR> Graffletopia used to be completely free. You could download as many stencils as you wished. Now its $24/year; you can download only one free stencil a month. $24 isn't a problem. But charging for community contributed stencils is disgraceful. As someone on Quora ([http://qr.ae/nm6Vf](http://qr.ae/nm6Vf)) said: _Actually, most of the good new stencils on Omnigraffle are no longer free. But the problem is these are not included with the subscription = lame. After downloading so many free stencils over the years, I don 't mind paying a small annual fee. But what's the point if I still have to pay $20-50 for any of the good ones?_

</comment></comment><comment><USER7> Now if only there were a free omnigraffle alternative that can read these stencils

<comment><AUTHOR> There are a few: draw.io, Google drawing, LibreOffice drawing etc. But nothing comes close to omnigraffle.

</comment></comment><comment><USER8> Looks good! I might give it a spin.</comment>


Why an Agile Project Manager is Not a Scrum Master - <AUTHOR> http://www.javacodegeeks.com/2012/06/why-agile-project-manager-is-not-scrum.html <comment><USER2> Scrum, Agile, all overrated buzz words.

This project management evangelists will never understand that methodologies won't make things happen.

Motivation drives people. Not daily scrums, agile, obscure names or roles. Motivation comes through great people, a MVP mindset and none existant project managers, no methodology and just some basic tools like group chat, Github and whiteboards.

<comment><USER3> In an all technical team, sure. But any time you have to interface with normals, tools like Scrum are often the only thing standing between you and "That page that gets their facebook photos and creates a PDF collage out of them will only take you a few hours right?"

<comment><USER2> Scrum as an interface?? You are talking about good specifications/good requirements.

For precise requirements I don't need Scrum or any other esoteric methodology. I just need some people who are able express themselves precisely, clearly and who can put their specifications in some tool and maintain a proper dialogue with devs (i.e. Pivotal). They don't need weird names, roles and daily standups. Scrum is heavy routine which actually wears people out and invented by people who desperately want to upvalue their shitty projectmanagement job.

The key is: just try to keep people out of your project and company who have absolutely no clue about tech/Internet/online.

<comment><USER4> I'd hire you in a flash. You were hit with the clue stick at birth!

<comment><USER5> Let's be honest, the clue stick beatings probabally came later in life.

</comment></comment></comment></comment></comment><comment><USER6> I came here to post a grumpy dismissal of all this tinplated ritual but I'm happy to see that every poster in this thread has beaten me to it.

Good engineers are instantly suspicious of this kind of ceremony because it's almost always an excuse for an insecure project manager to draw enough attention to him/herself to justify the office and salary. Certifications with vaguely mystical Eastern sounding names count -2x.

</comment><comment><USER7> Read the title and tell me that Agile vs. Scrum vs. whatever isn't the new buzzword-laden bureaucracy. All that time you're spending talking about how your going to do things and what names you are going to call each other is an unnecessary distraction from the doing of things.

Leave me and my team out.

<comment><USER8> Agreed. When you're spending as much time dealing with the methodology governing a project as you are actually _doing_ something, something is wrong.

</comment></comment><comment><USER4> An agile project manager is the stupidest concept ever.

A lean process requires no ceremonial positions, bean counters or project trolls.

If you have an agile project manager, welcome to extreme waterfall programming.

Note: this is from experience of watching lots of people fuck up the whole agile thing miserably in the following way: <http://www.halfarsedagilemanifesto.org/>

<comment><USER9> I liked this quote from the article:

> Explain that velocity is not a productivity metric

Velocity isn't always about productivity. Sometimes it's about survival.

I worked in a startup (making enterprise software) right after the first bubble, and the company "practiced" its own version of scrum. We had a small runway and small sales, and the development team insisted on setting its own pace. The fact that we had to have stuff to _sell_ to make money for our paycheques didn't seem to matter to development.

Needless to say, I managed to get the hell out of Dodge before the bank accounts went empty, but a good chunk of the dev team stuck around, plodding for months without pay. It took months before the dev team realized no more money (including backpay) was coming and most of the remaining people finally left. With no major release to sell.

If velocity is required to make development a financially sustainable effort, it can't be dismissed outright.

</comment></comment><comment><USER10> "Agile Project Manager"? That's a no-op. A non sequitur.

"Agile", in my opinion, is really about the following things (tactically, not philosophically, cuz we all know "people over processes" etc.):

* Don't churn requirements * Ship quality software

One could probably argue that the first probably is a necessity of the second (how can you ship anything of quality if the requirements are shifting under your feet?) but let's keep them separate.

"Don't churn requirements" -> keep tasks small. don't change your mind 1000 times a week. keep tactical horizons short (e.g. don't estimate long-range).

"Ship quality software" -> automated tests. code reviews/pair programming. just-in-time lightweight specs. integrated business users, design, ux, qa with development. retrospectives/kaizen.

</comment><comment><USER11> Sounds like most of the commenters here would subscribe to this philosophy: <http://programming-motherfucker.com/>

Myself included.

</comment><comment><USER12> I quit my last job on account of them moving to an _agile process_ , but not before my company had spent thousands of dollars being trained on the process. Snake oil.

</comment><comment><USER13> I was confused how Agile made it back to HN, but after a quick glance the reason is clear.

Enterprise Java has discovered Agile.

<comment><USER4> Enterprise java these days is like Germany after the wall came down I.e. A much better place. agile is still Auschwitz.

<comment><USER14> Sorry, but that analogy is disgusting.

<comment><USER4> Just a FYI, I've been to Auschwitz personally, had two relatives starve to death there as children (much worse than getting gassed if you ask me) under Soviet control in Jan '45 and three survive. TBH the two who are still alive laugh about it and pull their trousers down to Germans regularly in Koln where they live. Getting the trousers up again is quite hard in your late 80's...

Please don't be disgusted for other people who are perfectly capable of handling it. Most of the crap you here is from the 1% of vocal people on the subject. The others think it was a dark time, but shit happens (yes really bad shit but there you go).

(I've also suffered 5 years of agile).

</comment></comment></comment></comment><comment><USER15> My natural barf reflex was triggered by seeing both 'Agile' and 'Scrum master' in the same sentence. Thanks for upvoting guys.

</comment><comment><USER16> The whole agile methodology is so abused nowadays that old waterfall approach looks like kidding in comparison.

</comment><comment><USER17> Yawn!</comment>


Ask HN: Would you donate the equivalent of 1 hour of your work? - <AUTHOR> Let's say my fee is $80/hour. I would donate $80 if some company donates 3 times the amount, in this case, $240.
The total of $320 would be given to charities.
We'd been doing it with a great involvement of employees, around 80%, in my former company. <comment><USER2> I would donate even if my company wouldn't donate. But for me the goal has high priority. Not just abstract charity. Implementation matters (Country wide): \- system for healthcare with centralized health records (Hospitals/Insurance/Pharmacy) - if that would reduce all that US burden around healthcare and insurance \- android apps / systems for local governance (to get it out of stone age) \- bank 2 bank fund transfers - just hate when in US money from one bank to another travel for 2-3 weeks and recipient might get a paper check after all. Didn't ever think US is so far from some even undeveloped countries \- etc...</comment>


China's Market Eclipse - <AUTHOR> http://www.bloomberg.com/gadfly/articles/2016-07-05/china-vanke-s-eclipse-casts-light-on-market-dysfunction <comment><USER2> What the author is describing is similar to what was going on with the Mt Gox exchange a few years ago.

You have a product that should be fungible trading at two exchanges. In one exchange you have a free market and what should be very close to the true price of the product. In our example this would be most of the bitcoin markets and the Hong Kong stock exchange. In another market for some reason the price has become dislodged from its true price, this is usually due to liquidity issues. In our example this is the Mt Gox exchange and the Shenzhen stock exchange.

Now there is what appears to be an obvious arbitrage opportunity but if you look closer, and you really don't need to do any real in-depth analysis to figure this out, you'll see that the market is acting rationally by pricing in the liquidity issues to the shares on one exchange.

I mean, if the shares don't trade and you aren't sure when the regulators will let them trade you essentially have an illiquid asset that you can only really trade by brokering a trade yourself with a counter party outside of the stock exchange. And if you ever wondered what liquidity was worth, well you're going to find out that liquidity can be very expensive.

It will be interesting to see just how far the Chinese regulators will push this as it could have spill over effects onto the American and European stock markets.

So if you've ever thought about what you actually own if you buy a share of a Chinese company on a US stock market, say Alibaba( BABA US Equity), you aren't actually getting a share in Alibaba in the same way you can buy a share in Microsoft.

You are actually getting something called an ADR. see: [http://www.investopedia.com/university/adr/](http://www.investopedia.com/university/adr/)

It's somewhat similar to the brief case full of IOU's that Jim Carey hands over in the movie Dumb and Dumber. Given that you don't own actual shares in the Chinese company but a proxy for them, the Chinese regulators can have a very chilling effect on these shares by shutting down trading of the actual shares in their market as described in the article.

If regulators were to say suspend trading for Alibaba shares in China, its not entirely clear how to go about pricing their respective ADR shares that trade in the US, well at least it isn't to me. Fundamental valuations would still be useful but what discount would you give them, given that the underlying shares are no longer trading on their home exchange, and what exactly would you own in this case?

TL/DR Or put another way liquidity has value, maybe the HFT value proposition was right after all? :)

<comment><USER3> Except in this case it's the Hong Kong stock exchange with the lower share price. Almost all A-H dual-listed companies are relatively overvalued in A-share market and undervalued in H-share market, and it has been the case for the last 10 years.

There's no effectively way to arbitrage this other than waiting for "all future cash flows" to be realised and discounted to present. It's the same share in the same company, with equal voting and distribution rights, but you just can't take one share bought in Hong Kong to Shenzhen to sell.

Among the Chinese investors, it's commonly accepted that A-share has a price premium because its price is likely to go up more in a bullish market. Given the largely speculative nature of the Shanghai/Shenzhen markets (compared to the more "rational" western-style Hong Kong market), having the same voting and distribution rights is far from enough to cause a convergence in share price.

<comment><USER4> _" Almost all A-H dual-listed companies are relatively overvalued in A-share market and undervalued in H-share market, and it has been the case for the last 10 years."_

This may reflect the lower value of non-exportable yuan.

<comment><USER5> China is a huge captive market with few investment options, actually. Your choice is either to figure out how to get your money out into a convertible currency, invest in bubbly real estate, or play in a market with heavy insider trading problems.

</comment></comment></comment></comment><comment><USER6> This sounds spooky, but it sounds like more of what has been talked about for a long time; there is a huge element of fiction in the Chinese economy, and no one appears to be sure how much and where.

</comment><comment><USER7> Not limited to equities, or to China. Regulators worldwide are weighing the systemic risks of now enforcing the transparency that should have always been present. Dodd-Frank is paying off now, finally as capital flees risk for the perceived, possibly very real safety of US markets.

<comment><USER8> Exactly how is Dodd-Frank paying off now? Most investors have seen US markets as the safest for at least a few decades, if not longer. Dodd-Frank may have expedited the decrease in the number of listed companies (through the imposition of large fixed costs which hurt small companies), but I see little evidence that investors feel any 'safer'.

<comment><USER7> Dodd-Frank != Sarbanes-Oxley. You are confused.

<comment><USER8> Both have compliance costs, please give Dodd-Frank a thorough read, and you will see what I mean. There are some especially bad laws in there, such as the conflict mineral reporting requirement,

</comment></comment></comment></comment><comment><USER9> Sheesh, it's 1986 all over again, except that instead of "Japan this" and "Japan that" and "Yet another Japan thing", it's China.

Even Chinese tourists, instead of Japanese ones.

<comment><USER10> Well up until recently, Japan had the number 2 economy in the world. China just recently passed them so now it's all about China.

Due to their population size, there's a lot of potential but their economy is a giant mess. There are many separate huge bubbles forming in China right now, some larger than the U.S's largest economic bubble. Real estate and banking among them. Only, instead of popping organically as happens in every other economic system, the PRC has complete control of every facet so they just don't let it pop. Which causes it to grow even bigger. And that means when it does burst, it's going to come down a lot harder. They're putting off the inevitable. But they should realize that no economy can grow forever. It's never happened in 3,000 years of recorded history.

Japan never really suffered from this (oh they had bubbles, but they popped when they were supposed too) so I expect to see the country back in the number 2 spot when China's house of cards comes tumbling down. It _will_ come tumbling down, that much is certain. The 'when', however, is not so clear. The PRC can do a lot to stave a collapse off, but I don't think it can do much about manufacturing moving out of the country, or people choosing other countries to have their cheap products made (Vietnam, Philippines, Singapore, etc). That's one of the few things they can't stop and have little control over. And China relies on its cheap exports -- It's a massive part of the country's income. If that goes, not even the government can plug the hole. It will cause a cascade effect, allowing all those bubbles to pop. It will be a sight to behold.

Basically, China is different from Japan. In size, scope and because their economy operates differently since it's under the complete control of the government. Japan's economy was more "normal", if there is such a thing.</comment></comment>


Is This the Age of Irresponsibility? - <AUTHOR> http://counsellingresource.com/features/2009/07/30/is-this-the-age-of-irresponsibility/ [...] In the Victorian era, the social climate or zeitgeist was indeed very repressive. If there were a motto that best described the social milieu of the time, it would have been "Don't even think about it!" People were over-controlled and were programmed to be very apprehensive [...]<p>[...] our age is most characterized by its degree of permissiveness, license and privilege. And if there were a motto to best describe the Western cultural milieu, it would be as the once popular commercial advocated: Just, do it!. As a result, [...] modern culture has created a situation in which increasingly more people are not hung-up enough to restrain themselves from simply doing what they want to do and with a fair degree of indifference to the consequences [...] <comment><AUTHOR> _[...] In the Victorian era, the social climate or zeitgeist was indeed very repressive. If there were a motto that best described the social milieu of the time, it would have been "Don't even think about it!" People were over- controlled and were programmed to be very apprehensive [...]_

 _[...] our age is most characterized by its degree of permissiveness, license and privilege. And if there were a motto to best describe the Western cultural milieu, it would be as the once popular commercial advocated: Just, do it!. As a result, [...] modern culture has created a situation in which increasingly more people are not hung-up enough to restrain themselves from simply doing what they want to do and with a fair degree of indifference to the consequences [...]_

</comment><comment><USER2> > The person commented that disturbed characters "seem to infest every area of > life" even including "the ranks of doctors, therapists and public > officials."

The person says that like the idea that "disturbed characters" infest politics is a _new_ thing.</comment>


How to Buy an iPhone at the Worst Possible Time - <AUTHOR> http://designdare.com/how-to-buy-an-iphone-at-the-worst-possible-ti <comment><USER3> Or you could just buy an Android phone supported directly by Google, like the Nexus One or Nexus S, get first dibs on OS upgrades, and not be locked into a carrier contract, not pay extra for tethering, and not pay a subsidy tax every month for two years...

<comment><USER4> Where in the US can I get an unsubsidized Nexus S that won't cost me a couple hundred over the retail price? Can I just go to Best Buy and say I don't want the contract - just the phone? I find it odd that Google sells dev phones but only the ADP2 and the Nexus One. Why not sell the Nexus S directly to devs as well?

Edit: Thanks for the responses about being able to go to BB and get just the phone at the unsubsidized price. This was the answer I was looking for as it wasn't clear to me that this was an option and I was just more confused that Google wasn't selling it as an option for devs in their marketplace.

<comment><USER5> Unless I'm missing something?

 _Introducing the Nexus S. Starting at $199.99. Only at Best Buy®. Also available without a 2-year contract for $529.99_

[http://www.bestbuy.com/site/Google+-+Nexus+S+Mobile+Phone+-+...](http://www.bestbuy.com/site/Google+-+Nexus+S+Mobile+Phone+-+Black+\(T-Mobile\)/1484107.p?id=1218262482328&skuId=1484107&contract_desc=REPLACE)

</comment></comment></comment><comment><USER6> Nice trolling attempt :) Of course anyone that visits HN knows that after buying an Android phone, you just go to xda-developers and get free rom updates from the 4-5 different cooks there. Next phone I'm buying will be determined by the number of people that visit the phone's rom development forum :)

<comment><USER7> Heh, now that you say it, I actually looked at the topic counts on the device forums over at CyanogenMod recently to help me decide which phone to buy. Was considering the HTC Desire HD, but it seemed to have way too few threads, so I went with the Nexus S. Figured that's a safe bet :)

</comment></comment><comment><USER8> The genius of the troll here is that he shows the release cycle of the iPhone, but below he shows the release cycle of every Android manufacturer, thus creating visual clutter which instantly gives a negative impression on the viewer and thus you feel an instant pang of agreement.

<comment><USER9> while I agree with you assessment, isn't the smart phone battle presented as between "iPhone" and "Android" in almost every comparison, including (especially by) Android fans?

<comment><USER10> No, only by John Gruber.

As an Android fan I don't even know if the iPhone still exists. It can't run any software I need and won't work with the infrastructure I have setup, so its existence is meaningless. Except, perhaps, that it will lower the price of nice 3" screens.

Really, my eye is on the Next Cool Android Phone instead, because I can buy one and I won't have to make any compromises.

<comment><USER11> _Really, my eye is on the Next Cool Android Phone_

I think that was FraaJad's point; you probably don't care about the next HTC Android phone but the next Android phone regardless of vendor. While the iPhone gets one new model per year, there are three or four Next Cool Android Phones per year. Thus the original article isn't a troll; it reflects the way Android users actually think.

</comment></comment></comment></comment><comment><USER12> Alternative interpretation: I can buy an Android phone at any time and be sure I'm getting something reasonably current, whereas I have to track iPhone release schedule rumors to avoid paying full price for a year-old model. Trolling is fun!

</comment><comment><USER13> No mention that Apple's free iOS updates will turn your phone into a sluggish brick.

<comment><USER14> At least Apple gives you the option. Many Android phones either can't be upgraded at all, or aren't on any kind of unified release schedule, so you never know when you're going to be able to upgrade.

The only Android phone analogous to the iPhone is the Nexus line directly from Google, and it's future is far from a sure thing.

<comment><USER15> If you aren't happy with your release schedule, you can download a custom ROM for your device that implements the new version of Android. I use CyanogenMod 6.1 (Android 2.2) on my G1, which officially is still on 1.6. It's not very bad for me, a bit sluggish but definitely still usable and workable.

<comment><USER14> I know Android supports OTA updates -- which I really love, btw -- but I'm entirely unfamiliar with the custom ROM process. Does it involve anything like "jailbreaking" your device, or are you able to simply install it.

What I'm wondering is if it's something I could have my mom do. That's kind of my litmus test for whether something is a mass-market solution, or if it's something only geeks will do. If it's not a mass-market solution, I'd argue that the chart designer's point still stands. Jailbreaking or rooting your device to support an upgrade cycle isn't exactly a solution for most people.

<comment><USER15> It does have its issues and is probably not something that you should recommend universally. However, most Moms don't even know when a new version of Android has released unless you tell them, so it's probably not really an issue for them.

The process differs from device to device. Most of the time it will include pretty advanced stuff from the Mom's perspective, things like commandline usage and installation of bootloaders.

That said, it's not very hard for your average Linux user. Just follow the tutorials.

<comment><USER14> I disagree strongly regarding the Android version issue and moms. My mom, cousins, and co-workers (non-programmer) all seem to upgrade their iPhone iOS without issue. I would expect the same from Android.

<comment><USER15> Uh, do they all use jailbroken iPhone versions? A standard Android upgrade is easy and great. Rooting your phone and replacing with a custom ROM isn't at "Mom" skill level, just like jailbreaking an iPhone isn't "mom" skill either.

<comment><USER14> No, they don't run jailbroken iPhones. That's my point. They don't need to in order to get the latest OS, but you often do with Android.

<http://techcrunch.com/2011/01/17/ios-android-breakdown/>

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER16> Who are the 42 people who upvoted this, that's what I'd like to know. This is so obviously trolling that I can't even begin to refute it.

</comment><comment><USER17> This reminds me of an analysis I heard on the radio yesterday.

The synopsis is that it's better to offer people less choices, because given more choices, people are more likely to be unhappy with the one they choose. Apple does a really good job of handling that.

If you were to only single out one manufacturer and line, say Motorola Droids, the chart would look exactly the same. But since there are 5-6 manufacturers offering good Android phones, it seems disappointing that there are new ones coming out right after yours. Really, it's a good thing.

I felt regret at buying the Android phone I did, as the Nexus S came out 3 months later. But you have to buy one sometime!

<comment><USER18> This is also why America has only two large political parties.

<comment><USER19> That's widely considered a side effect of the system:

[http://en.wikipedia.org/wiki/Plurality_voting_system#Effect_...](http://en.wikipedia.org/wiki/Plurality_voting_system#Effect_on_political_parties)

<comment><USER18> That was sarcasm.

As an aside, Duverger's law appears to not hold in Canada, which is FPTP (regrettably IMO) but nevertheless has seen repeated rise of major third parties (CCF/NDP, Reform/Alliance who now de facto hold power).

</comment></comment></comment></comment><comment><USER20> Not trolling, really, but - In America, what is the upside to buying an unlocked Nexus? AFAIK, my plan options will be the same price whether I have a contract or not, so really I'm just costing myself an extra $330 dollars. OK, so I get to "choose" carriers, but again, if I choose not to use T-Mobile, won't I be stuck with edge?

<comment><USER21> Canada here, but - here's one scenario.

One of the local carriers, Wind Mobile, added a plan in December that includes a lot of goodies - unlimited calling in home areas, unlimited text, unlimited data, etc. - for $40/mo.

[http://shop.windmobile.ca/ProductCatalog/VoicePlans/PlanDeta...](http://shop.windmobile.ca/ProductCatalog/VoicePlans/PlanDetails.aspx?id=Holiday+Miracle+Plan%28WINDCA%29)

This is miles ahead of the competition - the catch is that Wind coverage isn't great, but not an issue if you rarely leave the metro area.

At least 3 of my friends have told me they would love to switch, but don't want to pay the cancellation fees for their current providers.

That alone is reason enough for me to avoid multi-year contracts - even if I don't end up ahead financially by switching to better deals, I have almost a philosophical objection to this kind of coercion.

<comment><USER22> To relate back to the original post, the salesman at Wind told me that they have no plans to update any of their phones. Which is why I'm on Mobilicity which offers a Google phone.

<comment><USER18> Which is why you upgrade your phone yourself.

</comment></comment></comment></comment><comment><USER23> Android device support really depends on your carrier. Some carriers are MUCH better than others at releasing updates.

If you're willing to go the unsupported route, most popular phones seem to have unofficial ROMs available. You can definitely get Froyo on your G1, and you might even be able to have Gingerbread.

I think that the trick with Android is to get a phone that seems like it will be pretty popular. Don't get a cool but oddball device (I have a Droid Pro, which fits into that category, and a Droid2, which seems to be pretty well supported by the ROM community).

</comment><comment><USER24> He really needs the WebOS chart where the next phone comes out exclusively on a different carrier.

</comment><comment><USER25> iOS 3 update wasn't free - $5 or $10 IIRC.

<comment><USER26> Only for iPod Touches, not for iPhones.

</comment></comment><comment><USER27> troll...

</comment><comment><USER28> Yeah, but then you have to have an iPhone.

</comment><comment><USER29> Or you could buy an android phone and have it break with 5 months to go on your contract. You can't get it repaired because the warranty is expired, you can't upgrade without paying full price of $500, and you can't leave the contract.

So you learn to live with 1/4 of your "touch"screen being not touchable at all --move your apps to the left side, get an app that lets you answer the phone without swiping, and forget about playing games or ever pressing the search button--and you hope that the new verizon iphone will create a glut in the used market.

<comment><USER30> Is there some reason you can't pay for an out of warranty repair? I don't think products breaking outside the warranty period is Android specific - <USER2> you dunk an iPhone a year after purchase you'll be spending dollars on a repair.

<comment><USER29> Wasn't offered at Verizon, but it could have been an unhelpful salesperson. If not them I can probably find a 3rd party repair. Don't know why I wasn't considering this, thanks for pointing it out.

I wasn't trying to trash android, just annoyed at the whole lock-in/upgrade scheme.

<comment><USER15> They usually want you to pay for insurance so they make these things difficult without such plans. The insurance has been worth it in my experience -- it adds $7-$10/mo, but covers a lot of damage that the warranty excludes, and you don't have these hassles or worries after warranty expires. I agree that it's a greedy, crappy scheme, but that's how the carriers want you to get your phone "repaired".</comment></comment></comment></comment>


Why the iPad Will Change Everything - <AUTHOR> http://www.newsweek.com/id/235565 <comment><USER2> I think this is actually a fairly even-handed look at the potential (both positive and negative) of the iPad. Even if the iPad is simply the spark that kicks off the touch-tablet craze (iphone to nexus one anyone?) - it's still a significant game-changer.

Will it displace desktops and laptops? Not for people who use them to get work done. Anyone writing a book probably won't use an iPad - nor will someone writing software, or doing accounting/etc. It will, however, one day replace the things "most" people use computers for - consuming media, articles, videos, writing short blogs, etc.

At it's heart - it's a consumption device with some capabilities of creating. The content creation capabilities will simply get stronger with time (UX mockup tools, Keynote/presentation tools, sketching applications, etc). As the tablet/touch-interface market expands and more companies get in the game, things will only get better. More tablets will come, more 3rd party hardware addons will come - creators will be enabled.

Do I wish it ran OSX-Full, instead of iPhone OS? Yes, but while I _wish_ that, I also see the fact that using an OS _designed_ for a touch-based system is superior than a normal OS with touch-based interactivity "bolted on".

Anyone doubting how intuitive and natural a touch-based system is to work with should try handing an iPhone to a two year old. They can figure it out much more quickly than a mouse and keyboard. My poor daughter (my guinea pig) can pick up my iphone, switch screens, find and start her game(s) in seconds. She's still flummoxed by my laptop/touchpad and the little mouse icon on the screen.

<comment><USER3> "Even if the iPad is simply the spark that kicks off the touch-tablet craze (iphone to nexus one anyone?) - it's still a significant game-changer."

Great point. I disagree about the iPad not being good for writing books. Won't Incase or Mophie come up with a $100 keyboard/case that turns this into a laptop? I think the app ecosystem will be greatly augmented by a hardware eco- system. I'm bullish on the platform.

<comment><USER2> Later in that comment I pointed that out - content creators will be enabled by third-party add ons, and other companies building "competitors" which further enable creation. I don't think that there is anything _blocking_ creation on it - just that that's not the short-term aim.

<comment><USER4> Nothing is blocking creation on the iPad except maybe Apple as the arbitrary gatekeeper of the App Store.

<comment><USER2> And in that; nothing is blocking apps aimed at creators from the app store. Day one will see a new version of Pages and Keynote and at least one artist/painter/sketching application.

The applications will make or break the device ultimately.

</comment></comment></comment></comment></comment><comment><USER5> Apple does have a history of entering markets ripe for explosion and then facilitating that explosion. It seems a bit different this time. In the past it seems the Apple was answering consumers' demands rather than publishers' demands. Portable music sold one at a time came to the delight of consumers and the detriment of the music industry. The iPhone's internet connected apps encourage using more and more bandwidth is to the detriment of cell phone carriers. Consumers outside of tech geeks and Apple fans seem to care less about tablets. It is the publishers who pine for a new platform to sell the wares on that seem to be the most vocal about how revolutionary the iPad is. Even friends that have pre-ordered it do not exactly know what they will be doing with it yet, only that the iPad is somehow revolutionary. Time will tell if the iPad will be a success but I personally hope a more open platform will win out.

<comment><USER6> There's "no demand" for a simple, easy to use tablet-sized internet/app device that doesn't crash, just like there was "no demand" for a simple, easy to use, pocket-sized music device 10 years ago.

There's plenty of demand for it. Most people just don't know they want one yet. As I argued in my article after the iPad launch, people who don't actually want a computer will be buying this in droves (instead of wasting money on ugly, powerless netbooks - which are priced in the same region!).

<comment><USER5> There was demand for a pocket-sized music device over 20 years ago. The Sony Walkman was a success which was followed by the proprietary Sony MiniDisc format which flopped. Then iRiver and Phillips had crappy mp3 players. The iPod was the first portable music player since the Walkman that was a success.

I am not prescient, the iPad may be a huge success, but there currently not a large market for tablets. There were large markets for portable music players and smart phones when Apple entered those fields.

</comment></comment></comment><comment><USER7> Does anyone else find it weird and a little sad that Woz has to pre-order one (or three) like a normal person?

Edit: To clarify, I mean sad that they don't just send him some. Since, you know, he helped start the company and everything.

<comment><USER8> Pretty sure they'd be happy to send him ten if he wanted them. Sounds like he's in it for the fun of the launch.

</comment></comment><comment><USER9> _I want one. ... It has a nice 9.7-inch screen, weighs only one and a half pounds, and can play movies for 10 hours on a single battery charge. Right away I could see how I would use it. I'd keep it in the living room to check e-mail and browse the Web. I'd take it to the kitchen and read The New York Times while I eat breakfast. I'd bring it with me on a plane to watch movies and read books._

I also pre-ordered, but I will note that I do all of the above with my rather old tc1100 tablet. (A p4 just barely squeaks by with video.) Even now, I'm posting to HN from bed using the phenomenal Windows XP Tablet handwriting recognition.

The iPad will change everything not because it's new. It will change everything because of its execution.

<comment><USER2> Precisely. The point people need to realize is that having a well-executed, easy-to-use touch-based interface _is the killer feature_ of the iPad. That "shiny user interface" _is the point_ of it.

</comment></comment><comment><USER10> _as long as you're willing to do it the Steve Jobs way._

<comment><USER11> Enough are - look at Apple's market cap.

</comment></comment><comment><USER12> Excuse my ignorance and non-appleness for this simple, non-technical, potentially stupid question:

While my 13" MacBook Pro rests on my knees, I can smoke with one hand, hold my coffee in the other hand, and read this article at the same time. I started it in the kitchen and finished in the bedroom.

How is iPad going to make my life better? Will I be able to do 3 things at a time with it? I suspect one hand will always be busy holding the thing. It will get tired, let alone that I won't be able to smoke and have my coffee and read an article (and always have the UNIX shell at hand) at the same time.

This seems like an iPad killer in the bad sense to me. I'm sorry.

<comment><USER13> Maybe you should quit smoking.

Seriously though, the iPad isn't made for the likes of us on Hacker News (though I still kinda want one). It's for people like my father who don't really give a damn what a computer is as long as it's easy to use.

<comment><USER12> He probably should care that his arm is going to get tired holding his iPad for more than 15-20 minutes. Seriously, I'm failing to picture any long-time user of this device being really, really happy with it.

<comment><USER9> I'm jazzed about my Windows slate at 3 pounds, and yes, I can browse and post on HN all day with it, even from bed. You don't have to hold it up to use it in any context you can use a hardcover book.

My iPad will weigh half as much and won't require a stylus.

</comment></comment></comment></comment><comment><USER8> I think that the iPad and similar devices down the road really could change everything as far as print and multimedia consumption is concerned. But it's going to take a long time (years, perhaps) until these publishers are producing content that's compelling enough that people will actually buy it as opposed to finding similar stuff for free on the web.

My hunch is that these first attempts by magazines will look cool but feel _really_ gimmicky, to the point that they're a bit of a joke. Think cheesy quizzes and games and video content that's been available on the web for weeks (remember, these digital magazines will take time to produce). I wrote about this topic a bit more back in January if you're interested: [http://techcrunch.com/2010/01/26/apple-tablet-book- revolutio...](http://techcrunch.com/2010/01/26/apple-tablet-book-revolution/)

<comment><USER14> And I'm sure the magazines will be littered with canvas tag-based banner ads that would be much more intrusive than a full page print ad. No ad-blocker plugins on an iPad.

</comment></comment><comment><USER15> (Ignoring the article itself here) The top picture reminded me of this ok/cancel comic strip: <http://okcancel.com/comic/3.html>

I know it's not the environment where you'd normally use the device, but still - how are you even supposed to hold it?

</comment><comment><USER16> The killer app on the iPad is DRM. Without it, there would be no for-purchase TV shows, movies, books, or magazines as it would be impossible to license the content.

The closed and managed system is not an option for Apple, it is essential to the success of the product.

<comment><USER17> One reason I would avoid those products.

It may be elegant but I would rather a more open device even at the cost of some "Ease of use."

</comment></comment><comment><USER18> Just three words needed: One million sold. [http://ipadtest.wordpress.com/2010/03/27/one-million- ipads-s...](http://ipadtest.wordpress.com/2010/03/27/one-million-ipads-sold/)

</comment><comment><USER19> This article should be renamed:

"Why we wish the iPad will change everything for us publishers".

<comment><USER20> Right. Much of what the article said the iPad would "transform", I already do on my PC and lots of others do on their laptops or tablets -- things like watching TV, for example. The only thing the iPad does is puts it in a small, portable shell with a slick interface that channels revenue to Apple.

There's a market for that. But I don't see it as being "transformational" in any significant way.

<comment><USER9> _There's a market for that. But I don't see it as being "transformational" in any significant way._

The combination of features, form factor, and App Store, will put all of the above within reach with practically _no requirement_ for being tech savvy. That's "transformational". I can do everything the iPad will do with my 4 year old tablet. But only the truly tech savvy would tinker with it as much as I have to get it to that point, and it's still not as slick as the iPad appears to be. I suspect my 71 year old Dad will be able to pull his out of the box, turn it on, and have an even slicker experience.

Lots of non tech-savvy folks with money will want one.</comment></comment></comment>


Basics of Compiler Design - <AUTHOR> http://www.diku.dk/hjemmesider/ansatte/torbenm/Basics/ <comment><AUTHOR> As I've been working on my pet compiler project recently, I began to collect some compiler-related online resources, this book is one of the hidden gems. The author (Torben Mogensen) has been so generous to give it out for free, that's why I think it needs more exposure.

It's heavy on the compiler theory part, so I mainly use it as a reference book. It also has a "companion" book (my take) -- Programming Language Concepts for Software Developers [1] by Peter Sestoft. The PLCSD book, also free, is more on hand-on projects, where you can implement a Micro-C compiler.

[1] <http://www.itu.dk/courses/BPRD/E2010/plcsd-0_50.pdf>

<comment><USER2> Perfect. The two together act to provide theory + practice (especially cool that PLCfSD uses F# :-)

What other resources have you found, and how do they compare? Have you read the Dragon Book? I hear it isn't actually all that wonderful in reality, though there seems to be differing opinions on that.

I have both a pet compiler project which has never really got off the ground and am attempting to contribute (albeit slowly!) to the go programming language so have a special interest :)

<comment><AUTHOR> The intriguing part of PLCSD to me is that it uses F#, which is the language I use as the tool for my project, :)

I took compiler course during my university days, so the theory part is not a big issue to me. That's why this free book comes so handy as a reference book. I'm aware of the dragon book, tiger book, as well as, the whale book and had a quick read in bookstore. They all are good books, but I don't find my need to keep them on my bookshelf. So I don't think I am in the position to comment on them. YMMV.

<comment><USER3> I hadn't heard of a whale book -- that turns out to mean Muchnick's.

</comment></comment></comment></comment><comment><USER4> Nifty, I actually worked a TA under Torben Mogensen recently, teaching out of this book. It's a really succinct description of the fundamental theory, although a little light on the practice.

<comment><AUTHOR> Out of curiosity, what projects the students have to implement for the course?

<comment><USER4> They are given a functioning compiler (written in Standard ML) for a small subset of a relatively simple artificial language (a new one every year), and they have to extend the compiler to support the full language. This involves changing every level (lexing, parsing, type-checking and code generation). For the final exam, they have to add a new feature to the language.</comment></comment></comment>


Show HN: BundleCop: Keep an eye on your webapp's size - <AUTHOR> https://bundlecop.com/ <comment><AUTHOR> Hi! I am using this to keep an eye on which commits cause my webpack chunks to increase (in particular the initial one), and it was hoping others might find it useful too.

(Fair Warning: Some stuff listed on the page (like the Github pull request integration) is not enabled yet.)

What I think would be really cool is if it had something like webpack-bundle- analzyer built in, so you could just browse the bundle at any point in time, and see the diff with the previous version.

Since it's really just storing and comparing file sizes, there might be other use cases that don't involve webpack/JS at all.</comment>


Fun with your bank's secret questions & answers  - <AUTHOR> http://tongodeon.livejournal.com/890323.html

<comment><USER2> This is really hilarious, and I think it qualifies as system hacking.

<comment><USER3> It's also somewhat cruel to whoever has to read your "clever" message.

<comment><USER4> Cruel? It might make a callcenter employee smile. It's borderline humanitarian.

</comment></comment></comment><comment><USER5> What happens if

Security Question: Can you repeat the question?

Answer: Can you repeat the question?

</comment><comment><USER6> Yeah this is quite clever and funny. Plus Zardoz?!? Wow, nice crazy old movie reference, that's awesome.

</comment><comment><USER7> This is great. Reminds me of that guy who kept signing his credit card statements with increasingly random signatures.

<http://news.ycombinator.com/item?id=1311710>

</comment><comment><USER8> all my emails from etrade start with:

      Hey, you big stud!

</comment><comment><USER9> A simple "speech injection attack." The bank obviously needs to sanitize the input before serving it up.

</comment><comment><USER10> Ah... I'd really like someone on the other end to say: "You can change your balance yourself, just use my login - the staff password is sexy_stud."

</comment><comment><USER11> [http://www.schneier.com/blog/archives/2010/04/fun_with_secre...](http://www.schneier.com/blog/archives/2010/04/fun_with_secret.html)

<comment><USER12> I like this one:

Q: If I said you had a beautiful body, would you hold it against me?

A: Sorry, you're not my type.

</comment></comment><comment><USER13> This is excellent! I wish more systems would let you configure the secret question instead of giving a set of predefined questions... (eg. mother's maiden name, first school, first friend, first town, etc), and most of those questions can be found out about a person with some small talk. For places like that my solution has been to give scrambled answers (eg. city of birth, the answer 'colombo' becomes 'obmoloc'. Also this isn't my scrambling method or answer, just an example).

<comment><USER14> Another simple but effective hack: have a standard untrue answer for your mother's maiden name, first school, etc. Only you will know them. Your mother's maiden name is McLuvin.

<comment><USER15> I usually provide an untrue date of birth to places where it doesn't matter. The only places that really need to know are the places that also need your SSN. Everybody else just likes having sensitive info lying around.

</comment></comment></comment><comment><USER16> I answer every single security question with a nonsensical answer.

CSRs get really confused.

</comment><comment><USER17> I'm changing my bank to this one immediately.</comment>


Ask HN: How do sites with little unique content get SEO clout? - <AUTHOR>

How do you get search engines to notice you when you are just a portal to other content? Think a service like padmapper.com or hipmunk? <comment><USER2> Start a blog. Add unique content.

Ask nicely from other sites like yours. 'Resource' or 'Links' pages are your friend.

Ask your friends to link from their blogs. If your friends are tech folks, they probably have more than one blog or site they can add your site too.

If you post your site I'll give you some specific strategies and examples.

<comment><AUTHOR> whitewatermap.com. So far its extremely regionally specific but that will change in the near future. Any recommendations for content options that don't diminish the effect of the full screen map?

<comment><USER2> Interesting. I would recommend adding a link to the blog in the box w/ the text.

As far as link opportunities, this seems to me like something that could go in: \- lists of google maps mashups \- lists of cartographic resources \- lists of outdoors resources \- On the resources page of every rafting club and outdoors club in the country - especially clubs at colleges.

Is the whitewater data you're laying over the Google Map yours, or does it come from somewhere else?

<comment><AUTHOR> The data comes from public KML files. Technically AmericanWhitewater.org publishes them.

</comment></comment></comment></comment><comment><USER3> Create original content. Build links. Dominate the social scene.

See <http://www.seomoz.org/blog/how-i-would-do-seo-for-hipmunkcom> for Tom Critchlow's take on it. (Did this post inspire your question? Seems coincidental.)

<comment><AUTHOR> I actually had never seen that link but its kinda useful. I guess I was looking for something other than "write a blog" and the targeted pages were new. Thanks.

</comment></comment><comment><USER4> Be awesome and get people to talk about you. If you're solving a real problem for people then people will link to you and share content. I run a niche job board and I get a lot of link juice from our customers who love our service and have blogged linking us.

</comment><comment><USER5> Ummm... perhaps your concept is wrong. Search engines can be thought of as a "portal to other content" and may, rightly, view your aggregation of links as noise. Perhaps you should focus on unique, useful, not-available-elsewhere content.

</comment><comment><USER6> The only way to get SEO these days is great inbound links or great unique content. If you don't have really great inbound content, then you better have a ton of really high profile sites linking to you.</comment>


Google warns 20,000 websites they could be infected with malware  - <AUTHOR> http://www.networkworld.com/news/2012/041912-google-warns-20000-websites-they-258454.html

<comment><USER2> >"Specifically, we think that JavaScript has been injected into your site by a third party and may be used to redirect users to malicious sites"

Funny, there was just a discussion here yesterday asking "Why would anybody use NoScript?"

<comment><USER3> I'm pretty sure anyone savvy enough to install NoScript would notice if he/she was redirected to a malicious site.

<comment><USER4> Nothing says the current user of the machine is the tech-savvy installer. My mother's comp has no-script on it, but she is in no way strong technologically.

A "mom-level" user may well enable scripts when prompted by an artful redirect like say:

1\. A clone of your bank website (if you were headed there)

2\. A page-not-found or server error page that looked normal..

3\. the old "virus alert" scam</comment></comment></comment>


The Descent to C - <AUTHOR> http://www.chiark.greenend.org.uk/~sgtatham/cdescent/?HN_20140803

<comment><USER2> [https://news.ycombinator.com/item?id=7134798](https://news.ycombinator.com/item?id=7134798)

[https://news.ycombinator.com/item?id=8127499](https://news.ycombinator.com/item?id=8127499)</comment>