Rob Pike on regular expressions in lexing and parsing - <AUTHOR> http://commandcenter.blogspot.com/2011/08/regular-expressions-in-lexing-and.html <comment><USER2> Regular expression systems are designed for a very wide variety of text processing tasks. Lexing and parsing systems are optimized for a much more limited range of tasks--lexing and parsing. When you are writing software that is intended as a production system, not a one-off quickie, you are almost always better off using a more specialized system optimized for your specific task, if one is available. The unoptimized, general purpose system would be Plan B. I think that was Pike's point.

Unfortunately, it will probably be misinterpreted as a general criticism of regular expression systems. There are so many cases where my knowledge of a powerful, general-purpose system has saved me from having to learn a new software library for each random task that I intend to keep using such things as regular expressions, scripting languages, unix command line pipelines, Excel, etc. Writing my own code or learning to use a new, specialized system is Plan B for simple, non-production tasks.

Only when I can amortize the cost (time/effort) over a lot of use (as in a production system), or when the advantage of the specialized system is noticeable to the user (including me), does the specialized system become my Plan A.

General purpose tools, special purpose tools, and custom coding all have different costs and benefits, and there are circumstances under which each one is the best choice.

<comment><USER3> One thing I've found is that using a bit of a subset of regular expressions seems to work very nicely for doing many lexing tasks. Things like using ^[a-zA-Z][a-zA-Z0-9]+ (or whatever shorthand your RE system has for those classes) can make a lexer nicer to work with. Now I wouldn't be trying to do lots with most of the other features in them for a lexer but it ends up a compact but simple to understand way of describing the input the lexer is looking for to pull tokens out.

</comment></comment><comment><USER4> I've found Ragel ( <http://www.complang.org/ragel/> ) to be a good compromise: it's less error-prone and easier to maintain a Ragel grammar than a handwritten lexer, but Ragel lets you use regular expressions for all the little places near the leaves of a grammar where it's easy to represent token rules as regular expressions. In contrast to most regex APIs, it does the state machine compilation at build time rather than runtime, and the generated code can be quite fast (although you have to make a speed-vs-code-size tradeoff).

</comment><comment><USER5> _Some people, when confronted with a problem, think "I know, I'll use regular expressions." Now they have two problems._

~jwz

<comment><USER6> I thought you were just having a cheap shot, but then I read the story, and that's exactly his sentiment.

</comment></comment><comment><USER7> Interesting that he claims that it's difficult to change a regular expression to cope with new requirements but easy to change a hand-coded loop.

In my experience, the opposite is true (up to the point when your grammar becomes non-regular).

<comment><USER8> Many regular expression implementations don't deal properly with unicode character classes and collation. So adding something like unicode identifiers to go may be impossible or impractical with regexs whereas with a loop you can rely on more standard collation support.

<comment><USER9> Some libs such as PCRE can match by character properties, for example \pL would match any unicode letter.

Doing that in a loop seems to be much more difficult. And unless you work on utf-32 data, you have to handle variable-size encoding of characters.

</comment></comment></comment><comment><USER10> I'm a bit confused, aren't lexers implemented using similar techniques as regular expressions? (i.e: NFA to DFA conversion and DFA minimization).

Or maybe I'm also confusing lexers with lexer generators.

<comment><USER11> The term "regex" usually means not "regular expressions" in the formal languages sense but rather the trickier operational notion that makes use of backtracking.

Look at [http://cstheory.stackexchange.com/questions/448/regular- expr...](http://cstheory.stackexchange.com/questions/448/regular-expressions- arent)

</comment></comment><comment><USER12> Amen, stop putting regex everywhere, they are way too error-prone and difficult to read!

Use parsers combinators where you can. There you can put sub-expressions in variables, have meaningful identifiers in your grammar, explorable and debuggable parsing etc.

</comment><comment><USER13> Does anyone else have a little trouble with the following sentence: "A regular expression library is a big thing. Using one to parse identifiers is like using a Ferrari to go to the store for milk." I for one would rather read that as "A regular expression library is freaking awesome!" But I think the author meant to say it was overkill! ;)

</comment><comment><USER9> > And when we want to adjust our lexer to admit other haracter types, such as > Unicode identifiers

Actually regex libraries such as PCRE have a good unicode support and are better than me when it comes to do things like matching character properties (letters, uppercase letters, numbers, punctuation, etc).

</comment><comment><USER8> I understand this point on lexers and parsers and i think it makes a lot of sense. And regular expressions definitely have a sweet spot in short off the cuff applications (sed, vim, grep). But there's a large amount of potential use cases in between. I wouldn't want to write a lexer/parser to deal with validating random user supplied data in a web application (phone numbers, email addresses).

So i guess the point is that the whole point of a parser/lexer is to look at and validate text so you shouldn't 'outsource' it. but maybe in applications where minor parsing/text validation tasks are more peripheral then regexs are more appropriate?

<comment><USER14> To be fair, I wouldn't want to write a regex to validate an email address either. <http://www.ex-parrot.com/pdw/Mail-RFC822-Address.html>

</comment></comment><comment><USER15> Maybe I'm thinking of something different, but I've never considered regular expression to be hard to write or hard to write well, and considering that they may be compiled to machine code (either AOT or JIT), they're not _necessarily_ inefficient and a good implementation is likely to handle unicode better than your average programmer. So, while I agree that they aren't a panacea, they certainly are a powerful and useful tool, and for simple pattern matching and/or extraction, they are often easier to visually verify than the hand-written equivalent in my experience.

<comment><USER16> Have you read Friedl's book? I didn't think they were hard to write or hard to write well until I did, and then I regretted my past life of sin.

<comment><USER17> Then again reading said book made me believe they're fairly easy to write well. You need to keep in mind what a quantifier really does ("this will gobble up the whole string and then yield bits until the pattern matches"), but in the end I find it not fundamentally any more taxing than a having in my head a rough idea of the behaviour of a few nested loops or recursions.

<comment><USER16> I feel like it's more error-prone. This regexp lexer took me several minutes to get right, and I'm still not totally sure it's bug-free:

        >>> replace = lambda text, env: ''.join(env[item[1:]] if item.startswith('$') else item[1:] if item.startswith('\\') else item for item in re.findall(r'[^\\$]+|\\.|\$\w+|\\$', text))
        >>> print replace(r'This $line has \stuff \\in it that costs \$50 and some $variables.', {'line': 'LINE', 'variables': 'apples'})
        This LINE has stuff \in it that costs $50 and some apples.

If I were to write out an explicit loop over the characters of the string, I would be a lot more sure that I wasn't accidentally dropping characters due to an inadvertent failure to make the regexp exhaustive (I originally forgot the \\\$ case! Although that reduces to the empty string anyway) and I wouldn't have to forget and rediscover which lexical category each token belonged to.

And, although it's not present in this case or in all regexp engines, it's a lot easier to accidentally write an exponential-time algorithm in a regexp than in a nested loop. And my experience has been that it's harder to debug it, too.

<comment> St-Clock I agree and disagree:

"If I were to write out an explicit loop over the characters of the string, I would be a lot more sure that I wasn't accidentally dropping characters due to an inadvertent failure to make the regexp exhaustive"

This is why regex comments exist. For any non-trivial regex (more than a two or three characters), you should break down and document your regex. Otherwise, it's worse than a 1000 character-long perl one-liner.

"it's a lot easier to accidentally write an exponential-time algorithm in a regexp than in a nested loop"

So true. I did not realize it was possible until I made that mistake. Debugging these cases is extremely difficult. For two strings that look similar, the same regex can go crazy on one. But this happened to me only once in the past three years (time when I started to heavily rely on regular expressions for a project).

<comment><USER16> > This is why regex comments exist.

Regex comments don't help much with inadvertently writing a non-exhaustive regex (i.e. one for which some possible input could fail to match), or a few other kinds of regexp bugs. Or, how would you write the regexp in the above code with comments so that it would be obvious if you left out the \\\$ case?

</comment></comment></comment></comment></comment></comment><comment><USER18> Bonus: Pike explains a regexp matcher: [http://www.cs.princeton.edu/courses/archive/spr09/cos333/bea...](http://www.cs.princeton.edu/courses/archive/spr09/cos333/beautiful.html)

</comment><comment><USER19> You haven't lived until you've written a hand-rolled lexer and recursive descent parser.

<comment><USER20> ... or at least, you haven't done second-year CS. Pick one.

<comment><USER19> Oh, that I could have been so lucky! I was stuck using lex and yacc. I had to work on an actual production parser before I saw the light.

</comment></comment></comment><comment><USER9> > They also result in much faster, safer, and compact implementations.

If you are using a scripting language, regular expressions are often faster than a hand-written parser because the regular expression library itself will work at a lowest-level.

</comment><comment><USER21> I don't understand: "Standard lexing and parsing techniques are so easy to write so general, and so adaptable there's no reason to use regular expressions".

Isn't using the lex family of generators pretty 'standard' in terms of writing lexers? Aren't they generally considered fast and in most cases better than hand-written code?

Aren't bad regular expressions just a consequence of someone not properly learning regular expressions? Couldn't the same argument be applied to every programming language?

Also: <http://swtch.com/~rsc/regexp/regexp1.html>

</comment><comment><USER22> Ten years ago I wrote a lexer myself for my own language, Q, because I didn't know about regular expressions.

It was actually quite powerful. But I literally just implemented a state machine. To be honest, I think there were a couple things it did that a regular expression wouldn't be able to do.

However, for the most part, the "efficiency" argument is off base. You can COMPILE regular expressions into tight code.

<comment><USER23> You don't have to implement a state machine yourself. You can use a lexical parser generator such as flex.

<comment><USER24> The lex program and its descendants are generally regarded poorly by Ken and Rob and others from their tribe of Unix systems programmers. The blog post suggests why. It is often faster and easier to write the lexer in C by hand. But they do tend to put great value on yacc. Once you wrap your head around LALR shift-reduce parsing, yacc is expressive in ways that are hard to replicate with a hand-written parser.

</comment></comment></comment><comment><USER25> Pike's argument is relevant for compiled system level languages (C, C++, Java, etc).

The regular expression libraries used in scripting languages today (Perl's and PCRE) are optimized C code. A lexer in interpreted code is hardly going to beat them in speed.

Note that a large part of the reason to use scripting languages is the development speed. Regexps FTW, etc.

Edit: I should add -- if I have a problem where I need to parse something complex like a "real" programming language, I go to the LALR libs of course. The right tool for the job. (On second thought -- this is probably what Pike talks about; he doesn't go around solving simple problems, like I do. But no complaints, I've got a job interview tomorrow... :-) )

<comment><USER16> He has several arguments; performance is only one of them.

<comment><USER25> >>He has several arguments; performance is only one of them.

First -- I noted that I limited my comment to scripting languages.

I did discuss most of the relevant arguments in the article, so I really don't see what your point is?

It doesn't matter for the scripting languages that a regexp lib is large (it is linked in and used anyway), so I ignored that. I also ignored the Unicode point, since it is generally supported in scripting languages' regexp engines.

I did touch on speed and development speed. Shorter code is also generally easier to understand; a simple regexp can often replace 10-20 lines.

In my edit, 10++ hours before your comment, I discussed when full grammars where a better alternative -- so I handled Pike's accusation of treating Regexps as a "panacea for all text processing". (I have used regexps as lexers for grammars. But only for parsers of less than a few hundred lines.)

(I am not going to touch Pike's argument if I and others _really_ grokk regular expressions, because I thought I did understand them until I read "Mastering Regular Expressions". Maybe there are some more satori insights waiting? I do consider myself a little bit familiar with them from automata theory, implementing state engines and usage [Edit: and my considered opinion is that re:s are often a good solution for scripting languages.])</comment></comment></comment>


Facebook-Microsoft Alliance to challenge Google Search - <AUTHOR> http://www.theprovince.com/technology/Microsoft+links+Bing+search+engine+Facebook/3671685/story.html?cid=megadrop_story

<comment><AUTHOR> "Starting Wednesday, Bing will take data posted on Facebook -- such as users' "likes" or preferences -- and use that information to provide more relevant search results."

This is a step ahead of Google's Social Circle, which currently pulls data form buzz and Twitter.

</comment><comment><USER2> Does this mean we're going to see a FB + MS pairup vs Twitter + Google? Is this the new battle over the long haul?

Seems like the cultures somewhat line up... FB is a super hacker-centric company with world domination ambitions, and Twitter seems to be more open-ey what with its main product being pretty utilitarian until recently... not to mention the whole Ev connection.

Granted, I'm sort of grasping at straws here, but would love to hear what HNers think.

<comment><USER3> looking at this decade alone, google seems to have far, far more 'world domination' ambitions than MSFT or Facebook. it's not like they release open source stuff with the primary intent of doing the public good.

</comment></comment><comment><USER4> Does pulling information from facebook likes really lead to better search results? How is this measured objectively?

</comment><comment><USER5> In the long run this seems the obvious direction search will head. The facebook "like", or a twitter link is the new hyperlink. This is just a new way to use crowdsourcing to judge the quality of a link or domain. Backlink, "like", its all the same.

</comment><comment><USER6> I'm curious: how many of you would be swayed to use Bing over Google because of this?

<comment><USER7> Not that I was terribly likely to use Bing before, but this actually makes me want to use it less.</comment></comment>


Show HN: Interviews can be stressful – give take-home projects instead - <AUTHOR> https://www.remoteinterview.io/takehome <comment><USER2> This is taking interviewing and hiring in the wrong direction. At my company, we have a relatively short interview process, followed by a 3 month full salary probation period.

The interview is used to weed out "red flaggers" and identify high-level strengths and weaknesses. It is completely non-technical. We assume that the person applying for the job has the technical requirements. (We also make judicious use of references, something people seem to have forgotten about. Quick phone call to previous employers can save you tons of money!)

The 3 month probation period is the real "test" and the applicant will be put on a small project with another employee mentoring. If someone lacks technical ability, it will be evident in the first few days and we can move on to the next applicant. If the person doesn't like the work environment, they can walk away no questions asked (maybe the commute is too long or they don't like the office space, etc. etc.).

This is cheaper than having directors and executives try to plan around all sorts of interview stages. It is much less stressful, as the applicant, once into the probation period, is treated like a regular employee and has much more opportunity to "prove their worth". Stop adding more shit to the interview! Don't make it longer, do the opposite!

<comment><USER3> Why would I ever accept a job at a company who treated the first 3 months of my employment as an extended interview, after which I could be released without penalty?

When a software accepts a full-time position, they are taking a _mammoth_ pay cut from what they could make 1099 contracting. They do that because the full- time position comes with a commitment to continued employment, so even though they could be making 2x-3x more per day consulting, they don't have to hustle to keep their utilization up.

The "3 month probation" thing is essentially a temp-to-perm contacting position. Do you pay people 2x-3x more during that period?

<comment><USER4> > _The "3 month probation" thing is essentially a temp-to-perm contacting > position. Do you pay people 2x-3x more during that period?_

What about paying people the 2x-3x more in the event you decide not to keep the hire? Then the mechanism becomes a 3-month option to make the job either full time or a fairly compensated contract.

<comment><USER5> > What about paying people the 2x-3x more in the event you decide not to keep > the hire?

This could be pretty easily abused, since it doesn't sound hard to get through the door if they are assuming you're qualified technically. So all you have to do is know just enough to fake it a little bit, then coast for as long as you can to get a big payout at the end.

<comment><USER3> If you're running easy-in, fast-fire, then you should pay 2x-3x _continuously_ during the trial period, just like you would to a freelance consultant doing the same work.

But if you're running a real hiring process with accuracy as a goal, as you probably should be, then the 2x-3x payout at the end of the trial program at least fixes some of the incentive problems, and, if you're running that program in good faith, you're doing it because you expect most people you extend offers to will pan out --- so you'll rarely pay that bonus.

</comment></comment></comment></comment></comment><comment><USER6> I've always liked take-home projects for 2 reasons: 1) I get extremely anxious during interviews, so I don't perform well in them (to the point of crying sometimes - [http://blog.pamelafox.org/2013/09/technical-interviews- make-...](http://blog.pamelafox.org/2013/09/technical-interviews-make-me- cry.html)) 2) I like getting to see what it'd be like to do the sort of work that the company would be asking me to do for my job - it's a good taste to see if the company would be a good fit for me. The best take-homes are the ones that _very closely_ approximate the actual job.

However, not everyone likes take-home projects or has the time to do them. So it seems that companies should ideally offer each candidate multiple paths, and develop consistent rubrics to evaluate how each candidate does across paths.

I also do like the idea of probation, as it once again seems like it could be both good for the company and good for the employer, to give them both a sense of fit. Jobs are such big decisions, it's stressful for both the employer and employee to figure out if it's the right path for them.

<comment><USER7> I like them too, but if and only if they come with a review after turn-in. If I botched it in your eyes I want to know why!

</comment></comment><comment><USER8> This is silly. First of all, are interviews stressful, yes. That's not a bad thing. The real world is stressful. Soft skills matter. Interviews are useful to see how candidates react.

Second, most people aren't born with the perfect skill set for a job. Most don't even acquire it in university, or at a previous job. Most important is the ability to learn and grow.

Third, in this day and age, you may send hundreds of resumes, and have multiple interviews. Are you going to do 2 weeks of take-home projects? What if you have multiple interviews in one day, and both places expect you to finish take-home projects. What then?

This just seems like another hoop to make candidates jump through, with no good outcomes.

<comment><USER9> Interviews, for many people, are stressful in a way that jobs are not.

Personally, the most stressful things I've had to do as a developer (including speaking in meetings large and small) are an order of magnitude less stressful than even the best interviews.

edit: typo

<comment><USER10> During my last job search I did a take-home project, spent far more time on it then was specified, and then was rejected without an explanation. What a wonderfully productive use of 40 hours.

I'd much rather study for interviews.

<comment><USER11> I always end up enjoying the projects and learning a lot about a new domain in the process. I think for me, I see them as an opportunity to write code outside of my current role or comfort zone. I have been rejected once, the team really liked the project and I spent a considerable amount of time on it, but they found a more experienced engineer.

</comment></comment></comment></comment><comment><USER12> As someone who has had "8 hour" take home projects with requirements like "build a responsive site for this fictitious product and include creative UX features as well as a full summary of thought process during the project" I think this idea is horrible. I refuse to do take home tests, they are a waste of my time.

I've been to 50+ interviews on both sides of the table and can tell you there is always a few questions you can ask to determine someones development competency.

Not to mention I have literally thousands of hours of work I can send you from PSDs to code if you want to dive into my work.

</comment><comment><USER13> I'm not a fan of take-home projects. But comparing that to mumbling at a white-board, give me take home interviews any day of my life.

<comment><AUTHOR> Agreed, ideally there should be a choice between the two because some actually prefer whiteboard type interviews.

This is a good post discussing both: [http://blog.triplebyte.com/take-home- interviews](http://blog.triplebyte.com/take-home-interviews)

</comment></comment><comment><USER14> Best interview I have had was an interview, then it was followed up by a paid "side-project". I got paid to do some work for them and everybody was happy. Eventually, I figured out there where some politics/business practices that I didn't care for and I politely declined the position. It just wasn't a fit, not the end of the world, nobodies feelings where hurt. (it also helps having a job, when looking for one.)

The thought of 3 month probation period is just silly, the author clear doesn't understand the concept of risk.

I know when I walk into an interview and someone asks me the difference between php 5.6 and 5.7, that's when I walk out. (true story)

<comment><USER15> You cannot legally do paid side jobs with visa holders.

<comment><USER16> > You cannot legally do paid side jobs with visa holders.

An alien admitted under a visa can do paid side jobs if the visa they have permits work in the United States (unless the visa only allows restricted work not compatible with the particular paid side job.)

<comment><USER15> H1-B & TN visa holders can only do work for their specified employer(s) in their visas. I would be pleasantly surprised if this was actually not the case.

If you have a green card or general work permit then it doesn't matter of course. To get a general work permit although, you're usually pretty close to getting a green card.

<comment><USER16> > H1-B & TN visa holders can only do work for their specified employer(s) in > their visas

Right, and I'm not saying that that is inaccurate for those particular visas. Just that the generalization to "visa holders" is overbroad.

<comment><USER15> The vast majority of people you could hire that have visas are those two categories. L visas are even worse because I think you cannot transfer employers with them. The few H1-B spouses, O-* holders and others are few and far between.

</comment></comment></comment></comment></comment></comment><comment><USER17> Sure but expect to half your candidate pool.

Can't I talk you some code that I have already written instead of wasting half a day doing some trival project in a rush?

<comment><AUTHOR> I think companies should optimize on giving options between all of the proven methods. Some may like the classic interviews but some may have a good github profile which is also a good signal on how the applicant is.

<comment><USER17> Proven methods? Are there any?

<comment><AUTHOR> As in the classic interview, discussion based on previous project, discussion on take-home interview. I think there can be good results when candidate is given the choice.

</comment></comment></comment></comment><comment><USER18> Disagree completely.

There are simply too many things that you learn in the interview that are too difficult to learn any other way. Having someone code is NOT about seeing what kind of code they produce (well maybe a little); it's about getting to know as much as you can about them in as little time as possible.

I want to put the candidate under some amount of stress to see how they respond. After all, responding to stress (deadlines, technical issues, people issues, etc.) is a very important part of a programmers job.

Just a few of the things I usually learn with a short coding test in an interview right away:

\- Did he understand the problem?

\- How did he approach the problem?

\- Does he appear as if he has any idea what he's doing?

\- Can he explain what he has done?

\- Can he defend what he has done?

\- Does he understand the concepts of order, cleanness, iteration, branching, recursion, etc., etc., etc...

\- Based on this small sample, do I think he can do the work we need done?

\- Do I like him? Will he fit in and be a good team member?

Of course, I share all of this with the candidate. There's no secret agenda. We just want to make a fair assessment of fit. This is good for everyone.

All a take-home project does is separate those who can build superior code from those who can build good code under ideal conditions. This is nice to know, but not a very important metric in the real world.

Ideal conditions rarely happen on real projects. Why simulate them in the screening process?

<comment><USER3> The problem is that most interviewers _think_ they can assess these things, but (a) really can't, and (b) are having their readings confounded by the nervousness and discomfort of candidates.

Also: we banned "Do I like him? Will be fit in" at Matasano several years ago, and were far better off for doing so.

Answer honestly: for _every bullet_ in this list, did your team have a _standardized rubric_ by which you could mechanically evaluate candidates? Matasano had a much smaller list of considerations for candidates, but developing working rubrics for assessing them took _years_. If you really have a working rubric for this ambitious list, you could be making a lot of money with it.

Note also the pronoun you've chosen to use through your comment. I know it isn't deliberate, but that choice also has an impact.

<comment><USER19> This is an excellent point and it's one of the ways we tried to reduce the interview subjectivity at my last employer.

Don't get me wrong: in the end, an interview is highly subjective, but when five people are evaluating someone, you want to have some way of calibrating their expectations so they are at least close to being on the same page.

It helped to even out the process. If Tim says the candidate was a 3 out of 5 on multithreading concepts, I know what questions he could have been asked (there may be 6 to choose from on the list) and what quality of answers he gave (typical candidate responses are shown, in increasing order of "quality").

Did it help get us better developers? I don't think so. In the end we concluded that our biggest problem wasn't screening good from bad candidates -- we usually made pretty good hires. The problem was getting good candidates to apply in the first place.

</comment></comment></comment><comment><USER20> Take-home projects can be time consuming, give them Interviews instead

</comment><comment><USER21> Nope. I've had several of these, all were at least a full days work, which I had to cram into late evenings etc on top of day job. And to top it all, they weren't even discussed in the interview!

Never again.

</comment><comment><USER22> Take-home projects are not the best way to judge skills (atleast in their current form). In the future, there would be some centralized solution so we don't waste time again and again.

<comment><USER23> create a commonapp style application process? one project, multiple companies, and companies can add their supplementary challenges or something

<comment><USER24> That assumes that you know how to test and judge engineers. I think the larger issue in engineering interviews these days is that we simply don't know how to effectively separate good engineers from the bad, and given the nature of our job there is a sometimes significant lasting negative impact of a bad/messy programmer.

<comment><USER17> In addition people are better at different things. I joined a startup recently. The code is well organised at the top level, and the logging was superb. However the architecture was terrible, horriby overengineered. And it looked like a Java programmer doing python - camel case instead of underscores (well a mix just to be inconsitent). Joins done at the application level rather than the database. And just doing way more work that was needed to calculate something.

</comment></comment></comment></comment><comment><USER25> Is there a mechanism for paying the test taker? A developer shouldn't be expected to work for free. The outside chance he'd be the 1/100 applicant to land the job isn't enough.

<comment><USER26> Absolutely. Since a good dev can get somebody a job, he can just set up a business gaming this system :-P

</comment></comment><comment><USER27> The best company I ever worked for did a two hour pair programming session on a simple rails app (since they were a rails shop) with every candidate that got passed the "chat over coffee" stage. You'd get to see the kind of code the candidates write and how they work through problems, how they work with another developer, and get a really good overall sense of how they would work as part of the team.

<comment><USER28> I like pair programming sessions a lot because both the candidate and the employer are investing an equal amount of time. While the intention behind take-home projects is good, the result is often that the candidate invests several hours into a project just to be quickly passed over by the employer.

</comment></comment><comment><USER29> Ain't nobody got time for that.

</comment><comment><USER30> Interesting I'll add it to [https://github.com/lukasz-madon/awesome-remote- job](https://github.com/lukasz-madon/awesome-remote-job) could be especially useful for companies that hire remote workers.

(shameless plug intensifies)

</comment><comment><USER31> Personality, communication skills, etc., all matter. A lot. There's more to being a good software engineer than knowing how to code well. You have to be able to back up your ideas, convince others of your ideas, hear others view points and objections to your ideas, etc. Take home projects are great, but they're not a substitute for a face-to-face interview.

<comment><USER32> We do both: \- A take-home project that takes a few hours \- We assess their code; if it looks good, we bring them in for a few more interviews, both tech and non-tech \- If it doesn't look good, we move on to the next candidate

<comment><USER33> OMG. A few more interviews? Can't you handle everything in one or two interviews?

Seriously, from my (quite senior) point of view - if you've seen my CV, done research on me (StackOverflow, GitHub etc.), seen my code from the take-home project, did a technical and non-technical interview and still unsure if I'm a match or not, I'd really doubt your competence. I must be really so exceptionally interested in the position to do a "few more interviews".

</comment></comment></comment><comment><USER34> bug report: The browser's back button goes into infinite loop.

1\. Click "CodePad".

2\. Click "Create New Pad".

3\. Press browser's back button.

4\. Repeat #3 to infinity.

Ubuntu 14.04, latest Chrome

</comment><comment><USER35> I like it. Let the company decide what's best. This is just another tool they can choose to use.

The term "take home project" is wrong. It's a take home "interview"! I keep my interview challenges at a 1-2 hour time frame if you know what you're doing. If you take longer, you may not be qualified.

</comment><comment><USER36> I'm all for trying out new ways to interview people, but IMHO people should err on the side of caution and be willing to let "good" candidates slip through the cracks than risk hiring someone who is a dud. Personally, I think it matters if someone can come up with a solution in 1 hour vs 15 hours.

<comment><USER37> There's a difference between being able to come up with a solution in 1 hour vs being able to come up with a solution in 1 hour on a whiteboard, with no internet access, and with someone breathing down your neck the whole time.

If your company has a genuine need to screen candidates for their ability to perform under the special conditions of the latter, then by all means, set up your interviews with those special conditions. Otherwise, I feel you'd be better served by an interview that more closely emulates the actual working environment that your employees work under.

My ideal technical interview would be one where the candidate is assigned a problem that is representative of the work that they're being hired for, and then left alone to work on it for an hour or two in their own preferred working environment, and _then_ the interviewer and candidate can meet up and demo the solution and discuss implementation details.

<comment><USER36> >There's a difference between being able to come up with a solution in 1 hour vs being able to come up with a solution in 1 hour on a whiteboard, with no internet access, and with someone breathing down your neck the whole time.

You can usually fit about 15 lines of code on a whiteboard which tends to bias questions to simple logic questions. Why do you need internet access? Also why does someone have to breath down your neck? They could leave you alone for 45 minutes and come back.

>Otherwise, I feel you'd be better served by an interview that more closely emulates the actual working environment that your employees work under.

Why do you feel this way?

I think that programmers should have as many programming related patterns memorized as possible - For e.g. How to compose different kinds of programs when given different resource restrictions, or having basic algorithmic concepts in their head so they can quickly choose one over the other. A programming test should immediately indicate if the candidate has such things on the tip of their tongue or not.

In much the same way as you don't open a high school physics textbook when you go about building an airplane, you don't want someone who has to google basic programming knowledge that they should just have in their head.

IME, when you start with a very high level of intrinsic knowledge it's much easier to make connections and come up with unique solutions, because your not constantly 'searching' for things. Instead you're just 'picking' the right building blocks. Or maybe trying to quickly eliminate obviously bad choices. Ofcource there is much more nuance, but you can't state everything in a single comment ! :")

<comment><USER37> > Why do you feel this way?

For the same reason why I feel whiteboarding and having someone breathing down your neck are generally not reasonable conditions to place on a candidate (and you seem to agree with me for these two points, so I'll only expand on the point about internet access): It doesn't make sense to place artificial limits on candidates' abilities to perform their work, when that's exactly what you're trying to gauge in a technical interview.

RE: internet access.

As with natural language, developers tend to have an active vocabulary (concepts and APIs that they come across often and can recall on demand), a passive vocabulary (concepts and APIs that they may be aware of and recognize, but cannot apply immediately or without reference), and a set of completely unknown vocabulary.

Yes, developers should strive to build and maintain our active vocabulary to the greatest extent possible (i.e. memorize all the things) since it would make us more productive. However, human active memory is severely limited compared to the vast amount of knowledge developers often need to perform their work, and as such the vast majority of most developers' knowledge tends to reside in their passive vocabulary.

To force candidates to develop without internet access means limiting them to their active vocabulary, when this is only a tiny subset of their total vocabulary. This means you won't get an accurate picture of their ability to perform their work, which is exactly what we're trying to gauge in a technical interview.

</comment></comment></comment></comment><comment><USER38> this is equivalent to asking math students to provide their answer but not show their work. I interview people to figure out how they think and learn how they solve problems. I don't actually care about the answer they give to the technical question nearly as much as how they got there. No way in hell I'd do this, nevermind pay for it.

<comment><AUTHOR> The idea is to then discuss the project in the interview. This closely relates to what applicant will be actually doing at the job.

</comment></comment><comment><USER39> Really interesting concept!</comment>


The In-game Economics of Ultima Online (1999) - <AUTHOR> http://www.mine-control.com/zack/uoecon/uoecon.html <comment><AUTHOR> A bit more back story on UO, ported from my comment in the Nomic thread.

UO was divided into Trammel (safe from PvP) and Felucca (not), and some servers only had Fel. Originally, there was only Fel, and Trammel was made to entice more players who might not want such a cutthroat environment.

In Fel as soon as you left a city you were more or less fair game to die from an _extremely_ hostile environment. PC Gamer (If I recall) described the learning curve of UO as "A frozen wall of acid." Brutal, but _very exciting._ A wild west MMO.

(In Fel) UO was a rare RPG where anyone could kill anyone, for any reason, but with the repercussion that they would be branded a "Bad person" (visible with a gray or red name instead of blue). Stealing from good people corpses also did this. Anyone can attack and kill bad persons, and if you killed even more people you were a murderer and it took a very long time to return to normal.

This Blue/Gray/Red name system created a sort of cautiousness among travelers. Being near a pack of "blue" people might be safe, since any person that tried to steal or kill would turn gray and they'd immediately kill him.

Unless of course, all those blue-named people were conspiring, and you are the target.

You want to use super awesome powerful gear? None of this sissy MMO stuff. Die and you lose it, and your enemy (or his enemy!) gets the spoils. An insurance system was added later (2005ish?) where you could pay a certain amount per item to not lose it, probably also as an attempt to make the game less harsh.

The problem I have with a lot of MMOs is that the power of your character is simply how much time you sink into the game. Essentially, MMOs are games that reward wasting time.

UO had so much more than that. UO was a game where treachery and sneakiness really paid off, if you wanted them to. Lots of ways to nearly instantly kill or entrap people lead to a lot of very exciting plots where guilds might be laden with spies. Absolutely nothing like the ridiculously limited PvP found in games like WoW.

In a lot of ways it was the Diplomacy (diplomatic back-stabbing board game) of MMOs. And it was great.

\</comment><comment>--

For more also see outworlder's excellent comment in the Nomic thread: <http://news.ycombinator.com/item?id=4890513>

The criticisms of UO wiki page is also a very good read, especially the housing and economy sections: <http://en.wikipedia.org/wiki/Criticism_of_Ultima_Online>

<comment><USER2> Many of these mechanics weren't in there during alpha/beta/launch, but came shortly after. But the essential part that was always in there is 'you will die, and get looted' which never happens in modern games.

<comment><USER3> > 'you will die, and get looted' which never happens in modern games.

DayZ? Minecraft?

<comment><USER4> As a game developer, I'd explain this as follows: the whole gaming industry has been following the same path towards casualness. Arrows in FPS games telling you where to go, auto adjusting difficulty, forgiving aiming, tons of quick save checkpoints, the list goes on.

The majority of players prefer these be in place - otherwise, why keep doing it? But there's still a significant segment of the market that pines for the day when games really kicked you in the ass.

So enter DayZ, and some psycho player with an Axe chases you while playing a creepy loop of a seven year old girl singing, and exploits bugs to kill you completely unfairly, there's a small but significant number of players that will fall in love with a game like that.

But you're not going to convert even a large share of Black Ops 2 players with that experience. It would be like making every car an Ariel Atom.

<http://www.youtube.com/watch?v=yldW2FXy7IY>

<comment><USER5> On the other hand, a lot of games now are large and diverse with hours worth of content/art, instead of 5 minutes of content constant replayed for hours until the button patterns are memorized.

<comment><USER3> Gran Turismo (first version, on Playstation one) was hard. It was good, but it was hard.

Later versions (GT4 especially) were much easier, with many more races and tracks and cars. But is it more fun? Is it fun for advanced players to grind through easy races? (even with weird features to increase difficulty and earnings from races?)

</comment></comment></comment></comment></comment></comment><comment> w-ll The UO economy was crazy, for a while I had a 11x14 inside of Luna Walls and was making like 30m a week. Selling to a gold dealer using my friends paypal (I didn't even have a bank account at the time, I was in middle school and said friend was in HS). I was making just shy of a grand a month in 8th grade running a mall in a video game.

I later sold my account and the in-game land for 5k, I regret it to this day :(

What am I doing now you might ask...

Bitcoin :)

<comment><USER6> I remember when one of the house placement land rushes occurred on the server I played on at the time. I had saved up for a Tower deed, and stay up ALL night in preparation. Had friends waiting to camp my spot with me and so forth. I got my spot, got my tower, then a friend of mine told me they were selling on Ebay for $600 to $1,000 depending on the placement and size of house. The Tower being the second largest structure in the game, I ended up selling it two days later for a cool $800.

I was 14 at the time I believe. I used my dad's Paypal account that I had convinced him to get months before, so he had never used it, and then had to explain how I made $800 on the Internet selling virtual goods. My parents insisted I was actually selling drugs for a good month.

Good times.

</comment></comment><comment><USER7> I'm pretty sure different economic systems could be simulated in MMO games for the benefit of mankind.

Let one game shard run with central banks, and the current economic system where banks create new credit through thin air. See how that affects the economic system in the game. How it affects house and asset prices, commodity prices and purchase power.

Let another shard run free banking with competing currencies. See how that affect asset prices, commodity and purchase power.

In yet another game world within the same system evaluate Bitcoin and see how that affects house prices, commodity prices etc.

And in another world use hard currencies such as gold and silver as purchase coin, simulate that there are limits on how much gold and silver can me mined.

Let the game worlds run for quite some time! Then evaluate which of all those system lead to the most prosperous game population, choose that for the real economy.

<comment><USER8> Too many variables will be missing. For example, you won't see how different economic systems affect scientific research and (thus) overall technological progress, which is the means by which a relatively poor person today is better off than a king 400 years ago.

</comment></comment><comment><USER9> Pure Gold:

"Human NPCs serve an extremely important role within the economy of UO because they are permanent. That is, real players disconnect to go to bed or (heaven forbid) to go to work and are therefore not online the majority of the time"

Other insights:

"An interesting economic phenomenon occurred concerning the fee charged by vendors. When they were first implemented, vendors charged a fee based on the resource price of their inventory regardless of their sales. With this in mind, clever players realized that they could set the price for the goods to be extraordinarily high and thus prevent anyone from buying them. This, it turned out, was a very effective way of creating a safety-deposit box since the vendors can not be robbed. Players started buying vendors for the sole purpose of increasing their hoarding space. This exacerbated hoarding problems and also resulted in a form of suburban sprawl where people built tents and attached vendors consuming valuable land. The designers ultimately fixed these problems with an elegant economic solution: the vendors now charge a fee based on the value of the goods assigned by the player. Thus, players can still set the values too high, but they will be charged rent proportionately thus deterring this practice dramatically."

And another

"...almost all characters are forced to be entrepreneurs of some type, a fact which doesn't correspond well to these player's real life. In other words, most people in real life generate their incomes from employment contracts and thus they understand these arrangements. Unfortunately, such employment contracts are not implemented in UO. Therefore, there is naturally disappointment when players are forced into being entrepreneurs and find that this job is not effortless."

<comment><USER10> Never played UO, but I saw this material given as a talk at GDC.

When they set their designs on a closed-loop material economy, they didn't account for extravagant hoarding behavior such as building houses from stacks of leather boots. Occasionally, they couldn't spawn new monsters because the world had ran out of the raw materials the monsters were supposed to drop as loot! They had to inflate the economy several times while trying to find yet another policy fix.

</comment></comment><comment><USER6> When I think back to the days when I played Ultima Online (14 to 17 I believe), for whatever reason my fondest memory is a "profession" I picked up, which was that of a blacksmith. We're not talking a cookie cutter category I selected on character creation here, we're talking I happened to have a certain level in a skill that allowed me to repair my own armor and weapons, so I turned it into a public service. I'd stand around in the forge area of the most popular town, about 3 - 4 blocks from the bank in town (the most popular area in the town), and advertise that I could do repairs (generally resulting in a tip).

This required that another player literally GIVE me his or her items. As in, I could run off with them if I wanted. There was no trade window that allowed me to repair their items without actually taking them. They had to trust that I would give their items back after repairing them.

I found this absolutely fascinating (and still do) for multiple reasons:

1\. Over time you built up a reputation of being trustworthy. At first people would only give a smith they didn't know crappy items, but as you became more trustworthy and well known they would start to hand over their rare items for repair. I was literally building a reputation in an online game, in a real community, for providing a quality service (think about the impact this likely had on a 14 year old, who at that time had no idea what having a job, or building a reputation, was like).

2\. Items HAD to be repaired in UO, so this wasn't something players could get around (unless they wanted to train up that skill themselves, and you were limited on how many skills you could train to their max level at any given time). You simply had to trust a blacksmith in order to repair your items. After players came back to town after long sessions of dungeon crawling and killing monsters (in many cases because their armor was in need of repair), they would funnel back into town and the first stop was generally to get repairs.

3\. For whatever reason everyone stuck to a tips based system. I don't remember a market rate at any time for repairs that players set as a community. It was just assumed you tipped your smith.

4\. The better armor/weapons you personally wore while smithing, the more likely you'd get customers who had better weapons/armor, resulting in higher tips (because they likely had more money). It was my first experience with advertising and building a personal brand.

This is just a taste of why UO was such an amazing game and social experiment. It's a shame nothing has come close to recreating such an experience (EVE Online being an exception, but it still falls short in many ways).

If I had to pick a second fondest aspect of the game, I'd probably say Rune Books. You could "mark" runes in the game with a spell, which coded that rune with the location you were standing in when you marked it, which you could then place in a book. You could then travel back to that exact location at will, at the cost of a spell and its required reagents (hell, the reagents system was amazing as well!). Players made Rune Books of all the towns, dungeons, and crazy hidden locations (or their own homes, favorite shops, etc) for personal use, but also to sell so other players didn't have to make them on their own (as it would require a significant amount of traveling on foot to do so). I remember being very fond of my Rune Book collection.

What a game.

<comment><USER11> I remember setting up a rune collection in my house/shop (you could set up NPC vendors to sell stuff to other players). The collection was free to use but my vendors sold copies of it you could take away or (for much less) runes that would bring you back to my shop. This meant that people would use my shop as a transport hub, and buy stuff while they were at it.

Such a harsh game, but that's what made it exciting. Interactions felt real, and just like in real life while people could be dicks most people chose to be decent.

</comment></comment><comment><USER12> I was an Elder in UO. We created short quests while Seers were responsible for rendering larger storyline quests. Seers weren't really allowed to venture off script and do things but Elders were encouraged to. We Elders had insane powers. We could appear anywhere in the world at anytime, walk around invisible indefinitely and into players locked dwellings. We were able to color and rename items. And make serpents and spiders appear in mass and in the middle of town by waving a beast making wand. This was actually encouraged under the Britain is under the plaque storyline. Being an Elder was boss, but stealing from mages in Occlo with my thieves guild in Baja. Also.

</comment><comment><USER13> Wow, that is an awesome read. Comparing it to the economics in the World of Warcraft is also interesting. I particularly like the discussion of 'drains' vs 'closed circuits' there is a lot of interesting questions about building virtual economies where those are implemented in various ways. In particular comparing them to 'drains' in the physical world like fashion changes or upgrade cycles.

<comment><USER14> ... and it's not just about virtual economies, either. The section "Hoarding and The Failure of the Closed Economy" is basically a lesson on the so-called Paradox of Thrift and the virtue of government deficits:

In the closed system, the "pumping up" of resources from the bottom of the top was done by the game administrators, and is very much akin to a government that spends exactly what it taxes.

In the open system, the "drain" at the bottom is still government taxation, while the "faucet" at the top is still government spending.

What the developers of UO learned is that those need to be tuned independently of each other to get a well working economy. If only people understood this about the real economy!

</comment></comment><comment><USER15> If you're interested in in-game economics, another very interesting read is _Lessons From Habitat_ <http://www.fudco.com/chip/lessons.html>, a game created in the early 80's by Chip Morningstar and F. Randall Farmer that ran on Commodore 64.

Despite being written more than two decades ago, most of the conclusions from _Lessons.._ are still fresh and applicable to game mechanics today.

Randall (@frandallfarmer) is also the author of _Building Web Reputation Systems_ , published by O'Reilly - <http://www.oreillynet.com/pub/au/3900>. Another nice reading, if you're interested in the subject.

</comment><comment><USER16> I have always wanted to play Ultima Online, since years ago. But too bad it still has monthly fees... $10 a month might not sounds too steep, but I'm in a place I can't pay using credit card nor get dollars... and it sucks.

</comment><comment><USER17> I remember playing UO and getting PKed the first foot out of the city. It was quite frustrating. At some point a group of players formed a mob and would hunt down those PKers who played dirty with low level players.

</comment><comment><USER18> Interesting.

I would like to see an article like this, about the economics behind Eve Online. That would be interesting.

</comment><comment><USER19> More UO nostalgia:

I started playing UO when I was about 13. My parents paid the monthly fees for a few months, but I couldn't convince them to keep paying for it. ("Do your homework, Sean!") Frustrated, my friends and I found the free, independent, emulator-run servers and started playing on them. Finding the nuances between the free servers (called "shards"), we settled on one in particular and stayed there for the next few years.

I got bored with playing pretty quickly -- I was never really good at games that required skill or huge amounts of time investment. I started talking with the shard staff, gained their trust, and "graduated" into being a moderator for the server when I was 14 by doing some minor policing (resolving disputes, busting people leveling up with macro programs like UOAssist, etc). I could warp anywhere and got to choose special colours of clothing and had this awesome counselor robe. Looking back, according to [1], I best fit a "socialiser" player.

Then I got bored of simply moderating. There were a backlog of feature requests to the lone shard developer, who was busy trying to, y'know, have a family and work his day job. At the behest of the Italian couple who ran the server, I began developing for the shard when I was 15 (having not programmed before). I did lots of simple things like create weapons and funny items with special abilities [2], my favourite being a bag of skipping stones. Then I started doing more complicated things like recreate Triple Triad from FFVIII [3], and useful things like remake the guild system.

These UO people were my friends, we hung out on IRC every day, and even did a couple of international Secret Santa gift exchanges.

I got bored again around 16 or 17 and forgot about UO, but it had a lasting impact on me:

At 18, I began studying computer science. I realized that the code I wrote to sort a list back in the UO days (not knowing any better) was actually the standard sorting algorithm, bubblesort.

(actually relevant to the link -->) At 19, I started to learn about economics and related it back in a very real way to my days playing UO. I was tempted to go back and try to design a method of distributing wealth that reduced inflation.

And by 20 I realized I was a very good programmer. (I'm now 26.) I still attribute UO at the Italians I've never met in person for that.

[1] <http://www.mud.co.uk/richard/hcds.htm>

[2] <http://neverlands-library.com/index.php?id=4lvl> \- can't believe I found this / someone documented this

[3] <http://neverlands-library.com/index.php?id=cards>

<comment><USER20> I wrote a good deal of code for a two of the popular emulators; starting with SphereServer, and then RunUO, primarily just on the server side and less so on the actual shard side.

Shards (emulators) let the server owner script and customize the game a lot, such that it rarely resembled the real game. A lot of shards were really shitty, in that the minute you created your character you instantly got full skills, tons of gold, free armor, etc. Thus, there was no pride or value in anything.

SphereServer scripting was very clumsy, written in its own quasi-object language. It had a LOT of flaws, but was leaps and bounds above UOX, the first major emulator.

The RunUO came around, written in C#, and was much faster than SphereServer. It scaled to thousands++ of online users. Scripting was all done in C# as well, meaning things were compiled rather than interpreted at runtime as in SphereServer (some things in Sphere were cached, but not many). Most of the active emulator shards today use RunUO.

</comment></comment><comment><USER21> I miss UO. I used to work from home, so I had a box set up that would run a simple c program that would send keys to repeat a macro based on what I wanted to train, but at random intervals. I did this because I wanted to keep up with my buddy, who had the time to actually play. But I also had a small bit of code that I injected that would scrape the text. I would set my character on a boat, and then when a mod would show up, it would beep from the PC internal speaker (so I wouldn't get caught if my wife turned down the volume on the speakers). I can't count the number of times talking to some directoe or VP while this high pitched noise came across on a conference call while I panicked to explain the noise, have a conference call, and type enough to convince that I was a person. I miss that.

<comment><USER22> Nowadays they have macro programs to help with this that use LUA or custom scripting languages to run the game.

I remember learning LUA by writing scripts. Probably my pinnacle here was one to harvest wood / mine automatically, recalling to a bank to drop everything off and then head to a new mine, rotating mines so they wouldn't run out and playing a siren noise and sending me an email when the anti-macro popup would come up.

At this point I was more playing to see how advanced and resilient I could make my script, and less so to play the game, heh.</comment></comment>


Want To Give Pinboard A Try? You'll Have To Pay $2.84  - <AUTHOR> http://www.techcrunch.com/2009/07/12/want-to-give-pinboard-a-try-youll-have-to-pay-284/ <comment><USER2> Nah I'll just use one of the many other free alternatives. Charging for something is fine but it helps to do something unique if you actually want people to pay.</comment>


PyPad - Python 2.7.1 for teaching on iOS - <AUTHOR> http://itunes.apple.com/us/app/pypad/id428928902?mt=8

<comment><USER2> FINALLY. I bet there is no way to program the phone with it through -- at least not to the extent one could program python on an android phone with SL4A. Someone please let me know if I'm wrong.</comment>


Ask HN: Which advertising network do you use and why? - <AUTHOR>

I see lots of short term websites popping up on HN. Sites that ride on the tail of a meme or one that has a lot of traffic but only for a week or two.
I often have smart or funny ideas* for a site of that description but I wouldn't really want to spend money on it.
Do ads recoup the cost of the domain name and hosting? I know it might not be much, especially if noone actually uses the site but it could potentially be a few hundred euro if the site took off.
Assuming that ads do cover the costs, what advertising network do you use? Are some better than other for monetising a short term site? The only ad network I know of is Adsense. <comment><USER2> If you only know of Adsense, go to google and type in "Advertising Network" and you can see a huge list pop up.

You know of Adsense bc it's one of the best ones and in terms of revenue generation.

If you have a niche topic that is going viral, it could be wiser to go to an affiliate advertising network and go for the commissions - make sure you know your demographic though.

Clicksor Adknowledge Adbrite Adonion Bing/Yahoo!/Adcenter

There is a ton more...

</comment><comment><USER3> Working in media for a full digital agency: several but below are two that might be useful to you.

Addynamo - Good traffic, good reporting, good agency commission.

Admob - Biggest mobile ad network, owned by Google, great analytics.

List on the Google Display network if you fall short of other options.</comment>


Wrong man was executed in Texas, probe says - <AUTHOR> http://news.yahoo.com/wrong-man-executed-texas-probe-says-051125159.html

<comment><USER2> Categorically opposed to the death penalty for every possible reason: moral, religious, practical. But: what the hell is this article doing on HN?

No shit, you say? Capital punishment is inane, unjust, and difficult to justify in a message board thread? Wow! Glad we sorted that out! Can we move on now?

If every legitimate social justice issue had a hearing on HN, the front page would be _nothing but_ social justice issues.

Flagged.

<comment><USER3> This isn't just a business-as-usual government-screws-up story. I have not seen a case of wrongful execution as bad as this. So while it's not evidence of an interesting new phenomenon, it's new evidence of an interesting phenomenon.

<comment><USER2> In most cases where social justice stories are plastered to the top of the front page, the people voting them up also believe that the story is uniquely interesting or important. Likelihood that this particular comment of yours is going to be used to justify many future stories like that, most of which will not be germane to the site: _extremely high_.

I also object to the idea that miscarriages of justice in death penalty cases are a "new phenomenon". This isn't a new phenomenon. It's just pithy, enough to capture the front page.

<comment><USER4> Having lived in Texas for 12 years I totally agree that this case isn't unique, in that Texas regularly appoints public defenders who are unqualified for the job or so overworked they can't possibly provide an adequate defense (1). And courts are happy to convict people whose lawyer sleeps during the trial (2). I express no opinion on whether this is suitable for HN.

(1) "Appointed as his advocates were a general practice lawyer who never had handled a major felony case and a veteran who had a heavy case load." (2) "The presiding judge said that the Constitution guarantees a defendant a lawyer, but it does not guarantee that the lawyer must be awake." <http://www.secondclassjustice.com/?p=196>

</comment></comment></comment></comment><comment><USER5> People are saying lots of things about the death penalty, with which I wholeheartedly disagree, but IMHO the really bad thing is this:

 _DeLuna was sentenced to death and executed based only on eyewitness accounts despite a range of signs he was not a guilty man, said law professor James Liebman_

Eyewitness accounts are proven to be wrong again and again. They should not be used to convict anyone. Much less to the death penalty. The brain has this amazing (if a little buggy) capacity of sometimes being completely wrong and at the same time being absolutely sure that it is right (like it happens, for instance, when you see an optical illusion).

I believe there was a time when they were useful (but even then, they were used to falsely accuse - sometimes intentionally, sometimes not). But now there's enough technology to make me think convictions should be made based only on material evidence. Eyewitness accounts should be used only to give context to the material evidence found. If there's not any, tough luck. I would have a very hard time reconciling if, for instance, being accused of rape/assault like it happened to this guy: [http://articles.latimes.com/2011/jun/27/local/la-me- accused-...](http://articles.latimes.com/2011/jun/27/local/la-me- accused-20110628)

<comment><USER6> Eyewitness testimony is the leading cause of wrongful convictions, according to the Innocence Project:

<http://www.innocenceproject.org/understand/>

</comment></comment><comment><USER7> I know it's tacky to meta-complain, but public interest articles are one of the reasons I stopped reading Reddit.

I am looking for one particular kind of "signal" in that signal-to-noise ratio, and while this might be of interest to a population including hackers, a population of hackers would likely not identify this article as relevant, outrageous though the article may be.

I've noticed HN's articles growing steadily less and less interesting. Perhaps it's time to move again. Any suggestions? MeFi? Other sites? Does a new one need to be created?

<comment><USER8> this is why there's a flag link for each submission

 _Please don't submit comments complaining that a submission is inappropriate for the site. If you think something is spam or offtopic, flag it by going to its page and clicking on the "flag" link. (Not all users will see this; there is a karma threshold.) If you flag something, please don't also comment that you did._

<http://ycombinator.com/newsguidelines.html>

<comment><USER7> I can do that (I think I can flag, but this was not always the case), and silently grow to hate the content on HN as flagging fails to catch the borderline cases (and possibly devalue each individual flag).

Or I can break that rule to voice displeasure and hope that someone shared that opinion enough to point me somewhere better, create somewhere better, or effect change.

> If your account is less than a year old, please don't submit comments saying > that HN is turning into Reddit. (It's a common semi-noob illusion.)

My account is over a year old (and I've lurked for longer). Therefore, I'm good!

But in all seriousness, while it probably is not turning into Reddit, I'm certainly planning to relegate HN, for the purposes of my news browsing / time management, to a category shared with other spammy news sites, for somewhat similar reasons.

<comment><USER9> I've been here over 4.5 years, and I can say that HN is not turning into Reddit. The quality of the front page has not declined.

The volume of stories submitted has grown dramatically, making it harder for the good content to get on the front page.

But this article definitely has no place on the front page. I flagged it, hopefully it will be gone soon.

</comment></comment></comment></comment><comment><USER10> Related: California has placed the Savings, Accountability and Full Enforcement for California Act (the SAFE California Act) on the November 2012 ballot.

The move to get this on the ballot was led by a former warden of California's San Quentin prison, Jeanne Woodford:

"California is on the brink of replacing the death penalty with a sentence of life in prison with no chance of parole as the maximum punishment for murder," Woodford said. "In November, voters will have the first opportunity ever to decide between the death penalty and a sentence of life in prison with absolutely no chance of parole. Back in 1978, we did not have an alternative sentence that would keep convicted killers behind bars forever. We certainly did not know that we would spend $4 billion on 13 executions. Our system is broken, expensive and it always will carry the grave risk of a mistake. SAFE California offers a solution with savings at a time when we're laying off teachers and cutting vital services."

I urge support of this measure:

<http://www.safecalifornia.org/act>

</comment><comment><AUTHOR> OP here - apologies to the folks who don't think it's HN-worthy. I usually shy away from posting anything like this on HN, but IMHO the death penalty stimulates my intellectual curiosity. Usual politics and "gee whiz poll numbers" not so much.

The death penalty is a deep issue, and I posted this on HN because I'd like to hear what the thoughtful folks in this community have to say on the topic.

Also, since I see a lot of people misquoting the Hacker News submission guidelines here they are in full, emphasis mine.

"Hacker News Guidelines

What to Submit On-Topic: Anything that good hackers would find interesting. That includes more than hacking and startups. If you had to reduce it to a sentence, the answer might be: _anything that gratifies one's intellectual curiosity._

Off-Topic: _Most_ stories about politics, or crime, or sports, unless they're evidence of some interesting new phenomenon. Videos of pratfalls or disasters, or cute animal pictures. If they'd cover it on TV news, it's _probably_ off- topic."

<comment><USER9> I've often considered that a HN style politics page would be interesting if you could somehow keep it from turning into a flaming mess.

That said, I don't believe this is Hacker News and I flagged it. It's going to devolve into politics in about 8 seconds and we try to avoid that.

<comment><AUTHOR> That's a fair point. Unfortunately, it's too late for me to delete. Lesson learned - I'll refrain from posting this kind of story in the future.

</comment></comment></comment><comment><USER11> This is the entire reason people petition against the death penalty. And while I've said I can see the need for capital punishment, as a juror I would never sentence someone to death because mistakes happen, the wrong people are accused, evidence gets fucked (accidentally, intentionally), etc.

The mistake isn't worth someone's life.

<comment><USER12> The death penalty is required. The judiciary just needs to have conclusive proof of the crime.

For example, right now, in one of India's jails lies a man named Ajmal Kasab. He is one of the terrorists involved in the 26/11 attack on Mumbai. He was caught on tape mercilessly killing people with his AK47.

If you do away with the death penalty entirely, how do you execute this fucker and others like him?

<http://en.wikipedia.org/wiki/2008_Mumbai_attacks>

Edit: A lot of people asking similar questions. So I'll just answer them here.

Being identified by eye-witnesses and being caught on camera are not the same thing. People make mistakes while remembering. Cameras don't. Further, he was apprehended by the army/police in the middle of his killing spree. There is NO WAY it was someone else.

Handing him a life imprisonment just puts the system at risk. It will just lead to terrorists resorting to kidnap to release him as has been done in the past. A well known example: <http://en.wikipedia.org/wiki/Indian_Airlines_Flight_814>

<comment><USER13> This should be a contreversal comment, so I'm going to say now that I'm just posting this rebuttal and not getting into an e-argument.

The criminal system isn't for punishment. It's for reform. It's not to 'get back' at people, it's to help integrate people back into society and act as a deterrent to stop crime in the first place. You're not trying to 'execute this fucker and others like him', you're trying to figure out what's wrong, how you can reform (if possible), and deter other crimes like it. If you're just executing it's a hell of a deterrence, but you're not reforming at all.

Remember the Oslo masscare? Read up on the perpetrator and his sentence. <http://en.wikipedia.org/wiki/Anders_Behring_Breivik>

<comment><USER12> If I were king of the world, I wouldn't sentence Anders to death either. It's sad that he turned out a madman and did what he did but surely you can't kill people who are mentally ill for being so.

There's a big difference between somebody who's medically declared insane and a member of a terrorist organisation who killed people as his masters wanted for the sole purpose of creating terror.

</comment></comment></comment></comment><comment><USER14> "Eye witness testimony is the lowest form of evidence in science. Which is sad, because it's the highest form of evidence in the court of law."

Neil D. Tyson

</comment><comment><USER15> My view is that when someone is wrongly executed, justice is not finished - someone must be held accountable and punished. It's not just "oh well." Sometimes prosecutors are happy to see a conviction even if they know it's wrong, just for their reputations. These people, I'd argue, could be put to death too. Murder.

You can't have the death penalty in a broken system, it enables the same crimes it attempts to deter, and with death there's no recourse. 20 rights and 1 wrong is just unacceptable. In reality, the false positives are apparently much, much more despicable. Just do a search on final statements for "innocent" or "innocence."

</comment><comment><USER16> Carlos' last words: [http://www.tdcj.state.tx.us/stat/dr_info/delunacarloslast.ht...](http://www.tdcj.state.tx.us/stat/dr_info/delunacarloslast.html)

</comment><comment><USER17> The original source seems to be this more sophisticated article at The Guardian which was published earlier today. [http://www.guardian.co.uk/world/2012/may/15/carlos-texas- inn...](http://www.guardian.co.uk/world/2012/may/15/carlos-texas-innocent-man- death)

</comment><comment><USER18> And this is why the death penalty should be outlawed. If there is even a 0.001% chance we could execute an innocent person than its not worth it. Unfortunately I think the probability of executing an innocent person is much higher. Humans are imperfect and should not play god.

</comment><comment><USER19> I'm not entirely sure how this relates to HN.

<comment><USER11> Does it need to "relate" to this site? If it doesn't get the votes, it'll be gone in no time.

<comment><USER20> I bet that's what they used to say on Reddit...

</comment></comment></comment><comment><USER21> I'll bet that >100 other innocents who were treated far worse by the system were killed by prison during the time Texas took to execute this guy. And, this would continue even if the US abandoned the death penalty.

If you're looking to save innocents from being killed by prison, death row is the wrong place to look. No matter how inadequate you feel the process is for death penalty cases, it's significantly worse for folks who get other sentences. Since prison kills people (and not just with "live without parole)....

No, they don't "get a [meaningful] chance to prove their innocence. They just die, killed by prison just as much as someone intentionally executed.

</comment><comment><USER22> Would people be so torqued up if this were a wrongful imprisonment in Massachusetts instead of a wrongful execution in Texas? I doubt it, which suggests that this story may be pushing buttons in your brain other than _injustice == bad_.

Wrongful conviction is bad in any case; the death penalty aspect is simply inflammatory. Executions are irreversible, but so is imprisonment. They might let you out if they discover you're innocent, but there's no way to refund you your lost time.

</comment><comment><USER23> With the American justice system it would have been cheaper to keep him in prison for life than to execute him for murder, due to the number of appeals allowed.

</comment><comment><USER24> In Ontario, Michael Rafferty & Terri-Lynn McClinic were recently convicted of the murder of Tori Stafford. Lots of _pro-capital punishment_ banter as of late. In this case, it was pretty open/shut. You could say he is the perfect candidate for capital punishment. I wouldn't, but many would.

My problem is – our judicial system pretending these people can be reconditioned, like a refurbished electronic or something. Terri-Lynn McClintic was wicked well before the murder, even doing things like microwaving a small dog until it cried.

These people cannot be reconditioned. And when you take a look at the science, it comes down to bad wiring. So does killing them make sense knowing that they are mentally damaged and not simply moonlighting as maniacs?

With tongue in cheek, we need Minority Report-style decommissioning. Technology to the rescue.

</comment><comment><USER25> Not for the first time: [http://www.newyorker.com/reporting/2009/09/07/090907fa_fact_...](http://www.newyorker.com/reporting/2009/09/07/090907fa_fact_grann)

</comment><comment><USER26> ..and that's just like Linux vs Microsoft.

</comment><comment><USER27> Great - independent, open-source defense lawyering.

Make sure when you read a press release like this, though, that if there isn't a press release saying the courts have convicted an innocent man, there isn't a press release at all. Which means Professor Liebman and his students have wasted their time. Which they don't want to do, so they are anything but disinterested investigators.

I mean, you're not going to see a headline that says "Murderer walks free in Texas, probe says." I mean, duh. That happens all the time, so who cares? There's certainly no market for an independent, open-source prosecution...

</comment><comment><USER28> F __king ridiculous.

Doesn't this make the prosecutors in this case guilty of murder?

<comment><USER29> Murder means intent. It makes them guilty of manslaughter, at best. I don't know if there's any legal precedent for that (I'm not a lawyer), but I'd imagine that a civil case by the deceased's family for a wrongful death would be in order.</comment></comment>


The Startup Kids Documentary - <AUTHOR> http://techcrunch.com/2013/03/22/the-startup-kids-documentary-itunes/

<comment><USER2> I saw this in Amsterdam and a lot of it was more of the same that you already read about here, but it was still really helpful, especially my less technical friends and hopefully future cofounders.</comment>


Ethiopian 737 Max: Aircraft Accident Investigation Preliminary Report [pdf] - <AUTHOR> http://www.ecaa.gov.et/documents/20435/0/Preliminary+Report+B737-800MAX+%2C%28ET-AVJ%29.pdf <comment><USER2> Comments moved to [https://news.ycombinator.com/item?id=19573893](https://news.ycombinator.com/item?id=19573893).

Edit: which turns out to have been the basis for this article and discussion yesterday: [https://news.ycombinator.com/item?id=19570378](https://news.ycombinator.com/item?id=19570378).

</comment><comment><USER3> > Also, the airspeed, altitude and flight director pitch bar values from the > left side noted deviating from the corresponding right side values. The left > side values were lower than the right side values until near the end of the > recording

Hmm. Seems not just faulty AoA sensors? Perhaps some major malfunction in the flight computer? I've felt this entire time there's been a rush to claim 'faulty AoA sensors' because 'sensors' are known to fail, and it's an easy scapegoat.

My question is, does the flight recorder have raw inputs from the sensor, or just what the flight computer is reporting from the sensor? Seems like the latter.</comment>


Regular Expression Engine in 14 lines of Python - <AUTHOR> http://paste.lisp.org/display/24849 <comment><USER2> Sometimes loops forever, sometimes takes exponential time. Avoiding these problems:

        """
        Regular-expression matching by the Thompson construction.
        Explained in C at http://swtch.com/~rsc/regexp/regexp1.html
        """

        def match(re, s): return run(prepare(re), s)

        def run(states, s):
            for c in s:
                states = set.union(*[state(c) for state in states])
            return accepting_state in states

        def accepting_state(c): return set()
        def expecting_state(char, k): return lambda c: k(set()) if c==char else set()

        def state_node(state): return lambda seen: set([state])
        def alt_node(k1, k2):  return lambda seen: k1(seen) | k2(seen)
        def loop_node(k, make_k):
            def loop(seen):
                if loop in seen: return set()
                seen.add(loop)
                return k(seen) | looping(seen)
            looping = make_k(loop)
            return loop

        def prepare(re): return re(state_node(accepting_state))(set())

        def lit(char):     return lambda k: state_node(expecting_state(char, k))
        def alt(re1, re2): return lambda k: alt_node(re1(k), re2(k))
        def many(re):      return lambda k: loop_node(k, re)
        def empty(k):      return k
        def seq(re1, re2): return lambda k: re1(re2(k))

From <https://github.com/darius/sketchbook>

I'm not sure how the original comes to 14 lines. btw.

<comment><USER3> It comes to 14 if you ignore the first line of each function (e.g. `def iconcat(xs, ys):`). I'm not sure why one would count lines like that, though.

</comment></comment><comment><USER4> Transliteration into Haskell brings type-safety into it, and cuts it down to 7 lines:

      import Prelude hiding (seq)

      seq l r s = concatMap r (l s)
      alt l r s = l s ++ r s
      star e s = s : seq e (star e) s
      plus e = seq e (star e)
      char c (x:xs)
        | x == c    = [xs]
        | otherwise = []
      char _ _ = []

      example = seq (char 'c') $ seq (plus (alt (char 'a') (char 'd'))) (char 'r')

      main = do
        line <- getLine
        mapM_ (putStrLn . ("Match with remainder: "++)) (example line)

(Note the original implementation failed to reuse itertools.chain, in Haskell I just use (++)).

</comment><comment><USER5> Why call it a regexp engine? Those are actually parser combinators with backtracking. They can parse much more than just regular languages, but are exponentially slower compared to a proper regexp engine.

<comment><USER4> regexps aren't regular, so most "proper" engines also do backtracking.

Indeed, if you stick with the regular subset, you can be really fast:

<http://swtch.com/~rsc/regexp/regexp1.html>

</comment></comment><comment><USER6> Here's a complete PEG parser in 3 lines:

        consume = lambda c: lambda inp: inp[1:] if inp and inp[0] in c else None
        ordered_choice = lambda va,vb: lambda inp: va(inp) or vb(inp)
        concatenate = lambda va,vb: lambda inp: (lambda x: x and vb(x))(va(inp))

Used like this (something I hacked together recently): <http://pastebin.com/dUyTZttZ>

If you want it to actually do something useful, just wrap the functions with another functions that do what you need.

<comment><USER2> In va(inp) or vb(inp) couldn't va(inp) succeed with an empty string as the result (at the end) which Python interprets as false, making it mistakenly try vb(x)? I haven't tried to run it.

Also I think c seems misleading as a variable name -- it suggests a single character, but the 'in c' means it's a set of characters.

<comment><USER6> Yes, that is a bug and yes, those variable names do suck. Thanks for the correction.

        ordered_choice = lambda va,vb: lambda inp: (lambda a,b: a if a or a=="" else b)(va(inp),vb(inp))

Would work, but starts to be so ugly that I'd have to write proper functions and the code would not be 3 lines anymore. ;)

<comment><USER2> How about

        eat = lambda chars: lambda s: s[1:] if s and s[0] in chars else None
        alt = lambda p, q: lambda s: (lambda r: q(s) if r is None else r)(p(s))
        seq = lambda p, q: lambda s: (lambda r: None if r is None else q(r))(p(s))

going a bit far in the short-names direction, oh well. :) It's too bad Python has this artificial statement/expression distinction.

</comment></comment></comment></comment><comment><USER7> This was a bit of a joke at the time. I'm surprised no-one yet pointed out the bug with unproductive left recursion which haunts this and all other similar matchers. Harper's classic Proof-Directed Debugging paper shows how to pre- normalize regular expressions to eliminate the issue: [http://citeseer.ist.psu.edu/viewdoc/summary?doi=10.1.1.19.56...](http://citeseer.ist.psu.edu/viewdoc/summary?doi=10.1.1.19.5602)

<comment><USER8> How can this left-recurse? Can you give an example? I thought regular expressions weren't recursive at all.

<comment><USER7> The Kleene star is a fixed-point/recursion operator. Here's an example that causes an infinite loop:

        (|a)*

<comment><USER8> That's not left-recursion.

<comment><USER7> I probably shouldn't have used the 'left' adjective. But that expression results in an unproductive recursion (unproductive meaning that no characters are consumed through each cycle) analogous to what happens when you recursive- descent parse a context-free grammar containing left-recursive productions.

</comment></comment></comment></comment></comment><comment><USER9> The one-character variable names remove any value this snippet might possibly have for documentation or demonstration.

</comment><comment><USER10> Interesting that it's on paste.lisp.org. It strongly reminds me of Emacs' rx macro. I suspect this a port of another toy regexp implementation -- otherwise I would expect the author to use `itertools.chain` rather than implementing something that does exactly the same (`iconcat` in this sample).

<comment><USER11> Pasted by: psykotic When: 5 years, 2 months ago

I don't know Python, but was your preferred choice available at that time?

<comment><USER12> It was, though it was fairly new at that time, so the author may not have had it on his system.

Also, the Python standard library is quite large. Most people will never really know about all its nooks and crannies, and it's easy to forget they exist. I've accidentally reimplemented parts of it more than once.

</comment></comment></comment><comment><USER13> Work on a new implementation might be a good idea. :-)

The last months I've used the PCRE regexp library, which afaik is the one used in most implementations wanting Perl compatibility. I was a bit shocked... I thought it was good?

I ran into a couple of cases with bugs (one regexp couldn't be ported from Perl and another over complex one from a newbie hung(!) the process). It seems inefficient with backtracking, or? (Maybe this is fixed now?)

<comment><USER14> Even though this implementation is short, don't think it is efficient.

Matching regular expressions can be done in linear time.

This presented algorithm takes exponential time.

<comment><USER15> > Matching regular expressions can be done in linear time.

Unfortunately, the expression "regular expression" has been so abused in the context of programming that, unless otherwise specified, we cannot expect it to correspond to the well-defined regular expression that is used in formal language theory.

For example, statements such as "you can use regular expressions to parse HTML" and "regexps aren't regular" (mentioned in this thread) are now acceptable. I wish people would at least say "extended regular expressions", but it's a lost cause.

</comment></comment></comment><comment><USER16> in 14 lines of ... python?? really? Ok, I can do better. A Regular Expression Engine in 1 line of Javascript: string.match();

<comment><USER17> You're confusing the implementation of algorithm with a library call.</comment></comment>


Ask HN: My 10 year old wants to develop games. Which platform is the best? - <AUTHOR> My 10 year old wants to develop computer games (preferably something that will also work in a phone). She can program in scratch comfortably. Is this a good time to introduce a physics engine/javascript? Any thoughts? <comment><USER2> My son is learning using Scratch ([https://scratch.mit.edu/](https://scratch.mit.edu/)) and it's pretty well tuned-in to the way he thinks. Haven't put any "code" in front of him yet, but he can make some pretty neat games very rapidly using Scratch. Might be worth a look.

</comment><comment><USER3> Unity is pretty decent with lots of paid/free assets and stuff.

<comment><AUTHOR> But is it suitable for a 10 year old?

<comment><USER4> could be, depending on the level of english the kid is able to read but [https://html5gameengine.com/](https://html5gameengine.com/) might have some options especially the first one on the list since it doesn't really require coding from what I can tell, and if the kid finds it fun, you can always switch to a game engine that requires more coding</comment></comment></comment>