Which Is Best: More Customers Visiting Once or Fewer Customers Visiting More Often? - <AUTHOR> http://www.jslogan.com/content/view/49/ <comment><USER2> Well, no one would pick the former choice because it's essentially an actual customer base versus stray eyeballs.

But keep in mind that web apps and advertisements are different beasts when it comes to frequency versus reach. For a web app, site, or service - of course you're going to want frequency. When it comes to advertising, the prevailing choice isn't as distinct. I'd contend that reach would have a greater chance of finding potential customers. Frequency can be seen as a nagging effect on deaf ears and blind eyes (the uninterested).

<comment><AUTHOR> Reach with frequency has a good opportunity to find new customers, but the lower the frequency, the less the effect. This is where many companies fail in their lead generation campaigns - they look at the campaign as a single event.</comment></comment>


OpenMetrics project accepted into CNCF Sandbox - <AUTHOR> https://opensource.googleblog.com/2018/08/openmetrics-project-accepted-into-cncf.html <comment><USER2> Can someone explain to me, what the pros/cons of this are as opposed to OpenTracing?

<comment> nevi-me EDIT: Tracing deals with annotating methods, and how long they take. E.g. function a took 3 seconds, and make 2 DB calls that took 1.4 seconds. Metrics deal with how many times you made calls, how long they took, etc. So two importantly related parts of application monitoring.

Metrics is different to Tracing in this case. From the surface (or from someone who doesn't closely follow), OpenCensus contains metrics and tracing (and maybe more).

OpenMetrics appears to be the standardising the metrics part, hence the sentence that OpenMetrics is being integrated with OpenCensus.

Now, you might ask, because I've also asked: what are the pros/cons of Census over Tracing for tracing purposes?

It does look like both standards compete and perhaps diverge. I once sensed a bit of bashing of the latter in the OpenCensus Gitter channel; which convinced me to just stay with OpenTracing as they already had more integrations at the time.

At least Metrics is being promoted by the same interests as those working on Census, so we won't see another set of standards targeting the same applications (LightStep, StackDriver, etc).

In short, why can't OC and OT combine their efforts?

<comment><USER2> Well, actually that's what I'm mainly interested in - what the technical differences between OpenTracing and the tracing part of OpenCensus are.

Excuse me for my badly worded question.

However, why they can't combine their efforts:

Probably a lot of infrastructure is built on both of them, having the aforementioned potential technical differences, which could make any unification backwards incompatible, as in: traces produced by the unified standard wouldn't be parsable by todays infrastructure tooling of the companies using them.

There may also be different tradeoffs chosen, which again boils down to the question regarding the technical differences.

<comment> nevi-me The thing is that they both target similar applications. Zipkin has been around for years, and as both support sending data to it, surely there's enough common ground. I understand what you mean regarding infra, but I think the parties could work towards unity in future.

gRPC supports OpenCensus for example, I recall that OpenTracing has been working on an RPC for reporting traces. gRPC by being the RPC framework, it would make sense if it could self-report to OT in addition to OC.

</comment></comment></comment></comment><comment><USER3> The blog post mentions that OpenCensus supports .NET, but on [https://opencensus.io/language-support/](https://opencensus.io/language- support/) .NET is not mentioned. What is true? Has anyone used opencensus with .NET ?

</comment><comment><USER4> Google accepted spam comment on their blog :D

[https://opensource.googleblog.com/2018/08/openmetrics- projec...](https://opensource.googleblog.com/2018/08/openmetrics-project- accepted-into-cncf.html)</comment>


Tech startups: The lure of the big number - <AUTHOR> http://www.economist.com/news/business-and-finance/21651136-true-unicorns-are-still-rareeven-magical-world-technology-lure-big-number?fsrc=scn/tw_ec/the_lure_of_the_big_number

<comment><AUTHOR> If a public company loses half of its value, investors lose half of the money they put in, he explains. But should this happen to a unicorn, investors may not lose anything—as long as the total value of the firm does not fall below the amount protected by the liquidation preference.</comment>


Swedish police may soon be allowed surveillance via trojans - <AUTHOR> https://www.thelocal.se/20191023/swedish-police-ready-for-increased-powers-of-surveillance <comment><AUTHOR> According to the Police "90% of (criminal?) traffic is encrypted".

There's also the recently enforced "Data storage directive" that record all ISP traffic for 6 to 24 months. (Accessible without warrant)

The "Data storage directive" law has been deemed illegal by the EU, but was forced through anyway and applied 1 October 2019.

<comment><USER2> Not agreeing with the ever increasing surveillance (nor this proposal), but, I don't think the 90% stat was gathered that way. When I saw the press conference [0], I got the impression that the 90% came from failed attempts at "hemlig avlyssning av elektronisk kommunikation" (secret surveillance of electronic communication). That's basically a network tap gathering data, SMS or phone calls.

[0]: [https://www.youtube.com/watch?v=TILOlZRga6U](https://www.youtube.com/watch?v=TILOlZRga6U)

<comment><AUTHOR> Yeah, makes sense that it's based on actual wiretapping attempts.

</comment></comment></comment><comment><USER3> At first glance I am kinda ok with this. I mean, as long as a citizen / criminal is allowed to encrypt anything and everything and take any step they want to to protect their communication / devices, I kinda think the police should be able to do anything they can to get evidence. As long as they get a proper warrant and the crime suspected is grave enough of course. I might be missing something and is up to change my mind on this though. As long as they aren't arguing for something stupid like a backdoor or making encryption illegal or something.

</comment><comment><USER4> How do they plan on loading up an iphone or android phone with a 'trojan'? This seems to be more an excuse to get police some sort of excuse to be able to snoop even more than what they currently can.</comment>


Show HN: Posthook – Job Scheduling as a Service - <AUTHOR> https://posthook.io <comment><AUTHOR> Hello all! I built Posthook as a simple solution for web applications to schedule one-off tasks. It lets you schedule a request back to your application for a certain time, with an optional JSON payload.

It can be an alternative to running your own systems like Sidekiq, Celery, or Quartz and the operational overhead that comes along with them. Cron jobs and cloud provider tools like CloudWatch Events are also used for job scheduling but lack observability and may force you to frequently make expensive queries to your data store just to see if there is any work to do.

Questions and feedback are greatly appreciated :)

<comment><USER2> Neat! We've been doing something like this at work, and another simple solution (curious if your Google cloud infrastructure is built on something similar) is Azure Service Bus queues with a function app trigger.

Unlike AWS SQS, Azure SB queues can have items scheduled to be queued at an arbitrary time, see [https://docs.microsoft.com/en- us/dotnet/api/microsoft.azure....](https://docs.microsoft.com/en- us/dotnet/api/microsoft.azure.servicebus.message.scheduledenqueuetimeutc?view=azure- dotnet). SQS can only delay up to 15 minutes, so you have to implement your own hack to schedule for later than that.

And also unlike SQS, they can trigger a function so you don't have to deal with the logic to listen yourself - or even the transactional message handling, if the function fails the message is automatically re-queued.

<comment><AUTHOR> Thanks! I do heavily use Google Cloud Pub/Sub but it does not support scheduling items so I only schedule items when they are due for processing.

<comment><USER3> Wouldn't it be good to have your own pub/sub mechanism instead of relying on a third party like gce for a critical component. Take for example - [https://github.com/grpc/grpc/issues/13327](https://github.com/grpc/grpc/issues/13327). It took them fair amount of time to fix that issue.

</comment></comment></comment></comment><comment><USER4> This is definitely a needed service. As a business, if something isn't core to the business model, it's better to pay money and avoid distractions, at least until it's clear that something is actually core to the business.

That said, I've wanted something akin to a distributed cron, without the complexities of workflow engines like airflow or azkaban, so I started writing a little HTTP API for scheduling jobs [1], which is mostly complete except for configurable retry logic and failure notifications. To schedule a periodic job that runs every 30 seconds,

        curl http://$HOSTNAME/jobs \
        -d '{ "title"      : "Healthcheck"
            , "code"       : "curl example.com"
            , "frequency"  : "30 seconds" }'

[1] [https://github.com/finix-payments/jobs](https://github.com/finix- payments/jobs)

<comment><USER5> Hey. I just made this. It's a coincidence that the OP and me are building it at the same time, but AFAICT the OP service does not support recurring tasks, but Pocketwatch does. Per your example, try:

       curl https://api.pocketwatch.xyz/v1/timer/new \
       -H "Content-type: application/json" \
       -d '{ "apiKey"             : "i_am_hn_and_proud"
           , "url"                : "https://host.tld/?my_param=my_val"
           , "method"             : "POST"
           , "interval_unit_type" : "second"
           , "interval_unit_count": 1
           , "duration_unit_type" : "minute"
           , "duration_unit_count": 6 }'

You can use curl but the Node.js client library[1] is probably easier. I've made a free demo API key for HN readers that should be enough to let everyone try it out a bit[2].

[1] [https://www.npmjs.com/package/%40dosy%2Fpocketwatch](https://www.npmjs.com/package/%40dosy%2Fpocketwatch) [2] [https://news.ycombinator.com/item?id=17353486](https://news.ycombinator.com/item?id=17353486)

</comment></comment><comment><USER6> Just curious, why would I use this instead of Sidekiq, Celery or any other job / background worker service that runs along side my app?

Almost all popular job processing libraries in all major frameworks support executing 1 off tasks with robust retry policies and even support recurring tasks.

The operational complexity is really just running the sidekiq or celery command in another Docker container or if you don't use Docker, then setting up 1 extra systemd unit file.

I once had a Celery / Flask app running on a $20 / month digitalocean server. It handled over 3 million background jobs per month and it also hosted my DB server, cache server and web front end.

On your pricing page, your highest tier supports up to 100,000 requests per month at $129/month. How much would it cost to do 3 million requests per month?

Also how would you set up custom retry policies?

<comment><AUTHOR> I understand Posthook may not be the best solution for everyone's scheduled tasks. It seems like for that project you had the expertise, time, and risk tolerance to set up that one DO server to handle all your needs. Other developers may be in a different situation and may find Posthook useful.

> How much would it cost to do 3 million requests per month?

If you want to use Posthook to schedule 3 million requests a month please email support [at] posthook.io and we can work out a special plan and support contract. Do keep in mind that background jobs and scheduled jobs are two different things and what I aim for Posthook to solve are scheduled jobs. I suspect a big part of those 3 million were background jobs.

> Also how would you set up custom retry policies?

The retry policy is fixed at the moment but it seems like this a common request so I want to roll that feature out soon.

<comment><USER6> > I suspect a big part of those 3 million were background jobs.

Yep, almost all of them were tasks that should be executed as soon as possible, but Celery also allows for you to schedule tasks at a future date, and deal with running scheduled tasks at a repeated time (every Sunday at 3am, etc.).

As for the price, I was just curious for the sake of wanting to compare it to the $20 / month DO set up.

Thanks for the answers.

</comment></comment></comment><comment><USER7> I would be really interested in a distributed job running system with a scheduler. I want a UI where I can set jobs via a schedule that run on one of many workers. In that same UI I want to be able to look at failed jobs and see all of the output from that job. I don't want to have to ssh over to the box where the job ran to retrieve a log file. I once had a job where we used Tidal Enterprise Scheduler which poorly implemented this. What solutions exist for this workflow in the open source world?

<comment><USER8> Would you pay for this? How much and what pricing scheme would you prefer?

<comment><USER7> As a service that runs on Heroku I'd pay around $25/month. that'd be $25/month * 2 (stage+prod). At the bootstrapping stage i'm at now, that's what I'd pay. If I were profitable, I'd pay up to $500 - $1000 a month and happily so, figure out market segmentation so that I'd be forced to pay that much. I would probably run around 1000 jobs a month, which seems like a number that would put me in a higher pricing range. This service would be super useful for around 60 jobs a month even. Output from the jobs would be tiny, maybe 10k at most. I would be happy if output was saved on private S3 buckets.

I guess the big point is, I'm looking to use this for regularly scheduled jobs, not just as a queuing service to spread load.

My email is in my profile if you want to get in touch.

</comment></comment></comment><comment><USER9> This is a good niche with the rise of serverless as there isn't a super simple way to do this if you want to hit some job every so often. You could set a header key that all requests require and bam, you got security for an ongoing task. I use kue.js myself with jobs for node and glue stuff together with redis across servers but serverless seems like it could work for a lot of use cases coupled with this idea with less code overall.

<comment><AUTHOR> Thanks sebringj! I agree and the rise of serverless was definitely one of the motivating factors to me building this service. Security is handled through signing the payload that includes a timestamp with an HMAC to prevent replay attacks as well.

<comment><USER9> i wonder how you ensure its 24-7 reliable? i am definitely interested in using it

<comment><USER9> ah ok -> [https://aws.amazon.com/batch/](https://aws.amazon.com/batch/) that's probably how

</comment></comment></comment></comment><comment><USER10> How do you compare/compete against something like Azure Scheduler which gives you 500 jobs with unlimited executions (max 1 per minute) for $14/month.

The Azure service also supports logging (60 days), retry options, and advanced recurrence schedules (every other Tue, etc.)?

<comment><USER5> Hey. That's interesting. I'll answer as well since coincidentally at same time as OP I just built a recurring webhook scheduler[1].

That Azure pricing is very competitive / cheap! That's 22 million requests per month for USD 14.

1 timer on my service can do max request 1 per second (2.6 million per month) and that 1 timer would cost $26.

On my biggest plan you can get 4000 timers and 100 million requests per month for $495.

Still Azure is beating my prices by about 10 times. But they can't do second resolution (yet?), so they're still only a "distributed cron".

</comment></comment><comment><USER11> Nice simple solution to a sure problem. Kudos, and good luck!

Lately, we've had great experience using Hangfire ([https://www.hangfire.io](https://www.hangfire.io)) for async job processing in .NET. It's an awesome piece of software.

</comment><comment><USER12> I use cloudwatch events, you can configure cron/frequency and define json to send out.

<comment><USER13> We use CloudWatch events too. It works really well but does require a bit more technical know-how and debugging than a simple cURL so there is that.

</comment></comment><comment><USER14> This is neat, nice work OP.

I haven't seen this mentioned yet but you could leverage Zapier.com for this type of work. They have a "schedule" app you can pipe it into an outgoing webhook to achieve the same result.

It's pretty powerful. for example, at work, we wanted to run a specific task at different times: nightly, after deploys, and using a slack command. This was all achieved using 3 Zaps (Zapier apps) and it took no more than 30 minutes for the whole thing.

</comment><comment><USER15> The status page is currently timing out for me: [https://status.posthook.io/](https://status.posthook.io/)

<comment><AUTHOR> Seeing the same, thanks. Looks like updown.io is having an issue with TLS. It also lives here if you wanted to take a look: [https://updown.io/fj8q](https://updown.io/fj8q)

</comment></comment><comment><USER16> That's something I will definitely use soon! Great simple idea indeed. Just one question regarding the criteria to come up with the current pricing tiers: isn't it a bit too expensive for an one-off scheduling? I mean, over time I could totally see this as a commodity service going by the same price of popular VPS offerings (in tandem: they increase it, you increase it too, prices go down yours goes too).

<comment><AUTHOR> Thank you! Pricing is still subject to change as I get more data from the market but the aim was to price it by the value it provides developers over just going off what the servers cost to run.

</comment></comment><comment><USER17> This is pretty brilliant and fits an immediate need we have. Signing up today.

<comment><AUTHOR> Thank you! I was surprised something like this didn't exist when I needed it. Feel free to use the live chat or email support for any specific questions you have.

</comment></comment><comment><USER18> This is perfect addition to a Serverless stack. Thank you.

</comment><comment><USER19> How does this compare to [http://atrigger.com](http://atrigger.com)?

</comment><comment><USER20> Nice work! I built [https://cronally.com](https://cronally.com) a few years ago for a similar use-case.

Keep it up!

</comment><comment><USER21> Cron-as-a-service?

<comment><USER22> Seems more like at-as-a-service?

</comment></comment><comment><USER23> How does it compare to [https://www.setcronjob.com](https://www.setcronjob.com)?

</comment><comment><USER24> Does anyone know of a free service that lets you run a few scheduled CURL requests each week?

<comment><USER25> Check EasyCron [https://www.easycron.com/](https://www.easycron.com/)

<comment><USER24> Awesome. Thank you

</comment></comment></comment><comment><USER26> We do something similar at Iron.io but with Docker containers.</comment>


Wake your dad with a text - <AUTHOR> http://www.hidaddyiloveyou.com/

<comment><USER2> Kirill, _come on!_ do you really need all these fb access rights to run your app?

"Access my basic information, Send me email, Post to my Wall, Access my data any time, Manage my advertisements, Access my profile information"

This is way too intrusive even to give it a one-time shot.

<comment><AUTHOR> Michal, you're right, I don't need all these rights. I asked for so many to see if people complain. So far you were the only one. A couple people clicked "Accept" w/out reading the permissions, and then when I told 'em just how much data I could grab, they were shocked to find out. So yes, I completely agree, the next time I will go with "Basic permissions" and that's enough for most purposes. Thank you for letting me know, appreciated!</comment></comment>


How to lie with statistics - <AUTHOR> http://books.google.com/books?id=5oSU5PepogEC

<comment><USER2> Step 1: Use them.

Step 2: There is no step 2. ;)

All statistics "lie" to some degree. All data leaves out factors that might influence the data.</comment>


Regexper: Beautiful regexp visualizations - <AUTHOR> http://www.regexper.com/ <comment><USER4> Hello, I'm the creator of this (trevmex is one of my co-workers).

I just want to thank everyone for the feedback so far, I am looking into the issues that have been brought up (they'll have to wait until this evening to be fixed though...I have a day job).

<comment><USER5> You should build a tool that is the inverse of this. Let people build regular expressions by creating the visual diagram (some sort of click and drag GUI).

<comment><USER6> I've always wondered why there doesn't already exist such a tool. Regex strings are so opaque and hard to read. We can't really fix that for code (short of building objects), but what we could fix is a better way to write them.

A graphical tool to create them, built of units instead of just strings, and the output could still be a regex string.

<comment><USER7> KDE 3 had KRegExpEditor (<http://www.blackie.dk/KDE/KRegExpEditor/>) but it only recently found someone interested in porting it to KDE 4.

</comment></comment></comment></comment><comment><USER8> Just a thought- on the page if the author showed a few example regexes that would be really useful. Rubular does a great job of having a 'key' at the bottom, so I don't have to remember (or look up) anything.

<comment><USER9> This is a good point. Also mentioning which flavour of regexp is supported would be nice.

<comment><USER10> The input bar says Javascript style before you start typing something in..

<comment><USER8> Since I do mostly back-end stuff, I'm quite bad at Javascript regexes off the top of my head :)

</comment></comment></comment></comment><comment><USER11> This is very cool! If you put enter this regexp, you get a visualization of a Telnet protocol lexer.

      (?:[^\xFF\x0D]+|\x0D(?:\x0A|\x00)|\xFF(?:[\xFA\xFB\xFC\xFD\xFE].|\xFF|.))*

The graphic makes it obvious why Telnet requires NUL to follow any bare CR's: if it didn't, you would need a byte of lookahead!

</comment><comment><USER12> This website is awesome! Instantly bookmarked.

Suggestions

1) it would be really nice if you could allow the user to share a regex with a friend. IE <http://www.regexper.com/shared/some_unique_slug> will display my saved regex

2) Add a Favicon

Great work thanks for creating this

<comment><USER13> Why not simply the regex itself in the url instead of some unique slug? - <USER2> need to store state on the server

<comment><USER14> I'd avoid it to prevent url encoding problems.

<comment><USER15> You could base64 it first.

<comment><USER16> With an effective maximum URL length of 2000 characters, you'd be stuck with a regex limit of around 1300 characters once you take into account the overhead of base64 encoding.

<comment><USER15> I'm pretty sure this webapp isn't capable of rendering any regex with 1300 character anyway (or if it could, the client couldn't handle that much SVG). I don't expect URL length would be the limiting factor here.

</comment></comment></comment></comment></comment></comment><comment><USER17> Interesting concept. As with all regex related online tools, it is important to test exhaustively before trusting it.

Real-time character-by-character output would be important. If you are working on a long regex expression real-time output can help you think like the FSA and see how it behaves.

Here's an example of this tool failing:

        ([^<]*)<\/a>

<http://gskinner.com/RegExr/?339m7>

or

<http://rubular.com/r/pCXZs0DnSS>

take your pick, same results.

If you remove the "?" from the regex you'll get different results.

One version selects all anchor tags individually and make a selection group of their contents and the other version ends-up selecting everything between the first opening anchor tag and the last closing anchor tag into the selection group except for "<" characters.

The graph in Regexper does not show any difference between the two expressions even when any one of these forms is used:

        #([^<]*)#msi

        #([^<]*)#gsi

        /([^<]*)<\/a>/msi

        /([^<]*)/gsi

It looks like the flags are not being processed.

Also, please change the textbox to a fixed width font.

In addition to that, regex authoring tools, ultimately, are only useful if you can also enter some input text and see the result, preferably in real-time.

Other than that, it's an interesting concept.

<comment><USER18> With regards to the real-time analysis, you might find perl's Regexp::Debugger fun to use:

[http://search.cpan.org/~dconway/Regexp- Debugger-0.001011/lib...](http://search.cpan.org/~dconway/Regexp- Debugger-0.001011/lib/Regexp/Debugger.pm)

That's the documentation for using the module in your code. If you're interested in the standalone tool, you'll want rxrx:

[http://search.cpan.org/~dconway/Regexp- Debugger-0.001011/bin...](http://search.cpan.org/~dconway/Regexp- Debugger-0.001011/bin/rxrx)

</comment></comment><comment><USER19> Nice, but would be nicer if it had a couple of samples to show what the site can do for you.

</comment><comment><USER20> This is great. Has this approach to visualizing loops and structures been tried with general programs? Indentation and color coding seems to be the limit to the visual expressiveness in most IDEs because they tend to stay withing the format of plain text files.

I'd really like to see some more creative visuals like this for making program structure visible.

<comment><USER21> I don't know if there's a name for this style of display, but it's the same as the way JSON is depicted on its official site. <http://www.json.org>

Edit: It's called a Railroad Diagram.

<comment><USER22> No, it's called a finite state machine ;)

<comment><USER21> The FSM is the model, the Railroad Diagram is the view, to borrow terms.

<comment><USER23> Well, as long as we're picking nits, an FSM accepts only regular languages, while a rail road diagram accepts context-free languages[1]. So a railroad diagram isn't really the "view" for an FSM. A state-diagram or state- transition table is (although the whole MVC analogy is itself kind of weird. Where's the controller?).

[1] <http://en.wikipedia.org/wiki/Syntax_diagram>

<comment><USER24> >Where's the controller?

The MV* family of patterns seems to be okay without the notion of a controller.

</comment></comment></comment></comment></comment></comment><comment><USER25> looks neat. can you please make the regex textbox use a fixed width font, reading a regex is already hard enough, let alone when it is in Verdana

</comment><comment><USER26> Pretty! I had wanted to make something like this, but one feature I thought of and you didn't do: For the entered regexp, generate 20 random matches and show them to me.

Use case for this feature: You encounter an undocumented regexp in someone else's code and want get a few quick examples of what it matches.

<comment><USER27> Since I'm usually starting with a bunch of data I want to parse/match/chop, I like going the other way - pasting a couple of lines into Patterns ( <http://krillapps.com/patterns/> ) and tweaking my expression until it looks right. This would be a useful addition to help debug in place.

</comment></comment><comment><USER28> I tried the most complicated regular expression in the world:

<http://ex-parrot.com/~pdw/Mail-RFC822-Address.html>

and the server 500'd. Just joking, it's a very good tool.

P.S. it crashed with three nested groups (((a*)))

<comment><USER17> As an aside. I quit using regex to validate email addresses a while ago. Part of it is that a regex that large is simply incomprehensible and very difficult to maintain and fix if something is broken. The best solution, in my opinion, is a state machine based analyzer that checks the email address character-by- character and confirms compliance with RFC5322. This would also include checking DNS for MX records (which might require following CNAME to find it).

<comment><USER29> No, stop validating emails. Not only it's error prone and will frustrate legit users but it utterly pointless.

<comment><USER30> The thing is, email input validation is only one use case for regex and emails. Here's a more sinister one: you want to build a web scraper looking for emails to add to your spam list. Put a bit more generically, you need a script that can import email addresses from a broad and unknown host of formats, and it is impractical to condition the data beforehand.

I agree that the value derived from email validation for something like a new account registration is almost nil. Just do something like an email verification round trip, which not only validates the email _could_ be real, but also provides assurance that the user has control of the email address used.

Note: My last name is O'Kelley, which is a bit of a pain when it comes to poorly designed computer services. Some sites will strip the ' and others will escape it so that I become Mr. O\'Kelley. The worst I've seen is that my school used my complete last name in my email address, so most sign up forms refuse to even try to accept it, even though it is a valid email address. It can be a pain if I need to use my .edu address for academic discounts or verification.

<comment><USER17> Landing page. Visitor accidentally enters invalid email address and clicks send.

No validation = Gone. You lost them. You can't email them for a correction.

With validation = You catch the issue before the visitor leaves and you ask them to fix it.

Sure, it doesn't verify the 1 to 1 relationship between the email and the person. That requires a round-trip verification. I get it. At least you ensure that it isn't all garbage-in to begin with.

The other aspect of email verification is that you don't have to choose to bug the user with the results. Depending on what it is, if someone enters an obviously junky address you can simply tag that email as potential crud in your database. Someone would then manually look at these every so often for cleanup or re-categorization.

I don't like the idea of looking signups or customers in a transaction where both parties are interested in transferring the information accurately. That's a use-case where validation works well.

Now, regarding your last name. The issue is cause by programmers who simply go around grabbing code off the internet without vetting it in any way. There are email "validation" regex expressions out there that are horribly wrong, yet people post them on blogs and others use them without question. It's unfortunate.

</comment></comment></comment></comment></comment><comment><USER31> Useful. Thanks to this, found a bug in regex I did just today.

`/[^0-9^\\+]+/` -> `/[^0-9\\+]+/` # Thought that I needed to negate "+" as well.

<comment><USER32> You typically don't need to escape a + inside a character class, either, so /[^0-9+]+/ should work.

<comment><USER31> Thanks! :)

</comment></comment></comment><comment><USER33> Be nice to be able to include the regex in the url so I could link it in documentation or have some sort of rest interface.

</comment><comment><USER34> Breaks for:

      ^([^\W]+)@((?:[a-zA-Z0-9-]+\.)+(?:org|com|net|gov))$

Though this works:

      ^([^\W]+)@([a-zA-Z0-9-]+\.)+(?:org|com|net|gov)$

And it doesn't seem to be the nested groups that are screwing it up because this works:

      ((?:tinker|tailor)+(?:soldier|spy))

<comment><USER35> This is the minimal breaking input:

        ^(([a-z]+)+)$

</comment></comment><comment><USER36> Truly brilliant. Please add some examples, so I don't have to find some to impress people with.

</comment><comment><USER37> Nice! Small glitch: shows the visualization twice if enter is pressed twice quick enough.

</comment><comment><USER38> That's really neat.

My only suggestion is that it would be nice to have each regex diagram be separately addressable, by either putting the regex as a query param and yielding a raw image/svg/whatever, or by using a shortened url like a gist after saving.

Very nice work.

</comment><comment><USER39> The parser is failing for regexps containg the caret or dollar chars:

<http://cl.ly/image/3q0B2X1F3i1H>

<http://cl.ly/image/171f172x0P1M>

<comment><USER40> You need to remove the slashes from that example

<comment><USER39> Ha, thanks. Old habits die hard.

</comment></comment></comment><comment><USER41> To try it out, I made up the regex "car[pet] cleaner", but what I really meant was "car(pet)? cleaner". When I visualized it, I instantly noticed the bug. It's an indicator that this can turn out to be very useful.

</comment><comment><USER42> it breaks with this: (?:(?:\r\n)?[ \t]) _(?:(?:(?:[^() <>@,;:\\\".\\[\\] \000-\031]+(?:(?:(?:\r\n)?[ \t] )+|\Z|(?=[\\["()<>@,;:\\\".\\[\\]]))|"(?:[^\"\r\\\\]|\\\\.|(?:(?:\r\n)?[ \t]))_"(?:(?: \r\n)?[ \t]) _)(?:\\.(?:(?:\r\n)?[ \t])_ (?:[^()<>@,;:\\\".\\[\\] \000-\031]+(?:(?:( ?:\r\n)?[ \t])+|\Z|(?=[\\["()<>@,;:\\\".\\[\\]]))|"(?:[^\"\r\\\\]|\\\\.|(?:(?:\r\n)?[ \t])) _"(?:(?:\r\n)?[ \t])_ )) _@(?:(?:\r\n)?[ \t])_ (?:[^()<>@,;:\\\".\\[\\] \000-\0 31]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\\["()<>@,;:\\\".\\[\\]]))|\\[([^\\[\\]\r\\\\]|\\\\.) _\ ](?:(?:\r\n)?[ \t])_ )(?:\\.(?:(?:\r\n)?[ \t]) _(?:[^() <>@,;:\\\".\\[\\] \000-\031]+ (?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\\["()<>@,;:\\\".\\[\\]]))|\\[([^\\[\\]\r\\\\]|\\\\.)_\\](?: (?:\r\n)?[ \t]) _))_ |(?:[^()<>@,;:\\\".\\[\\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z |(?=[\\["()<>@,;:\\\".\\[\\]]))|"(?:[^\"\r\\\\]|\\\\.|(?:(?:\r\n)?[ \t])) _"(?:(?:\r\n) ?[ \t])_ ) _\ <(?:(?:\r\n)?[ \t])_(?:@(?:[^()<>@,;:\\\".\\[\\] \000-\031]+(?:(?:(?:\ r\n)?[ \t])+|\Z|(?=[\\["()<>@,;:\\\".\\[\\]]))|\\[([^\\[\\]\r\\\\]|\\\\.) _\\](?:(?:\r\n)?[ \t])_ )(?:\\.(?:(?:\r\n)?[ \t]) _(?:[^() <>@,;:\\\".\\[\\] \000-\031]+(?:(?:(?:\r\n) ?[ \t])+|\Z|(?=[\\["()<>@,;:\\\".\\[\\]]))|\\[([^\\[\\]\r\\\\]|\\\\.)_\\](?:(?:\r\n)?[ \t] ) _))_ (?:,@(?:(?:\r\n)?[ \t]) _(?:[^() <>@,;:\\\".\\[\\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\\["()<>@,;:\\\".\\[\\]]))|\\[([^\\[\\]\r\\\\]|\\\\.)_\\](?:(?:\r\n)?[ \t])* )(?:\\.(?:(?:\r\n)?[ \t]) _(?:[^() <>@,;:\\\".\\[\\] \000-\031]+(?:(?:(?:\r\n)?[ \t] )+|\Z|(?=[\\["()<>@,;:\\\".\\[\\]]))|\\[([^\\[\\]\r\\\\]|\\\\.)_\\](?:(?:\r\n)?[ \t]) _))_ ) _:(?:(?:\r\n)?[ \t])_ )?(?:[^()<>@,;:\\\".\\[\\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+ |\Z|(?=[\\["()<>@,;:\\\".\\[\\]]))|"(?:[^\"\r\\\\]|\\\\.|(?:(?:\r\n)?[ \t])) _"(?:(?:\r \n)?[ \t])_ )(?:\\.(?:(?:\r\n)?[ \t]) _(?:[^() <>@,;:\\\".\\[\\] \000-\031]+(?:(?:(?: \r\n)?[ \t])+|\Z|(?=[\\["()<>@,;:\\\".\\[\\]]))|"(?:[^\"\r\\\\]|\\\\.|(?:(?:\r\n)?[ \t ]))_"(?:(?:\r\n)?[ \t]) _))_ @(?:(?:\r\n)?[ \t]) _(?:[^() <>@,;:\\\".\\[\\] \000-\031 ]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\\["()<>@,;:\\\".\\[\\]]))|\\[([^\\[\\]\r\\\\]|\\\\.)_\\]( ?:(?:\r\n)?[ \t]) _)(?:\\.(?:(?:\r\n)?[ \t])_ (?:[^()<>@,;:\\\".\\[\\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\\["()<>@,;:\\\".\\[\\]]))|\\[([^\\[\\]\r\\\\]|\\\\.) _\\](?:(?:\r\n)?[ \t])_ )) _\ >(?:(?:\r\n)?[ \t])_)|(?:[^()<>@,;:\\\".\\[\\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\\["()<>@,;:\\\".\\[\\]]))|"(?:[^\"\r\\\\]|\\\\.|(?:(?:\r\n)? [ \t])) _"(?:(?:\r\n)?[ \t])_ ) _:(?:(?:\r\n)?[ \t])_ (?:(?:(?:[^()<>@,;:\\\".\\[\\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\\["()<>@,;:\\\".\\[\\]]))|"(?:[^\"\r\\\\]| \\\\.|(?:(?:\r\n)?[ \t])) _"(?:(?:\r\n)?[ \t])_ )(?:\\.(?:(?:\r\n)?[ \t]) _(?:[^() <> @,;:\\\".\\[\\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\\["()<>@,;:\\\".\\[\\]]))|" (?:[^\"\r\\\\]|\\\\.|(?:(?:\r\n)?[ \t]))_"(?:(?:\r\n)?[ \t]) _))_ @(?:(?:\r\n)?[ \t] ) _(?:[^() <>@,;:\\\".\\[\\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\\["()<>@,;:\\\ ".\\[\\]]))|\\[([^\\[\\]\r\\\\]|\\\\.)_\\](?:(?:\r\n)?[ \t]) _)(?:\\.(?:(?:\r\n)?[ \t])_ (?:[^()<>@,;:\\\".\\[\\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\\["()<>@,;:\\\".\\[ \\]]))|\\[([^\\[\\]\r\\\\]|\\\\.) _\\](?:(?:\r\n)?[ \t])_ )) _|(?:[^() <>@,;:\\\".\\[\\] \000- \031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\\["()<>@,;:\\\".\\[\\]]))|"(?:[^\"\r\\\\]|\\\\.|( ?:(?:\r\n)?[ \t]))_"(?:(?:\r\n)?[ \t]) _)_ \<(?:(?:\r\n)?[ \t]) _(?:@(?:[^() <>@,; :\\\".\\[\\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\\["()<>@,;:\\\".\\[\\]]))|\\[([ ^\\[\\]\r\\\\]|\\\\.)_\\](?:(?:\r\n)?[ \t]) _)(?:\\.(?:(?:\r\n)?[ \t])_ (?:[^()<>@,;:\\\" .\\[\\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\\["()<>@,;:\\\".\\[\\]]))|\\[([^\\[\ ]\r\\\\]|\\\\.) _\\](?:(?:\r\n)?[ \t])_ )) _(?:,@(?:(?:\r\n)?[ \t])_ (?:[^()<>@,;:\\\".\ [\\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\\["()<>@,;:\\\".\\[\\]]))|\\[([^\\[\\]\ r\\\\]|\\\\.) _\\](?:(?:\r\n)?[ \t])_ )(?:\\.(?:(?:\r\n)?[ \t]) _(?:[^() <>@,;:\\\".\\[\\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\\["()<>@,;:\\\".\\[\\]]))|\\[([^\\[\\]\r\\\\] |\\\\.)_\\](?:(?:\r\n)?[ \t]) _))_ ) _:(?:(?:\r\n)?[ \t])_ )?(?:[^()<>@,;:\\\".\\[\\] \0 00-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\\["()<>@,;:\\\".\\[\\]]))|"(?:[^\"\r\\\\]|\\\ .|(?:(?:\r\n)?[ \t])) _"(?:(?:\r\n)?[ \t])_ )(?:\\.(?:(?:\r\n)?[ \t]) _(?:[^() <>@, ;:\\\".\\[\\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\\["()<>@,;:\\\".\\[\\]]))|"(?:[^\"\r\\\\]|\\\\.|(?:(?:\r\n)?[ \t]))_"(?:(?:\r\n)?[ \t]) _))_ @(?:(?:\r\n)?[ \t])* (?:[^()<>@,;:\\\".\\[\\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\\["()<>@,;:\\\". \\[\\]]))|\\[([^\\[\\]\r\\\\]|\\\\.) _\\](?:(?:\r\n)?[ \t])_ )(?:\\.(?:(?:\r\n)?[ \t]) _(?:[ ^() <>@,;:\\\".\\[\\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\\["()<>@,;:\\\".\\[\\] ]))|\\[([^\\[\\]\r\\\\]|\\\\.)_\\](?:(?:\r\n)?[ \t]) _))_ \>(?:(?:\r\n)?[ \t]) _)(?:,\s_ ( ?:(?:[^()<>@,;:\\\".\\[\\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\\["()<>@,;:\\\ ".\\[\\]]))|"(?:[^\"\r\\\\]|\\\\.|(?:(?:\r\n)?[ \t])) _"(?:(?:\r\n)?[ \t])_ )(?:\\.(?:( ?:\r\n)?[ \t]) _(?:[^() <>@,;:\\\".\\[\\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[ \\["()<>@,;:\\\".\\[\\]]))|"(?:[^\"\r\\\\]|\\\\.|(?:(?:\r\n)?[ \t]))_"(?:(?:\r\n)?[ \t ]) _))_ @(?:(?:\r\n)?[ \t]) _(?:[^() <>@,;:\\\".\\[\\] \000-\031]+(?:(?:(?:\r\n)?[ \t ])+|\Z|(?=[\\["()<>@,;:\\\".\\[\\]]))|\\[([^\\[\\]\r\\\\]|\\\\.)_\\](?:(?:\r\n)?[ \t]) _)(?:\\.(?:(?:\r\n)?[ \t])_ (?:[^()<>@,;:\\\".\\[\\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+| \Z|(?=[\\["()<>@,;:\\\".\\[\\]]))|\\[([^\\[\\]\r\\\\]|\\\\.) _\\](?:(?:\r\n)?[ \t])_ )) _|(?: [^() <>@,;:\\\".\\[\\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\\["()<>@,;:\\\".\\[\ ]]))|"(?:[^\"\r\\\\]|\\\\.|(?:(?:\r\n)?[ \t]))_"(?:(?:\r\n)?[ \t]) _)_ \<(?:(?:\r\n) ?[ \t]) _(?:@(?:[^() <>@,;:\\\".\\[\\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\\[" ()<>@,;:\\\".\\[\\]]))|\\[([^\\[\\]\r\\\\]|\\\\.)_\\](?:(?:\r\n)?[ \t]) _)(?:\\.(?:(?:\r\n) ?[ \t])_ (?:[^()<>@,;:\\\".\\[\\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\\["()<> @,;:\\\".\\[\\]]))|\\[([^\\[\\]\r\\\\]|\\\\.) _\\](?:(?:\r\n)?[ \t])_ )) _(?:,@(?:(?:\r\n)?[ \t])_ (?:[^()<>@,;:\\\".\\[\\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\\["()<>@, ;:\\\".\\[\\]]))|\\[([^\\[\\]\r\\\\]|\\\\.) _\\](?:(?:\r\n)?[ \t])_ )(?:\\.(?:(?:\r\n)?[ \t] ) _(?:[^() <>@,;:\\\".\\[\\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\\["()<>@,;:\\\ ".\\[\\]]))|\\[([^\\[\\]\r\\\\]|\\\\.)_\\](?:(?:\r\n)?[ \t]) _))_ ) _:(?:(?:\r\n)?[ \t])_ )? (?:[^()<>@,;:\\\".\\[\\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\\["()<>@,;:\\\". \\[\\]]))|"(?:[^\"\r\\\\]|\\\\.|(?:(?:\r\n)?[ \t])) _"(?:(?:\r\n)?[ \t])_ )(?:\\.(?:(?: \r\n)?[ \t]) _(?:[^() <>@,;:\\\".\\[\\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\\[ "()<>@,;:\\\".\\[\\]]))|"(?:[^\"\r\\\\]|\\\\.|(?:(?:\r\n)?[ \t]))_"(?:(?:\r\n)?[ \t]) _))_ @(?:(?:\r\n)?[ \t]) _(?:[^() <>@,;:\\\".\\[\\] \000-\031]+(?:(?:(?:\r\n)?[ \t]) +|\Z|(?=[\\["()<>@,;:\\\".\\[\\]]))|\\[([^\\[\\]\r\\\\]|\\\\.)_\\](?:(?:\r\n)?[ \t]) _)(?:\ .(?:(?:\r\n)?[ \t])_ (?:[^()<>@,;:\\\".\\[\\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z |(?=[\\["()<>@,;:\\\".\\[\\]]))|\\[([^\\[\\]\r\\\\]|\\\\.) _\\](?:(?:\r\n)?[ \t])_ )) _\ >(?:( ?:\r\n)?[ \t])_)) _)?;\s_ )

<comment><USER43> To anyone that sees this email validation regex, DO NOT USE IT. Hope that was clear, if not obvious.

Use something like `^([^\s]*)@([^\s]*\\.[^\s]*)$` which will do most of the work for you, then check second group for common domain typos, and what have you.

<comment><USER17> >Use something like `^([^\s]*)@([^\s]*\\.[^\s]*)$` which will do most of the work for you

I don't understand. How does this expression do anything even remotely close to email validation?

For example, how does it tell you that:

        These are valid:
          email@example.com
          <comment>~@nasa.gov
          firstname.lastname@example.com
          email@example.com

and that:

        These are NOT valid
          email@example.com    (no MX RR)
          test@-nasa.gov
          test"@nasa.gov
          firstname.lastname@example.com-
          test
          email@example.com
          email@example.com..uk
          firstname.lastname@example.com
          email@example.com

You'd have to write all the validation logic yourself all over again. And that's just a few examples.

Barring anything else, the RFC822 expression isn't so bad that someone should replace it with the kind of thing you are suggesting.

Sorry if I don't see it.

<comment><USER44> Honestly I don't understand why people get all flustered over email validation, I would probably use something along the lines of this just to check that the email address is along the lines of firstname.lastname@example.com, obviously this could do with a little tweaking.

The best way to validate an email address is to send an email to whatever address is supplied to you, if it is a true email address the user will receive an email and it will be validated, if not then their account or query will go unused/unanswered and that will be down to them.

<comment><USER17> > Honestly I don't understand why people get all flustered over email > validation

Multiple reasons, and, yes, context is important.

Landing Page: You have one, and ONLY ONE, opportunity to capture a potential new customer's contact info. If they make a mistake entering their email and you didn't catch it you'll loose them forever. You can't send an email to let them know they entered two periods by mistake, can you? They are gone and you screwed-up.

Every single potential customer is sacred. Thou shalt not loose them by being careless.

Forum signup: In general terms, if someone is visiting a forum it probably means that they want to sign-up. In this case, it is OK to make them enter their address twice, make sure they match and send them a confirmation email. They'll probably try to log-on later on and discover something went wrong and re-register.

While I said "that's OK", I also think it is bad form not to at least do enough validation of all input data, including email, to catch innocent mistakes. I think people who are against email validation might have that position because they don't understand it or gat bitten by a crappy regex expression and that is that.

Now your forum sign-up user is angry because they have to enter all of their information again and go through the process one more time. Who knows, they might make a mistake once again. While I don't have any data to back this up I would venture to guess that the drop-off rate for making a visitor enter all of their data multiple times is significant.

Payment Confirmation: Must check as much as you can.

From my vantage point taking ANY action that might loose or annoy a visitor is simply --to be kind-- programming. There's no excuse for that in my book.

</comment></comment></comment></comment></comment></comment><comment><USER45> Unfortunately it seems to be broken now, but reAnimator was an awesome regex visualization tool (<http://osteele.com/tools/reanimator/>). It let you enter a regex and would display the state machine similar to this, but then it would also let you enter a string and it would animate the progress of the string matching (or not) against the state machine as you enter each character.

</comment><comment><USER46> Nice! Like Edi Weitz's "The Regex Coach", but Web-based. It might be worth looking at TRC's way of illustration of the regex matching a specific string.

</comment><comment><USER47> Pretty neat, although I have a nitpick with your choice of a block labeled "None of:" in for example this regex: /foo(.*)[^0-9]+bar/

It ends up saying "none of [0-9]" when that's not actually a good english description of what should go there. It should say something equivalent to "one or more things that are not [0-9]". The empty string fits the description "none of [0-9]" but does not match that part of the regex.

</comment><comment><USER48> Hey, this is really cool, and a great visual language for regular expressions! Must have been fun to build.

I found a couple of bugs: it doesn't seem to handle `?>` and inline modifiers (<http://www.regular-expressions.info/modifiers.html>), and with a couple of expressions it simply gave me a server error with no further information.

</comment><comment><USER49> Doug Crockford would love this tool (despite disliking regexex in general). He uses a railroad diagram of a regex on page 67 of _Javascript: The Good Parts_ to walk through a regex that validates URLs.

[http://books.google.com/books?id=PXa2bby0oQ0C&pg=PA67...](http://books.google.com/books?id=PXa2bby0oQ0C&pg=PA67&lpg=PA65&ots=HIqon2w7iK&dq=doug+crockford+regex)

</comment><comment><USER50> Nice. I wrote something similar a while back (with the goal to show lint errors): regexp.quaxio.com

Note: if you type [b-a], you fail to say it's an invalid range.

<comment><USER50> (a)(a)(a)(a)(a)(a)(a)(a)(a)(a)\10 incorrectly gets parsed as ... + back reference to group 1 + "0".

</comment></comment><comment><USER51> Its beautiful to see the "check if number is a prime" regex as a diagram:

        ^1?$|^(11+?)\1+$

I noticed two glitches:

1) It gives me the same output image for these two different regexes:

        ^[a-x]*yo$
        ^[a-x]+yo$

2) It gives me a server error for this:

        a(b*(c*(d*)*)*)

Edit: I was wrong about glitch 1. See below.

<comment><USER52> >Gives me the same output image for these two different regexes:

> ^[a-x]* yo$

> ^[a-x]+yo$

No, it does not.

When you do (something)* , pay attention to the additional path that is created that circumvents the (something) box. With (something)+, there is no such path.

<comment><USER51> Aaaahh! Yes! Awesome!

</comment></comment></comment><comment><USER53> Chokes with a "Server error" on the following URL validator:

\b(?:((?:https?|mailto|s?ftp): )(?:\/{1,3} |[a-z0-9%]) |w{2,3}\d{0,3}[.]|[a-z0-9\\.\\-]{1,40}[.][a-z]{2,4}\/)(?: \&[a-z]{2,8}\; | [\w\\(\\)\\.\/\:\@\\#\?\=\&\\-\\!\~\;\'\\[\\]] | \%[0-9]{2})+

This is after changing ?> to ?: since it doesn't understand the non- backtracking syntax.

</comment><comment><USER54> The following valid js regexp gives a server error: ^/([^/]+)[/]?(_((site)?(edit|show|manager))(/|$))?([\s\S]*)

<comment><USER55>
      [\s\S]*

Doesn't this match every possible character, 0 to infinity times?

<comment><USER56> It's an idiom for `.*` when you want `.` to mean any character, including newlines. `.` never matches a newline in JS regexes.

</comment></comment></comment><comment><USER57> +1 for indicating what type of regular expression it is.

Also, this would be a very useful tool in a Formal Automata class.

</comment><comment><USER58> Fails on non-ASCII characters.

</comment><comment><USER59> Great tool! I also find the title funny (which referes to following quote by Jamie Zawinski: Some people, when confronted with a problem, think "I know, I'll use regular expressions." Now they have two problems.)

</comment><comment><USER60> It would be great if it started with a Regex example in the text box to instantly see what the tool is capable of, that or include a simple 'show me a demo' button, which adds a example regex and queries it.

</comment><comment><USER61> very nice! makes me wonder, would it be helpful to have a natural english style language to express regular expressions in? Since the diagrams say "one of" and things like that, it makes it so much clearer. Maybe a library that let you say the same things would help too, since I think the ultra- hieroglyphics are part of what makes regex so confusing. Nothing very ambitious, just changing dots and stars and weird parentheticals to english words so when you read a regular expression, it reads more like the output of this site, which is nice.

</comment><comment><USER62> Definitely bookmarked, this is the easiest method i've found so far that actually _shows_ the exact rules of regex. It's infinitely more useful than a book.

</comment><comment><USER63> but it doesn't show what is matched - as in a example matched string - <USER3> is the most important thing

also how do you put in the case insensitive flag?

<comment><USER64> I think this would be quite valuable -- provide a second input for sample text and highlight the path taken. Bonus points for figuring out a way to handle multiple sample inputs, showing capturing group output, etc.

<comment><USER65> Extending this idea, a regex (code)coverage tool would be great...

</comment></comment></comment><comment><USER66> Heh, I was actually thinking about creating something that teaches Finite State Machines using something like this! Very cool. :)

</comment><comment><USER67> OK I'm missing something...if this post wasn't labeled "Beautiful regexp visualizations" I don't know if I would've poked around for more than a couple minutes (and I _love_ regexes)...perhaps the opening screen should by default have one of the more appealing examples? I hadn't known that I had to hit "Enter" for anything to happen...I still don't know what I'm supposed to be seeing, though I'll support anything that makes regexes more accessible.

</comment><comment><USER68> Really cool. Any chance you could add an export-as-<some vector graphics format> option?

<comment><USER69> Well the image already is a SVG element. But it's dynamically generated so you need a slight bit of trickery to grab it, enter this in your JS console: (tested with Opera and Chrome)

        document.location.href = 'data:image/svg+xml,'+encodeURIComponent(document.getElementById('paper-container').innerHTML)

it'll open just that element as a SVG document, where you can save it with ctrl-S.

I'd like to have used window.open instead of document.location so it wouldn't replace the RegExper tab, but that gets caught in my browsers' pop-up blockers :) Inserting it as a regular link in the original document would've been the best solution, in a pinch, I am offering this quick hack :)

Also, poking at this puzzle, I just noticed the _badass_ colour green he's using for the diagrams and header.

</comment></comment><comment><USER70> Cool! Though it would be great if you could have a shareable link to the regex visualized.

</comment><comment><USER71> This visualization looked particularly good to me:

\b((regexs)[=]+([AWESOME])(am|i|right|\??))?

</comment><comment><USER72> I wonder if the code behind this uses regexes on the entered regexes...

</comment><comment><USER73> ((((a))))

-> server error. Looks like 3 groups within each other are the limit

</comment><comment><USER74> nice tool. bookmarked and upvoted. :-) thank you.

Next level would be to turn this around and build a (nice) visual regexbuilder. The would be f __ __ing nice.

</comment><comment><USER75> Wow that's awesome.

</comment><comment><USER76> Requires www.

</comment><comment><USER77> Really nice.

</comment><comment><USER78> Can't seem to handle UTF-8. WTF?

</comment><comment><USER79> this statement triggers server error: ^(\d(\\.(\d(\\.(\d)?)?)?)?)?$

</comment><comment><USER80> It would be gorgeous if upon clicking enter you'd be redirected to an URL that you could share with someone.

Even better: make it directly a "tiny" URL.

I realize that then some kind of a database would be needed but it would be really sweet.

<comment><USER81> Or, you know, a good old query string.

</comment></comment><comment><USER82> bookmarked

</comment><comment><USER83> Neither beautiful nor a visualization really.

For the uninitiated, there's an isomorphic relationship between regular expressions (not PCREs which are way more complicated) and finite state automata proving that if you have a regexp you can generate a FSA for it, and visa versa.

What we have here is a system that generates a graph of the finite state automaton for any given regular expression.

Neat project, misleading headline.

edit: let me add more constructive comments. What is useful about things like rubular (<http://rubular.com/> ) is the ability to see how a Regexp behaves with a given input. Where a finite state chart can be useful is giving users a better impression of the internal workings of a particular regular expression. Where rubular can indicate where a match can be found, it would be cool to have a tool like this if you can show the path a particular input would take (and possibly fail out on) through a given regular expression.

<comment><USER84> This type of useless comment is why I tend to avoid HN for periods at a time. Congrats, you've shown the world you know what "isomorphic" and "FSA" means, but did you initially bring anything constructive? No. You had to edit in things _barely_ constructive because your impulse was to tear down rather than build up.

> Neither beautiful nor a visualization really.

This is the least helpful thing I can imagine. "Here's a beautiful regex visualizier" "LOL NO IT'S NOT". Geez.

<comment><USER69> > This type of useless comment is why I tend to avoid HN for periods at a > time.

There's useless pedants like this all over the Internet. You should probably wear a hat.

I'd agree if this were the top two comments without many disagreeing replies or down-votes.

I see this comment in down-voted grey, somewhere way down the page and assume a lack of social skills, bad-morning-before-coffee-grumps.

Or simply Internet Asshat Background Radiation, it's everywhere. Legends say you can trace its echos back to the First Flame from which the Internet was born.

<comment><USER84> It wasn't grey when I commented. I wouldn't take the time to get on a soapbox if the community had already buried it.</comment></comment></comment></comment>


Ask HN: need feedback on using Vultr VPS hosting for production? - <AUTHOR> I've seen a lot of basic reviews online but I'm looking for real production use feedback/review<p>I'm considering vultr, digitalocean, linode <comment><USER2> I have used digital ocean and vultr. From a technical review I used vultr and digital ocean differently. On vultr I ran some large Windows servers. For digital ocean I run all sorts of things.

In my experience, Digital Ocean's support is much much better than Vultr. Towards the end of my used of Vultr I had some pricing disputes for some IP address blocks.

On the other hand Digital Ocean gave me account credit when I reached out.

I can't speak for linode.

<comment><USER3> Good to see a first hand review. Just checking if anybody in the forum has used and/or considered server4you.com. Their prices and server specifications look very competitive</comment></comment>


No more tenures for schools in California - <AUTHOR> http://www.cnn.com/2014/06/10/justice/california-teacher-tenure-lawsuit/index.html?hpt=hp_t1

<comment><USER2> Great news! I hope this is upheld. Politicians and union leaders rush to defend awful teachers without worrying about the consequences.</comment>


Please Review My New Site - <AUTHOR> http://gopollgo.com/the-company-you-choose-remains-the-other-ceases-to-exist-you-decide-to-live-with-

<comment><USER2> Very nicely done. Nice and fairly clean looking site. Your animations upon answering are nice.

I wasn't familiar with PollDaddy, so know nothing about this market, but on their homepage, they don't have a poll. They don't show me what it is like from a user perspective.

I think you've done a nice job with the social sharing aspect. The widget is key with many brands, as is either a good look, or the ability for them to style it. The easier it is for them to set-up a poll and stick it onto their site, the better.

I think you need to find a place that isn't using polls, yet where they would be a great fit. Engadget is always doing these "how would you change" [http://www.engadget.com/2010/12/31/how-would-you-change- webo...](http://www.engadget.com/2010/12/31/how-would-you-change-webos-2-0/) posts. The content fits into a poll, but they need the ability for their audience to suggest what the polling options should be.

I never read those posts because it is just going through a bunch of comments, and you don't get any idea of what the cons are of the device. That could be very valuable information to the manufacturer. Of course, they'd still need comments, so hooking up with the disqus api would be a good idea.

Just an idea.

<comment><USER3> > I wasn't familiar with PollDaddy, so know nothing about this market, but on > their homepage, they don't have a poll. They don't show me what it is like > from a user perspective.

gopollgo doesn't have one on their homepage either. The link is actually one to an existing poll. Try <http://gopollgo.com> and you'll see.

I do think you should add a poll on the homepage or at least a screenshot of one to draw users in.

I love the site though. The UX is fun and engaging.

<comment><AUTHOR> Thanks -- I'll definitely put one on the home page. Thanks for the feedback!

</comment></comment></comment><comment><USER4> love the analytics feature preview. The site rocks!

</comment><comment><USER5> Solid UI and simple concept. Great job!

</comment><comment><AUTHOR> Two others and I just launched <http://gopollgo.com> a few days ago and its starting to pick up a bit of steam! There are two parts to it: There is the free, consumer side of the site for anyone to use. People can create, vote and comment on polls. Think of giving your Twitter followers a poll to vote on rather than getting a bunch of @replies. All polls can be embedded as widgets where javascript is allowed. If anyone from Twitter is reading, we'd love to be able to embed polls that slide out pane as well ; )

Soon we will be launching a premium, paid version ( <http://gopollgo.com/images/marketing/analytics.png> ) where you can dive into your votes and segment data on a variety of mediums. The screenshot shows the USA, but we're collecting data for individual cities, counties, countries and will also have the ability to segment by arbitrary areas like political divisions, zip codes, or sales regions. Think of it as PollDaddy on steroids. I built this site because I was so fed up with PollDaddy. We hope to work with big brands on Twitter and websites with large amounts of traffic: Think FoxNews, CNN, and large technology/media blogs.

I also hope to release an API at one point so that websites can easily integrate polls into their websites.

All suggestions, comments, feedback, criticism more than welcome. I also would love advice for how to get this more publicity and users. Everyone seems to like it, but I just am having trouble getting it in front of people. I would appreciate it if you liked it, that you tell your Twitter / Facebook friends.</comment>


In 2 Months Lollipop will be the dominant Android SDK version - <AUTHOR> http://www.appbrain.com/stats/top-android-sdk-versions <comment><USER2> And yet their latest OS won't... Google and all of the OEM's need to get their act together and fix the OS update cycle...</comment>


German woman found a hidden camera in her California Airbnb rental - <AUTHOR> http://fusion.net/story/246334/hidden-camera-airbnb-lawsuit/?utm_source=twitter&utm_medium=social&utm_campaign=ThisIsFusion&hootPostID=0be4c13c423cc311274ed1b53005fd3c <comment><USER2> There have been several cases of hidden cameras filming guests at Airbnbs, prompting a blog post about a script to detect cameras and disconnect them from WiFi [0] that was posted here about a month ago [1].

0: [https://julianoliver.com/output/log_2015-12-18_14-39](https://julianoliver.com/output/log_2015-12-18_14-39)

1: [https://news.ycombinator.com/item?id=10763732](https://news.ycombinator.com/item?id=10763732)

</comment><comment><USER3> _"Airbnb takes privacy issues extremely seriously," said an Airbnb spokesperson via email._

Which is apparently why they let the couple who surreptitiously taped Mrs. Schumacher to continue to rent their apartment through their service.

</comment><comment><USER4> This might be in poor taste, not just because of my proposal but also because I'm using a pun combined with a Hacker News cliche, but it doesn't need to be a bug - _it can be a feature_.

A lot of people are exhibitionists, a lot of people are voyeurs. Just advertise that your Airbnb is encrusted with hidden cameras. Solved.

<comment><AUTHOR> Sounds like a tinder / airbnb mashup waiting to happen.</comment></comment>


The Things He Carried - <AUTHOR> http://www.theatlantic.com/magazine/archive/2008/11/the-things-he-carried/7057

<comment><USER2> It's worth noting that since this article was published (2008) the TSA has begun scanning boarding passes at the checkpoint (at least at some airports) and confirming the name that appears on their screen is also the name on the boarding pass and the ID. The 2D barcode on the pass is encrypted to prevent the Photoshop trickery.

But, it's only taken them years to even attempt to plug this gap. Schneier wrote about the issue in 2003[1] and a kid got his house raided by the FBI in 2006[2] for making a fake boarding pass generator. I wonder why they didn't raid Adobe's headquarters, I hear they produce a wonderful tool for forgery too...

[1]<http://www.schneier.com/crypto-gram-0308.html#6>

[2][http://www.boingboing.net/2006/10/28/fbi_returns_to_fake_.ht...](http://www.boingboing.net/2006/10/28/fbi_returns_to_fake_.html)

<comment><USER3> It's not really a fix. It's nothing a convincing fake ID couldn't fix--then everything matches.

</comment></comment><comment><USER4> The worst part is that George Carlin's rants from the '90s can be taken, copy- pasted verbatim and still not lose poignancy.

" _Airport security is a stupid idea, it's a waste of money, and it's there for only one reason: to make white people feel safe!_ "

</comment><comment><USER5> There's a growing protest against the TSA's unconstitutional new searches:

Check out <http://www.wewontfly.com>

<comment><USER6> excellent point. there was also a revolt against the TSA back in 2004/2005 and i think this one is shaping up even louder. airlines, pilots, flight attendants, passengers, parents, and seniors are all on the same side here. and politically, beating up on the TSA is going to be popular on both sides of the aisle.

</comment></comment><comment><USER7> There's a reason this article could only be written by a Jeffery Goldberg and not an Ahmed Jalaluddin.

<comment><USER8> Please elaborate

<comment><USER6> My guess is that he meant that the TSA typically treats _certain people_ differently because they're threats, and adult white guys somewhat more deferentially than everybody else.

Goldberg has continued to blog and write about this topic. here's his most recent: "The War on Terror Meets the War on Pedophilia; Terror Wins" [http://www.theatlantic.com/national/archive/2010/11/the- war-...](http://www.theatlantic.com/national/archive/2010/11/the-war-on- terror-meets-the-war-on-pedophilia-terror-wins/66515/)

</comment></comment></comment><comment><AUTHOR> Its probably articles like this one that pushed TSA to go to its current extreme.

What I don't understand is that they are screening pilots for weapons, who once they pass the screening, are then given control of the plane, which itself can be used as a weapon.

Some have said we can trust those working at the airport not to place dangerous things on planes because they get background checks. But apparently the ones entrusted with a very big weapon, the pilots, have to be checked for small objects.

And, of course, these scanners don't do cavity checks, another way things can get on the plane. Are those coming next?

I'm just glad they have those bomb-proof plastic bags to contain exploding liquids that do get through security.

Of course, the real problem is that we are afraid to give more attention and ask a lot of questions of passengers that certain criteria. Seems all we know how to do is look up names on a list. Anything more complex is too much to handle.

As I understand it, Israeli security asks suspicious passengers a lot of questions about their trip and background. And they've had very few problems considering the hostile situation they're in.

[http://news.yahoo.com/s/ap/20101102/ap_on_re_mi_ea/ml_israel...](http://news.yahoo.com/s/ap/20101102/ap_on_re_mi_ea/ml_israel_airport_security)

<comment><USER9> _What I don't understand is that they are screening pilots for weapons, who once they pass the screening, are then given control of the plane, which itself can be used as a weapon._

Because there are often two or three crewmembers in the cockpit. If one of them is subverted and can carry weapons aboard, they can establish control over the cockpit by killing the other crew. By preventing weapons from entering the cockpit, one is preventing this scenario. The problem is that searches of the pilots are easy to circumvent. Simply have a terminal worker sneak in a weapon and give it to the pilot. If this ever happened, the reinforced cockpit doors would very much work against us. This is a very good reason to pay pilots much more than $19,000 a year!

<comment><AUTHOR> Yes, one pilot could kill the other. Or they could just do the deed while the other pilot is in the bathroom.

<comment><USER9> This only works if that pilot leaves behind the keys to the reinforced door, or if there's a way of disabling/jamming the lock.

</comment></comment></comment></comment><comment><USER10> From the article.. \-- I once asked Michael Chertoff, the secretary of Homeland Security, about this. "We actually ultimately do have a vision of trying to move the security checkpoint away from the gate, deeper into the airport itself, but there's always going to be some place that people congregate. So if you're asking me, is there any way to protect against a person taking a bomb into a crowded location and blowing it up, the answer is no." \--

The implications of this reality have to be considered. The TSA does not exist to protect passengers on the planes. The TSA exists to prevent terrorists from getting the planes themselves and using them as weapons. If terrorists wanted to kill you, the passenger, then they would indeed do so as you waited to be screened through security. I guess you can take some comfort in the fact that they don't hate you, personally, for your freedoms...

<comment><USER11> > We actually ultimately do have a vision of trying to move the security > checkpoint away from the gate, deeper into the airport itself

Like at Schiphol?

<comment><USER12> Or CDG or ATH or many other (if not most) European airports.

Screening at the gates makes a lot of sense in terms of traffic flow, as you don't get huge queues, and airline personnel hold the plane if there are security hold-ups. Also if there is a breach, you only have to shut down a few gates rather than the whole airport.

The problem is that this will br very difficult to retrofit in many existing US airports because they weren't designed with that in mind.</comment></comment></comment>


Whats 20% helped you most - <AUTHOR> I've been doing some thinking about the 20/80 rule, and was curious to see what others thought too! I spent about 1 year building up a product without getting any clients, did this happen to anyone else? I finally managed to transition and make the first sales when I focused on the following:
I'd like to preface this by saying that I was in charge of the business side of things and product design, but not development.
Looking back realised that my 20% was sales and lead gen: •Hiring Sales Development Representative to generate new leads. •Doing some lead gen myself. •Doing PPC based lead generation. •Looking for companies and targeting them personally.
If you guys also have good ideas on how I could generate more leads for our product that would be awesome https://www.parceltracker.com/internal-parcel-tracking
I want to move away from the PPC and paid acquisition channels! <comment><USER2> I have been putting at least 20% into dev/biz learning even when having full time employment. Sorta took a page from the 20% idea at Google and applied it to myself.

I have seen similar results, in capabilities and revenues (income). I advocate for it at places I contract, but, unfortunately, many never adopt and realize the long term gains.

So I would say, to answer your question, self/skill/career development has been the 20% with the best ROI

Maybe, somewhat to the opposite side (and morbidly), I have pondered as I make choices with my time, "is this something, when I'm on my death bed, that I will wish I spent more or less time doing?" So I stopped watching video based entertainment (tv, movies, Netflix, et al) and that may actually be far more significant of the choices, as it gave me the time to use for self improvement. Not sure where I'm going with this, food for thought anyhow.

<comment><AUTHOR> I feel like the 20% from google is very different to the 2080 rule / Pareto principle.

Oh, that hit deep: "is this something, when I'm on my death bed, that I will wish I spent more or less time doing?

I'm going to implement that into my life outside work.

</comment></comment><comment><USER3> OT: I literally mis-read 20% as %20 (space)--as in a URL escape.

<comment><AUTHOR> lol</comment></comment>


Ask HN: What does your team do to socialize now that we are fully remote? - <AUTHOR> These days that we are fully remote I'd like to know what other teams are doing to keep non-work-related social interactions alive. For example, we have arranged some online games and coffee meetings but I'm getting bored and would like to do something new. <comment><USER2> It's funny. I work from home as a distributed employee of a firm. Most of my other colleagues worked in an office until last week. Now it's been a boon for me socially! We've setup afternoon and morning coffee chats, and people are just more communicative in general with me. Sorry to say it, that it took a disaster like this to get some love for remote folks.

</comment><comment><USER3> My team didn't socialize before this and aren't socializing now either. In the office we didn't even sit together! We saw eachother at standups for 15-30 mins and then whenever we needed to work together.

Now it's a conferencing call for the 15-30 mins during the day and people can contact whoever they need to work with.

I'm not even sure how socializing in a conference call works. Aren't people speaking over each other and won't it just end up with the biggest talkers taking over? Not that that is too dissimilar to face-to-face interactions..</comment>


Google's Real Problem – GTD? - <AUTHOR> http://gigaom.com/2010/10/31/google%e2%80%99s-real-problem-%e2%80%93-gtd/?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+OmMalik+%28GigaOM%3A+Tech%29 <comment><USER2> Dup: <http://news.ycombinator.com/item?id=1855420>

And the link there doesn't have all the "feedburner" crap in the URL.</comment>


Sing to Me: Karaoke is self-compromise as spectacle - <AUTHOR> http://reallifemag.com/sing-to-me/ <comment><USER2> Singing at social gatherings is pretty much universal across human societies.

The real question isn't "Why is karaoke popular?" it's "Why is breaking into song at other bars frowned upon?".

<comment><USER3> In most social gatherings you have people who are at least vaguely talented singing alone or you have a group singing (maybe even most of the people present); Karaoke is different: it highlights individual lack of talent.

</comment></comment><comment><USER4> Hm - I think Karaoke is a chance to express yourself with the help of a ready- made backing track. Together with the reverb of the mic and an audience of friendlies, karaoke is your best singing self, not a self-compromise.

</comment><comment><USER5> I think "self-compromise as spectacle" is a recurring theme in a lot of Japanese pastimes -- from karaoke to Those Wacky Japanese Game Shows to just getting shitfaced with your coworkers (and boss!) on Fridays. Japanese people have few opportunities in their daily work and social lives to let their freak flag fly. The tension that comes from having to maintain your tatemae nearly 24/7 has got to be released somewhere and karaoke bars are one of the "safe spaces" created for that purpose.

<comment><USER6> Why target the Japanese with that? I've yet to meet a group of people that didn't act like that in some way, many did it quite a lot when they weren't having to represent something professionally... And some did it even then.

I worked at a company where the local manager would occasionally dress in a cow costume and let employees throw foam brick at his udder.

Sure, the Japanese "invented" and named karaoke, but it has been whole- heartedly adopted by so many peoples.

<comment><USER5> Karaoke is not nearly as prevalent in, for example, America as it is in Japan. Maybe Korea or the Philippines can come close.</comment></comment></comment>


The Podbike: four wheels, electric assist and protection against the weather - <AUTHOR> https://www.podbike.com/en/ <comment><AUTHOR> Given all the recent threads about "Neighborhood Electric Vehicle", thought I'd post this.

Interesting how the projected price isn't much cheapers than current existing NEVs.

When I used to commute to work, I used a bicycle a lot, and indeed spring was the time I skipped the bicycle and used the train a lot because of the rain, and often wished for something like the podbike.</comment>


As More Electric Cars Arrive, What's the Future for Gas-Powered Engines? - <AUTHOR> https://www.npr.org/2019/02/16/694303169/as-more-electric-cars-arrive-whats-the-future-for-gas-powered-engines <comment><USER2> Death-cycle. Fewer petrol stations, fewer mechanics, fewer sales stores, fewer places where they are allowed to be used.

But, its not happening fast enough. We need to ban fossil-fuel powered non- commercial vehicles starting from 2021. We have the technology, we just need a massive ramp-up of manufacturing and production.</comment>


Google Protocol Buffers - Open Sourced - <AUTHOR> http://google-opensource.blogspot.com/2008/07/protocol-buffers-googles-data.html

<comment><USER2> It's great to see Google has a brain about this. XML is sooo hyped up, and there are so many fan boys that it's generally not worth getting into a discussion about its merits lest you start a religious war. It's cool that Google looks beyond the Dilbert-like mission statement of XML and recognizes its failures as well as its strengths, not to mention the suitability to purpose for their needs.

<comment><USER3> Agreed.

When I first saw Gmail without folders, I was like wtf!idiots. Then I used it a bit and I was like wtf!sweet.

Same thing with Google Maps. I heard about them using Javascript and I was like wtf!idiots. And then I finally got to see it and I was like wtf!whoa.

</comment></comment><comment><USER4> Similar to Facebook Thrift (<http://developers.facebook.com/thrift/>)?

<comment><USER5> Thrift was inspired by ProtocolBuffers. Ex-googlers who went to Facebook reimplemented PB. In fact Facebook was hiring people explicitly with experience with PBs.

<comment><USER6> And by inspired, you mean "a complete rip-off of".

I talked with some of the guys who did Thrift, and while they didn't steal code, the concept and config files are close to identical.

<comment><USER7> Identical config files aside, the concept itself is rather simple and it has been around for ages in a form of ASN.1 and its encodings. It also routinely used in custom network protocols, e.g. IPC or RPC ones. Anyone outside of the XML fanboy club is aware of these things :-)

File format wise, it's hardly a rocket science too. When developers re- implement something, it's only natural to recycle an established implementation element. Changing it _just_ for the sake of being different from another vendor is frankly quite dumb.

So I'd be careful with bold "rip-off" statements. In the end " _Everything new is a well-forgotten old_ ".

<comment><USER6> Well, I don't believe calling it a rip-off was necessarily criticism. As they say: mediocrity borrows, genius steals.

Also, I must plead ignorance. I've never worked with ASN.1 or these other formats. From my cursory examination, ASN.1 seems to be far more complex.

<comment><USER8> this is a complete fabrication. facebook's thrift was inspired by pillar, an rpc library written in ml by former cto adam d'angelo.

<comment><USER6> I understand why Facebook might want to claim this in public, but don't call people 'fabricator', okay? You never know when someone might just have conclusive evidence that proves you wrong.

Don't get me wrong, I admire Facebook for making Thrift public, and it was a great thing for the internet. It was probably foolish of me to focus on the lineage of the thing.

</comment></comment></comment></comment></comment></comment></comment><comment><USER9> Is this something like modern day ASN.1?

<comment><USER7> Yep, in encoding part it's quite similar to PER - <http://en.wikipedia.org/wiki/Packed_Encoding_Rules>

</comment></comment><comment><USER10> Neat!

There's an article in the 1997 Game Developer's Conference Proceedings on a similar technique that Naughty Dog used in their Crash Bandicoot tool-chain.

Given a binary file format, it's nice to be able to specify a reader or writer code-generator with a declarative syntax that looks just like the file format spec.

</comment><comment><USER11> I am sure the scale of their applications demands optimizing as much as possible. I use yaml for this purpose. And find it very flexible and easy to use. It is also more readable than xml.

</comment><comment><USER12> Never trust an IDL whose performance is benchmarked against XML.

<comment><USER13> The fact that Google said this is much faster than XML does not mean that the IDL might be slower than other IDLs. In fact, the way it is implemented, I see no reason why it should be slow. If you want to find a drawback, it's the fact that the IDL is actually less _expressive_. Whether that matters or not in real life is another matter completely.

<comment><USER12> I'm not saying it's slow. I'm saying, "an order of magnitude faster than XML" isn't saying much. They've chosen the wrong benchmark. How does it compare to Pickle, Marshal, XDR, DCE, and Thrift?

For that matter, how does it compare to JSON?

I'm not really making a point about how fast Protocol Buffers are. I'm making a point about how the article was written.

<comment><USER6> Pickle, Marshal, and JSON are all ways of describing the kind of data structures used by Perl, Python, Javascript, and so on. They are way more flexible than Protocol Buffers -- you can have arbitrarily nested structures of arbitrary length. The format of the messages are basically self-describing. By which I mean that if they want to put an array there, they just put a '[' in and away we go.

I think XDR describes fixed-format structures. I don't know anything about DCE.

As described above, Thrift is a clone of Protocol Buffers.

Thrift and PB are a lot like XDR or god forbid, CORBA, but with a few twists. They define message templates for how to parse and emit relatively simple binary structures, which are then compiled into RPC clients and servers. PBs are cross-platform, at least by Google standards: they work with both C++ and Java very well. So you get the speed and convenience of making RPC calls with binary data structures in a mixed C++/Java environment.

But the best thing about PBs is that the message structure can evolve. If a message suddenly has a new field the receiver can't use, the receiver doesn't panic or read in garbage, unlike many other binary formats. So you can upgrade clients and servers in a gradual manner, without any downtime. This is what makes PBs especially suited to cloud computing.

<comment><USER12> You know, forwards compatibility is a feature of a great many binary protocols, including DNS, RADIUS, SNMP, and DHCP (itself a forward-compatible hack on BOOTP). So, PB is just another TLV encoding?

<comment><USER6> It seems like it. Before today I only had a vague idea how ASN.1 worked, and I never really delved into the wire format of PBs either. But yes, it seems to write a tag then a serialization of whatever the value is. Lengths are only required for strings because the format doesn't define any other type with variable length.

<comment><USER12> ASN.1/BER (you really mean BER) is the example most protocol dorks bring up when they argue against IDLs and structured protocols. It's incredibly complicated (though not as hard to implement as CORBA/IIOP).

<comment><USER14> indeed. i have used ASN.1 for a bunch of SNMP and GGSN accounting stuff, it is a pita. xdr on the other hand, is a breeze. also, xdr does support variable length fields.

<comment><USER15> hey signa11, are you a software developer for cellphone networking equipment?

<comment><USER14> yes.

<comment><USER15> kool, Im a SGSN/GGSN engineer by trade, but I don't do the programming aspect.</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment>


Grappple - - #inspirational - #socialBUCKETLIST - #photo #feed - - <AUTHOR> http://grappple.com/

<comment><AUTHOR> Does anyone want to ask any questions about what we are going to do?</comment>


Q&A with someone successfully employing the 400 $1-a-day projects approach - <AUTHOR> http://www.reddit.com/r/IAmA/comments/a74ba/not_an_adsense_millionaire_but_i_make_upwards_of/

<comment><AUTHOR> Interesting as this guy seems to follow a similar approach to that suggested in the recent popular post, "How to become rich even if nobody is following you on Twitter" [<http://news.ycombinator.com/item?id=1107828>]</comment>